Route zstd compression config through EVCacheTranscoderProperties bundle (stacked on #197)#199
Closed
joegoogle123 wants to merge 2 commits into
Closed
Conversation
added 2 commits
July 2, 2026 11:27
…atible reads
Hashed-key values are wrapped in an EVCacheValue envelope
(key, value, flags, ttl, createTime) currently serialized with Java
ObjectOutputStream, adding ~50-80 bytes of structural overhead per item.
This adds a compact, length-prefixed binary format for the envelope while
remaining fully backwards-compatible on reads, plus a first-class typed
property bundle (EVCacheTranscoderProperties) that lives on the base
transcoder and drives the transcoder's runtime behavior.
## Wire format
[magic 0x0C][reserved 0x00][int keyLen][key UTF-8][int valLen][value]
[int flags][long ttl][long createTime]
- Leading magic byte disambiguates from Java's 0xAC 0xED.
- Reserved byte at index 1 is read-and-ignored today; future readers
can branch on it for backwards-compatible format changes.
- Reads auto-detect format by leading byte: 0xAC 0xED -> Java, 0x0C -> binary.
## New classes
- EVCacheValueSerde (com.netflix.evcache.pool): the codec. Bounds-checks
length prefixes before allocating; corrupt payloads warn-log the failing
field + truncated hex dump (via Apache Commons Hex.encodeHexString, capped
at 1024 bytes) and return null. Matches BaseSerializingTranscoder's
resilience contract: corruption -> cache miss -> caller refills.
- EVCacheTranscoderProperties (com.netflix.evcache.config): typed access
to the FastProperties that govern transcoder behavior. Three keys today
(BINARY_SERIALIZATION_ENABLED, MAX_DATA_SIZE_BYTES, COMPRESSION_THRESHOLD_BYTES).
The Key enum is the extension point for future keys. Per-app -> global ->
static-default resolution chain; static value cached at construction for
hot-path reads.
## EVCacheSerializingTranscoder integration
The bundle is a protected final field on the base class:
protected final EVCacheTranscoderProperties properties;
Subclasses inherit one source of truth for FP resolution. Legacy
constructors are preserved and internally build a null-appName bundle.
## Feature Property rollout gate
Resolved through EVCacheTranscoderProperties with the three-level fallback:
- Per-app: <appName>.binary.serialization.enabled
- Global: default.evcache.binary.serialization.enabled
- Default: false
Read once at client construction and cached as a primitive on the
(immutable) transcoder ==> deploy/restart required to take effect; NOT a
live runtime toggle. Flip the property, then redeploy the consuming app.
## Compatibility
- New clients decode existing Java-serialized values unchanged (dual-format read).
- With the FP off (default), wire output is byte-identical to today.
- Corrupt binary payloads degrade to cache miss (null), matching the Java path.
- Public EVCacheTranscoder constructors (0-arg, int, int int) preserved.
## Testing
- EVCacheValueSerdeTest — 17 cases via public transcoder API: binary
round-trip edge cases, wire-shape assertions, backwards-compat with
legacy Java bytes, non-EVCacheValue passthrough, size-win, malformed
binary paths, forward-compat with optional additive fields.
- EVCacheTranscoderPropertiesTest — 15 cases (5 per key) covering the
three-level resolution chain. Property keys are asserted as string
literals so an enum rename fails the tests loudly.
- ./gradlew :evcache-core:test green.
- Smoke-tested end-to-end against a locally-published snapshot in a Spring
Boot Netflix sample app; round-trip put/get/delete on hashed keys passes.
Chunked-payload integration remains covered by EVCacheClient's existing
byte-level assembleChunks + CRC path; the binary format introduces no new
chunking risk by construction.
…sion through EVCacheTranscoderProperties Combines PR #196 (binary EVCacheValue serde + EVCacheTranscoderProperties bundle) with PR #197 (zstd compression on EVCacheSerializingTranscoder) and eliminates PR #197's setter-based Property injection in favor of constructor-time bundle resolution. ## Merge resolution - EVCacheImpl.java: kept PR #196's envelope-transcoder wiring (uses EVCacheTranscoderProperties, COMPRESSION_THRESHOLD_BYTES local constant). - EVCacheSerializingTranscoder.java: kept PR #197's zstd machinery (CompressionAlgorithm enum, ZSTD_MAGIC, compress/decompress overrides, compressionRatioSummaries) AND PR #196's protected final properties field + (EVCacheTranscoderProperties, int) constructor. Dropped: * public setCompressionAlgorithmProperty / setCompressionLevelProperty * Property<String> compressionAlgorithmProperty / Property<Integer> zstdLevelProperty (as mutable, setter-set fields) * protected final String appName (now read from properties.getAppName()) * (String appName, int max) constructor (redundant with bundle path) - EVCacheTranscoder.java: dropped PR #197's PropertyRepository config constructors and the private getProperty helper; kept PR #196's (EVCacheTranscoderProperties) constructor path and binary-serde dispatch in serialize()/deserialize(). - Base(Ascii)ConnectionFactory.java: getDefaultTranscoder() now builds an EVCacheTranscoderProperties(appName, repository) and passes it to EVCacheTranscoder. ## New properties in the bundle EVCacheTranscoderProperties.Key now includes: COMPRESSION_ALGORITHM ("compression.algorithm", "default.evcache.compression.algorithm") COMPRESSION_ZSTD_LEVEL ("compression.zstd.level", "default.evcache.compression.zstd.level") With static defaults DEFAULT_COMPRESSION_ALGORITHM="GZIP" and DEFAULT_COMPRESSION_ZSTD_LEVEL=3. Global-key names match what PR #197's description advertised; PR #197's Java source used "evcacheclient.compression.*" — those are normalized to "default.evcache.compression.*" here. ## Property<T> handle pattern (setter replacement) EVCacheSerializingTranscoder resolves both compression properties to Archaius Property<T> handles once at construction: private final Property<String> compressionAlgorithmProperty; private final Property<Integer> zstdCompressionLevelProperty; // in ctor: this.compressionAlgorithmProperty = properties.resolveProperty( Key.COMPRESSION_ALGORITHM, String.class, DEFAULT_COMPRESSION_ALGORITHM); this.zstdCompressionLevelProperty = properties.resolveProperty( Key.COMPRESSION_ZSTD_LEVEL, Integer.class, DEFAULT_COMPRESSION_ZSTD_LEVEL); Two private getters return the handles; compress() invokes .get() on them each call, so live FP updates propagate without JVM restart (same dynamic-read semantics we verified end-to-end via the throwaway LOG_PUT_ENABLED FP smoke test). ## Bundle additions - EVCacheTranscoderProperties.getAppName() — public accessor so cross-package callers (EVCacheSerializingTranscoder) can read the bundle's app name; keeps a single source of truth (dropped the parent-class appName field). - EVCacheTranscoderProperties.resolveProperty(Key, Class, T) — returns the Property<T> handle instead of T. ## Testing ./gradlew :evcache-core:test BUILD SUCCESSFUL. - EVCacheSerializingTranscoderTest (PR #197): 22 tests pass, rewritten to build transcoders via EVCacheTranscoderProperties instead of the removed setters. Global keys use default.evcache.compression.*, per-app keys use <appName>.compression.*. - EVCacheTranscoderPropertiesTest (PR #196): 15 tests pass. - EVCacheValueSerdeTest (PR #196): 17 tests pass. No public API on the primary EVCache transcoder path changed except: - EVCacheSerializingTranscoder loses setCompressionAlgorithmProperty and setCompressionLevelProperty (only PR #197 introduced them; no prod call site can exist yet since PR #197 hasn't merged). - EVCacheTranscoder(String, PropertyRepository, ...) constructors are dropped; callers pass an EVCacheTranscoderProperties instead.
janewang1680
reviewed
Jul 2, 2026
| public Transcoder<Object> getDefaultTranscoder() { | ||
| return new EVCacheTranscoder(appName, | ||
| client.getPool().getEVCacheClientPoolManager().getEVCacheConfig().getPropertyRepository()); | ||
| return new EVCacheTranscoder(new com.netflix.evcache.config.EVCacheTranscoderProperties(appName, |
There was a problem hiding this comment.
We should one instance of EVCacheTranscoderProperties per appName.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Combines PR #196 (binary
EVCacheValueserde +EVCacheTranscoderPropertiesbundle) with PR #197 (zstd on
EVCacheSerializingTranscoder) and replacesPR #197's public setter-injection API with typed
Property<T>handles ownedby the bundle.
What changes vs #197
setCompressionAlgorithmPropertyandsetCompressionLevelPropertyfrom
EVCacheSerializingTranscoder. Also removed the two mutableProperty<T>fields those setters wrote to.protected final String appNamefield onEVCacheSerializingTranscoder— single source of truth isproperties.getAppName().EVCacheTranscoder(String, PropertyRepository, ...)constructoroverloads. Callers pass an
EVCacheTranscoderPropertiesinstead.CompressionAlgorithmenum fromEVCacheSerializingTranscoderto
EVCacheTranscoderPropertiesso the bundle can expose it as a typedreturn value.
evcacheclient.compression.*(Java source) todefault.evcache.compression.*(matching PR Add Zstd support for EVCache #197's own PR description and the naming convention already
used by
default.evcache.max.data.size/default.evcache.compression.threshold).The bundle now owns the compression config
New keys and defaults in
EVCacheTranscoderProperties:The bundle stores two live
Property<T>handles as final fields, resolvedonce at construction:
The
Property<CompressionAlgorithm>is built via Archaius's built-inProperty.map(Function<T, S>)..get()on the mapped handle re-parses theunderlying String on each call, so live FP updates propagate without JVM
restart and callers never see a stale enum. The upper-case wrap keeps the
FP case-insensitive.
EVCacheSerializingTranscoder.compress()becomes trivialNo null-checks, no
toUpperCase()+valueOf()boilerplate, no local fieldplumbing.
Testing
./gradlew :evcache-core:test— BUILD SUCCESSFUL.EVCacheSerializingTranscoderTest(PR Add Zstd support for EVCache #197's suite): 22 tests pass,rewritten to build transcoders via
new EVCacheTranscoderProperties(appName, repo)instead of the removedsetters. Tests exercise all four combinations of {global, per-app} × {algo,
level} keys plus cross-decode via magic-byte detection.
EVCacheTranscoderPropertiesTest(PR EVCacheValue: opt-in compact binary serialization with backwards-compatible reads #196's suite): 15 tests pass.EVCacheValueSerdeTest(PR EVCacheValue: opt-in compact binary serialization with backwards-compatible reads #196's suite): 17 tests pass.Dynamic-FP behavior end-to-end verified separately on a Spring Boot Netflix
sample app (via
metatron curl -a gate POST /fastproperties/upsertand observedpropagation into the running JVM's REMOTE Archaius layer within ~30–45s).
Migration notes for downstream callers
PR #197 hasn't merged yet so the blast radius is small — the removed
setters have no external call sites. The renamed Archaius keys
(
default.evcache.compression.*instead ofevcacheclient.compression.*)similarly can't be set in prod yet.
If a downstream caller of
EVCacheTranscoderwas passing(appName, PropertyRepository)(only possible on PR #197's WIP branch),replace with:
Compatibility
Public API on
EVCacheSerializingTranscoder:setCompressionAlgorithmProperty(Property<String>),setCompressionLevelProperty(Property<Integer>).EVCacheSerializingTranscoder(EVCacheTranscoderProperties, int).EVCacheSerializingTranscoder(),EVCacheSerializingTranscoder(int max).Public API on
EVCacheTranscoder:(String),(String, PropertyRepository),(String, int),(String, PropertyRepository, int),(String, int, int),(String, PropertyRepository, int, int)(all PR Add Zstd support for EVCache #197 additions).EVCacheTranscoder(EVCacheTranscoderProperties).(),(int max),(int max, int compressionThreshold).