feat: Add the file-based override source - #2053
kinyoklion wants to merge 5 commits into
Conversation
f2a06ab to
bab7cc6
Compare
b2a0f2f to
f3a0ea8
Compare
bab7cc6 to
c224cd1
Compare
f3a0ea8 to
2b97a8c
Compare
c224cd1 to
ed51797
Compare
1cab12b to
ada9b35
Compare
|
@launchdarkly/js-sdk-common size report |
|
@launchdarkly/js-client-sdk size report |
|
@launchdarkly/js-client-sdk-common size report |
ed51797 to
1f0e0c9
Compare
ada9b35 to
7c91764
Compare
1f0e0c9 to
ee3c511
Compare
7c91764 to
5d8aa0c
Compare
ee3c511 to
e4656e8
Compare
5d8aa0c to
60f7458
Compare
e4656e8 to
cd8301d
Compare
60f7458 to
9765955
Compare
cd8301d to
f8bd9a5
Compare
9765955 to
c720aed
Compare
f8bd9a5 to
3a6273a
Compare
c720aed to
bb5486f
Compare
3a6273a to
5095497
Compare
bb5486f to
e630c23
Compare
5095497 to
7fdfeaf
Compare
e630c23 to
3fdf2c8
Compare
7fdfeaf to
1fe976f
Compare
3fdf2c8 to
f12acb8
Compare
1fe976f to
764637d
Compare
f12acb8 to
51a7a0d
Compare
764637d to
21504bf
Compare
51a7a0d to
a908d4b
Compare
|
bugbot review |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit a908d4b. Configure here.
| sink.setOverrides( | ||
| result.flags.map((flag) => flag.item), | ||
| result.segments.map((segment) => segment.item), | ||
| ); |
There was a problem hiding this comment.
Override layer ignores document map keys
Medium Severity
The file source merges and de-duplicates flags and segments by map key, but apply forwards each item as written and the overlay stores it under item.key. When those differ, the override is served for the wrong key, and two map keys that share one key property silently overwrite each other instead of failing the reload.
Additional Locations (2)
Reviewed by Cursor Bugbot for commit a908d4b. Configure here.
Adds the file-based override source described by the OVERRIDE specification,
configured as { type: 'file', paths: [...] } in the overrides option of the
data system options.
The source reads one or more JSON or YAML files in the file data source
document format and combines their entries in the configured order. Duplicate
keys across files fail the reload by default or keep the first file's entry
with the ignore handling. Change detection is one of two modes: polling, the
default, examines the files once per second with a one second minimum, and
watching reacts to change notifications for the files' directories. A
configured file that does not exist contributes no overrides, so a file can be
created later and deleting a file removes its overrides. A file that exists
but cannot be read or parsed fails that reload, keeps the last good overrides,
logs the failure, and retries. Every applied change is logged at Info level
with the overrides in effect and what each file supplied. The initial load
completes before the client evaluates anything.
An invalid configuration, such as no file paths or an unrecognized change
detection mode, fails client construction. An unrecognized duplicate keys
handling and a polling interval below the minimum are logged and replaced by
their defaults.
The Node.js SDK reads YAML files without configuration through the yaml
package, which it supplies to the shared code as a platform default. A
configured yamlParser takes precedence.
…d use the default
21504bf to
41a072c
Compare
a908d4b to
965b3da
Compare


Summary
This PR is based on the events branch because it completes the feature that the earlier branches build up, and it should merge after them.
This change adds the file-based override source described by the OVERRIDE specification to the Node.js server SDK. It is configured as
{ type: 'file', paths: [...] }in theoverridesproperty of the data system options, alongside the source object and factory forms that the earlier change added.The source reads one or more JSON or YAML files in the file data source document format, with optional
flags,flagValues, andsegmentsmembers, and combines their entries in the configured order. Duplicate keys across files fail the reload by default, or keep the first file's entry with theignorehandling. Change detection is one of two modes. Polling, the default, examines the files once per second, with a one second minimum, and works on every filesystem. Watching reacts to change notifications for the directories that contain the files. Change detection is in place before the initial load, so a change made between the two is not missed.A configured file that does not exist contributes no overrides, so a file can be created later and deleting a file removes its overrides. A file that exists but cannot be read or parsed fails that whole reload: the last good overrides stay in effect, the failure is logged once, and the source retries after one second. Every applied change is logged at Info level with the overrides in effect and what each file supplied, and an unchanged reload logs nothing. The initial load completes before the client evaluates anything, so an override present at startup takes effect from the first evaluation.
An invalid configuration fails client construction, the way an invalid file initializer does: no file paths, an unrecognized change detection mode, or a platform without filesystem support. An unrecognized duplicate keys handling is logged and replaced by its default. A polling interval below the minimum is logged and raised to one second, and one that is not a finite number or exceeds 2147483 seconds is logged and replaced by the default.
The specification requires YAML support without extra configuration. The Node.js SDK package gains a dependency on the
yamlpackage (2.9.1, already present in the lockfile as a transitive dependency of other workspaces) and supplies its parser to the shared code as a platform default. A configuredyamlParsertakes precedence. The shared server package has no new dependency.The existing file data sources keep their current behavior. The override source has its own document handling (format detection by file extension, validation of the document shape, entries keyed by their map key, value expansion to a flag that is on and serves the value by fallthrough, missing-file handling, retry) and none of it applies to the FDv1
FileDataSourceor the FDv2 file data initializer.The tests mirror the Go file source tests over a mock filesystem: initial load, YAML, ordered merge, duplicate handling, missing files that appear and disappear, the Info log on each change, quiet watching of an absent file, reloads in both modes, retention across a malformed edit with recovery through the retry, and close. A server-node test drives the source through the client on real files, including the built-in YAML parser.
SDK-3247
Note
Overview
Adds a file-based override source for experimental flag overrides via
dataSystem.overrides: { type: 'file', paths: [...] }. Local JSON/YAML files (same shape as file data sources:flags,flagValues,segments) are merged in path order, hot-reloaded via polling (default, 1s) or filesystem watching, and pushed into the existing override layer so evaluations can win over LaunchDarkly data withoverrideAffected: true.Shared server code introduces
FileOverrideSource, override-specific document parsing/validation, and expandedcreateOverrideSourcewiring (path validation, duplicate-keyfail/ignore, poll interval bounds). Missing files contribute nothing; bad reloads keep the last good snapshot with retry/logging. The Node server SDK adds theyamldependency and registers a defaultyamlParseron the platform (overridable per config).Coverage is mostly new unit/integration tests over mock and real filesystems;
TestLoggergainsgetMessagesfor log assertions.Reviewed by Cursor Bugbot for commit a908d4b. Bugbot is set up for automated code reviews on this repo. Configure here.