Skip to content

chore(appsec): tag spans with refreshed rc client id - #19819

Closed
litianningdatadog wants to merge 1 commit into
tianning.li/3-1-remoteconfig-identity-clientfrom
tianning.li/3-2-appsec-rc-client-id
Closed

litianningdatadog wants to merge 1 commit into
tianning.li/3-1-remoteconfig-identity-clientfrom
tianning.li/3-2-appsec-rc-client-id

Conversation

@litianningdatadog

@litianningdatadog litianningdatadog commented Aug 23, 2026 •

Copy link
Copy Markdown
Contributor

Stacked PRs:

Description

Split from #19780.

AppSec reports the Remote Config client id on spans. This PR changes span tagging to read the live Remote Config client id at span finalization instead of copying the id into asm_config when AppSec RC is enabled.

This is necessary because MicroVM runtime identity refresh rebuilds the Remote Config client with a fresh client id. A cached asm_config._rc_client_id would keep tagging spans with the old id after refresh. asm_config now only tracks whether AppSec RC client-id tagging is enabled; the id itself comes from the current RC client so spans reflect identity refreshes.

Testing

  • scripts/lint style ddtrace/appsec/_asm_request_context.py ddtrace/appsec/_remoteconfiguration.py ddtrace/internal/settings/asm.py tests/appsec/appsec/test_asm_request_context.py tests/appsec/appsec/test_remoteconfiguration.py
  • git diff --check
  • scripts/run-tests --venv 106f2d7 tests/appsec/appsec/test_asm_request_context.py tests/appsec/appsec/test_remoteconfiguration.py -- -- tests/appsec/appsec/test_asm_request_context.py tests/appsec/appsec/test_remoteconfiguration.py attempted, but the riot env failed before tests due missing native extension: ModuleNotFoundError: No module named 'ddtrace.internal.native._native'

Stack

Draft split branch. Stacked on the Remote Config split PR.

@litianningdatadog litianningdatadog added changelog/no-changelog A changelog entry is not required for this PR. aws-microvm Work related to AWS MicroVM onboarding labels Aug 23, 2026
@litianningdatadog litianningdatadog changed the title fix(appsec): tag spans with refreshed rc client id chore(appsec): tag spans with refreshed rc client id Aug 23, 2026
@datadog-prod-us1-4

datadog-prod-us1-4 Bot commented Aug 23, 2026 •

Copy link
Copy Markdown
Contributor

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🔄 Datadog auto-retried 1 job - 1 passed on retry View in Datadog

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b55c391 | Docs | View more details | Give us feedback!

@litianningdatadog
litianningdatadog force-pushed the tianning.li/3-1-remoteconfig-identity-client branch from 3e763c0 to ff3e857 Compare August 24, 2026 02:24
@litianningdatadog
litianningdatadog force-pushed the tianning.li/3-2-appsec-rc-client-id branch from 4d1e2f9 to f070e18 Compare August 24, 2026 02:24
@litianningdatadog
litianningdatadog force-pushed the tianning.li/3-1-remoteconfig-identity-client branch from ff3e857 to 25a94b2 Compare August 24, 2026 02:31
@litianningdatadog
litianningdatadog force-pushed the tianning.li/3-2-appsec-rc-client-id branch from f070e18 to eda6c45 Compare August 24, 2026 02:31
@litianningdatadog
litianningdatadog force-pushed the tianning.li/3-1-remoteconfig-identity-client branch from 25a94b2 to 997466f Compare August 25, 2026 13:22
@litianningdatadog
litianningdatadog force-pushed the tianning.li/3-2-appsec-rc-client-id branch from eda6c45 to 0369998 Compare August 25, 2026 13:23
@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against tianning.li/3-1-remoteconfig-identity-client using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

ddtrace/appsec/_asm_request_context.py                                  @DataDog/asm-python
ddtrace/appsec/_remoteconfiguration.py                                  @DataDog/asm-python
ddtrace/internal/settings/asm.py                                        @DataDog/asm-python
tests/appsec/appsec/test_asm_request_context.py                         @DataDog/asm-python
tests/appsec/appsec/test_remoteconfiguration.py                         @DataDog/asm-python

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 25, 2026 •

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 3 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.llmobs -> ddtrace.llmobs._evaluators -> ddtrace.llmobs._evaluators.format -> ddtrace.llmobs._experiment -> ddtrace.llmobs
ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector
ddtrace.appsec._asm_request_context -> ddtrace.appsec._iast._iast_request_context_base -> ddtrace.appsec._iast._iast_env -> ddtrace.appsec._iast.reporter -> ddtrace.appsec._exploit_prevention.stack_traces -> ddtrace.appsec._asm_request_context

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 25, 2026 •

Copy link
Copy Markdown

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 250 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 250 highest severity)
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=135)
ddtrace.llmobs._integrations.mcp -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)
ddtrace.profiling.scheduler -×-> ddtrace.trace  (product:profiling -> product:tracing, score=133)
ddtrace.profiling.collector.pytorch -×-> ddtrace.trace  (product:profiling -> product:tracing, score=133)
ddtrace.llmobs._integrations.vllm -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=133)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates AppSec span tagging to use the current Remote Config client ID after runtime identity refreshes.

Changes:

  • Replaces cached client-ID state with an enablement flag.
  • Reads the live client ID during span finalization.
  • Adds tests for refreshed IDs, disabled tagging, and lazy imports.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Summary
tests/appsec/appsec/test_remoteconfiguration.py Tests live and disabled client-ID tagging.
tests/appsec/appsec/test_asm_request_context.py Verifies lazy Remote Config worker imports.
ddtrace/internal/settings/asm.py Adds the tagging enablement flag.
ddtrace/appsec/_remoteconfiguration.py Manages Remote Config client-ID tagging state.
ddtrace/appsec/_asm_request_context.py Tags spans with the live client ID.
Suppressed comments (2)

ddtrace/appsec/_asm_request_context.py:404

  • finalize_asm_env runs on every finalized AppSec request, so this function-level import performs an import-cache/module lookup on every span while RC is enabled. That adds avoidable overhead to a hot path and hides the dependency from normal module initialization; register a live client-id accessor (or client reference) during enable_appsec_rc() and clear it in disable_appsec_rc(), then read that accessor here so the worker is not imported per request.
            from ddtrace.internal.remoteconfig.worker import remoteconfig_poller

ddtrace/appsec/_remoteconfiguration.py:88

  • Please add a regression assertion for the enabled-to-disabled transition here. The new test at test_remoteconfiguration.py:297 only covers the never-enabled case, so it would still pass if this assignment were removed; it does not verify that spans stop receiving _dd.rc.client_id after AppSec RC is disabled.
    asm_config._rc_client_id_enabled = False

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@pr-commenter

pr-commenter Bot commented Aug 25, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-08-25 13:43:28

Comparing candidate commit 0369998 in PR branch tianning.li/3-2-appsec-rc-client-id with baseline commit 3bb9ecd in branch main.

📊 Benchmarking dashboard

Found 0 performance improvements and 2 performance regressions! Performance is the same for 82 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

  • 🟩 = significantly better candidate vs. baseline
  • 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:iastaspectsospath-ospathbasename_aspect

  • 🟥 execution_time [+143.346µs; +150.770µs] or [+36.929%; +38.841%]

scenario:iastaspectssplit-rsplit_aspect

  • 🟥 execution_time [+23.461µs; +27.927µs] or [+17.194%; +20.467%]

@litianningdatadog
litianningdatadog force-pushed the tianning.li/3-2-appsec-rc-client-id branch from 0369998 to e849ad0 Compare August 25, 2026 15:06
@litianningdatadog
litianningdatadog force-pushed the tianning.li/3-1-remoteconfig-identity-client branch from 6e15d59 to e9a58bc Compare August 25, 2026 15:10
@litianningdatadog
litianningdatadog force-pushed the tianning.li/3-2-appsec-rc-client-id branch 2 times, most recently from 56a237b to f38c7a7 Compare August 25, 2026 15:16
@litianningdatadog
litianningdatadog force-pushed the tianning.li/3-2-appsec-rc-client-id branch from f38c7a7 to b55c391 Compare August 25, 2026 15:19
@litianningdatadog
litianningdatadog marked this pull request as ready for review August 25, 2026 15:21
@litianningdatadog
litianningdatadog requested a review from a team as a code owner August 25, 2026 15:21
@litianningdatadog
litianningdatadog requested review from avara1986 and removed request for a team August 25, 2026 15:21

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b55c391340

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

if asm_config._rc_client_id is not None:
entry_span.set_tag(APPSEC.RC_CLIENT_ID, asm_config._rc_client_id)
if asm_config._rc_client_id_enabled:
from ddtrace.internal.remoteconfig.worker import remoteconfig_poller

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Remove the deferred Remote Config worker import

When an AppSec request is finalized with RC tagging enabled, this imports the worker inside the hot-path function to avoid an import-time dependency, leaving _asm_request_context structurally coupled to the RC worker and even adding a test that enforces the workaround. Extract or inject a lightweight current-client-ID accessor instead; repository policy explicitly forbids leaving deferred imports in place to conceal import-graph problems.

AGENTS.md reference: AGENTS.md:L20-L21

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Codex’s comment makes sense. WDYT @christophe-papazian?

# Fetch the current id from the RC client at span finalization. asm_config only
# tracks whether AppSec RC enabled tagging; mirroring the id there would go stale
# when runtime identity refresh rebuilds the RC client.
rc_client_id = remoteconfig_poller._client.id

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve the polling client ID in forked workers

In a forked worker, the runtime-ID callback renews RemoteConfigClient.id and drops the inherited native client (client.py:142-159), while RemoteConfigPoller.reset_at_fork() disables agent polling and consumes the parent's snapshots through inherited shared memory (worker.py:144-165). Reading the renewed local id here therefore tags worker spans with a client ID that has never polled or registered with the agent, so AppSec traces from common prefork deployments cannot be correlated with the RC client that supplied their configuration; retain/expose the origin poller's ID for shared-memory consumers instead.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Codex’s comment makes sense. WDYT @P403n1x87?

@litianningdatadog

Copy link
Copy Markdown
Contributor Author

close it as it is out of the scope

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aws-microvm Work related to AWS MicroVM onboarding changelog/no-changelog A changelog entry is not required for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants