Skip to content

Address misreported value on exemplars for counters.#776

Merged
jdmarshall merged 1 commit into
prometheus:mainfrom
jdmarshall:exemplarCounters
Jul 15, 2026
Merged

Address misreported value on exemplars for counters.#776
jdmarshall merged 1 commit into
prometheus:mainfrom
jdmarshall:exemplarCounters

Conversation

@jdmarshall

@jdmarshall jdmarshall commented Jul 13, 2026

Copy link
Copy Markdown
Contributor

After one AI user filed a PR to fix this the wrong way, I finally went tracing through the code to understand what the heck is wrong as implemented. The real issue turned out to be pretty dumb: mislabeled variables resulting in mischaracterization of the data it represents.

We're calling the increment step 'value' but we use the 'value' to call setDelta in incWithoutExemplars() and delegating to it from incWithExemplars() but then using the passed-in delta as a 'value' again when updating the 'value' field in the exemplar, which is not the same meaning of 'value'.

Within minutes of updating the ticket to set the labels and the problem, a completely separate user filed a PR. Maybe I should not have set helpWanted when a PR was already open. However, both PRs had an identical new unit test, and both of them were identically asserting the wrong thing, and solving the issue the wrong way.

At this point I had invested more time in both PRs than the authors had. The entire problem was front-of mind, and the existing test is wrong but written in a way where either interpretation would be correct. Clearly they are not.

Fixes #621

@jdmarshall jdmarshall requested a review from zbjornson July 13, 2026 02:55
Fixes prometheus#621

Signed-off-by: Jason Marshall <jdmarshall@users.noreply.github.com>
@jdmarshall

Copy link
Copy Markdown
Contributor Author

I initially had planned to rename the variables to make the distinction clear, but unfortunately the 'value' is exposed as part of the public API and I don't want to break existing code. Even though technically any existing code is already broken.

@jdmarshall

Copy link
Copy Markdown
Contributor Author

@zbjornson This is pretty much the last PR I really want in 0.16 if you could take a look.

@zbjornson zbjornson left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense.

@jdmarshall jdmarshall merged commit 73cc3ee into prometheus:main Jul 15, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Exemplar values in counter metrics are always 1

2 participants