Skip to content

[query] Normalize options for aggregated histograms#3556

Open
arnikola wants to merge 13 commits into
masterfrom
process-le-rollup-series-together
Open

[query] Normalize options for aggregated histograms#3556
arnikola wants to merge 13 commits into
masterfrom
process-le-rollup-series-together

Conversation

@arnikola

@arnikola arnikola commented Jun 11, 2021

Copy link
Copy Markdown
Collaborator

What this PR does / why we need it:
Due to the way the aggregator and prom style histograms interact, there is an edge case where histogram bucket datapoints arrive with slight drift; this is fine and should not impact data quality, but can lead to counter resets which cause incorrect and unpredictable behavior when querying. This PR gathers all aggregated histograms by series, then normalizes them in such a way that bucket value at each timestamp is monotonically increasing across le tags (when viewed in descending le size; i.e. +Inf should always have a smaller value than 0.1)

TODO:

  • test for histogram gathering
  • integration test
  • criteria for normalizing counter series

@codecov

codecov Bot commented Jun 15, 2021

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 55.9%. Comparing base (28c1d95) to head (81da16a).
⚠️ Report is 542 commits behind head on master.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #3556     +/-   ##
========================================
- Coverage    56.2%   55.9%   -0.4%     
========================================
  Files         550     550             
  Lines       61886   61506    -380     
========================================
- Hits        34799   34391    -408     
- Misses      23992   24006     +14     
- Partials     3095    3109     +14     
Flag Coverage Δ
aggregator 57.1% <ø> (ø)
cluster ∅ <ø> (∅)
collector 54.3% <ø> (ø)
dbnode 60.3% <ø> (-0.5%) ⬇️
m3em 46.4% <ø> (ø)
metrics 19.8% <ø> (ø)
msg 74.4% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.


Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 28c1d95...81da16a. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants