[aggregator] efficiently GC expired aggregation versions#3842
[aggregator] efficiently GC expired aggregation versions#3842rallen090 wants to merge 13 commits into
Conversation
rallen090
changed the title
| var cascadeDirty bool | ||
| var ( | ||
| cascadeDirty bool | ||
| expiredTimeNanos = make([]int64, 0) |
There was a problem hiding this comment.
Would it be better to make this a resettable field on GenericElem so that we don't have to re-alloc it each Consume?
| ) | ||
|
|
||
| type onForwardedAggregationDoneFn func(key aggregationKey) error | ||
| type onForwardedAggregationDoneFn func(key aggregationKey, expiredTimeNanos []int64) error |
There was a problem hiding this comment.
nit: i'd prefer using xtime.UnixNano for new code if possible.
| for k, v := range agg.buckets { | ||
| v.values = v.values[:0] | ||
| v.prevValues = v.prevValues[:0] | ||
| for i := 0; i < bucketLen; i++ { |
| b.prevValues = append(b.prevValues, prevValue) | ||
| if annotation != nil { | ||
| b.annotation = annotation | ||
| for i := 0; i < len(agg.buckets); i++ { |
|
|
||
| // internal consume state that does not need to be synchronized. | ||
| toConsume []timedAggregation // small buffer to avoid memory allocations during consumption | ||
| toExpire []int64 |
There was a problem hiding this comment.
lol we're gonna have a name conflict with the upcoming dirty changes
There was a problem hiding this comment.
well I guess all of this will be a bit of a semantical merge with those changes. no big deal.
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #3842 +/- ##
========================================
- Coverage 57.0% 56.8% -0.3%
========================================
Files 552 552
Lines 63129 63129
========================================
- Hits 36033 35887 -146
- Misses 23907 24038 +131
- Partials 3189 3204 +15
Flags with carried forward coverage won't be shown. Click here to find out more. Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
3 participants
What this PR does / why we need it:
Avoid scanning to cleanup expired agg versions.
Special notes for your reviewer:
Does this PR introduce a user-facing and/or backwards incompatible change?:
Does this PR require updating code package or user-facing documentation?: