feat(storage): Add full object checksum validation when resuming async bidi uploads#16138
feat(storage): Add full object checksum validation when resuming async bidi uploads#16138v-pratap wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request adds support for resuming unbuffered uploads with persisted data checksums in the asynchronous GCS client. When resuming, if CRC32C validation is enabled and the server provides persisted checksums, the hash function is initialized with the existing CRC32C value and size. Feedback highlights a critical bug in MakeAppendableWriter where a fallback to CreateNullHashFunction is needed when persisted_size > 0 but validation cannot be seeded, preventing incorrect checksum calculations. Additionally, the new unit test should be enhanced to explicitly assert that checksums are not sent when validation is disabled.
cdb9a70 to
10e29d0
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #16138 +/- ##
==========================================
- Coverage 92.72% 92.71% -0.01%
==========================================
Files 2356 2356
Lines 220517 220617 +100
==========================================
+ Hits 204468 204543 +75
- Misses 16049 16074 +25 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
When taking over or resuming an async bi-directional upload (resumable or appendable) via QueryWriteStatus or MakeAppendableWriter, if the server provides persisted_data_checksums, seed the client-side CRC32C hash function from the acknowledged persisted state and forward the checksums directly to AsyncWriterConnectionImpl. This enables complete end-to-end full object integrity verification upon finalization.