Repository navigation
[Backport release-1.19] fix: accept workflow work items larger than gRPC's 4MB default (#1860) - #1865
Merged
javier-aliaga merged 1 commit intoOct 22, 2026
Conversation
…1860) The workflow runtime builds its sidecar channel with the SDK's inbound message limit, dapr.grpc.max.inbound.message.size.bytes, which defaults to 4MB. The sidecar sends work items up to its own max body size, which apps raise for large payloads, so a work item over 4MB (for example a fan-out of five 1MB activity results) failed the GetWorkItems stream with RESOURCE_EXHAUSTED. The worker reconnected, the sidecar redelivered the same work item, and it failed again, so every workflow on the worker stalled. The history fetch after a stateful history cache miss had the same limit. Signed-off-by: joshvanl <me@joshvanl.dev> (cherry picked from commit ae6016d) Signed-off-by: joshvanl <me@joshvanl.dev>
javier-aliaga
approved these changes
Oct 22, 2026
javier-aliaga
changed the title
Contributor
There was a problem hiding this comment.
🟢 Approval recommended
The narrowly scoped override addresses both affected receive paths, and the regression test exercises actual transport message-size enforcement.
No open issues 🎉
What changed in this PR
This PR prevents oversized workflow work items and history responses from repeatedly failing the worker’s gRPC receive limit and stalling workflow processing.
Changes:
- Overrides the worker stub’s inbound message limit without changing the shared channel configuration.
- Adds a TCP-based regression test that verifies processing a 6 MB activity payload through a channel configured with a 4 MB limit.
| File | Description |
|---|---|
| durabletask-client/src/test/java/io/dapr/durabletask/DurableTaskGrpcWorkerLargeWorkItemTest.java | Tests successful processing of a work item exceeding the default receive limit. |
| durabletask-client/src/main/java/io/dapr/durabletask/DurableTaskGrpcWorker.java | Raises the worker stub’s inbound message limit for work items and history responses. |
Review effort: Balanced
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## release-1.19 #1865 +/- ##
==================================================
- Coverage 80.21% 80.11% -0.10%
Complexity 2798 2798
==================================================
Files 273 273
Lines 8278 8278
Branches 896 896
==================================================
- Hits 6640 6632 -8
- Misses 1270 1278 +8
Partials 368 368 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The workflow runtime builds its sidecar channel with the SDK's inbound message limit, dapr.grpc.max.inbound.message.size.bytes, which defaults to 4MB. The sidecar sends work items up to its own max body size, which apps raise for large payloads, so a work item over 4MB (for example a fan-out of five 1MB activity results) failed the GetWorkItems stream with RESOURCE_EXHAUSTED. The worker reconnected, the sidecar redelivered the same work item, and it failed again, so every workflow on the worker stalled. The history fetch after a stateful history cache miss had the same limit.
(cherry picked from commit ae6016d)
Description
Please explain the changes you've made
Issue reference
We strive to have all PR being opened based on an issue, where the problem or feature have been discussed prior to implementation.
Please reference the issue this PR will close: #[issue number]
Checklist
Please make sure you've completed the relevant tasks for this PR, out of the following list: