Feature Summary
The Python worker catches an exception while unpacking a data batch but only writes it to the worker log. It then consumes the remaining rows and can report successful completion with missing output.
Before: a row fails, later rows continue, and the coordinator receives no error.
Expected: the first failure is reported to the coordinator and the worker enters exception pause without consuming later rows.
Reproduction evidence:
- Send a two row data frame to a Python worker.
- Make tuple bookkeeping fail on the first row.
- Observe that both rows are attempted and the exception manager remains empty.
Version and commit evidence:
1.3.0-incubating-SNAPSHOT on main
Commit Hash (Optional)
50321e4
Relevant log output
RuntimeError: statistics backend unavailable
Proposed Solution or Design
Expected: the first failure is reported to the coordinator and the worker enters exception pause without consuming later rows.
Affected Area
Workflow Engine (Amber)
Feature Summary
The Python worker catches an exception while unpacking a data batch but only writes it to the worker log. It then consumes the remaining rows and can report successful completion with missing output.
Before: a row fails, later rows continue, and the coordinator receives no error.
Expected: the first failure is reported to the coordinator and the worker enters exception pause without consuming later rows.
Reproduction evidence:
Version and commit evidence:
1.3.0-incubating-SNAPSHOT on main
Commit Hash (Optional)
50321e4
Relevant log output
RuntimeError: statistics backend unavailable
Proposed Solution or Design
Expected: the first failure is reported to the coordinator and the worker enters exception pause without consuming later rows.
Affected Area
Workflow Engine (Amber)