Skip to content

Fix parser resource handling and generator integration issues#40

Merged
anthonyprintup merged 3 commits into
mainfrom
codex/parser-generator-fixes
Jul 13, 2026
Merged

Fix parser resource handling and generator integration issues#40
anthonyprintup merged 3 commits into
mainfrom
codex/parser-generator-fixes

Conversation

@anthonyprintup

@anthonyprintup anthonyprintup commented Jul 13, 2026

Copy link
Copy Markdown
Owner

Summary

  • Fix packed fixed-width parsing to validate claimed payload availability before allocation or commit.
  • Add aggregate byte, allocation, repeated-element, map-entry, and recursion limits while preserving protobuf-compatible defaults.
  • Make Reader::can_read part of the required reader contract.
  • Validate generator requests, direct generator options, descriptor complexity, virtual output paths, output sizes, and formatter policy.
  • Verify prebuilt protoc archives with pinned SHA-256 values, bounded download and extraction, and transactional replacement.
  • Tighten CMake option handling, reserved transport validation, and exact package-version behavior.
  • Remove obsolete parameter and debugger compatibility paths.
  • Preserve freestanding C++20 support, regenerate checked smoke outputs, and update the documentation.

Why

Malformed or oversized inputs could trigger avoidable allocation and processing before their complete payloads were known to be available. Embedded generator use also lacked operator-controlled request and output budgets, while CMake could accept runtime options it could not model consistently. The CI protoc bootstrap downloaded and extracted release archives without validating their configured digest or bounding archive work.

Impact

Default local generator behavior remains unchanged unless an embedding caller supplies a GeneratorPolicy. Runtime wire limits retain protobuf-compatible defaults; finite collection and allocation budgets are explicit application policy. Custom readers and scalar vectors must implement the now-required can_read, append_trivial_range, and resize_for_overwrite contracts. Protocyte remains pre-1.0, so callers should update directly rather than depend on removed aliases or shims.

Validation

  • clang-format --dry-run --Werror on all 7 touched C++ files
  • Isolated regeneration: all 10 checked outputs matched byte-for-byte
  • uv run ruff check .
  • uv run pytest -q: 339 passed
  • Review regressions for direct options and encoded CMake transport: 7 passed
  • Windows clang-cl hosted smoke: 41/41 passed
  • Windows MSVC hosted smoke: 41/41 passed
  • clang-cl and MSVC WDK driver builds passed
  • README local-link and public-control checks passed

- validate packed payload availability and enforce aggregate parsing and resource budgets
- validate generator requests, descriptor graphs, output paths, output budgets, and formatter configuration
- verify prebuilt protoc archives with pinned SHA-256 values, bounded transfer and extraction, and transactional replacement
- tighten CMake option handling and remove obsolete debugger and namespace compatibility paths
- regenerate smoke fixtures and expand documentation and regression coverage for hosted and freestanding C++20 builds
@anthonyprintup

Copy link
Copy Markdown
Owner Author

@codex review

@anthonyprintup
anthonyprintup marked this pull request as ready for review July 13, 2026 20:41

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0eddadb05c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/protocyte/parameters.py
Comment thread cmake/ProtocyteFunctions.cmake Outdated
Comment thread src/protocyte/runtime/runtime.hpp
- validate runtime and include prefixes for direct GeneratorOptions construction
- reject reserved transport parameters passed through CMake OPTIONS
- document the reserved parameter namespace and add regression coverage

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b862156e04

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/protocyte/cpp.py Outdated
Comment thread src/protocyte/cpp.py Outdated
Comment thread src/protocyte/cpp.py Outdated
Comment thread src/protocyte/cpp.py
- charge repeated and map limits before decoding or allocation
- route nested parsing through a private runtime access gate so ReaderRef entry points retain top-level budgets
- stream formatter stdout and stderr under the remaining output cap
- add regression coverage, regenerate smoke outputs, and document formatter budget behavior

Copy link
Copy Markdown
Owner Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 2f6e07ba61

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@anthonyprintup
anthonyprintup merged commit ef7104d into main Jul 13, 2026
8 checks passed
@anthonyprintup
anthonyprintup deleted the codex/parser-generator-fixes branch July 13, 2026 22:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant