Security hardening: Tier 1 fixes from SECURITY.md sweep#41
Open
MarketDataApp wants to merge 3 commits into
Open
Security hardening: Tier 1 fixes from SECURITY.md sweep#41MarketDataApp wants to merge 3 commits into
MarketDataApp wants to merge 3 commits into
Conversation
- Percent-encode caller-supplied symbols in request paths (path traversal / query smuggling via untrusted input); valid symbols unaffected - Neutralize dot-segments in options.lookup() path while keeping valid slashes (dates) intact - Token obfuscation no longer reveals token length or any characters of short tokens - Bound API error messages extracted from response bodies (max 500 chars) - Malformed rate-limit headers no longer crash a successful request - Pin GitHub Actions to full commit SHAs; test workflow token read-only
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #41 +/- ##
=========================================
Coverage 100.00% 100.00%
=========================================
Files 52 52
Lines 2293 2311 +18
=========================================
+ Hits 2293 2311 +18
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Keep pypa/gh-action-pypi-publish on a fixed release tag (v1.14.0) rather than the moving release/v1 branch ref.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Full security sweep per SECURITY.md. All fixes below are Tier 1 (API- and behavior-compatible for legitimate consumers). Tier 2 findings are held back for maintainer discussion per policy.
Credential handling
obfuscate_tokenused a variable-width mask ("*" * (len-4) + last4), revealing the exact token length in debug logs — and up to 80% of short tokens. Now a fixed-width mask that never reveals length, and never reveals any characters of tokens ≤ 8 chars.Injection into outbound requests
f"stocks/earnings/{symbol}/"). A consumer passing untrusted end-user input as a symbol (e.g. a ticker search box) allowedAAPL/../../userto redirect the authenticated request to a different endpoint, orAAPL?x=yto smuggle query params. Path segments are now percent-encoded (quote(v, safe="")). Valid symbols (alphanumerics,.,-) are byte-identical after encoding.options.lookup(): the lookup string was quoted withsafe='/', so..segments survived. Slashes are legitimate in lookup strings (dates like7/28/2023), so they are kept — but all-dot segments are now percent-encoded so they cannot traverse. Valid lookup strings are unaffected.Deserialization safety
except:withexcept Exception:.x-api-ratelimit-*headers on an otherwise-successful response raised a rawKeyError/ValueErrorto the consumer. Now logs a warning and keeps the previously known limits.Supply chain (CI/publish pipeline)
pypa/gh-action-pypi-publishwas pinned torelease/v1— a moving branch ref — on the OIDC trusted-publishing job. Now pinned to the fixed release tagv1.14.0. Other actions stay on readable major-version tags; Dependabot (github-actionsecosystem) keeps them current.test.ymlnow setspermissions: contents: read(default token was potentially write-scoped).Tests
test_path_injection.py: end-to-end respx tests asserting encoded wire paths for traversal/smuggling inputs and unchanged behavior for valid symbols and lookup strings.obfuscate_tokenexpectations.355 tests pass (
uv run pytest -n 4). A few files show formatting-only diffs from./lint.sh(Black).Not in this PR (Tier 2 — needs maintainer approval)
Held back per policy; will summarize separately for discussion.