Conversation
A header value cannot carry characters above Latin-1, so a challenge escapes
them as `\uXXXX` and the reference implementation decodes them back on parse.
readQuotedAuthParamValue treated the backslash as a plain RFC 9110 quoted-pair,
dropping it and keeping the rest verbatim, so a description serialized as
description="Payment — 50% off ☕"
parsed as "Payment u2014 50% off u2615".
Decode the escape, recombining the surrogate pairs that astral characters are
serialized as. An unpaired surrogate becomes U+FFFD, which is the closest value
a Go string can hold. A doubled backslash is still consumed first, so `\\u2014`
stays the literal text.
Characters at or below Latin-1 travel unescaped and already parsed correctly;
they are covered here to pin that down.
Contributor
Author
|
Companion fixes for the same escape, so the three SDKs land on the reference behaviour together: tempoxyz/pympp#247 (Python), tempoxyz/mpp-rs#418 (Rust). |
This was referenced Sep 21, 2026
Contributor
|
Superseded by #168, which expands this Unicode fix to parsing and formatting with surrogate, malformed-escape, and round-trip coverage, and is covered by tempoxyz/mpp-tools#240. Thank you for identifying this. |
brendanjryan
added a commit
that referenced
this pull request
Sep 21, 2026
## Motivation Quoted challenge parameters encoded by mppx use UTF-16 escapes, but the Go parser treated them as literal text. ## Summary - decode BMP and surrogate-pair escapes in quoted parameters - emit canonical UTF-16 escapes for non-Latin-1 text - cover literal, truncated, unpaired-surrogate, BMP, astral, and raw Latin-1 cases ## Key design considerations - preserve ordinary quoted-pair semantics - unpaired surrogates decode to U+FFFD - `description="Pay with 😀"` round-trips through `\ud83d\ude00` - supersedes [#154](#154) --------- Co-authored-by: Kanan <93033289+kriss39@users.noreply.github.com>
brendanjryan
added a commit
to tempoxyz/mpp-tools
that referenced
this pull request
Sep 21, 2026
## Motivation Challenge-header behavior has drifted across SDKs because Unicode escape and extension-method cases were absent from the shared vectors. ## Summary - add BMP, astral, realm, literal escape, truncated escape, and raw Latin-1 scenarios - add the canonical extended method identifier scenario - exercise parse, format, and round-trip behavior where applicable ## Key design considerations - mppx wire behavior remains the golden reference - SDK updates: [mpp-rs #428](tempoxyz/mpp-rs#428), [mpp-go #168](tempoxyz/mpp-go#168), and [pympp #256](tempoxyz/pympp#256) - original reports: [mpp-rs #414](tempoxyz/mpp-rs#414), [mpp-rs #418](tempoxyz/mpp-rs#418), [mpp-go #154](tempoxyz/mpp-go#154), and [pympp #247](tempoxyz/pympp#247) - keep this draft until the pinned SDK revisions contain the corresponding fixes --------- Co-authored-by: Kanan <93033289+kriss39@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A header value cannot carry characters above Latin-1, so
mppxescapes them as\uXXXXwhen it serializes a challenge and decodes them again when it parses one.readQuotedAuthParamValuetreats the backslash as a plain RFC 9110 quoted-pair — drop the backslash, keep the next byte — so the escape survives as literal text.Round-tripping a description through the reference implementation and then through mpp-go:
pymppandmpp-rsdecode it the same wrong way; companion PRs are open for both.Descriptionis excluded from the challenge binding, so a corrupted one is cosmetic. The same escape is applied to every quoted parameter though, includingrealm, which is slot 0 of the challenge ID HMAC — a realm above Latin-1 would come back altered and the binding check would then fail.Fix
Decode the escape, matching
readQuotedAuthParamValueinmppx:\uXXXXbecomes that code unit.😀→😀), so a high surrogate is combined with the escape that follows it viautf16.DecodeRune.\\u2014stays the literal text—— the same reasoning the reference implementation documents for why a bare\uis unambiguous.Only the parse side changes, and only the escape handling: raw bytes still go through
WriteByte, so characters at or below Latin-1, which the serializer leaves unescaped, are unaffected. They are covered in the tests to pin that down.Tests
Seven cases in
TestParseChallengeDecodesUnicodeEscapes: BMP escapes, an astral surrogate pair, raw Latin-1, both unpaired surrogates, a doubled backslash, and a truncated escape. Four fail onmain.go test ./...andgo vet ./...pass,gofmtis clean.The expectations were taken from
mppxrather than written by hand: the header shape in the tests is whatChallenge.serializeproduces, and each expected value is whatChallenge.deserializereturns for it.