[twiss]

For SHA-3, cSHAKE, and KMAC inputs and outputs, align the encoding of bit strings to byte sequences and the decoding of byte sequences to bit strings with the convention defined in FIPS 202 (Appendix B.1), i.e. putting the least significant bit first rather than last within the bytes.

(Based on https://github.com/panva/webcrypto-modern-algos/tree/wip.)

---

Preview | Diff

[panva]

Compared to my original wip, this drops the right shift on KMAC key import and the corresponding left shift on export. Those shifts keep partial-byte keys aligned with WebCrypto’s existing external representation while using Keccak bit ordering internally.

This matters for deriveKey, which passes the deriveBits output directly to key import. ECDH and X25519 put meaningful bits in the high positions of a partial final byte and zero-pad the low positions. For a 129-bit derivation, the final byte is 0x00 or 0x80: my wip preserves that last derived bit, whereas this PR imports a zero in both cases.

Export/import and wrap/unwrap work with either convention when the original length is supplied, but deriveKey exposes the difference.

[panva]

And for the record I'd rather see non-byte lengths purged from WebCrypto entirely.

[twiss]

Compared to my original wip, this drops the right shift on KMAC key import and the corresponding left shift on export. Those shifts keep partial-byte keys aligned with WebCrypto’s existing external representation while using Keccak bit ordering internally.

This matters for deriveKey, which passes the deriveBits output directly to key import. ECDH and X25519 put meaningful bits in the high positions of a partial final byte and zero-pad the low positions. For a 129-bit derivation, the final byte is 0x00 or 0x80: my wip preserves that last derived bit, whereas this PR imports a zero in both cases.

Export/import and wrap/unwrap work with either convention when the original length is supplied, but deriveKey exposes the difference.

Yeah. The problem is that if you call importKey directly then you might still expect it to match the convention for Keccak function inputs. Obviously you can shift the bits manually but it would still be unexpected that that's required.

When combining a function with big-endian bit convention with a function with a little-endian bit convention, there's indeed a mismatch, but I think it would be reasonable to just advise to not do that (with non-integer numbers of bytes).

And for the record I'd rather see non-byte lengths purged from WebCrypto entirely.

I somewhat agree, and we could do that for these new functions at least in a follow-up PR. It seems the official test cases also don't cover this: https://csrc.nist.gov/CSRC/media/Projects/Cryptographic-Standards-and-Guidelines/documents/examples/KMAC_samples.pdf.

But, we'll still need the Keccak encoding/decoding steps, also for KMAC keys, and that's the part that this PR adds.

[panva]

Somewhat agree that the encoding/decoding steps are needed to connect WebCrypto's bytes to NIST's bit-string notation. For whole-byte inputs, a byte-oriented implementation already follows that convention and needs no additional conversion though so adding any additional text may actually suggest something extra is needed for the very simple, existing implementations/libraries wiring, cases. A note saying that byte-oriented implementations require no additional bit reversal or byte transformation might be due.

I somewhat agree, and we could do that for these new functions at least in a follow-up PR.

I think we should go there straight away.

[twiss]

For whole-byte inputs, a byte-oriented implementation already follows that convention and needs no additional conversion though so adding any additional text may actually suggest something extra is needed for the very simple, existing implementations/libraries wiring, cases. A note saying that byte-oriented implementations require no additional bit reversal or byte transformation might be due.

Makes sense, I've added a note. Btw, a byte-oriented implementation should also already follow this convention even for non-integer numbers of bytes (it'd need an extra parameter for the length in bits, but would then only read the low-order bits of the last byte), so this applies regardless of whether we support non-integer number of bytes or not.

I think we should go there straight away.

That change is mostly orthogonal to this one, and we need all the text from this PR regardless, so can we merge this first so that the spec is at least correct? :)

[panva]

Approved, though i'm not happy about the interaction with deriveKey and its lost bits upon import, that alone is cause for a warning not to do that, which still leaves an API that is not right working instead of throwing.

[twiss]

Thanks! Makes sense, I'll make a followup PR. (Edit: #84)