feat(probes): add UTF fullwidth encoding probe#1864
Conversation
Signed-off-by: Varun Nuthalapati <nuthalapativarun@gmail.com>
|
Investigated the failing Root cause: unrelated infra flake, not caused by this PR's diff. Evidence:
No code changes made — this is a HuggingFace Hub rate-limit / GH Actions cache-miss flake on that one runner, unrelated to the probe implementation. Re-running the job should clear it once the cache is warm again or HF Hub rate limits reset. |
|
Just a friendly nudge on this one — CI's been green for a while and it's still mergeable. Happy to make any tweaks if there's feedback. |
|
Hi team — following up on PR #1864 (UTF fullwidth encoding probe). CI is green and the PR has been open for ~3 weeks with no review. Happy to address any feedback. Thanks for your time! |
There was a problem hiding this comment.
Can you provide any information on how this has been tested? The project needs to validate that the technique works, preferably against some live target, even if it is a minimal system. Something the team can reproduce is preferred.
The project recently added some guidance related to Scope/Naming/Substance criteria when adding new probe. While this PR looks to be attempting to fulfill an open issue, the original issue was vague. This probe may have intrinsic value to help determine if say an attacker can bypass the some alignment expectation or some guardrail protection of the target.
I guess a request here is for some clear documentation of how this adds signal and how practical it is for deployments to need to be evaluated in relation to this method of encoding. A small survey of a few recent targets may help establish the value from this probe.
Summary
InjectUTFFullwidthprobe togarak/probes/encoding.py, which encodes payloads by mapping ASCII printable characters (0x21-0x7E) to their Unicode fullwidth equivalents (U+FF01-FF5E), following the same pattern as the existingInjectZalgoandInjectLeetprobes.EncodingMixinbase,TEMPLATES, payload mechanism, tags, and detectors used by other encoding probes.Ref #152
AI assistance disclosure
This change was developed with AI assistance (Claude). The implementation follows the existing conventions of
garak/probes/encoding.py(mirroringInjectZalgo/InjectLeet). I reviewed the diff for correctness and style consistency before submitting.Test plan
python -m pytest tests/probes/test_probes_encoding.py -q- 226 passed, 1 skipped (skip is pre-existing/unrelated)python -m pytest tests/probes -q -k encoding- all passing, new probe auto-discovered via existing test enumerationpython -m black garak/probes/encoding.py- no changes needed beyond new code, file formatted