Skip to content

feat(probes): add UTF fullwidth encoding probe#1864

Open
nuthalapativarun wants to merge 1 commit into
NVIDIA:mainfrom
nuthalapativarun:feat/152-utf-fullwidth-encoding
Open

feat(probes): add UTF fullwidth encoding probe#1864
nuthalapativarun wants to merge 1 commit into
NVIDIA:mainfrom
nuthalapativarun:feat/152-utf-fullwidth-encoding

Conversation

@nuthalapativarun

@nuthalapativarun nuthalapativarun commented Jun 13, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds a new InjectUTFFullwidth probe to garak/probes/encoding.py, which encodes payloads by mapping ASCII printable characters (0x21-0x7E) to their Unicode fullwidth equivalents (U+FF01-FF5E), following the same pattern as the existing InjectZalgo and InjectLeet probes.
  • Reuses the existing EncodingMixin base, TEMPLATES, payload mechanism, tags, and detectors used by other encoding probes.

Ref #152

AI assistance disclosure

This change was developed with AI assistance (Claude). The implementation follows the existing conventions of garak/probes/encoding.py (mirroring InjectZalgo/InjectLeet). I reviewed the diff for correctness and style consistency before submitting.

Test plan

  • python -m pytest tests/probes/test_probes_encoding.py -q - 226 passed, 1 skipped (skip is pre-existing/unrelated)
  • python -m pytest tests/probes -q -k encoding - all passing, new probe auto-discovered via existing test enumeration
  • python -m black garak/probes/encoding.py - no changes needed beyond new code, file formatted

Signed-off-by: Varun Nuthalapati <nuthalapativarun@gmail.com>
@nuthalapativarun

Copy link
Copy Markdown
Contributor Author

Investigated the failing build (ubuntu-latest, 3.13) check (run 27480787824).

Root cause: unrelated infra flake, not caused by this PR's diff.

Evidence:

  • The job log shows Cache not found for input keys: garak-test-resources-shared (cache restore failed for this leg specifically), so this run had to fetch all HF Hub models/datasets cold instead of using the warm .cache/huggingface from a hit cache.
  • That triggered repeated 429 Too Many Requests responses from huggingface.co (e.g. httpx.HTTPStatusError: Client error '429 Too Many Requests' ... garak-llm/chatgpt_paraphraser_on_T5_base), cascading into 85 failed tests across completely unrelated suites: test_detectors_refusal.py, test_detectors_unsafe_content.py, test_huggingface.py, test_langprovision.py, test_plugin_load.py, test_probes_atkgen.py, test_buffs_paraphrase.py, test_probes_packagehallucination.py, etc.
  • tests/probes/test_probes_encoding.py and tests/detectors/test_detectors_encoding.py — the suites that actually exercise this PR's change (InjectUTFFullwidth in garak/probes/encoding.py) — passed fully, with no reference to fullwidth anywhere in the failure output.
  • The diff itself is a minimal, self-contained 29-line addition (one helper function + one probe class) using only basic bytes/str operations — nothing that would behave differently on Python 3.13 specifically, and the bare build (3.13) matrix leg (different runner config, same Python version) passed cleanly.

No code changes made — this is a HuggingFace Hub rate-limit / GH Actions cache-miss flake on that one runner, unrelated to the probe implementation. Re-running the job should clear it once the cache is warm again or HF Hub rate limits reset.

@nuthalapativarun

Copy link
Copy Markdown
Contributor Author

Just a friendly nudge on this one — CI's been green for a while and it's still mergeable. Happy to make any tweaks if there's feedback.

@nuthalapativarun

Copy link
Copy Markdown
Contributor Author

Hi team — following up on PR #1864 (UTF fullwidth encoding probe). CI is green and the PR has been open for ~3 weeks with no review. Happy to address any feedback. Thanks for your time!

@jmartin-tech jmartin-tech left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you provide any information on how this has been tested? The project needs to validate that the technique works, preferably against some live target, even if it is a minimal system. Something the team can reproduce is preferred.

The project recently added some guidance related to Scope/Naming/Substance criteria when adding new probe. While this PR looks to be attempting to fulfill an open issue, the original issue was vague. This probe may have intrinsic value to help determine if say an attacker can bypass the some alignment expectation or some guardrail protection of the target.

I guess a request here is for some clear documentation of how this adds signal and how practical it is for deployments to need to be evaluated in relation to this method of encoding. A small survey of a few recent targets may help establish the value from this probe.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants