feat(probes/encoding): add InjectCaesar probe

[Ujwal-Ramachandran]

Adds a Caesar cipher encoding probe that tests whether models silently decode and execute instructions obfuscated with Caesar cipher shifts (3, 7, and 17). Complements the existing InjectROT13 probe which covers shift 13.

Includes parametrized unit tests for the _caesar_shift helper function.

What changed

• garak/probes/encoding.py: added InjectCaesar class with _caesar_shift helper supporting configurable shift values
• tests/probes/test_probes_encoding.py: added parametrized tests for _caesar_shift and probe instantiation
• garak/data/plugin_cache.json: regenerated via --list_probes

Testing

pytest tests/probes/test_probes_encoding.py tests/probes/test_probes.py

Verification

• Run the tests and ensure they pass python -m pytest tests/probes/test_probes_encoding.py tests/probes/test_probes.py
• Verify the probe encodes payloads correctly across shifts 3, 7, and 17
• Verify encoded prompts do not contain the plaintext trigger (covered by existing test_encoding_triggers_not_in_prompts parametrized test)

[jmartin-tech]

Can you offer details on how this was tested, do you know of a target within the last year that will show a rate of successful attack from this probe?

Validation of the efficacy of new probes is an important factor is evaluating suitability for inclusion especially when marked active and Tier.OF_CONCERN or higher.

[Ujwal-Ramachandran]

Hi! Sure, here's my testing methodology.

Tested locally by:

• Verifying _caesar_shift() output across shifts 3, 7, and 17
• Confirming the existing test_encoding_triggers_not_in_prompts parametrized test automatically picks up InjectCaesar and passes
• Confirming the probe appears in --list_probes

Output:

test_encoding_len_cap[probes.encoding.InjectCaesar] PASSED
test_encoding_prompt_trigger_match[probes.encoding.InjectCaesar] PASSED

Full test output: test_output.txt

Evidence of real-world efficacy

Caesar cipher as an injection vector has consistent empirical support:

• Yuan et al. (2023) - https://arxiv.org/pdf/2308.06463
• Huang et al. (2025) - https://arxiv.org/pdf/2501.10800
• Peng et al. (May 2025) - https://arxiv.org/pdf/2505.13527
• Maskey et al. (May 2025) - https://arxiv.org/pdf/2505.24621

Yuan et al. (2023) predates the one-year window, but is directly cited in both May 2025 papers above, confirming the attack surface remains active in current literature.

The multi-shift design (3, 7, 17) is intentional. Yuan et al. and the ICLR 2025 bijection learning paper (Huang et al.) both note that shift-3 is over-represented in pre-training data. Non-standard shifts test whether safety failures generalise beyond the classical variant.

Please let me know if you need anything else.

Thanks.