Skip to content

Add NonEngagement detector (#975)#1852

Open
azrabano23 wants to merge 1 commit into
NVIDIA:mainfrom
azrabano23:detector-non-engagement
Open

Add NonEngagement detector (#975)#1852
azrabano23 wants to merge 1 commit into
NVIDIA:mainfrom
azrabano23:detector-non-engagement

Conversation

@azrabano23

Copy link
Copy Markdown

Summary

Implements #975 — a non-engagement detector. Where mitigation.MitigationBypass and mitigation.Prefixes detect genuine safety refusals, this detector flags responses that deflect or change the subject without taking a safety stance — the issue's canonical "ma'am, this is a Wendy's" non-answer. This is a complementary eval signal: used as an extended detector alongside jailbreak/refusal probes, it separates "refused on safety grounds" from "simply did not engage", which the existing detectors conflate or miss.

Changes

  • garak/detectors/mitigation.py — new NonEngagement(StringDetector). Mirrors the existing Prefixes structure (StringDetector, lang_spec="en", matchtype="str", hit_desc/pass_desc, RST docstring). Unlike the refusal detectors it does not invert detect() — a hit means a deflection phrase is present. The trigger list is deliberately conservative and topic-change oriented (explicit subject changes, declining-to-discuss without a safety rationale, evasive non-answers), grouped with inline comments so it does not overlap the safety-refusal phrasing in the other detectors. Tagged avid-effect:performance:P0204.
  • tests/detectors/test_detectors_mitigation.py — added test_non_engagement_init, test_non_engagement_strings (deflections → 1.0; a genuine answer and a plain safety refusal → 0.0, confirming it doesn't overlap MitigationBypass), and test_non_engagement_none_output (None → None).

Not a duplicate

I checked garak/detectors/mitigation.py and the rest of garak/detectors/: the existing detectors (MitigationBypass, Prefixes, ModernBERTRefusal) all target refusals and invert to score compliance. None detect non-safety deflection. No open PR matches non-engagement/engagement (the only "engagement"-adjacent open PR, #1742, is an unrelated adaptive-attacks probe).

Testing

pytest tests/detectors/test_detectors_mitigation.py tests/test_docs.py -q   # 667 passed
pytest tests/detectors/ -q                                                  # 713 passed, 34 skipped

The generic tests/detectors/ and tests/test_docs.py suites auto-validate the new detector's tags, docstring, and plugin loading. All tests are offline (no model/network access required for this detector).

Note

AI assistance (Claude) was used in developing this change; I reviewed and validated every line, ran the tests above, and am able to defend the implementation. The detector is purely defensive — it classifies a target's output and ships no attack payloads.

@patriciapampanelli patriciapampanelli self-requested a review June 8, 2026 17:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant