Make sub-validators safe to share between schemas

[fivetime]

Problem

_WithSubValidators (Any, All, Union, SomeOf) keeps the compiled sub-validators on the instance. __voluptuous_compile__ resets self._compiled to an empty list and then appends to it. _run reads self._compiled on every call, and overwrites it when a discriminant is set.

A validator instance is often shared. A common pattern is a module-level Any(...) reused inside schemas that are built per request. Then two things can overlap:

• one thread compiles a new schema around the shared validator;
• another thread runs a schema that was compiled earlier around it.

In that case the running thread sees an empty or half-filled list, and valid data fails:

• no valid value found (an Any with no alternatives left);
• an error from the wrong alternative, e.g. extra keys not allowed @ data['search'] when only the first alternative of an Any has been compiled yet.

We hit this in production with Gnocchi's REST API under a threaded uWSGI. Gnocchi has a module-level Any(str, float, int, bool, None) and a class-level Any request schema, and both are compiled into a new Schema on every request. Valid resource searches and /v1/aggregates requests were rejected intermittently with HTTP 400:

• no valid value found for dictionary value @ data['and'][0]['or'][0]['=']['ended_at']
• extra keys not allowed @ data['search']

Reproducer

import threading, voluptuous

AttrValue = voluptuous.Any(str, float, int, bool, None)   # shared
Fetch = voluptuous.Any({"operations": str},
                       {"operations": str, "search": dict})

def schema():  # built per request
    return voluptuous.Schema({voluptuous.Any("=", "<="): {str: AttrValue}})

errors = {}
def worker():
    for _ in range(20000):
        for s, d in ((schema(), {"=": {"ended_at": None}}),
                     (Fetch, {"operations": "x", "search": {"a": 1}})):
            try:
                voluptuous.Schema(s, required=True)(d)
            except voluptuous.Invalid as e:
                errors[str(e)] = errors.get(str(e), 0) + 1

ts = [threading.Thread(target=worker) for _ in range(8)]
[t.start() for t in ts]; [t.join() for t in ts]
print(errors or "no errors")

Results for 320,000 validations with 8 threads:

Python	0.16.0 / master	With this change
3.9.25	about 1,000 failures	no errors
3.14.8	about 1,000 failures	no errors

The failures include no valid value found, extra keys not allowed @ data['search'] and expected str for dictionary value @ data['=']['ended_at']. With one thread there are no errors before or after the change.

Fix

The compiled sub-validators now live in the closure that __voluptuous_compile__ returns, not on the instance. Compiling one schema therefore never changes what another, already compiled, schema runs. The discriminant path uses a local list too.

The new test, test_shared_any_unaffected_by_compiling_another_schema, is deterministic and single-threaded. It runs an already compiled schema while a second schema sharing the same Any is in the middle of compiling. It fails on master and passes with this change.

The full test suite passes (183 tests on Python 3.9 and 3.14). flake8, black and mypy are clean.

🤖 Generated with Claude Code