Implement providers and compatibility policies

doc/changelog.rst

@@ -32,6 +32,39 @@ Added
   :meth:`ResourceType.from_resource <scim2_models.ResourceType.from_resource>` publishes the
   necessity under ``schemaExtensions.required``, which :rfc:`RFC7643 §6 <7643#section-6>`
   defines. See :doc:`how-to/define-custom-models`. :issue:`105`
+- :class:`~scim2_models.ScimProvider` describes a SCIM service: the models it builds its
+  resources from, and the :class:`~scim2_models.Schema`, :class:`~scim2_models.ResourceType` and
+  :class:`~scim2_models.ServiceProviderConfig` objects its discovery endpoints answer
+  (:rfc:`RFC7644 §4 <7644#section-4>`). Its ``models`` hold bare resources and extensions alike,
+  and its ``resource_types`` bind them as :rfc:`RFC7643 §6 <7643#section-6>` defines: a name or
+  an endpoint answers the composed model, a schema URI answers the bare one. Two resource types
+  may therefore share a base schema and differ by the extensions they carry, each served under
+  its own endpoint. A service whose models share a schema, whose resource types share a name or
+  an endpoint, or whose resource type names a schema no model describes, is refused with
+  :class:`~scim2_models.ScimProviderError`. See :doc:`how-to/describe-a-scim-service`.
+  :issue:`108`
+- :meth:`ScimProvider.from_discovery <scim2_models.ScimProvider.from_discovery>` builds a
+  provider from what a service publishes, turning each resource type into a model carrying the
+  extensions it declares, required ones included. A resource type naming a schema the service
+  does not publish is refused with an error naming that schema.
+- :class:`~scim2_models.ScimPolicy` states how much a payload may depart from
+  :rfc:`RFC7643 <7643>` and :rfc:`RFC7644 <7644>` and still be read. Pass it as the
+  ``scim_policy`` argument of :meth:`~scim2_models.BaseModel.model_validate`,
+  :meth:`~scim2_models.BaseModel.model_dump` and :meth:`PatchOp.patch
+  <scim2_models.PatchOp.patch>`, or open a ``with`` block on a policy or on a
+  :class:`~scim2_models.ScimProvider` carrying one. Every setting defaults to the strict reading,
+  so nothing changes until one is chosen. See :doc:`explanation/policies` and
+  :doc:`how-to/tolerate-a-nonconformant-peer`. :issue:`108`
+- :attr:`ScimPolicy.unknown <scim2_models.ScimPolicy.unknown>` reads a payload carrying attributes
+  no model declares, unmodelled extensions included. ``ignore`` drops them and ``keep`` writes
+  them back with the spelling the peer used; both leave them readable on
+  :attr:`~scim2_models.BaseModel.unknown_attributes`, at the level they were found. :issue:`85`
+- :attr:`ScimPolicy.remove_value_as_filter
+  <scim2_models.ScimPolicy.remove_value_as_filter>` reads the ``value`` of a PATCH ``remove`` as
+  the selection :rfc:`RFC7644 §3.5.2.2 <7644#section-3.5.2.2>` spells in the ``path``, which is
+  the form `Microsoft Entra ID
+  <https://learn.microsoft.com/en-us/entra/identity/app-provisioning/application-provisioning-config-problem-scim-compatibility>`_
+  sends to remove a group member.
 - lark is a new dependency.
 
 Changed

doc/explanation/index.rst

@@ -10,4 +10,5 @@ written to be read away from the keyboard, rather than consulted while completin
 
    filters
    patch
+   policies
    scim-contexts

doc/explanation/patch.rst

@@ -145,7 +145,10 @@ Write the selection in the path instead, as ``emails[value eq "work@example.com"
    non-conformant
    <https://learn.microsoft.com/en-us/entra/identity/app-provisioning/application-provisioning-config-problem-scim-compatibility>`_
    and the ``aadOptscim062020`` flag, added to the tenant URL of the application, has Entra send
-   a filter path.
+   a filter path. An application serving that client without the flag reads the ``value`` as a
+   selection with
+   :attr:`ScimPolicy.RemoveValue.apply <scim2_models.ScimPolicy.RemoveValue.apply>`; see
+   :doc:`../how-to/tolerate-a-nonconformant-peer`.
 
 Primary values
 --------------

doc/explanation/policies.rst

@@ -0,0 +1,86 @@
+Validation policies
+===================
+
+Vendors implement SCIM from the same specifications and still send payloads those specifications
+do not describe. scim2-models refuses such a payload by default. A
+:class:`~scim2_models.ScimPolicy` states, for one application, how much of it to accept anyway.
+
+Why a policy sits beside the context
+------------------------------------
+
+A :class:`~scim2_models.Context` and a policy answer two different questions, and the difference
+decides what may become a policy setting at all.
+
+The context says what a payload is: a creation request, a query response. Both ends of an exchange
+read it the same way, because :rfc:`RFC7644 <7644>` defines what each one contains. It travels
+with the message.
+
+The policy says how the payloads are treated here. Nothing on the wire carries it, and the peer
+never learns of it. Two applications talking to each other may hold opposite policies and still
+agree on every message they exchange.
+
+A setting belongs in a policy when it changes how a payload is read or written, and when the
+specification leaves the choice open. Page sizes, base URLs and authentication schemes are
+configuration too, and they belong to the application that serves the requests.
+
+Why every default is strict
+---------------------------
+
+§5.3 of the SCIM interoperability profile asks a service provider to reject the attributes and the
+schema URNs it does not define, and the strict reading is what the library applies when it is
+given no policy.
+
+Two consequences follow from that choice.
+
+A service built on scim2-models keeps refusing the payloads of a client that sends unknown
+attributes until its author picks :attr:`~scim2_models.ScimPolicy.Unknown.ignore` or
+:attr:`~scim2_models.ScimPolicy.Unknown.keep`. What the library ships is a documented way to
+tolerate, and the tolerance itself stays a decision.
+
+The strict default also applies to a client reading a response, which reaches further than the
+profile does: §5.3 addresses service providers receiving requests and says nothing about clients.
+Telling the two apart would take a notion of role, and a model has none — a request is validated
+by the client that wrote it as readily as by the server that received it, so the context cannot
+stand in for one.
+
+What a policy leaves alone
+--------------------------
+
+**PATCH paths stay strict.** An operation whose ``path`` names an attribute no model declares
+raises :class:`~scim2_models.PathNotFoundException`, whatever the policy says. Path resolution and
+unknown attributes are two separate mechanisms, and making them uniform would take a third. The
+default that would come out of it is the wrong one: a server would answer 200 to a modification it
+never applied, where :rfc:`RFC7644 §3.5.2 <7644#section-3.5.2>` asks for an error. Inside the body
+of a resource the trade is different, since dropping one unknown attribute still lands everything
+the peer and the model both knew.
+
+**Building a model in Python stays strict.** ``User(bogus=1)`` and ``user.bogus = 1`` raise under
+every policy. Pydantic only offers a hook for extra keys during validation, so a keyword argument
+has nowhere to go. A policy governs payloads, and a payload comes from a peer.
+
+Where a policy is read
+----------------------
+
+Three layers answer, in order:
+
+1. the ``scim_policy`` argument of the call;
+2. the policy of the innermost open block, from ``with policy:`` or ``with provider:``;
+3. the strict reading.
+
+The resolution happens for each pass, which gives the layers a visible consequence. A model
+validated inside a block and serialized outside it is serialized under the strict reading. Under
+:attr:`~scim2_models.ScimPolicy.Unknown.keep` the attributes themselves survive on the instance
+and :attr:`~scim2_models.BaseModel.unknown_attributes` still reads them, so the dump can be made
+again inside a block; the other settings leave nothing behind.
+
+Blocks nest, and each one restores what it interrupted. Each thread and each asyncio task carries
+its own, so a server may open one per request.
+
+What a kept attribute costs
+---------------------------
+
+An attribute no model declares has no :class:`~scim2_models.Returned` and no
+:class:`~scim2_models.Mutability` annotation, since those come from a schema. Nothing can filter
+it by context, and it is written back in all of them. A client reading from one service under
+:attr:`~scim2_models.ScimPolicy.Unknown.keep` and creating on another pushes the first service's
+attributes to the second. A proxy wants exactly that, and ``keep`` is a setting an author picks.

doc/how-to/describe-a-scim-service.rst

@@ -0,0 +1,194 @@
+Describe a SCIM service
+=======================
+
+Use this guide when an application needs to know, in one place, which resources a SCIM service
+serves and what it supports. A :class:`~scim2_models.ScimProvider` describes both, whether the
+application exposes the service or queries it.
+
+Describe the service you expose
+-------------------------------
+
+A :class:`~scim2_models.ScimProvider` holds the attributes that describe a service. ``models``
+and ``resource_types`` cover the resources it serves and the endpoints it serves them under.
+
+``models`` is the catalogue of what the service can build: bare resources and extensions, each
+identified by its schema URI. ``resource_types`` binds extensions to a resource and gives it an
+endpoint, as :rfc:`RFC7643 §6 <7643#section-6>` describes. Build one from a parameterized model
+with :meth:`ResourceType.from_resource <scim2_models.ResourceType.from_resource>`, which supplies
+default values:
+
+.. doctest::
+
+   >>> from scim2_models import EnterpriseUser, Group, ResourceType, ScimProvider, User
+   >>> provider = ScimProvider(
+   ...     models=[User, EnterpriseUser, Group],
+   ...     resource_types=[
+   ...         ResourceType.from_resource(User[EnterpriseUser]),
+   ...         ResourceType.from_resource(Group),
+   ...     ],
+   ... )
+   >>> for schema in provider.schemas:
+   ...     print(schema.id)
+   urn:ietf:params:scim:schemas:core:2.0:User
+   urn:ietf:params:scim:schemas:extension:enterprise:2.0:User
+   urn:ietf:params:scim:schemas:core:2.0:Group
+
+The provider composes the two, and an endpoint serves the composed model. Look it up by the name
+the resource type declares, the name ``meta.resourceType`` carries on every resource
+(:rfc:`RFC7643 §6 <7643#section-6>`). Those are the defaults at work: here the name is ``"User"``,
+the last segment of the schema URI, and the endpoint is ``/Users``, that segment with an ``s``
+appended. Write the :class:`~scim2_models.ResourceType` yourself when a service publishes
+something else.
+
+.. doctest::
+
+   >>> provider.model_for("User") is User[EnterpriseUser]
+   True
+
+Leave ``resource_types`` out for a service whose resources carry no extension: the provider
+derives one per resource.
+
+Announce what the service supports
+----------------------------------
+
+``config`` is the ``/ServiceProviderConfig`` response: which operations the service supports, and
+within which bounds (:rfc:`RFC7644 §4 <7644#section-4>`). Pass it alongside the models:
+
+.. doctest::
+
+   >>> from scim2_models import Filter, Patch, ServiceProviderConfig
+   >>> config = ServiceProviderConfig(
+   ...     patch=Patch(supported=True),
+   ...     filter=Filter(supported=True, max_results=200),
+   ... )
+   >>> provider = ScimProvider(
+   ...     models=[User, EnterpriseUser, Group],
+   ...     resource_types=[
+   ...         ResourceType.from_resource(User[EnterpriseUser]),
+   ...         ResourceType.from_resource(Group),
+   ...     ],
+   ...     config=config,
+   ... )
+   >>> provider.config.filter.max_results
+   200
+
+Serve two variants of one resource
+----------------------------------
+
+Two resource types may share a base schema, declare different extensions, and serve them under
+different endpoints. Their resources carry identical ``schemas``, and ``meta.resourceType`` tells
+them apart (:rfc:`RFC7643 §3 <7643#section-3>`):
+
+.. doctest::
+
+   >>> from scim2_models import URI, Reference, SchemaExtension
+   >>> staff = ResourceType(
+   ...     id="Staff",
+   ...     name="Staff",
+   ...     endpoint=Reference[URI]("/Staff"),
+   ...     schema_=Reference[URI](str(User.__schema__)),
+   ...     schema_extensions=[
+   ...         SchemaExtension(
+   ...             schema_=Reference[URI](str(EnterpriseUser.__schema__)),
+   ...             required=True,
+   ...         )
+   ...     ],
+   ... )
+   >>> provider = ScimProvider(
+   ...     models=[User, EnterpriseUser],
+   ...     resource_types=[ResourceType.from_resource(User), staff],
+   ...     config=config,
+   ... )
+   >>> provider.model_for_endpoint("/Users") is User
+   True
+   >>> provider.model_for_endpoint("/Staff") is provider.model_for("Staff")
+   True
+
+Route a request to a model
+--------------------------
+
+:meth:`~scim2_models.ScimProvider.model_for` takes a resource type name or a schema URI, and
+:meth:`~scim2_models.ScimProvider.model_for_endpoint` takes an endpoint. A name gives the composed
+model, a schema URI gives the catalogue entry: the bare resource, or the extension the URI names.
+Neither method takes the name of a Python class:
+
+.. doctest::
+
+   >>> provider.model_for("/Staff") is None
+   True
+   >>> provider.model_for(str(User.__schema__)) is User
+   True
+   >>> provider.model_for(str(EnterpriseUser.__schema__)) is EnterpriseUser
+   True
+
+Both methods return :data:`None` for an unknown key, which a server turns into a 404. Both also
+resolve the three discovery resources, even though ``models`` never lists them:
+
+.. doctest::
+
+   >>> from scim2_models import Schema
+   >>> provider.model_for_endpoint("/Schemas") is Schema
+   True
+   >>> provider.model_for("Pet") is None
+   True
+
+Describe the service you query
+------------------------------
+
+A client queries the three discovery endpoints and validates each response, which gives it a
+``ListResponse[Schema]``, a ``ListResponse[ResourceType]`` and a
+:class:`~scim2_models.ServiceProviderConfig`. Pass the ``resources`` of the first two, and the
+third one, to :meth:`~scim2_models.ScimProvider.from_discovery`:
+
+.. doctest::
+   :hide:
+
+   >>> schemas, resource_types, service_provider_config = (
+   ...     provider.schemas,
+   ...     provider.resource_types,
+   ...     provider.config,
+   ... )
+
+.. doctest::
+
+   >>> discovered = ScimProvider.from_discovery(
+   ...     schemas, resource_types, service_provider_config
+   ... )
+
+Each schema becomes a model, a resource or an extension depending on how the resource types name
+it, and the resource types bind them back together:
+
+.. doctest::
+
+   >>> model = discovered.model_for("Staff")
+   >>> list(model.get_extension_models())
+   ['urn:ietf:params:scim:schemas:extension:enterprise:2.0:User']
+   >>> discovered.config.filter.max_results
+   200
+
+The provider reads back an extension a resource type declares required, and the rebuilt model
+refuses a creation request that leaves it out. See :doc:`define-custom-models` for what that
+annotation means.
+
+When a resource type references a schema the service did not publish,
+:meth:`~scim2_models.ScimProvider.from_discovery` raises
+:class:`~scim2_models.ScimProviderError` and names the missing schema.
+
+Diagnose a refused provider
+---------------------------
+
+A provider validates its whole description at construction, since nothing can change afterwards.
+Two models sharing a schema, or two resource types sharing a name or an endpoint, raise
+:class:`~scim2_models.ScimProviderError`: one key would designate two models.
+
+Passing an already parameterized model raises it too. List the bare resource and its extensions
+instead, and bind them with a resource type:
+
+.. doctest::
+
+   >>> from scim2_models import ScimProviderError
+   >>> try:
+   ...     ScimProvider(models=[User[EnterpriseUser]])
+   ... except ScimProviderError as exc:
+   ...     print(exc)
+   User[EnterpriseUser] is already parameterized: list the bare resource and its extensions, and bind them with a resource type, as ResourceType.from_resource builds one

doc/how-to/generate-models-from-schemas.rst

@@ -65,3 +65,7 @@ Parameterize the resource with the generated extension before parsing its payloa
 
 Keep the schema with the generated class. It remains the source of the schema URN and attribute
 characteristics used when the model validates or serializes a resource.
+
+:meth:`ScimProvider.from_discovery <scim2_models.ScimProvider.from_discovery>` does all of this
+at once for a whole service, pairing each schema with the resource type declaring it. See
+:doc:`describe-a-scim-service`.

doc/how-to/index.rst

@@ -10,5 +10,7 @@ ends where an application resumes its own work, and assumes the :doc:`../overvie
    access-resource-values
    build-filters
    define-custom-models
+   describe-a-scim-service
    generate-models-from-schemas
+   tolerate-a-nonconformant-peer
    validate-and-serialize