Merge branch 'main' into cursor-based-pagination

# Conflicts:
#	doc/changelog.rst

Author: azmeuk

doc/changelog.rst

@@ -44,11 +44,42 @@ Added
   :class:`~scim2_models.ResponseParameters` a client sent, instead of its ``attributes`` and
   ``excludedAttributes`` spelled out one by one. A :class:`~scim2_models.SearchRequest` is one,
   so a server answering ``POST /.search`` passes the request it received. :issue:`141`
+- :meth:`~scim2_models.BaseModel.model_validate`,
+  :meth:`~scim2_models.BaseModel.model_validate_json`,
+  :meth:`~scim2_models.BaseModel.model_dump` and
+  :meth:`~scim2_models.BaseModel.model_dump_json` take a ``scim_provider`` and a ``scim_spc``: the
+  :class:`~scim2_models.ScimProvider` describing the service a payload belongs to, and the
+  :class:`~scim2_models.ServiceProviderConfig` its peer publishes. A ``with`` block opened on a
+  provider lends both, as it already lends its policy, and ``scim_spc`` wins over the
+  configuration the provider carries. Rules the specification makes conditional on a declared
+  capability read them. See :doc:`how-to/describe-a-scim-service`.
+- :meth:`~scim2_models.PatchOp.build_from` builds the patch turning one resource state into
+  another. Only the attributes the wanted state names take part in the comparison, so what a peer
+  maintains and the caller does not model survives the modification — which is what a PATCH
+  offers over a PUT. See :doc:`how-to/build-a-patch`. :issue:`104`
+- Bulk messages are validated, in the new :attr:`~scim2_models.Context.BULK_REQUEST` and
+  :attr:`~scim2_models.Context.BULK_RESPONSE` contexts. Each operation's
+  :attr:`~scim2_models.BulkOperation.data` is checked as the single request it stands for: a
+  creation for a POST, a patch for a PATCH. See :ref:`helpers-bulk`. :pr:`149`
 - lark is a new dependency.
 - Support for :rfc:`RFC9865 <9865>`
 
 Changed
 ^^^^^^^
+- The bulk models take the resource type their operations carry, as in ``BulkRequest[User]`` or
+  ``BulkRequest[User | Group]``, and raise a :class:`TypeError` when used bare. A payload the type
+  parameter does not cover is now refused, and a bulk response no longer dumps ``path``.
+- :class:`~scim2_models.ListResponse` raises a :class:`TypeError` when used without the resource
+  type its entries carry, as :class:`~scim2_models.PatchOp` and the bulk models do. A bare
+  ``ListResponse`` used to answer a pydantic error naming ``Resource``, and whether it did depended
+  on what the calling module had imported.
+- A message type parameter must name resource types. ``ListResponse[str]`` used to build a class
+  that read anything as its entries.
+- ``ListResponse[Resource]``, ``PatchOp[Resource]`` and their bulk counterparts stay writable where
+  a type is expected, which is what an annotation covering any resource type needs, and raise a
+  :class:`TypeError` when they read or build a payload. ``Resource`` declares no attribute, so a
+  payload read against it fails on the first one it carries. ``PatchOp[Resource]`` used to be
+  refused as a type, and ``ListResponse[Resource]`` used to read payloads.
 - :attr:`SearchRequest.filter <scim2_models.SearchRequest.filter>` is a
   :class:`~scim2_models.ScimFilter` instead of a :class:`str`, so a malformed filter is rejected
   at validation time.
@@ -63,6 +94,9 @@ Changed
   when applied. It used to remove the entries equal to that ``value``, and to report no change
   when the ``value`` was a list or described an entry only in part. Set
   :attr:`~scim2_models.ScimPolicy.remove_value_as_filter` to keep reading it.
+- A PATCH reaching an extension attribute takes the extended resource type, as in
+  ``PatchOp[User[EnterpriseUser]]``. ``PatchOp[User]`` used to carry such an operation to the
+  endpoint, and now refuses a path its type parameter leaves out.
 - :meth:`SCIMException.from_error <scim2_models.SCIMException.from_error>` reconstructs
   :class:`~scim2_models.InvalidCursorException`, :class:`~scim2_models.ExpiredCursorException` and
   :class:`~scim2_models.InvalidCountException` from an :class:`~scim2_models.Error` carrying the
@@ -96,12 +130,21 @@ Deprecated
 
 Fixed
 ^^^^^
+- A bulk model indexed with something other than a resource type names itself in the error. The
+  rules of the :class:`~scim2_models.PatchOp` its operations carry used to answer for it, so
+  ``BulkRequest[str]`` told the caller to write ``PatchOp[User]``.
+- A subclass of a parameterized message, such as ``class Users(ListResponse[User])``, reads its
+  payloads with the type parameter it inherits. It used to raise an :exc:`IndexError`, a subclass
+  carrying no parameter of its own.
 - A PATCH operation targeting an attribute of an extension answers for the constraints that
   extension declares, where it used to look them up on the resource and find none. A refused
   operation no longer leaves the extension instantiated on the resource.
 - A PATCH operation carrying no ``path`` accepts a resource as its ``value``, and checks the
   attributes it names against the model. They used to go through unexamined, so a client naming
   an attribute it had misspelled was answered success.
+- A PATCH operation whose ``path`` names an attribute the resource schema does not declare is
+  refused with ``invalidPath``. It used to pass, so a client that misspelled an attribute was
+  answered success without anything being written. :issue:`164`
 - A refused PATCH ``add`` on a multi-valued attribute leaves the attribute as it was. The entry
   used to be appended before being validated, and outlived the failure.
 - A PATCH operation carrying no ``path`` marks the attributes it assigns as set, so

doc/explanation/patch.rst

@@ -168,5 +168,17 @@ model does not declare. Table 9 lists ``invalidFilter`` as applying to a "PATCH
 so it answers what goes wrong between the brackets. That covers an unknown sub-attribute, a
 comparison the attribute cannot take, and a selection over an attribute holding a single value.
 
+Building a patch rather than applying one
+-----------------------------------------
+
+An application holding both the state a peer has and the state it should have does not have to
+spell the operations out. :meth:`~scim2_models.PatchOp.build_from` compares the two states and
+builds them, restricted to the attributes the wanted state names, so what the peer maintains and
+the application does not model is left alone.
+
+What that builder can express follows from this page. A multi-valued attribute is replaced as a
+whole, since :rfc:`RFC7643 §2.4 <7643#section-2.4>` gives its entries no identity to match one
+state against the other. See :doc:`../how-to/build-a-patch`.
+
 To inspect or change a resource directly with the same path syntax, outside a PATCH request, use
 :doc:`../how-to/access-resource-values`.

doc/explanation/policies.rst

@@ -46,13 +46,13 @@ stand in for one.
 What a policy leaves alone
 --------------------------
 
-**PATCH paths stay strict.** An operation whose ``path`` names an attribute no model declares
-raises :class:`~scim2_models.PathNotFoundException`, whatever the policy says. Path resolution and
-unknown attributes are two separate mechanisms, and making them uniform would take a third. The
-default that would come out of it is the wrong one: a server would answer 200 to a modification it
-never applied, where :rfc:`RFC7644 §3.5.2 <7644#section-3.5.2>` asks for an error. Inside the body
-of a resource the trade is different, since dropping one unknown attribute still lands everything
-the peer and the model both knew.
+**PATCH paths stay strict.** An operation whose ``path`` names an attribute no model declares is
+refused with ``invalidPath``, whatever the policy says. Path resolution and unknown attributes are
+two separate mechanisms, and making them uniform would take a third. The default that would come
+out of it is the wrong one: a server would answer 200 to a modification it never applied, where
+:rfc:`RFC7644 §3.5.2 <7644#section-3.5.2>` asks for an error. Inside the body of a resource the
+trade is different, since dropping one unknown attribute still lands everything the peer and the
+model both knew.
 
 **Building a model in Python stays strict.** ``User(bogus=1)`` and ``user.bogus = 1`` raise under
 every policy. Pydantic only offers a hook for extra keys during validation, so a keyword argument

doc/how-to/build-a-patch.rst

@@ -0,0 +1,122 @@
+Build a patch from two resource states
+======================================
+
+Use this guide when an application holds the state a peer is believed to have and the state it
+should have, and must send the modification. :meth:`~scim2_models.PatchOp.build_from` compares the
+two and returns the :class:`~scim2_models.PatchOp` that closes the gap.
+
+Build the patch
+---------------
+
+Pass the state the peer holds first, then the state it should hold:
+
+.. doctest::
+
+   >>> from scim2_models import Name, PatchOp, User
+   >>> distant = User(user_name="bjensen", name=Name(given_name="Barbara", family_name="Jensen"))
+   >>> wanted = User(user_name="bjensen", name=Name(given_name="Babs", family_name="Jensen"))
+   >>> patch = PatchOp.build_from(distant, wanted)
+   >>> patch.model_dump()["Operations"]
+   [{'op': 'replace', 'path': 'name.givenName', 'value': 'Babs'}]
+
+A complex attribute is compared sub-attribute by sub-attribute, and each one gets its own path.
+Targeting ``name`` as a whole would replace it entirely and drop what the operation does not
+carry.
+
+Leave alone what the wanted state does not name
+-----------------------------------------------
+
+Only the attributes the wanted state names take part in the comparison. This is what separates a
+patch from the :meth:`~scim2_models.Resource.replace` it stands for: an attribute the peer
+maintains and the application does not model survives the modification.
+
+.. doctest::
+
+   >>> distant = User(user_name="bjensen", nick_name="Barb", title="CEO")
+   >>> wanted = User(user_name="bjensen", nick_name="Babs")
+   >>> patch = PatchOp.build_from(distant, wanted)
+   >>> patch.model_dump()["Operations"]
+   [{'op': 'replace', 'path': 'nickName', 'value': 'Babs'}]
+
+Naming an attribute with no value says the opposite. ``title=None`` reads as "clear the title",
+where an unnamed ``title`` reads as "leave it alone":
+
+.. doctest::
+
+   >>> wanted = User(user_name="bjensen", title=None)
+   >>> patch = PatchOp.build_from(distant, wanted)
+   >>> patch.model_dump()["Operations"]
+   [{'op': 'remove', 'path': 'title'}]
+
+The same rule reaches sub-attributes, and an extension is named by its schema URN:
+
+.. doctest::
+
+   >>> from scim2_models import EnterpriseUser
+   >>> distant = User[EnterpriseUser](user_name="bjensen")
+   >>> distant[EnterpriseUser] = EnterpriseUser(department="Tour")
+   >>> wanted = User[EnterpriseUser](user_name="bjensen")
+   >>> wanted[EnterpriseUser] = EnterpriseUser(department="Chess")
+   >>> patch = PatchOp.build_from(distant, wanted)
+   >>> str(patch.operations[0].path)
+   'urn:ietf:params:scim:schemas:extension:enterprise:2.0:User:department'
+
+Send nothing when nothing changed
+---------------------------------
+
+Two states that agree have no patch to describe: an ``Operations`` array must hold at least one
+operation, per :rfc:`RFC7644 §3.5.2 <7644#section-3.5.2>`. :meth:`~scim2_models.PatchOp.build_from`
+returns :data:`None`, so an application tests it before sending a request:
+
+.. doctest::
+
+   >>> PatchOp.build_from(wanted, wanted) is None
+   True
+
+Know how collections are compared
+---------------------------------
+
+A multi-valued attribute is replaced as a whole. Only the sub-attributes the wanted entries name
+decide whether it changed, so the sub-attributes the peer alone maintains do not read as a
+difference:
+
+.. doctest::
+
+   >>> from scim2_models import Email
+   >>> distant = User(emails=[Email(value="barb@example.com", type="work", primary=True)])
+   >>> wanted = User(emails=[Email(value="barb@example.com")])
+   >>> PatchOp.build_from(distant, wanted) is None
+   True
+
+When the collection does change, the whole of it is replaced and the peer's own sub-attributes go
+with it:
+
+.. doctest::
+
+   >>> wanted = User(emails=[Email(value="babs@example.com")])
+   >>> patch = PatchOp.build_from(distant, wanted)
+   >>> patch.model_dump()["Operations"]
+   [{'op': 'replace', 'path': 'emails', 'value': [{'value': 'babs@example.com'}]}]
+
+The builder cannot do better: :rfc:`RFC7643 §2.4 <7643#section-2.4>` gives the entries of a
+multi-valued attribute no identity, so nothing distinguishes an entry that changed from an entry
+that was removed and another that was added. An application that knows how to identify its own
+entries writes those operations itself, targeting a sub-attribute through a filter such as
+``emails[type eq "work"].value``.
+
+Read what the patch never carries
+---------------------------------
+
+A ``readOnly`` attribute is left out however much the two states differ:
+:rfc:`RFC7644 §3.5.2 <7644#section-3.5.2>` forbids a client to modify one, and naming it would
+make the patch invalid. That covers :attr:`~scim2_models.Resource.id`,
+:attr:`~scim2_models.Resource.meta` and :attr:`~scim2_models.User.groups`, along with the
+``readOnly`` sub-attributes of a complex attribute.
+
+An ``immutable`` attribute that holds no value yet is added, which
+:rfc:`RFC7644 §3.5.2 <7644#section-3.5.2>` allows. One that already holds a value cannot be
+modified, and :meth:`~scim2_models.PatchOp.build_from` raises a
+:class:`~scim2_models.MutabilityException` rather than building a request the peer must refuse.
+
+An attribute a server never returns, such as :attr:`~scim2_models.User.password`, reads as unset on
+the side of the peer. Every patch built from a state naming it carries it again.

doc/how-to/describe-a-scim-service.rst

@@ -72,6 +72,42 @@ within which bounds (:rfc:`RFC7644 §4 <7644#section-4>`). Pass it alongside the
    >>> provider.config.filter.max_results
    200
 
+Read the description during a validation
+----------------------------------------
+
+A rule the specification makes conditional on a capability needs what the service declares, not
+only the payload. Name the provider at the call, and every validator the pass reaches sees it,
+down to a nested attribute:
+
+.. doctest::
+
+   >>> payload = {"schemas": [str(User.__schema__)], "userName": "bjensen"}
+   >>> user = User.model_validate(payload, scim_provider=provider)
+   >>> user.user_name
+   'bjensen'
+
+A server states it once per request instead, by opening a block on the provider. The block lends
+its policy the same way:
+
+.. doctest::
+
+   >>> with provider:
+   ...     user = User.model_validate(payload)
+
+A client holds the configuration of a peer as soon as it has queried ``/ServiceProviderConfig``,
+before it builds any provider. Pass it alone:
+
+.. doctest::
+
+   >>> user = User.model_validate(payload, scim_spc=config)
+
+``scim_spc`` also wins over the configuration a provider carries, so one provider answers peers
+that declare different capabilities. Both parameters are taken by
+:meth:`~scim2_models.BaseModel.model_validate`,
+:meth:`~scim2_models.BaseModel.model_validate_json`,
+:meth:`~scim2_models.BaseModel.model_dump` and
+:meth:`~scim2_models.BaseModel.model_dump_json`.
+
 Serve two variants of one resource
 ----------------------------------
 

doc/how-to/index.rst

@@ -8,6 +8,7 @@ ends where an application resumes its own work, and assumes the :doc:`../overvie
    :maxdepth: 1
 
    access-resource-values
+   build-a-patch
    build-filters
    define-custom-models
    describe-a-scim-service

doc/how-to/validate-and-serialize.rst

@@ -36,6 +36,9 @@ in a request context, responses are validated and serialized in the matching res
    * - ``POST /Users/.search``
      - :attr:`~scim2_models.Context.SEARCH_REQUEST`
      - :attr:`~scim2_models.Context.SEARCH_RESPONSE`
+   * - ``POST /Bulk``
+     - :attr:`~scim2_models.Context.BULK_REQUEST`
+     - :attr:`~scim2_models.Context.BULK_RESPONSE`
 
 :attr:`~scim2_models.Context.DEFAULT` applies neither set of rules, and suits a resource held in
 application state rather than exchanged over HTTP.

doc/integrations/_examples/django_example.py

@@ -11,6 +11,7 @@
 from django.views.decorators.csrf import csrf_exempt
 from pydantic import ValidationError
 
+from scim2_models import BulkRequest
 from scim2_models import Context
 from scim2_models import Error
 from scim2_models import ListResponse
@@ -24,6 +25,7 @@
 from scim2_models import User
 
 from .integrations import delete_record
+from .integrations import execute_bulk
 from .integrations import from_scim_user
 from .integrations import get_record
 from .integrations import get_resource_type
@@ -397,13 +399,39 @@ def post(self, request):
 # -- search-root-end --
 
 
+# -- bulk-start --
+class BulkView(SCIMView):
+    """Apply a bulk job and answer one result per operation."""
+
+    def post(self, request):
+        try:
+            bulk_request = BulkRequest[User].model_validate_json(
+                request.body, scim_ctx=Context.BULK_REQUEST
+            )
+        except ValidationError as error:
+            return scim_validation_error(error)
+
+        try:
+            bulk_response = execute_bulk(
+                bulk_request, lambda record: resource_location(request, record)
+            )
+        except SCIMException as error:
+            return scim_exception_error(error)
+
+        return SCIMJsonResponse(bulk_response.model_dump(scim_ctx=Context.BULK_RESPONSE))
+
+
+# -- bulk-end --
+
+
 # -- urls-start --
 # ``/Users/.search`` comes first: the ``user`` converter would otherwise read
 # ``.search`` as a resource id.
 urlpatterns = [
     path("scim/v2/Users", UsersView.as_view(), name="scim_users"),
     path("scim/v2/Users/.search", UsersSearchView.as_view(), name="scim_users_search"),
     path("scim/v2/.search", RootSearchView.as_view(), name="scim_root_search"),
+    path("scim/v2/Bulk", BulkView.as_view(), name="scim_bulk"),
     path("scim/v2/Users/<user:app_record>", UserView.as_view(), name="scim_user"),
 ]
 # -- urls-end --

doc/integrations/_examples/fastapi_example.py

@@ -13,6 +13,8 @@
 from fastapi.responses import JSONResponse
 from pydantic import ValidationError
 
+from scim2_models import BulkRequest
+from scim2_models import BulkRequestContext
 from scim2_models import Context
 from scim2_models import CreationRequestContext
 from scim2_models import Error
@@ -32,6 +34,7 @@
 from scim2_models import User
 
 from .integrations import delete_record
+from .integrations import execute_bulk
 from .integrations import from_scim_user
 from .integrations import get_record
 from .integrations import get_resource_type
@@ -327,6 +330,19 @@ async def create_user(
 # -- collection-end --
 
 
+# -- bulk-start --
+@router.post("/Bulk")
+async def bulk(
+    request: Request, bulk_request: BulkRequestContext[BulkRequest[User]]
+):
+    """Apply a bulk job and answer one result per operation."""
+    bulk_response = execute_bulk(
+        bulk_request, lambda record: resource_location(request, record)
+    )
+    return SCIMResponse(bulk_response.model_dump(scim_ctx=Context.BULK_RESPONSE))
+# -- bulk-end --
+
+
 # -- discovery-start --
 # -- schemas-start --
 @router.get("/Schemas")