Skip to content

Fixes #812: Prevent field type markers from shadowing missing fields - #818

Merged
arthanson merged 2 commits into
mainfrom
812-field-type-markers
Oct 21, 2026
Merged

arthanson merged 2 commits into
mainfrom
812-field-type-markers

Conversation

@jnovinger

Copy link
Copy Markdown
Member

Fixes: #812

Field type markers (Devices.device_type = DeviceTypes, Permissions.users = [Users], Devices.config_context = JsonField) are plain class attributes. When a payload omits one of those fields, normal attribute lookup finds the class attribute, so device.device_type returns the DeviceTypes class and Record.__getattr__ never runs to lazy-load it. The post-hydration probe in __getattr__ that the issue points at is never reached for these keys.

Record.__init_subclass__ now wraps each marker in a non-data descriptor. Class access still returns the marked type, so _parse_values() and any external code reading Devices.device_type see no change. Instance access raises AttributeError, so an omitted field lazy-loads like any other field and raises if the server omits it too.

The probe also changes from getattr()/hasattr() to object.__getattribute__. It no longer re-enters __getattr__ twice for a key that is still missing after hydration, and it still returns falsy values and re-runs properties.

Behavior changes for the release notes:

  • Reading an omitted marker field on a record with a URL now fetches the full object (or raises AttributeError) instead of returning the class.
  • hasattr(record, "device_type") on an unhydrated record fetches like any other missing field. dir() driven introspection (inspect.getmembers, debugger panes) can trigger the same fetch.
  • Code that checked record.field is SomeClass to detect absence needs hasattr or try/except AttributeError instead.
  • Any class body attribute holding a Record subclass, a JsonField style class, or [RecordSubclass] is treated as a marker. Plugin code that keeps a non-field reference like that should read it through type(self). docs/extensions.md now says so.

Devices.has_details = True and Interfaces.interface_connection are dead: __init__ always overrides has_details, and no NetBox serializer emits interface_connection. I left both in place because removing them saves no requests and changes class-level surface. I can remove them in a follow-up, but I am leaning against it.

This touches pynetbox/core/response.py alongside #794, so whichever lands second will need a rebase.

@jnovinger

Copy link
Copy Markdown
Member Author

@claude /review Only report on issues of potential concern; we don't need commentary on anything that doesn't require further consideration.

@github-actions

github-actions Bot commented Oct 20, 2026 •

Copy link
Copy Markdown

Claude finished @jnovinger's task in 6m 18s —— View job


Review

  • Read changed files
  • Analyze implementation
  • Post review findings

pynetbox/core/response.py

_FieldTypeMarker descriptor design

The non-data descriptor (no __set__) is the right tool here. setattr(self, k, v) in _parse_values writes to instance.__dict__, which takes precedence over non-data descriptors in Python's lookup order, so present fields are returned from __dict__ and absent fields fall through to the descriptor (which raises). Clean.

_is_field_type_marker uses hasattr(value, "_json_field") not a value check

hasattr(value, "_json_field") returns True for any class carrying that attribute regardless of its value. A class with _json_field = False would still be treated as a field-type marker and wrapped. Since _json_field is an internal convention and the only classes that carry it are intentional JsonField-style markers, this is unlikely to bite anyone, but getattr(value, "_json_field", False) would be more precise.

__getattr__ change fixes a second, pre-existing bug

The old getattr(self, k, None) + hasattr(self, k) probe had an undocumented failure mode for properties: if a property raised AttributeError before hydration and then succeeded after, getattr(self, k, None) saved None as ret, and hasattr (returning True) caused __getattr__ to return None rather than the property value. object.__getattribute__ re-invokes the property descriptor so it gets the correct result. test_property_reruns_after_hydration covers this, but it may be worth a brief note in the release notes since it's a quiet behavior fix on top of the marker fix.

Interfaces.interface_connection = InterfaceConnection is now wrapped

The PR acknowledges this is dead code (no NetBox serializer emits interface_connection). With the descriptor in place, any access to interface.interface_connection on a record that has a URL will now trigger one fetch attempt (capped by has_details). That is worse than before (was a silent no-op returning the class), but since the field is never in any payload, no fetch completes successfully and AttributeError is raised. The risk is only if someone was using record.interface_connection is InterfaceConnection as an existence check, which seems unlikely. Leaving it is fine for now per the PR rationale.

Devices.has_details = True not wrapped (correct) but still confusing

has_details = True on Devices is a bool, so _is_field_type_marker correctly skips it. Record.__init__ always writes self.has_details = False as an instance attribute, shadowing the class attribute immediately. The class attribute is harmless but misleading; __init_subclass__ does not interact with it. Deferring cleanup is fine.


tests/unit/test_response.py

Coverage is thorough — missing field, present field, list marker, JsonField marker, single-underscore marker, inheritance, dynamic subclass creation, pickle round-trip, lazy-load, server-omits-field-after-fetch, hasattr-fetches-once, nested brief record, falsy-value return, and property-reruns-after-hydration are all exercised.

One observation: test_hasattr_on_missing_marker_fetches_once patches Request.get with return_value=iter([...]). If the mock were called a second time it would return an exhausted iterator and full_details() would raise StopIteration. The test is correct in asserting call_count == 1, but if the caching breaks the test failure message would be a confusing StopIteration from inside the context manager rather than a clear assertion failure. A minor robustness issue in the test itself — not in the production code.


docs/extensions.md

The new sentence is correct but is doing a lot of work in one line. It reads well enough for existing contributors but may be hard for a first-time extension author to parse. A follow-up to split it into two or three shorter bullet points would help, but it is not a blocker.


Summary: The core mechanism is sound. The only items worth a follow-up are (1) getattr → getattr/hasattr precision in _is_field_type_marker, (2) the docs sentence density, and (3) the dead has_details/interface_connection class attributes — all acknowledged in the PR description and none blocking merge.

@jnovinger

Copy link
Copy Markdown
Member Author

_is_field_type_marker uses hasattr(value, "_json_field") not a value check

This matches the three hasattr(lookup, "_json_field") checks in _parse_values(). Switching only the new one would let the two disagree about what counts as a JsonField.

__getattr__ change fixes a second, pre-existing bug

The old probe did not have this bug. After hydration, getattr(self, k, None) re-ran the property and returned its value. It only fell back to None when the property still raised, and then hasattr was also False, so it raised anyway. test_property_reruns_after_hydration guards against a regression to a self.__dict__ lookup, which would skip the property.

One observation: test_hasattr_on_missing_marker_fetches_once patches Request.get with return_value=iter([...]).

The #808 tests in this file use the same mock pattern, and a second fetch still fails the test. I am keeping it consistent with those.

The new sentence is correct but is doing a lot of work in one line.

Split into two bullets in ddfb71a.

@jnovinger
jnovinger requested review from a team and arthanson and removed request for a team October 20, 2026 14:01
Record subclasses declare field type markers as class attributes (e.g. Devices.device_type = DeviceTypes). When a payload omitted the field, normal attribute lookup found the class attribute, so the marker class came back as the value and Record.__getattr__ never ran to lazy-load it.

Record.__init_subclass__ now wraps each marker in a non-data descriptor. Class access still returns the marked type, so _parse_values() is unchanged. Instance access raises AttributeError, so an omitted field lazy-loads like any other and raises if the server omits it too. The descriptor has no __set__, so values that _parse_values() assigns on the instance take precedence over it.

The post-hydration probe in __getattr__ now uses object.__getattribute__ instead of getattr()/hasattr(). It no longer re-enters __getattr__, and it still returns falsy values and re-runs properties.
@jnovinger
jnovinger force-pushed the 812-field-type-markers branch from ddfb71a to 00ac00e Compare October 20, 2026 14:32
@arthanson
arthanson merged commit 56cc6e7 into main Oct 21, 2026
11 checks passed
@arthanson
arthanson deleted the 812-field-type-markers branch October 21, 2026 18:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Record.__getattr__ returns class-level type markers instead of raising AttributeError

2 participants