Skip to content

Fix xarray inferred-from data ordering - #8445

Merged
Aaron Smull (asmull) merged 5 commits into
mainfrom
aaron/fix-xarray-inferred-from-ordering
Sep 16, 2026
Merged

Aaron Smull (asmull) merged 5 commits into
mainfrom
aaron/fix-xarray-inferred-from-ordering

Conversation

@asmull

@asmull Aaron Smull (asmull) commented Aug 25, 2026 •

Copy link
Copy Markdown
Contributor

Dataset export to xarray currently fails under the condition.

This can occur when array-valued parameters. Users may wish to acquire and save data in a particular order, and no assumptions.

This failure occurs in two cases:

  1. With pandas-based exporting to xarray, indexing is not propagated to inferred-from relationships.
  2. With direct xarray exporting, the assumption is made that shape metadata can be used to imply that, e.g., the first column of an NxM sweep can be used to infer the axis of one of the sweep values. This contract is not made explicit in the API, and should not be relied upon. If data for an NxM sweep is created in a permuted order, this MR fixes the direct export to directly extract the axis information from the larger multi-dimensional array. Backwards compatibility of ordering is maintained for non-permuted cases.

@asmull

Copy link
Copy Markdown
Contributor Author

@microsoft-github-policy-service agree company="Microsoft"

@asmull Aaron Smull (asmull) changed the title Aaron/fix xarray inferred from ordering Fix xarray inferred-from data ordering Aug 25, 2026
@asmull
Aaron Smull (asmull) force-pushed the aaron/fix-xarray-inferred-from-ordering branch from 6029ed8 to 820eb16 Compare August 25, 2026 22:19
permuted data axis.

Fixing ordering of data when using permuted data axes

Revert "Updating documentation"

This reverts commit 60a4735.

Fixing linting
@asmull
Aaron Smull (asmull) force-pushed the aaron/fix-xarray-inferred-from-ordering branch from 820eb16 to 93cb275 Compare August 25, 2026 22:21
@codecov

codecov Bot commented Aug 25, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 71.90%. Comparing base (4a1350b) to head (8342065).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8445      +/-   ##
==========================================
+ Coverage   71.87%   71.90%   +0.02%     
==========================================
  Files         305      305              
  Lines       31997    32019      +22     
==========================================
+ Hits        22998    23022      +24     
+ Misses       8999     8997       -2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes ordering issues when exporting QCoDeS datasets to xarray in cases where data acquisition order is permuted relative to the inferred grid structure, and when inferred-from parameters need to be exported consistently alongside reordered data.

Changes:

  • Add an index plumbing path to _add_inferred_data_vars to support order-aware insertion of inferred-from variables in the pandas→xarray export route.
  • Rework the direct-to-xarray export (_xarray_data_set_direct) to derive coordinate axes and reorder data based on the actual dependency arrays rather than relying on shape-based axis inference.
  • Extend test coverage to validate permuted-grid ordering, direct-export grid validation, and inferred handling.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
src/qcodes/dataset/exporters/export_to_xarray.py Adds index-aware inferred var insertion and reimplements direct xarray export ordering/reconstruction logic.
tests/dataset/test_dataset_export.py Adds regression + validation tests for permuted direct export and _xarray_data_set_direct input validation.
tests/dataset/test_inferred_multiple_parents.py Updates direct calls to _add_inferred_data_vars for the new index argument.
tests/dataset/test_parameter_with_setpoints_has_control.py Updates _add_inferred_data_vars invocation to pass index=None.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread src/qcodes/dataset/exporters/export_to_xarray.py Outdated

Copilot AI commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

One or more custom setup steps configured for this repository failed during this Copilot code review run:

install pyright

Setup steps run before each review. If the review above is missing context, or no review was posted at all, the failing step above may be the cause. See the workflow run for failure details, fix your setup steps configuration, and re-request a review.

Note

You can configure setup steps for Copilot code review separately from Copilot cloud agent with a copilot-code-review.yml file. Read the docs for details.

@asmull
Aaron Smull (asmull) marked this pull request as ready for review August 26, 2026 08:22
@asmull
Aaron Smull (asmull) requested a review from a team as a code owner August 26, 2026 08:22
`Series(...).to_xarray()` expands a pandas index into one dimension per
index level. That is only compatible with the target xarray dataset when
the dataset dimensions are exactly the levels of the index. It is not the
case when the dataset uses a single `multi_index` dimension or the flat
index created by `DataFrame.reset_index()`, and a non unique MultiIndex
cannot be converted at all.

Only take the reindexing path when the index is unique and the dataset
dimensions match the index level names. In the remaining cases the data is
already in index order and can be reshaped directly.

Adds tests covering export of an inferred parameter with a `multi_index`
dimension and with a non unique MultiIndex.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 92aac34d-1a70-4ca9-900c-4ab563a731df
Fix inferred parameter export when index cannot be expanded
@asmull
Aaron Smull (asmull) added this pull request to the merge queue Sep 16, 2026
Merged via the queue into main with commit 49299f2 Sep 16, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants