Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,10 @@ jobs:
shards: 4
# Named rather than inherited: handing an entire secret store to another repository's
# workflow for one optional token is the wrong shape. Whether it ARRIVES is being measured
# on DataVault.jl#36 — a secret did not survive this call from a different ORGANISATION
# (lab-sotashimozono/ITensorAD.jl#20), and both this repo and that probe are same-org. If
# `collect` warns that no token reached it, add the caller-side upload job
# on DataVault.jl#36 — a secret did not survive this call when the caller was in a
# DIFFERENT organisation (measured once, on a private repo elsewhere), and both this repo
# and that probe are same-org. If `collect` warns that no token reached it, add the
# caller-side upload job
# (QAtlasHub/TestShards.jl/actions/upload-coverage). See QAtlasHub/TestShards.jl#48.
secrets:
CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
10 changes: 5 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# CLAUDE.md — ParamIO.jl

**Layer 1 of the infra HPC stack** (ParamIO → DataVault → ParallelManager):
**Layer 1 of the infra HPC stack** (ParamIO → DataVault → SweepRunner):
config TOML → `Vector{DataKey}`. Pure parsing + Cartesian enumeration; no IO, no
storage. See [`../CLAUDE.md`](../CLAUDE.md) for how the three layers fit together.

Expand Down Expand Up @@ -30,7 +30,7 @@ storage. See [`../CLAUDE.md`](../CLAUDE.md) for how the three layers fit togethe

- **`examples/inspect.jl`** — run it; prints exactly what `expand` produces (dotted keys, `format_path`, `canonical`). Start here.
- `README.md` — config schema + API.
- Full stack: [`../ParallelManager.jl/examples/`](../ParallelManager.jl/examples/).
- Full stack: [`../SweepRunner.jl/examples/`](../SweepRunner.jl/examples/).

## Source layout

Expand All @@ -39,6 +39,6 @@ storage. See [`../CLAUDE.md`](../CLAUDE.md) for how the three layers fit togethe

## Invariant when changing this package

**`canonical()`'s output schema is FROZEN** — `ParallelManager`'s `Manifest` and
the per-key lock use it as a directory-safe identity. Changing its format
silently orphans every existing manifest/lock. Run the test suite locally before pushing.
**`canonical()`'s output schema is FROZEN** — `SweepRunner`'s `Manifest` uses it
as a directory-safe identity. Changing its format silently orphans every
existing manifest. Run the test suite locally before pushing.
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
name = "ParamIO"
uuid = "938a3ac2-d340-473c-bcf1-88af577e4ccf"
version = "0.4.8"
version = "0.4.9"
authors = ["sota shimozono <shimozono-sota631@g.ecc.u-tokyo.ac.jp>"]

[deps]
Expand Down
14 changes: 7 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,21 +6,21 @@
[![Code Style: Blue](https://img.shields.io/badge/Code%20Style-Blue-4495d1.svg)](https://github.com/invenia/BlueStyle)

[![codecov](https://codecov.io/gh/QAtlasHub/ParamIO.jl/graph/badge.svg?token=57dh1RFl0t)](https://codecov.io/gh/QAtlasHub/ParamIO.jl)
[![Build Status](https://github.com/sotashimozono/ParamIO.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/sotashimozono/ParamIO.jl/actions/workflows/CI.yml?query=branch%3Amain)
[![Build Status](https://github.com/QAtlasHub/ParamIO.jl/actions/workflows/CI.yml/badge.svg?branch=main)](https://github.com/QAtlasHub/ParamIO.jl/actions/workflows/CI.yml?query=branch%3Amain)
[![License](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)

Read a config TOML and expand it into a list of `DataKey` — one per parameter
point in a sweep. Pure parsing and Cartesian enumeration: no IO, no storage, no
side effects.

ParamIO is **layer 1 of a three-package HPC stack**. It answers *what to
compute*; [`DataVault.jl`](https://github.com/sotashimozono/DataVault.jl) answers
compute*; [`DataVault.jl`](https://github.com/QAtlasHub/DataVault.jl) answers
*where results go* (`DataKey` → storage), and
[`ParallelManager.jl`](https://github.com/sotashimozono/ParallelManager.jl)
[`SweepRunner.jl`](https://github.com/QAtlasHub/SweepRunner.jl)
answers *do it* (`run!(work_fn, vault, keys)`, parallel and crash-recoverable).

```text
ParamIO DataVault ParallelManager
ParamIO DataVault SweepRunner
config.toml ──► Vault(config) ──► run!(work_fn, vault, keys)
│ expand
▼
Expand Down Expand Up @@ -147,11 +147,11 @@ src/

## See also

- [DataVault.jl](https://github.com/sotashimozono/DataVault.jl) — storage layer
- [ParallelManager.jl](https://github.com/sotashimozono/ParallelManager.jl) — the runtime
- [DataVault.jl](https://github.com/QAtlasHub/DataVault.jl) — storage layer
- [SweepRunner.jl](https://github.com/QAtlasHub/SweepRunner.jl) — the runtime
- [`../CLAUDE.md`](../CLAUDE.md) — how the three packages fit together

Issues / requests: [GitHub Issues](https://github.com/sotashimozono/ParamIO.jl/issues).
Issues / requests: [GitHub Issues](https://github.com/QAtlasHub/ParamIO.jl/issues).

## License

Expand Down
8 changes: 4 additions & 4 deletions examples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

ParamIO is **layer 1** of the HPC compute stack: it turns one config TOML into
a `Vector{DataKey}` — the list of parameter points a sweep will compute.
`DataVault` then maps each `DataKey` to storage, and `ParallelManager` runs the
`DataVault` then maps each `DataKey` to storage, and `SweepRunner` runs the
work over them. This example makes layer 1's output visible.

```bash
Expand Down Expand Up @@ -40,7 +40,7 @@ DataKey.params sample format_path canonical

- **`format_path` ≠ `canonical`.** `format_path` (the on-disk directory name)
uses only `path_keys`. `canonical` (the unique, Julia-version-stable key
identity used by `ParallelManager`'s manifest and lock) uses *all* params plus
identity used by `SweepRunner`'s manifest) uses *all* params plus
the sample index. Two distinct DataKeys can share a `format_path` but never a
`canonical`.

Expand All @@ -62,7 +62,7 @@ Optional: `[datavault] sweep_order` overrides the enumeration order;
| `load(path) -> ConfigSpec` | parse the TOML (and merge `[base] inherit`) |
| `expand(spec) -> Vector{DataKey}` | Cartesian product × samples |
| `format_path(key, path_keys) -> String` | compact directory segment |
| `canonical(key) -> String` | stable identity (used by ParallelManager) |
| `canonical(key) -> String` | stable identity (used by SweepRunner) |

See the full stack wired together in
[`ParallelManager.jl/examples/`](../../ParallelManager.jl/examples/).
[`SweepRunner.jl/examples/`](../../SweepRunner.jl/examples/).
2 changes: 1 addition & 1 deletion src/core/canonical.jl
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Return a string representation of `key` that is:
guaranteed stable within a Julia version.

This is intended for use as a directory-safe identity by downstream packages
that need a lookup key (e.g. `ParallelManager.Manifest`, `KeyLock`).
that need a lookup key (e.g. `SweepRunner.Manifest`).

## Schema(固定。変更不可)

Expand Down
Loading