Add detailed AGENTS.md files for most of the modules

AGENTS.md

@@ -0,0 +1,127 @@
+# AGENTS.md
+
+This file provides guidance to coding agents (Claude Code, and other tools that read `AGENTS.md`) when
+working with code in this repository.
+
+## What this is
+
+Kotlin DataFrame — a typesafe, immutable, hierarchical in-memory data-processing library for the JVM
+(`org.jetbrains.kotlinx:dataframe`). Every `DataFrame` operation returns a new instance, reusing underlying
+column storage where possible. Type-safe column access is provided through auto-generated extension properties
+(on-the-fly in notebooks, via the compiler plugin / KSP in projects).
+
+## Build & test commands
+
+Gradle (Kotlin DSL), multi-module. Use the wrapper (`./gradlew`, or `gradlew.bat` on Windows). A JDK must be on
+`JAVA_HOME`/`PATH` to launch the wrapper; the build daemon JDK is provisioned from `gradle/gradle-daemon-jvm.properties`
+(version controlled by `gradle-jdk` in `gradle/libs.versions.toml`).
+
+- `./gradlew build` — full build; runs all tests **and** the ktlint check.
+- `./gradlew <module>:test` — test a single module (e.g. `./gradlew core:test`); much faster during development.
+- `./gradlew <module>:test --tests "org.jetbrains.kotlinx.dataframe.SomeTest"` — run a single test class/method.
+- Add `-Pkotlin.dataframe.debug=true` when validating changes. It enables extra, heavier correctness checks that
+  are off in production. PR/CI builds run with this flag (`build -Pkotlin.dataframe.debug=true`), so match it locally.
+- `./gradlew publishToMavenLocal -PskipKodex` — publish locally for iteration. `-PskipKodex` skips KDoc
+  preprocessing (see below); the result has "broken" KDocs, so it is for local dev only.
+
+Module names for `<module>:` come from `settings.gradle.kts` (e.g. `core`, `dataframe-csv`, `dataframe-jdbc`,
+`dataframe-json`, `dataframe-arrow`, `dataframe-excel`, `dataframe-jupyter`).
+
+## Architecture
+
+Dependency graph is rooted at `core`, with I/O and integration split into optional modules (see `MODULES.md`):
+
+- **`core`** — base module, no deps. DataFrame/DataColumn/DataRow types, the operations API (`core/.../api/`),
+  aggregations, math, schema handling, codegen, and the `impl/` internals. Also holds deprecated csv/tsv and HTML
+  integrations that now live in dedicated modules.
+- **I/O modules** (each depends on `core`): `dataframe-csv`, `dataframe-json`, `dataframe-excel`,
+  `dataframe-arrow`, `dataframe-jdbc`. The root `dataframe` artifact re-exports Arrow/Excel/JDBC/CSV/JSON as `api`.
+  `dataframe-jdbc` ships built-in `DbType` support for **H2, MariaDB, MySQL, MS SQL Server, PostgreSQL, SQLite, and
+  DuckDB** (any other JDBC database works via a user-supplied `DbType`); see `dataframe-jdbc/AGENTS.md`.
+- **Integrations**: `dataframe-jupyter`, `dataframe-geo` → `dataframe-geo-jupyter`, `dataframe-openapi` →
+  `dataframe-openapi-generator`.
+- **Compiler/build tooling**:
+  - The **Kotlin DataFrame compiler plugin** — the recommended way to get compile-time type-safe extension
+    properties — is **NOT in this repo**. It is developed in the Kotlin repository at
+    `github.com/JetBrains/kotlin/tree/master/plugins/kotlin-dataframe`. This repo's `plugins/kotlin-dataframe`
+    is a **disabled, out-of-date legacy copy** (its tests are turned off; see issue #1290) — do not treat it as
+    the live plugin.
+  - `dataframe-compiler-plugin-core` — a shaded subset of `:core` bundled *inside* that compiler plugin (and
+    IntelliJ) to run compile-time interpreters of operations; it is not itself a compiler plugin. See its
+    `AGENTS.md`.
+  - `plugins/*`: `plugins/symbol-processor` (KSP codegen for `@DataSchema`), `plugins/dataframe-gradle-plugin`,
+    and support plugins (`expressions-converter`, `public-api-modifier`, `keywords-generator` — the last is a
+    separate build with its own Kotlin version).
+- **Build logic** lives in `build-logic/` and `build-settings-logic/` as convention plugins
+  (e.g. `conventions.plugins.dfbuild.*`), not inline in the module build files.
+
+Most modules ship their own `AGENTS.md` (`core`, `dataframe-csv`, `dataframe-json`, `dataframe-excel`,
+`dataframe-arrow`, `dataframe-jdbc`, `dataframe-geo`, `dataframe-compiler-plugin-core`, `samples`,
+`docs/StardustDocs`) with module-specific details. **Read the relevant module's `AGENTS.md` before working in
+it** — it usually answers structure/pattern questions directly, so you can skip broad code exploration.
+
+The public operations API is organized as one file per operation under
+`core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/api/` (e.g. `filter.kt`, `groupBy.kt`, `join.kt`). When adding
+or changing an operation, find the sibling file for the closest existing operation and follow its structure.
+
+## Generated sources & KDoc preprocessing (KoDEx)
+
+Public KDocs use KoDEx notations (`{@include [X]}`, `@set`/`@get`/`$`, `@sample`, `@ExcludeFromSources`,
+`@ExportAsHtml`). See `KODEX_KDOC_PREPROCESSING.md` and `KDOC_GUIDELINES.md`.
+
+- **KoDEx** (an external KDoc-preprocessing tool, applied via the `kodex` convention plugin) expands the templated
+  KDocs. `core:processKDocsMain` processes KDocs into `generated-sources`; `changeJarTask` makes `sources.jar` use
+  those.
+- **Don't hand-edit generated sources, and don't grep/crawl the tree.** Every module built with KoDEx (`core`,
+  `dataframe-csv`, `dataframe-excel`, `dataframe-geo`, `dataframe-jdbc`) has a tracked `generated-sources/` copy,
+  plus checked-in accessors under `**/src/generated-dataschema-accessors/` and generated **HTML** under
+  `docs/StardustDocs/resources/`. In `generated-sources/` the **implementation and public API are the same** as the
+  hand-written `src/main/kotlin/…` — KoDEx only expands the KDoc and strips the doc-only fragments (declarations
+  marked `@ExcludeFromSources`). So searching the tree (`**/generated-sources/**`,
+  `**/generated-dataschema-accessors/**`) just doubles every hit and bloats context — do code work from
+  `src/main/kotlin/…`. The one reason to open a generated file is the KDoc: it is the **fully-expanded**
+  documentation (all `{@include …}`/`@set`/`@get` resolved, URLs inlined). If you specifically need a symbol's
+  final rendered docs, open that single generated file deliberately — don't explore. A CI bot regenerates and
+  auto-commits all of these on `master` after merge — you don't run/commit generation yourself.
+- For KDocs: never write from scratch — reuse/`@include` an existing operation's KDoc and adapt it. Reusable KDoc
+  fragments are written once as `internal`/`private` interfaces and `typealias … = Nothing` declarations and
+  composed via `@include`; this pattern is used in every KoDEx module (the shared cross-module fragment library
+  lives in `core`'s `documentation/` package).
+
+## Documentation samples
+
+Documentation snippets are runnable tests. The `samples` module and `core`'s `samplesTest`/`korro` tasks execute
+sample code, save outputs, and inject them into the WriterSide docs under `docs/StardustDocs/`. Migration of samples
+into the `:samples` module is in progress; new documentation samples belong there.
+
+## Code style
+
+- Kotlin official style, enforced by **ktlint** (`ktlint_official`, experimental rules enabled) via `.editorconfig`.
+  IntelliJ Ktlint plugin recommended. In IDEA, delegate build actions to Gradle.
+- 4-space indent, LF, UTF-8, 120-column max line length (see `.editorconfig` for exact per-glob overrides).
+- **No wildcard imports** — star-import thresholds are set to effectively infinite.
+- Class/function signatures with ≥4 parameters are forced multiline.
+- **`ktlint_standard_chain-method-continuation = disabled`** — deliberate. Group operations on one line when
+  possible so a chain reads like a sentence; only start a new line when a *new* operation begins:
+
+  ```kt
+  df
+      .update { x and y }.where { true }.with { it + 1 }
+      .split { z }.by(",").into("z1", "z2")
+      .groupBy {
+          cols(a, b, c)
+      }.aggregate {
+          count { d } into "count"
+          mean { d } into "mean"
+      }
+      .print()
+  ```
+
+## Constraints
+
+- Base all PRs on `master`; the current checked-out branch may be a feature branch (`git status` at session start).
+- Bug fixes require a reproducing test; new public APIs require docs + tests.
+- Public API is guarded by **binary-compatibility-validator** in every publishable module — when you change public
+  signatures, run the `apiDump`/`apiCheck` Gradle tasks and commit the module's updated `api/*.api` dump.
+- DB tests in `dataframe-jdbc` are structured to run against either Dockerized or locally installed DBMS
+  (`io/local/` per-database tests, `io/db/` type tests, `io/h2/`). Do not point tests at production databases.

CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md

core/AGENTS.md

@@ -0,0 +1,133 @@
+# AGENTS.md — core
+
+Guidance for this module. See the root `AGENTS.md` for repo-wide build/style/KoDEx rules; only module-specific
+details are here. `core` is the base module (artifact `dataframe-core`) — it has no dependency on any other
+`dataframe-*` module; everything else depends on it.
+
+Paths below abbreviate `core/src/main/kotlin/org/jetbrains/kotlinx/dataframe/` as `…/`.
+
+## Central types & the column model
+
+Top-level public interfaces, each in its own file at `…/`:
+
+- **`DataFrame<out T>`** (`DataFrame.kt`) — immutable, ordered list of columns (distinct non-empty names, equal
+  sizes). `T` is a **schema marker** (covariant), not stored data — it resolves the generated typesafe extension
+  properties (see Codegen). Impl in `impl/DataFrameImpl.kt`.
+- **`DataColumn<out T>`** (`DataColumn.kt`) — named, typed column; its companion is the main factory
+  (`createValueColumn`, `createColumnGroup`, `createFrameColumn`, `createByInference`).
+- **`DataRow<out T>`** (`DataRow.kt`) — a single row (`index()`, `df()`, cell getters). Impl `impl/DataRowImpl.kt`.
+- Access hierarchy: **`ColumnsScope`** (`ColumnsScope.kt`, minimal `get(name)`) ⊃ **`ColumnsContainer`**
+  (`ColumnsContainer.kt`) ⊃ `DataFrame`. Generated extension properties hang off `ColumnsScope<Schema>`.
+
+**Column kinds** (`columns/ColumnKind.kt`: `Value`/`Group`/`Frame`) are what make schemas *hierarchical*:
+
+- `ValueColumn<T>` (`columns/ValueColumn.kt`) — plain values.
+- `ColumnGroup<T>` (`columns/ColumnGroup.kt`) — a nested set of columns; behaves as both a column **and** a
+  `DataFrame`. This is how nested/JSON-like structures are represented.
+- `FrameColumn<T>` (`columns/FrameColumn.kt`) — each cell is itself a `DataFrame` (produced by `groupBy`).
+
+Type aliases in `aliases.kt`: `AnyFrame`/`AnyRow`/`AnyCol` and the DSL lambda aliases `ColumnsSelector<T,C>`,
+`ColumnSelector<T,C>`, `RowFilter<T>`, `RowExpression`, etc. The `columns/` package holds the lower-level
+abstractions the DSL resolves against: `BaseColumn`, `ColumnReference`, `ColumnAccessor`, `ColumnSet`,
+`ColumnsResolver`, `SingleColumn`, `ColumnPath`, `ColumnWithPath`.
+
+## Operations API (`api/`) and the `api → impl` convention
+
+~125 files in `api/`, **one file per operation**, lowercase-named after the operation (`filter.kt`, `select.kt`,
+`groupBy.kt`, `convert.kt`, `move.kt`, `pivot.kt`, `join.kt`, `add.kt`, …). PascalCase files hold supporting API
+types (`ColumnsSelectionDsl.kt`, `DataFrameGet.kt`, `DataRowApi.kt`, `ParserOptions.kt`, `Defaults.kt`).
+
+A typical operation file exposes public extension functions on `DataFrame<T>` (plus overloads on `DataColumn`,
+`GroupBy`, `ColumnsSelectionDsl`), grouped with `// region …` comments, and usually offers overloads for a
+`ColumnsSelector` lambda, `vararg String`, and (deprecated) `KProperty`/`ColumnReference`.
+
+**Delegation convention — follow it when adding/changing operations:** the `api/xxx.kt` function is a thin façade;
+the real logic lives in `impl/api/xxxImpl.kt`. `impl/` mirrors the public packages (`impl/api/`, `impl/columns/`,
+`impl/aggregation/`, `impl/schema/`, `impl/codeGen/`, `impl/io/`) and holds all concrete classes and helpers
+(`DataFrameImpl`, `GroupByImpl`, `TypeUtils`, resolution decorators like `ColumnWithParent`). Keep new internal
+logic in `impl/`, not in the public file.
+
+Operations are annotated for the compiler / IntelliJ plugins (`@Refine`, `@Interpretable("…")`, `@AccessApiOverload`,
+`@RequiredByIntellijPlugin`, from `annotations/`). Preserve these annotations when editing signatures.
+
+## Column(s) Selection DSL
+
+The receiver of the selector lambdas (`ColumnsSelector`/`ColumnSelector`, defined in `aliases.kt`) is
+**`ColumnsSelectionDsl<out T>`** (`api/ColumnsSelectionDsl.kt`) — used by
+`select`/`move`/`convert`/`group`/`remove`/`update`/etc. — which inherits the smaller base
+**`ColumnSelectionDsl<out T>`** (`api/ColumnSelectionDsl.kt`). Whether a selector resolves to a **single** column or
+**multiple** is decided by the lambda's **return type** (`SingleColumn<C>` vs `ColumnsResolver<C>`), not by the
+receiver. `ColumnsSelectionDsl` is a large **composite interface**: it inherits one per-feature DSL interface per operation
+file (e.g. `all.kt` → `AllColumnsSelectionDsl`, `cols.kt` → `ColsColumnsSelectionDsl`). It carries a `@DslMarker`
+to prevent accessor leakage across nested `select`s. Lambdas resolve to a `columns/ColumnsResolver<C>`
+(`SingleColumn` or `ColumnSet`) against the frame via `impl/DataFrameReceiver.kt` (`getColumnsImpl`, driven by an
+`UnresolvedColumnsPolicy`). When adding an operation usable inside a selector, add its DSL interface in the same
+`api/` file and wire it into `ColumnsSelectionDsl`.
+
+## Typesafe access: compiler plugin vs codegen
+
+Extension properties on `ColumnsScope<Schema>` (returning `DataColumn<X>`) and `DataRow<Schema>` (returning `X`),
+each with a `@JvmName`, are what make `df.someColumn` type-safe — and why `T` is a schema marker. There are three
+ways they get produced, and picking the wrong one is a common mistake:
+
+- **Kotlin compiler plugin — the recommended path.** It generates accessors on-the-fly at compile time (and in the
+  IDE) with no `@DataSchema` declaration needed. It lives in the Kotlin repo
+  (`github.com/JetBrains/kotlin/tree/master/plugins/kotlin-dataframe`), not here. The operation annotations in this
+  module (`@Refine`, `@Interpretable("…")`, `@RequiredByIntellijPlugin`, `@AccessApiOverload`, from `annotations/`)
+  exist specifically to drive it, and it consumes `dataframe-compiler-plugin-core` for its interpreters.
+- **KSP symbol processor + Gradle plugin — the older `@DataSchema` build-time path (now legacy/deprecated).**
+  `@DataSchema` (`annotations/DataSchema.kt`) marks an interface as a schema (related: `ColumnName`,
+  `ImportDataSchema`, `HasSchema`, … in `annotations/`); `plugins/symbol-processor` + `plugins/dataframe-gradle-plugin`
+  invoke this module's `CodeGenerator` at build time to emit accessors. Note: `plugins/symbol-processor` is now
+  disabled because KSP1 is no longer compatible with Kotlin 2.3+ (`plugins/README.md`).
+- **Runtime `ReplCodeGenerator` — Jupyter/REPL only.** `impl/codeGen/ReplCodeGenerator` emits accessor source as
+  strings to be `eval`'d in the next notebook cell (used by the `jupyter/` helpers); it is not a path for ordinary
+  compiled projects. The underlying engine `CodeGenerator` (`codeGen/CodeGenerator.kt`, `ExtensionsCodeGenerator.kt`,
+  `SchemaProcessor.kt`, `Marker*.kt`, `NameNormalizer.kt`, `DefaultReadDfMethods.kt`; impl in `impl/codeGen/`) is
+  **shared** — it is what the build-time `@DataSchema` path above also calls.
+
+The runtime schema model (used for generation and schema comparison) is in `schema/` (`DataFrameSchema.kt`,
+`ColumnSchema.kt`, `CompareResult.kt`).
+
+**Checked-in accessors for `:core` itself:** `core/src/generated-dataschema-accessors/{main,test}/…` holds
+pre-generated `<SchemaSimpleName>$Extensions.kt` for `:core`'s own internal `@DataSchema` types (e.g.
+`ColumnDescription`, `ValueCount`). They are committed **only because `:core` cannot attach the compiler plugin to
+itself** — the plugin depends on a subset of `:core`, so doing so would be circular. `core/generated-sources/…` is
+the build-time KoDEx output. **Don't hand-edit any of these** — they are generated (the build registers the
+accessors dir as a source root in `build.gradle.kts`). In `core/generated-sources/` (358 files) the
+**implementation and public API match** `src/main/kotlin/…` (KoDEx only expands the KDoc and strips
+`@ExcludeFromSources` doc-only fragments), so don't grep/browse it for implementation — it only bloats context;
+work from `src/main/kotlin/…`. The one reason to open a generated file is to read a symbol's **fully-expanded
+KDoc** (KoDEx resolves all `{@include …}`/`@set`/`@get` there).
+
+## Other packages (one line each)
+
+- `aggregation/` — DSLs/contracts for reductions (`Aggregatable`, `AggregateDsl`, `NamedValue`), used by
+  `aggregate`/`groupBy`/`pivot`.
+- `math/` — statistical primitives (`mean`, `median`, `std`, `sum`, `percentile`, `cumsum`, …).
+- `dataTypes/` — rich cell value types for rendering (`IMG`, `IFRAME`).
+- `jupyter/` — helper utilities (`JupyterConfiguration`, `CellRenderer`, `KotlinNotebookPluginUtils`, …); the
+  actual Kotlin Notebook integration lives in `:dataframe-jupyter`.
+- `io/` — read/write helpers and the **deprecated** in-core csv/tsv/html integrations (`csv.kt`, `tsv.kt`,
+  `html.kt`, `guess.kt`, `Compression.kt`); prefer the dedicated `dataframe-csv`/`dataframe-json` modules.
+- `documentation/` — see below. `exceptions/` — custom exceptions. `util/` — deprecation-message constants.
+
+## KoDEx documentation-only pattern (core specifics)
+
+The general rule (reuse/`@include` fragments; see `KDOC_GUIDELINES.md` / `KODEX_KDOC_PREPROCESSING.md`) is in the
+root `AGENTS.md`. Core-specific: `core` hosts the **shared cross-module KDoc-fragment library** in its
+`documentation/` package (`SelectingColumns.kt`, `SelectingRows.kt`, `DocumentationUrls.kt`, `DslGrammar.kt`, …) —
+`internal`/`private` interfaces and `typealias … = Nothing` declarations with no runtime behavior, `@include`d into
+operation KDoc (also inline in operation files, e.g. `internal interface Select` in `api/select.kt`). Other KoDEx
+modules keep their own local fragments (e.g. csv's `documentationCsv/`).
+
+## Build specifics
+
+- Applies `buildConfig`, `kodex`, `keywordsGenerator` convention plugins plus `korro`,
+  `binary-compatibility-validator`, and `kotlinx.benchmark`. (For the `apiDump`/`apiCheck` flow, see root `AGENTS.md`.)
+- Extra source sets beyond `main`/`test`: **`samples`** (`src/test/kotlin`, compiled with the expressions-converter
+  compiler plugin; runs documentation samples via `samplesTest`/`korro` — see root `AGENTS.md`) and **`testJava16`**
+  (`src/testJava16/`, compiled at JDK 16 for Java-Records tests).
+- Run just this module: `./gradlew core:test` (add `-Pkotlin.dataframe.debug=true` to match CI). The debug checks
+  are relevant for **every** module (jupyter/example integration tests, runtime `DataColumn`-vs-`KType` checks in
+  jdbc, …), not just `core`.

core/CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md