From 30a038f5634e9aeab4e7b83cdb6278a13e491b06 Mon Sep 17 00:00:00 2001 From: Shivaji Kharse Date: Wed, 7 Oct 2026 11:27:11 +0530 Subject: [PATCH 1/2] docs(bulk): document the --tablet_placement flag Adds a Predicate Placement section to the bulk loader guide covering the placement file format, routing behavior, validation errors and the load-time-only scope, and adds the flag to the dgraph bulk CLI reference. --- docusaurus-docs/docs/cli/bulk.md | 1 + docusaurus-docs/docs/migration/bulk-loader.md | 52 +++++++++++++++++++ 2 files changed, 53 insertions(+) diff --git a/docusaurus-docs/docs/cli/bulk.md b/docusaurus-docs/docs/cli/bulk.md index 7bf138a2..88388d12 100644 --- a/docusaurus-docs/docs/cli/bulk.md +++ b/docusaurus-docs/docs/cli/bulk.md @@ -164,6 +164,7 @@ Flags: --skip_map_phase Skip the map phase (assumes that map output files already exist). --skip_reduce_phase Skip the reduce phase (stops after map phase completion). --store_xids Generate an xid edge for each node. + --tablet_placement string Path to a JSON file pinning predicates to groups, as an array of {"predicate", "group", "namespace"} entries. Pinned predicates are written to the output shard of their group (group N is out/); everything else is packed as usual. Placement is applied at load time only; until the cluster enforces pins, Zero's rebalancer may later move tablets. Use map_shards > reduce_shards so unpinned predicates still balance by size. --tls string TLS Client options ca-cert=; The CA cert file used to verify server certificates. Required for enabling TLS. client-cert=; (Optional) The Cert file provided by the client to the server. diff --git a/docusaurus-docs/docs/migration/bulk-loader.md b/docusaurus-docs/docs/migration/bulk-loader.md index 85af1dd2..069eac3a 100644 --- a/docusaurus-docs/docs/migration/bulk-loader.md +++ b/docusaurus-docs/docs/migration/bulk-loader.md @@ -153,6 +153,57 @@ Copy `p` directories directly for faster deployment: 3. Start all Alphas simultaneously 4. Verify all Alphas create snapshots with matching index values +## Predicate Placement + +By default, Bulk Loader packs predicates into reduce shards by size, so which group serves a given predicate varies between runs. The `--tablet_placement` flag pins chosen predicates to specific groups, so the cluster starts with a deterministic, operator-chosen layout. + +Create a placement file containing a JSON array of entries: + +```json +[ + {"predicate": "payload", "group": 2}, + {"predicate": "friend", "group": 3, "namespace": 0} +] +``` + +| Field | Description | +|-------|-------------| +| `predicate` | Predicate name, without a namespace prefix | +| `group` | Target Alpha group, from 1 to `--reduce_shards`. Group N's data is written to `out//p` | +| `namespace` | Optional namespace the predicate belongs to (default: `0`) | + +Pass the file to the loader: + +```sh +dgraph bulk \ + --files data.rdf.gz \ + --schema schema.txt \ + --zero localhost:5080 \ + --map_shards 6 \ + --reduce_shards 3 \ + --tablet_placement placement.json +``` + +### Behavior + +- A pinned predicate's data, index, and schema keys are all written to its group's output directory. This includes predicates that appear only in the schema and carry no data in the load. +- Predicates not listed in the file keep the default size-balanced packing. Use `--map_shards` greater than `--reduce_shards` for this: with equal values every map shard is dedicated to a group and size balancing is disabled (the loader logs a notice). +- The loader logs the routing of every pinned predicate, for example `pinned: 0-payload -> map shard 1 -> out/1/p (group 2)`, and warns after the map phase about pinned predicates that matched nothing in the schema or data — usually a typo in the placement file. + +### Validation + +The loader exits with an error before loading any data if the placement file contains: + +- a group below 1 or above `--reduce_shards` +- duplicate entries for the same namespace and predicate +- reserved (`dgraph.*`) predicates — these are always served by group 1 +- unknown fields, or a document that isn't a JSON array +- entries for namespaces that can never match when `--force-namespace` is also set + +:::note +Placement is applied at load time only. Once the cluster is running, Zero's automatic rebalancer (`--rebalance_interval`, default 8 minutes) may move tablets between groups; set a large rebalance interval on your Zeros to preserve the layout. `/moveTablet` can also move tablets at any time. +::: + ## Multi-tenancy By default, Bulk Loader preserves namespace information from data files. Without namespace info, data loads into the default namespace. @@ -267,6 +318,7 @@ Increase if you have RAM to spare: | `--xidmap` | Directory for XID→UID mappings | | `--format` | Force format (`rdf` or `json`) | | `--force-namespace` | Load into specific namespace | +| `--tablet_placement` | JSON file pinning predicates to groups | | `--encryption` | Encryption key file | | `--encrypted` | Input files are encrypted | | `--encrypted_out` | Encrypt output (default: true if key provided) | From 59aec567c9cf1c416232ab6aebbe0fedda562dbf Mon Sep 17 00:00:00 2001 From: Shivaji Kharse Date: Wed, 7 Oct 2026 16:36:15 +0530 Subject: [PATCH 2/2] docs(bulk): recommend --rebalance_interval=0 to preserve tablet placement --- docusaurus-docs/docs/migration/bulk-loader.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docusaurus-docs/docs/migration/bulk-loader.md b/docusaurus-docs/docs/migration/bulk-loader.md index 069eac3a..cbdedc88 100644 --- a/docusaurus-docs/docs/migration/bulk-loader.md +++ b/docusaurus-docs/docs/migration/bulk-loader.md @@ -201,7 +201,7 @@ The loader exits with an error before loading any data if the placement file con - entries for namespaces that can never match when `--force-namespace` is also set :::note -Placement is applied at load time only. Once the cluster is running, Zero's automatic rebalancer (`--rebalance_interval`, default 8 minutes) may move tablets between groups; set a large rebalance interval on your Zeros to preserve the layout. `/moveTablet` can also move tablets at any time. +Placement is applied at load time only. Once the cluster is running, Zero's automatic rebalancer (`--rebalance_interval`, default 8 minutes) may move tablets between groups; set `--rebalance_interval=0` on every Zero to disable automatic rebalancing and preserve the layout. `/moveTablet` can still move tablets at any time. ::: ## Multi-tenancy