[leedqin]

Motivation

Closes: CNS-210

Maintained Objects hid subsources and every table, so the only freshness a source showed was the source row's own. That number tracks the source's progress collection, which advances on a timer as it reads the upstream position, whether or not table data has been written. A source can therefore read 1s while its subsources and tables lag by a minute or more, and the console gave no way to see the per-table number. This also covers CNS-190.

The critical path had the same blind spot. It only followed edges between sources, MVs, indexes and sinks, so an MV reading CREATE TABLE ... FROM SOURCE tables stopped at the MV and never reached the lagging table. Sources had no critical path at all.

With Postgres sources now snapshotting a newly added table without holding back the rest, users need to see each table's freshness and snapshot status to see that improvement.

Description

List

• Subsources and source-fed tables appear as rows with type subsource or table. Each carries its own freshness and ingestion status, its source's cluster, and its source's connector type. A table that is still snapshotting reads "Snapshotting" next to siblings that read "Running". The Object Type, Cluster and Status filters cover them.
• Progress collections and tables written with INSERT stay hidden.

Drawer

• A source's Freshness tab lists its subsources and tables with their freshness over the lookback window and their status. One line explains why the source's own number can read fresher than its tables.
• A subsource or table shows a link to its source, its ingestion status, and its own freshness chart.

Critical path

• A subsource or source-fed table walks back to its source. The breakdown strip then separates the source's lag from the table's own ingestion delay. An MV's path continues past the table to the source.
• A source has no upstream inside Materialize, so its critical path draws the subsources and tables it writes to. The one that trails the source the most is marked as the bottleneck, and the source itself is marked when none trails it. The four slowest are drawn, with a toggle for the rest. An unreadable (still snapshotting) table always sorts first. The node panel shows cluster metrics, which replaces the separate "Source cluster metrics" section.
• Node tags read source, subsource or table.
• The critical path graphs and the object drawer show an error when their query fails, instead of spinning.

Data and cost

• The subsource and source-table to source mapping is one builder, buildSourceExportsQuery. The hydration-aggregate SUBSCRIBE, the critical path and the workflow graph all use it.
• The parent mapping rides on the page's existing hydration-aggregate SUBSCRIBE, which now also keeps subsource status rows. That adds an mz_tables_ind scan and an mz_object_dependencies join to that page-scoped subscribe. There is no new subscribe or poll.
• The source critical path reads its exports' binned lag by id from mz_wallclock_global_lag_recent_history (an index lookup) and labels nodes from the list.
• The drawer's freshness chart also moves to the id-keyed query, which drops its catalog joins. The query it used inner-joins mz_clusters on the object's own cluster, so it returned nothing for subsources and tables.

Fix in a shared component

• Canvas now sizes the graph in pixels. Chakra read the graph's numeric height as a theme size token when one matched, so a single-row graph (56px, the token for 14rem) rendered 224px tall and clipped its own node.

Not in this PR:

• The snapshot progress percentage in Source diagnostics is still missing for source-fed tables, because fetchSourceStatistics resolves stats through mz_sources. Subsources get a percentage.
• While any one table snapshots, the source's own status pill also reads "Snapshotting", because that is what the source-level snapshot_committed reports. The per-table rows show which table it is.

Screenshots

From a local Materialize. shop_src writes four tables and is snapshotting a fifth, page_views. The 15M-row snapshot saturates the small local replica, which is why the other tables lag too. crm_src has a legacy subsource, and order_summary reads one table from each source.

Source: per-table freshness and status, and the critical path out to its tables.

Source-fed table: the critical path back to its source.

Materialized view: the path now runs through the subsource and table to their sources.

Verification

Tests added or changed:

• queries.test.ts (new) covers the list builder:
  • Subsources and source-fed tables carry their source's cluster and connector type.
  • Progress collections and written tables are hidden.
  • A subsource is listed before its parent is known.
• MaintainedObjects.test.tsx: a snapshotting table renders next to a running one from the same source.
• criticalPathRenderable.test.ts covers the source critical path:
  • The export that trails the source most is primary, or the source is when none trails it.
  • Each node reads the bucket containing the anchor.
  • An unreadable export stays visible past the cap.
  • The cap and toggle work, and a source with no exports draws alone.
• hydrationAggregate.test.sql.ts:
  • Subsources are reported by status only.
  • Each subsource and source-fed table names its source and carries its own snapshot flag.
• criticalPath.test.sql.ts:
  • The path follows a source-fed table and a subsource back to their source, and leaves out a written table.
  • A subsource walks back to its source.

I checked all views in a browser against a local Materialize with Postgres sources. I also ran EXPLAIN on mz_catalog_server for the changed queries.

🤖 Generated with Claude Code

[jdonelson]

Really nice to see subsources and tables get their own freshness, and moving the parent lookup into one buildSourceExportsQuery cleans up three copies. Unit tests pass (39/39) and typecheck is clean. I didn't run the SQL tests. One design question on the critical path edge and one nit, otherwise looking good.

[jdonelson]

I wonder if the edge from a source to its subsources and tables should look different from the other edges in the critical path. The other edges mean "computed from": an MV can't be fresher than its input, so a slow input holds it back. A subsource isn't computed from its source. Both are written by the same ingestion, and the source's lag doesn't carry down to it. With the same arrow, the graph reads as if it does.

[leedqin]

Good call, it isn't a "computed from" edge, so it's now dotted, with a one-line legend under the graph (8389c8d). The source's lag does carry down for CREATE TABLE … FROM SOURCE tables, though. The source's own collection is the ingestion's progress collection (progress_id() in src/catalog/src/memory/objects.rs), and its tables are reclocked against it, so a table can't be fresher than its source. That's what the "1s from source + 2s self-delay" split relies on. It doesn't hold for legacy subsources, where the source is just another export and progress lives in <source>_progress, so those no longer get the edge.

[jdonelson]

Nit: subsources and tables do have an upstream for the critical path now (their source). Since this only picks the status to show, maybe drop the last clause?

[leedqin]

Done, dropped it.

[martykulma]

lgtm, thanks @leedqin!