Skip to content

Commit 0dbec57

Browse files
fix(kernel): server-sourced async state + cursor-state/metadata parity for use_kernel (#838)
fix(kernel): cursor-state tracking + metadata parity for use_kernel Brings the use_kernel=True path closer to Thrift parity on the cursor-state-tracking and metadata clusters from the API+CUJ gap audit, surfaces DML rowcount, and makes async command state server-sourced. Async command state — server is the source of truth (no local state): - get_query_state / get_execution_result now re-attach to the statement by id (Session.attach_async_statement, kernel side) and read state / await results from the server, instead of consulting a connector-side handle map. SEA keys GetStatementStatus/Result purely on the id, so this works whether or not the connector still holds the submitting handle — and get_execution_result is inherently re-callable (each call re-attaches), matching the Thrift backend's re-fetchable operation handle. - Deleted _closed_commands (and its bounded-FIFO machinery _record_closed / _CLOSED_COMMANDS_MAX): the connector no longer invents a CLOSED-vs-SUCCEEDED answer. The server reports CLOSED natively (verified: GET after close returns 200 state=CLOSED until TTL, then 404 NOT_FOUND, which maps to a terminal state). This is the local-state inconsistency removed. - _async_handles / _async_statements are retained but DEMOTED to a keep-alive-only registry: the submitting ExecutedAsyncStatement's Drop fires close_statement, which would kill the live async query, so we hold the handle until close_command / close_session. They are no longer consulted for state or results. Cursor-state tracking (T7): - Set cursor.active_command_id in the _make_result_set chokepoint so sync execute, async fetch, AND metadata all leave the cursor pointing at the command that produced the current result set (matches Thrift's unconditional set in _handle_execute_response). - On a failed sync execute, publish the server-issued statement id (read from the canceller's inflight slot) onto active_command_id before re-raising. Best-effort; never masks the original failure. Metadata: - Normalize wildcard/blank catalog ('%' / '*' / '' / None) to None (all-catalogs) across get_schemas/get_tables/get_columns (_catalog_or_none), making the three symmetric (fixes columns). - Normalize empty/whitespace-only pattern args to None (match-all) via _none_if_blank. % / * stay as real LIKE wildcards on patterns. - Drop the connector-side table_types drain + client-side refilter in get_tables; the kernel filters table_types itself (case-insensitively as of the batch-3 kernel change). Removed the dead _drain_kernel_handle / _StaticArrowHandle helpers + unused imports. - Fix the Cursor.columns() docstring: catalog_name=None is accepted on all backends. DML rowcount: - Surface the kernel's num_modified_rows as cursor.rowcount for DML instead of the hardcoded -1. None (SELECT / warehouses that don't report it) leaves rowcount at -1, matching Thrift; getattr-guarded. Bumps KERNEL_REV to the batch-3 kernel commit (case-insensitive table_types, HTTP error-envelope parsing, num_modified_rows getter, Session::attach_async_statement). Co-authored-by: Isaac Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>
1 parent 85f8ba3 commit 0dbec57

5 files changed

Lines changed: 803 additions & 272 deletions

File tree

KERNEL_REV

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
f4ee6fec78aabce8c0ea9c1ff47fc11b8191d013
1+
7ffb30d533c08651ca707b8dd13894c9e01cb68e

0 commit comments

Comments
 (0)