Commit 0dbec57
authored
fix(kernel): server-sourced async state + cursor-state/metadata parity for use_kernel (#838)
fix(kernel): cursor-state tracking + metadata parity for use_kernel
Brings the use_kernel=True path closer to Thrift parity on the
cursor-state-tracking and metadata clusters from the API+CUJ gap
audit, surfaces DML rowcount, and makes async command state
server-sourced.
Async command state — server is the source of truth (no local state):
- get_query_state / get_execution_result now re-attach to the
statement by id (Session.attach_async_statement, kernel side) and
read state / await results from the server, instead of consulting a
connector-side handle map. SEA keys GetStatementStatus/Result purely
on the id, so this works whether or not the connector still holds the
submitting handle — and get_execution_result is inherently re-callable
(each call re-attaches), matching the Thrift backend's re-fetchable
operation handle.
- Deleted _closed_commands (and its bounded-FIFO machinery
_record_closed / _CLOSED_COMMANDS_MAX): the connector no longer
invents a CLOSED-vs-SUCCEEDED answer. The server reports CLOSED
natively (verified: GET after close returns 200 state=CLOSED until
TTL, then 404 NOT_FOUND, which maps to a terminal state). This is the
local-state inconsistency removed.
- _async_handles / _async_statements are retained but DEMOTED to a
keep-alive-only registry: the submitting ExecutedAsyncStatement's Drop
fires close_statement, which would kill the live async query, so we
hold the handle until close_command / close_session. They are no
longer consulted for state or results.
Cursor-state tracking (T7):
- Set cursor.active_command_id in the _make_result_set chokepoint so
sync execute, async fetch, AND metadata all leave the cursor pointing
at the command that produced the current result set (matches Thrift's
unconditional set in _handle_execute_response).
- On a failed sync execute, publish the server-issued statement id
(read from the canceller's inflight slot) onto active_command_id
before re-raising. Best-effort; never masks the original failure.
Metadata:
- Normalize wildcard/blank catalog ('%' / '*' / '' / None) to None
(all-catalogs) across get_schemas/get_tables/get_columns
(_catalog_or_none), making the three symmetric (fixes columns).
- Normalize empty/whitespace-only pattern args to None (match-all) via
_none_if_blank. % / * stay as real LIKE wildcards on patterns.
- Drop the connector-side table_types drain + client-side refilter in
get_tables; the kernel filters table_types itself (case-insensitively
as of the batch-3 kernel change). Removed the dead
_drain_kernel_handle / _StaticArrowHandle helpers + unused imports.
- Fix the Cursor.columns() docstring: catalog_name=None is accepted on
all backends.
DML rowcount:
- Surface the kernel's num_modified_rows as cursor.rowcount for DML
instead of the hardcoded -1. None (SELECT / warehouses that don't
report it) leaves rowcount at -1, matching Thrift; getattr-guarded.
Bumps KERNEL_REV to the batch-3 kernel commit (case-insensitive
table_types, HTTP error-envelope parsing, num_modified_rows getter,
Session::attach_async_statement).
Co-authored-by: Isaac
Signed-off-by: Vikrant Puppala <vikrant.puppala@databricks.com>1 parent 85f8ba3 commit 0dbec57
5 files changed
Lines changed: 803 additions & 272 deletions
File tree
- src/databricks/sql
- backend/kernel
- tests
- e2e
- unit
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | | - | |
| 1 | + | |
0 commit comments