Add non-blocking DNS for the async API via c-ares event-loop integration#324
Draft
bjosv wants to merge 6 commits into
Draft
Add non-blocking DNS for the async API via c-ares event-loop integration#324bjosv wants to merge 6 commits into
bjosv wants to merge 6 commits into
Conversation
bjosv
added a commit
that referenced
this pull request
Jun 24, 2026
`getaddrinfo()` has no timeout mechanism. When DNS is slow or unresponsive, libvalkey hangs indefinitely with no way to cancel the lookup. This PR adds optional c-ares integration (`USE_CARES=1` / `-DENABLE_CARES=ON`) that replaces `getaddrinfo()` with c-ares driven by a `poll()` loop. DNS resolution respects `connect_timeout` (defaulting to 5 seconds if unset), so lookups that previously hung indefinitely now fail with a timeout error. The PR consists of two commits: the first moves existing DNS logic into `dns.[ch]`, and the second adds the c-ares integration. **Notes:** - The async API currently still blocks during DNS (now with a timeout). A follow-up PR #324 will make DNS fully non-blocking for the async API via event-loop integration. - Without c-ares enabled, behavior is unchanged (plain `getaddrinfo`) - IPv4/IPv6 preference flags and fallback behavior are preserved - IPv6 literals (e.g. `::1`) are detected and handled correctly - Targets Linux, macOS, FreeBSD (not Windows) --------- Signed-off-by: Björn Svensson <bjorn.a.svensson@est.tech>
Add attach_fn and attach_data to valkeyOptions, allowing the event-loop adapter to be specified as part of the connection options. When set, valkeyAsyncConnectWithOptions() attaches the adapter and registers the fd with the event loop internally. For TCP connections, connect is deferred past context initialization to ensure the context is fully set up before the connect call. The adapter receives a valid fd and works without modification. This mirrors the existing cluster API pattern (valkeyClusterOptions) and enables future work where the adapter drives non-blocking DNS resolution. The legacy path (attaching the adapter separately after connect) remains fully supported. Signed-off-by: Björn Svensson <bjorn.a.svensson@est.tech>
Signed-off-by: Björn Svensson <bjorn.a.svensson@est.tech>
Add async_connect_callback and async_disconnect_callback to valkeyOptions, allowing the connect and disconnect callbacks to be specified as part of the connection options. When set, valkeyAsyncConnectWithOptions() registers them on the context before the connect is initiated, so no connect event is missed when the event loop is already running at connect time. This is equivalent to calling valkeyAsyncSetConnectCallback() / valkeyAsyncSetDisconnectCallback() after connect, which remains fully supported. Signed-off-by: Björn Svensson <bjorn.a.svensson@est.tech>
Add src/timer.c with a pre-allocated pool of timers managed as a sorted linked list. Timers can be scheduled, cancelled, and processed one at a time (to allow callbacks to free the context). This is the foundation for running multiple concurrent timers (command timeout, connect timeout, cluster topology refresh, etc.) through a single adapter scheduleTimer hook. Callers that need periodic behavior reschedule in their callback. Uses CLOCK_MONOTONIC on Unix and QueryPerformanceCounter on Windows. Signed-off-by: Björn Svensson <bjorn.a.svensson@est.tech>
Connect and command timeouts are now separate timers: - connect_timer: scheduled during connect, cancelled on success - command_timer: scheduled on first write when connected, tracks reply activity to detect unresponsive servers valkeyAsyncHandleTimeout dispatches expired timers via valkeyProcessTimers. No adapter changes required. | Test | main | PR | |--------------------------|--------|--------| | 10000x PING pipelined | 0.004s | 0.005s | (avg from 4 runs) | 10000x INCRBY pipelined | 0.004s | 0.004s | (avg from 4 runs) | 10000x LRANGE pipelined | 0.793s | 0.782s | (avg from 4 runs) Note: changing command_timeout at runtime no longer affects an already-running timer. The new value takes effect after the current timer fires. Signed-off-by: Björn Svensson <bjorn.a.svensson@est.tech>
ee27b49 to
9cdafe2
Compare
When c-ares is enabled and the connection is set up via the options path (valkeyAsyncConnectWithOptions with attach_fn), DNS resolution is driven by the event loop instead of blocking the calling thread. For TCP, connect is deferred until the event adapter is attached. A c-ares aware adapter (libevent) initiates async DNS from its attach function, registering c-ares sockets and timers with the event loop. On DNS completion, socket() + connect() proceed as normal and the fd is registered for connect completion. The async path is opt-in: it only engages through the options path with an adapter that implements the c-ares hooks (addCaresSocket / delCaresSocket). All other cases, the legacy connect-then-attach path and adapters without the hooks, resolve DNS synchronously with the existing timeout-bounded path. Changes: - Add addCaresSocket/delCaresSocket hooks and dns_state to valkeyAsyncContext - Add the async DNS engine to dns.c (valkeyResolveAsyncStart / HandleEvent / Free), driven by the internal timer system - Add valkeyTcpConnectNonBlock to net.c for connect after async resolve - Attach the adapter before initiating connect in valkeyAsyncConnectWithOptions so async DNS can start on attach - Implement the c-ares socket handling in the libevent adapter, with lazy fd registration since no fd exists until DNS completes Signed-off-by: Björn Svensson <bjorn.a.svensson@est.tech>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Even with c-ares enabled for timeout-bounded DNS, the async API still blocked the calling thread during DNS resolution.
valkeyAsyncConnectWithOptions()resolved DNS synchronously before returning, using the sameblocking resolve path as the sync API. For applications using async I/O on a single thread, a slow DNS lookup stalls the entire event loop.
This PR builds on the c-ares sync DNS support (#323) and makes DNS fully non-blocking for the async API. When the connection is set up via the options path with a c-ares aware adapter (libevent), DNS resolution
is driven by the event loop, so no thread blocks during DNS.
How it works:
Async DNS is opt-in: it engages only when connecting via
valkeyAsyncConnectWithOptions()with an adapter set invalkeyOptions.attach_fn, and only when that adapter implements the c-ares hooks. When c-ares isenabled and such a connection is made:
valkeyAsyncConnectWithOptions()defers the TCP connect (VALKEY_CONNECT_DEFERRED): the endpoint is stored, but no DNS lookup or socket fd is created yet.event processing without blocking.
valkeyAsyncConnectWithOptions()resolves DNS synchronously with a timeout and connects before returning. This still blocks brieflybut is bounded by
connect_timeout. These adapters can be upgraded to full async DNS by implementing the c-ares hooks.The legacy path (
valkeyAsyncConnect()followed by a separate adapter attach) never defers, and always uses the blocking, timeout-bounded resolve, unchanged.The connect logic (source_addr binding, SO_REUSEADDR, MPTCP) is consolidated in
valkeyTcpConnectNonBlock()to avoid duplication between the async DNS path and the synchronous fallback.Design decisions:
requires no changes to existing adapters.
valkeyAsyncConnectWithOptions()(taken when the adapter did not start async DNS, i.e. dns_state == NULL), rather than a per-adapter macro. Adapters need no changes.prevents side effects when the channel is destroyed during context teardown.
valkeyAsyncConnectWithOptions()andvalkeyAsyncSetConnectCallback()skip write-interest registration; the write watch is added once the fd exists (aftersync connect, or from the DNS-completion callback). The shared _EL_ADD_READ/_EL_ADD_WRITE macros are unchanged.
Custom adapters:
Third-party adapters need no changes to remain correct: when c-ares is enabled, an adapter that does not implement the hooks gets a synchronous, timeout-bounded resolve performed by
valkeyAsyncConnectWithOptions()before the adapter is asked to operate on the fd. To get fully non-blocking DNS, an adapter implements the c-ares socket hooks (addCaresSocket/delCaresSocket) and callsvalkeyResolveAsyncStart()from its attach function, forwarding fd activity tovalkeyResolveAsyncHandleEvent(). See the libevent adapter for a reference.Testing:
ct_async_dnsintegration test exercising both the libevent async path and the standalone connect flow.