Background
When --use_io_uring=true, established RPC sockets already bypass
EventDispatcher for inbound reads and use their worker-local RingListener.
However, EventDispatcher is still initialized and uses epoll for:
- readiness on listening sockets, followed by
accept() until EAGAIN;
- temporary
EPOLLOUT waits when an established socket cannot continue writing;
- completion of nonblocking
connect();
- sockets and transports that do not use the io_uring receive path.
This leaves an extra dispatcher execution path and prevents io_uring mode from
owning the complete normal TCP lifecycle.
Goal
When io_uring mode is enabled, move normal TCP accept, connect, and
writable-wait handling from EventDispatcher to worker-local io_uring rings.
Preserve existing EventDispatcher behavior when io_uring is disabled and keep
an explicit fallback for unsupported transports.
Proposed direction
Accept
- Use
io_uring_prep_multishot_accept() for listening sockets.
- Prefer worker-local accept requests so accepted connections remain on the
worker that receives the CQE, avoiding a central accept-worker bottleneck and
cross-worker ring submission.
- Create accepted descriptors with
SOCK_NONBLOCK | SOCK_CLOEXEC.
- Keep the request active while CQEs contain
IORING_CQE_F_MORE.
- On server stop, cancel accept requests on their owning workers and wait for
terminal CQEs before closing the listening descriptor or destroying the
Acceptor.
- Handle
EMFILE, ENFILE, ENOBUFS, and ENOMEM with bounded/backoff
rearming rather than a tight loop.
Connect
- Replace nonblocking
connect() + EPOLLOUT with IORING_OP_CONNECT.
- Preserve timeout and exactly-once callback behavior, potentially using linked
timeouts.
- Keep connect completion and socket ownership on the selected worker ring.
Writable waits
- Replace temporary
EPOLLOUT registration with an io_uring POLLOUT request,
or let the io_uring send/write operation remain pending until writable.
- Preserve current timeout, failure, and queued-write semantics.
- Ensure cancellation cannot race with socket recycling or descriptor reuse.
Design constraints
- Rings may use
IORING_SETUP_SINGLE_ISSUER | IORING_SETUP_DEFER_TASKRUN; all
submit/enter operations must run on the owning worker pthread.
- CQE handlers run from the scheduler/module path and must not block waiting for
a bthread on another worker.
- Multishot operations must treat
IORING_CQE_F_MORE as a lifetime signal and
explicitly handle terminal CQEs.
- Accepted socket,
SocketId, Acceptor, and ring-request lifetimes must remain
valid across cancellation and shutdown.
- TLS, RDMA, health checks, and other non-ring transports need either an
io_uring implementation or a documented EventDispatcher fallback.
- The legacy path must remain unchanged when
--use_io_uring=false.
Suggested staging
- Move listening sockets to multishot accept while retaining
EventDispatcher for connect and writable waits.
- Move asynchronous connect to
IORING_OP_CONNECT.
- Move established-socket writable waits to io_uring.
- Avoid initializing global
EventDispatcher in io_uring mode when no fallback
consumer exists.
Acceptance criteria
- Normal io_uring TCP server traffic does not register listening or established
connection fds with epoll.
- Accept, connect, read, write-backpressure, cancellation, and server shutdown
have stress coverage.
- No lost wakeups, stale-fd CQEs, double callbacks, or shutdown hangs under
connection churn.
- Non-io_uring behavior remains compatible.
- Benchmarks report connection rate, request throughput/latency, CPU usage,
context switches, and relevant syscall counts before and after the change.
Background
When
--use_io_uring=true, established RPC sockets already bypassEventDispatcherfor inbound reads and use their worker-localRingListener.However,
EventDispatcheris still initialized and uses epoll for:accept()untilEAGAIN;EPOLLOUTwaits when an established socket cannot continue writing;connect();This leaves an extra dispatcher execution path and prevents io_uring mode from
owning the complete normal TCP lifecycle.
Goal
When io_uring mode is enabled, move normal TCP accept, connect, and
writable-wait handling from
EventDispatcherto worker-local io_uring rings.Preserve existing
EventDispatcherbehavior when io_uring is disabled and keepan explicit fallback for unsupported transports.
Proposed direction
Accept
io_uring_prep_multishot_accept()for listening sockets.worker that receives the CQE, avoiding a central accept-worker bottleneck and
cross-worker ring submission.
SOCK_NONBLOCK | SOCK_CLOEXEC.IORING_CQE_F_MORE.terminal CQEs before closing the listening descriptor or destroying the
Acceptor.EMFILE,ENFILE,ENOBUFS, andENOMEMwith bounded/backoffrearming rather than a tight loop.
Connect
connect() + EPOLLOUTwithIORING_OP_CONNECT.timeouts.
Writable waits
EPOLLOUTregistration with an io_uringPOLLOUTrequest,or let the io_uring send/write operation remain pending until writable.
Design constraints
IORING_SETUP_SINGLE_ISSUER | IORING_SETUP_DEFER_TASKRUN; allsubmit/enter operations must run on the owning worker pthread.
a bthread on another worker.
IORING_CQE_F_MOREas a lifetime signal andexplicitly handle terminal CQEs.
SocketId,Acceptor, and ring-request lifetimes must remainvalid across cancellation and shutdown.
io_uring implementation or a documented
EventDispatcherfallback.--use_io_uring=false.Suggested staging
EventDispatcherfor connect and writable waits.IORING_OP_CONNECT.EventDispatcherin io_uring mode when no fallbackconsumer exists.
Acceptance criteria
connection fds with epoll.
have stress coverage.
connection churn.
context switches, and relevant syscall counts before and after the change.