Introduced in 2019 (kernel 5.1) by Jens Axboe, io_uring (henceforth uring) is a system for providing the kernel with a schedule of system calls, and receiving the results as they’re generated. Whereas epoll and kqueue support multiplexing, where you’re told when you can usefully perform a system call using some set of filters, uring allows you to specify the system calls themselves (and dependencies between them), and execute the schedule at the kernel “dataflow limit”. It combines asynchronous I/O, system call polybatching, and flexible buffer management, and is IMHO the most substantial development in the Linux I/O model since Berkeley sockets (yes, I’m aware Berkeley sockets preceded Linux. Let’s then say that it’s the most substantial development in the UNIX I/O model to originate in Linux):

• Asynchronous I/O without the large copy overheads and restrictions of POSIX AIO
• System call batching/linking across distinct system calls
• Provide a buffer pool, and they’ll be used as needed
• Both polling- and interrupt-driven I/O on the kernel side

The core system calls of uring are wrapped by the C API of liburing. Windows added a very similar interface, IoRing, in 2020. In my opinion, uring ought largely displace epoll in new Linux code. FreeBSD seems to be sticking with kqueue, meaning code using uring won’t run there, but neither did epoll (save through FreeBSD’s somewhat dubious Linux compatibility layer). Both the system calls and liburing have fairly comprehensive man page coverage, including the io_uring.7 top-level page.

Rings

Central to every uring are two ringbuffers holding CQEs (Completion Queue Entries) and SQE (Submission Queue Entries) descriptors (as best I can tell, this terminology was borrowed from the NVMe specification). Note that SQEs are allocated externally to the SQ descriptor ring. SQEs roughly correspond to a single system call: they are tagged with an operation type, and filled in with the values that would traditionally be supplied as arguments to the appropriate function. Userspace is provided references to SQEs on the SQE ring, which it fills in and submits. Submission operates up through a specified SQE, and thus all SQEs before it in the ring must also be ready to go (this is likely the main reason why the SQ holds descriptors to an external ring of SQEs: you can acquire SQEs, and then submit them out of order). The kernel places results in the CQE ring. These rings are shared between kernel- and userspace. The rings must be distinct unless the kernel specifies the IORING_FEAT_SINGLE_MMAP feature (see below).

uring does not generally make use of errno. Synchronous functions return the negative error code as their result. Completion queue entries have the negated error code placed in their res fields.

CQEs are usually 16 bytes, and SQEs are usually 64 bytes (but see IORING_SETUP_SQE128 and IORING_SETUP_CQE32 below). Either way, SQEs are allocated externally to the submission queue, which is merely a ring of 32-bit descriptors.

System calls

The liburing interface will be sufficient for most users, and it is possible to operate almost wholly without system calls when the system is busy. For the sake of completion, here are the three system calls implementing the uring core (from the kernel’s io_uring/io_uring.c):

int io_uring_setup(u32 entries, struct io_uring_params *p);
int io_uring_enter(unsigned fd, u32 to_submit, u32 min_complete, u32 flags, const void* argp, size_t argsz);
int io_uring_register(unsigned fd, unsigned opcode, void *arg, unsigned int nr_args);

Note that io_uring_enter(2) corresponds more closely to the io_uring_enter2(3) wrapper, and indeed io_uring_enter(3) is defined in terms of the latter (from liburing’s src/syscall.c):

static inline int __sys_io_uring_enter2(unsigned int fd, unsigned int to_submit,
          unsigned int min_complete, unsigned int flags, sigset_t *sig, size_t sz){                                                                                                                                   
  return (int) __do_syscall6(__NR_io_uring_enter, fd, to_submit, min_complete, flags, sig, sz);                                                                                           
}                                                                                                                                   
                                                                                                                                    
static inline int __sys_io_uring_enter(unsigned int fd, unsigned int to_submit,                                                     
               unsigned int min_complete, unsigned int flags, sigset_t *sig){                                                                                                                                   
  return __sys_io_uring_enter2(fd, to_submit, min_complete, flags, sig, _NSIG / 8);                                                                                                            
}

io_uring_enter(2) can both submit SQEs and wait until some number of CQEs are available. Its flags parameter is a bitmask over:

struct io_uring_getevents_arg {
        __u64   sigmask;
        __u32   sigmask_sz;
        __u32   pad;
        __u64   ts;
};

Setup

The io_uring_setup(2) system call returns a file descriptor, and accepts two parameters, u32 entries and struct io_uring_params *p:

int io_uring_setup(u32 entries, struct io_uring_params *p);
struct io_uring_params {                                                                                                            
  __u32 sq_entries;                 // number of SQEs, filled by kernel
  __u32 cq_entries;                 // see IORING_SETUP_CQSIZE and IORING_SETUP_CLAMP
  __u32 flags;                      // see "Flags" below
  __u32 sq_thread_cpu;              // see IORING_SETUP_SQ_AFF
  __u32 sq_thread_idle;             // see IORING_SETUP_SQPOLL
  __u32 features;                   // see "Kernel features" below, filled by kernel
  __u32 wq_fd;                      // see IORING_SETUP_ATTACH_WQ
  __u32 resv[3];                    // must be zero
  struct io_sqring_offsets sq_off;  // see "Ring structure" below, filled by kernel
  struct io_cqring_offsets cq_off;  // see "Ring structure" below, filled by kernel
};

resv must be zeroed out. In the absence of flags, the uring uses interrupt-driven I/O. Calling close(2) on the returned descriptor frees all resources associated with the uring.

io_uring_setup(2) is wrapped by liburing’s io_uring_queue_init(3) and io_uring_queue_init_params(3). When using these wrappers, io_uring_queue_exit(3) should be used to clean up. These wrappers operate on a struct io_uring. io_uring_queue_init(3) takes an unsigned flags argument, which is passed as the flags field of io_uring_params. io_uring_queue_init_params(3) takes a struct io_uring_params* argument, which is passed through directly to io_uring_setup(2). It’s best to avoid mixing the low-level API and that provided by liburing.

Ring structure

The details of ring structure are only relevant when using the low-level API, and they are not exposed via liburing. They’re primarily used to prepare the three (or two, see IORING_FEAT_SINGLE_MMAP) backing memory maps. You’ll need set these up yourself if you want to use huge pages.

struct io_sqring_offsets {
    __u32 head;
    __u32 tail;
    __u32 ring_mask;
    __u32 ring_entries;
    __u32 flags;
    __u32 dropped;
    __u32 array;
    __u32 resv[3];
};

struct io_cqring_offsets {
    __u32 head;
    __u32 tail;
    __u32 ring_mask;
    __u32 ring_entries;
    __u32 overflow;
    __u32 cqes;
    __u32 flags;
    __u32 resv[3];
};

As explained in the io_uring_setup(2) man page, the submission queue can be mapped thusly:

mmap(0, sq_off.array + sq_entries * sizeof(__u32), PROT_READ|PROT_WRITE,
     MAP_SHARED|MAP_POPULATE, ring_fd, IORING_OFF_SQ_RING);

The submission queue contains the internal data structure followed by an array of SQE descriptors. These descriptors are 32 bits each no matter the architecture, implying that they are indices into the SQE map, not pointers. The SQEs are allocated:

mmap(0, sq_entries * sizeof(struct io_uring_sqe), PROT_READ|PROT_WRITE,
     MAP_SHARED|MAP_POPULATE, ring_fd, IORING_OFF_SQES);

and finally the completion queue:

mmap(0, cq_off.cqes + cq_entries * sizeof(struct io_uring_cqe), PROT_READ|PROT_WRITE,
     MAP_SHARED|MAP_POPULATE, ring_fd, IORING_OFF_CQ_RING);

Recall that when the kernel expresses IORING_FEAT_SINGLE_MMAP, the submission and completion queues can be allocated in one mmap(2) call.

Flags

The flags field is set up by the caller, and is a bitmask over:

When IORING_SETUP_R_DISABLED is used, the ring must be enabled before submissions can take place. If using the liburing API, this is done via io_uring_enable_rings(3):

int io_uring_enable_rings(struct io_uring *ring); // liburing 2.4

Kernel features

Various functionality was added to the kernel following the initial release of uring, and thus not necessarily available to all kernels supporting the basic system calls. The __u32 features field of the io_uring_params parameter to io_uring_setup(2) is filled in with feature flags by the kernel, a bitmask over:

Registered resources

Buffers

Since Linux 5.7, user-allocated memory can be provided to uring in groups of buffers (each with a group ID), in which each buffer has its own ID. This was done with the io_uring_prep_provide_buffers(3) call, operating on an SQE. Since 5.19, the “ringmapped buffers” technique (io_uring_register_buf_ring(3)) allows these buffers to be used much more effectively.

struct io_uring_rsrc_register {
    __u32 nr;
    __u32 resv;
    __u64 resv2;
    __aligned_u64 data;
    __aligned_u64 tags;
};

struct io_uring_rsrc_update2 {
    __u32 offset;
    __u32 resv;
    __aligned_u64 data;
    __aligned_u64 tags;
    __u32 nr;
    __u32 resv2;    
};

struct io_uring_buf_reg {
    __u64 ring_addr;
    __u32 ring_entries;
    __u16 bgid;
    __u16 pad;
    __u64 resv[3];
};

Registered files

Registered (sometimes “direct”) descriptors are integers corresponding to private file handle structures internal to the uring, and can be used anywhere uring wants a file descriptor through the IOSQE_FIXED_FILE flag. They have less overhead than true file descriptors, which use structures shared among threads. Note that registered files are required for submission queue polling unless the IORING_FEAT_SQPOLL_NONFIXED feature flag was returned.

struct io_uring_file_index_range {
    __u32 off;
    __u32 len;
    __u64 resv;
};

Personalities

Submitting work

Submitting work consists of four steps:

• Acquiring free SQEs
• Filling in those SQEs
• Placing those SQEs at the tail of the submission queue
• Submitting the work, possibly using a system call

The SQE structure

struct io_uring_sqe has several large unions which I won’t reproduce in full here; consult liburing.h if you want the details. The instructive elements include:

struct io_uring_sqe {                                                                                                               
  __u8  opcode;   /* type of operation for this sqe */                                                                              
  __u8  flags;    /* IOSQE_ flags */                                                                                                
  __u16 ioprio;   /* ioprio for the request */                                                                                      
  __s32 fd;   /* file descriptor to do IO on */   
  ... various unions for representing the request details ...
};

Flags can be set on a per-SQE basis using io_uring_sqe_set_flags(3), or writing to the flags field directly:

static inline
void io_uring_sqe_set_flags(struct io_uring_sqe *sqe, unsigned flags){                                                                                                                                   
  sqe->flags = (__u8) flags;
}

The flags are a bitfield over:

Prepping SQEs

Each SQE must be seeded with the object upon which it acts (usually a file descriptor) and any necessary arguments. You’ll usually also use the user data area.

User data

Each SQE provides 64 bits of user-controlled data which will be copied through to any generated CQEs. Since CQEs don’t include the relevant file descriptor, you’ll almost always be encoding some kind of lookup information into this area.

void io_uring_sqe_set_data(struct io_uring_sqe *sqe, void *user_data);
void io_uring_sqe_set_data64(struct io_uring_sqe *sqe, __u64 data);
void *io_uring_cqe_get_data(struct io_uring_cqe *cqe);
__u64 io_uring_cqe_get_data64(struct io_uring_cqe *cqe);

Here’s an example C++ data type that encodes eight bits as an operation type, eight bits as an index, and forty-eight bits as other data. I typically use something like this to reflect the operation which was used, the index into some relevant data structure, and other information about the operation (perhaps an offset or a length):

union URingCtx {
  struct rep {
    rep(uint8_t op, unsigned ix, uint64_t d):
      type(static_cast<URingCtx::rep::optype>(op)), idx(ix), data(d)                                                                                                                       
    {
      if(type >= MAXOP){
        throw std::invalid_argument("bad uringctx op");
      }
      if(ix > MAXIDX){
        throw std::invalid_argument("bad uringctx index");
      }
      if(d > 0xffffffffffffull){
        throw std::invalid_argument("bad uringctx data");
      }
    }
    enum optype: uint8_t {
      ...app-specific types...
      MAXOP      // shouldn't be used
    } type: 8;
    uint8_t idx: 8;
    uint64_t data: 48;
  } r;
  uint64_t val;
  static constexpr auto MAXIDX = 255u;
  URingCtx(uint8_t op, unsigned idx, uint64_t d):
    r(op, idx, d)
  {}
  URingCtx(uint64_t v):
    URingCtx(v & 0xffu, (v >> 8) & 0xffu, v >> 16)
  {}
};

The majority of I/O-related system calls have by now a uring equivalent (the one major exception of which I’m aware is directory listing; there seems to be no readdir(3)/getdents(2)). What follows is an incomplete list.

Opening and closing file descriptors

void io_uring_prep_openat(struct io_uring_sqe *sqe, int dfd, const char *path,
                          int flags, mode_t mode);
void io_uring_prep_openat_direct(struct io_uring_sqe *sqe, int dfd, const char *path,
                                 int flags, mode_t mode, unsigned file_index);
void io_uring_prep_openat2(struct io_uring_sqe *sqe, int dfd, const char *path,
                           int flags, struct open_how *how);
void io_uring_prep_openat2_direct(struct io_uring_sqe *sqe, int dfd, const char *path,
                                  int flags, struct open_how *how, unsigned file_index);
void io_uring_prep_accept(struct io_uring_sqe *sqe, int sockfd, struct sockaddr *addr,
                          socklen_t *addrlen, int flags);
void io_uring_prep_accept_direct(struct io_uring_sqe *sqe, int sockfd, struct sockaddr *addr,
                                 socklen_t *addrlen, int flags, unsigned int file_index);
void io_uring_prep_multishot_accept(struct io_uring_sqe *sqe, int sockfd, struct sockaddr *addr,
                                    socklen_t *addrlen, int flags);
void io_uring_prep_multishot_accept_direct(struct io_uring_sqe *sqe, int sockfd, struct sockaddr *addr,
                                           socklen_t *addrlen, int flags);
void io_uring_prep_close(struct io_uring_sqe *sqe, int fd);
void io_uring_prep_close_direct(struct io_uring_sqe *sqe, unsigned file_index);
void io_uring_prep_socket(struct io_uring_sqe *sqe, int domain, int type,                                                           
                          int protocol, unsigned int flags);                                                                        
void io_uring_prep_socket_direct(struct io_uring_sqe *sqe, int domain, int type,                                                    
                                 int protocol, unsigned int file_index, unsigned int flags);                                        
void io_uring_prep_socket_direct_alloc(st