>>> ids =FileIndex([-1,-1,3])>>> ids[1]-1>>> ids.update([4,5,6])>>> ids[1]5>>>list(ids)[4, 5, 6]>>>for id in ids:... id # do something e.g.`if id != -1: os.close(id)`...456
Note
Makes copy of the list provided & shares it with io_uring, so modifying python list will not effect FileIndex already submitting.
For io_uring_register_files / _files_update: each slot is a file descriptor to register, -1 an empty (sparse) slot a later update or a *_direct request (IORING_FILE_INDEX_ALLOC) may fill. A request uses a registered file by its slot index as fd with IOSQE_FIXED_FILE in sqe.flags.
ring is the memory itself, not its address: the ring keeps it locked from the register until io_uring_unregister_buf_ring or its exit (it can not be resized, closed or freed meanwhile).
The buffer ring's memory: a writable, page-aligned buffer (mmap.mmap) of at least ring_entries * 16 bytes, or None (with IOU_PBUF_RING_MMAP the kernel allocates it).
>>> reg =SyncCancelReg()>>> reg.addr=123# the user_data of the request (a pending read here)>>> reg.timeout=1.0# seconds to wait (default: until done)>>>io_uring_register_sync_cancel(ring, reg)0
Setup used to run this example
defpending_read(ns):'''A ring with a read (user_data 123) waiting on an empty pipe.''' ring =make_ring(ns) r, _w = os.pipe() sqe =io_uring_get_sqe(ring)io_uring_prep_read(sqe, r,bytearray(1)) sqe.user_data=123io_uring_submit(ring)
Not tested with a real device (needs a NIC whose driver supports zero-copy receive, like mlx5, bnxt, gve): only the fields and the error without a device are.
area and region are the memory itself, not addresses: the ring keeps them locked from the register until its exit (the device writes into area). region = None: the kernel allocates the refill queue.
Not tested with a real device (needs a NIC whose driver supports zero-copy receive, like mlx5, bnxt, gve): only the fields and the error without a device are.
The receive area: a writable, page-aligned buffer (mmap.mmap) the device writes into. Setting it sets area_len to its length (it may be lowered); None for none (a dma-buf: area_dmabuf_fd).
The refill queue's memory: a writable, page-aligned buffer (mmap.mmap). Setting it sets region_size to its length (it may be lowered) and IORING_MEM_REGION_TYPE_USER in region_flags; None clears both (the kernel allocates it).
>>> bpf =Bpf()>>> bpf.opcode=io_uring_op.IORING_OP_OPENAT>>> program = struct.pack("HBBI",0x06,0,0,0) # `BPF_RET | BPF_K` 0: deny every `openat`>>> bpf.filter= program # `struct sock_filter` instructions (8 bytes each), hold on to it>>>io_uring_register_bpf_filter(ring, bpf)0
Registers memory the ring uses itself (io_uring_register_region): flags for the registration and the region's description (region_*).
Example
>>> mem = mmap.mmap(-1,4096)>>> reg =MemRegionReg()>>> reg.region= mem # also `region_size = len(mem)` and `IORING_MEM_REGION_TYPE_USER` in `region_flags`>>> reg.flags=IORING_MEM_REGION_REG_WAIT_ARG>>>io_uring_register_region(ring, reg) # `ring` set up with `IORING_SETUP_R_DISABLED`0
The region's memory: a writable, page-aligned buffer (mmap.mmap). Setting it sets region_size to its length (it may be lowered) and IORING_MEM_REGION_TYPE_USER in region_flags; None clears both (the kernel allocates the region).
The raw io_uring_register(2) system call on fd: opcode is an IORING_REGISTER_*, nr_args the number of arguments at address arg (default None: none). Returns the call's result; UnsupportedError if the kernel does not know opcode.
Linux 5.13+ (IORING_REGISTER_BUFFERS2). Returns 0. Use the buffers with the *_fixed requests (io_uring_prep_read_fixed, ...) by their index in bufs; a request may cover any part of its buffer.
bufs: writable buffers; the kernel pins them until io_uring_unregister_buffers (or the ring's exit), so the ring holds them (locked: no resize) until then.
bufs must be anonymous memory (bytearray, mmap.mmap(-1, ...); file-backed memory fails with EOPNOTSUPP or EFAULT), each at most 1 GiB (EFAULT), at most 1024 of them (EINVAL). A buffer on a huge page pins the whole page. Before Linux 5.12 they count against RLIMIT_MEMLOCK (ENOMEM).
OSError: EBUSY when buffers are already registered: unregister them first (registering again is how to change the set). Before Linux 5.13 the registration waits for the ring to go idle.
tags needs one item for each buffer in bufs. A tag of 0 means none; for any other, once the buffer is unregistered (or replaced) and no request uses it any more, a CQE is posted with user_data set to the tag and every other field 0.
Update the registered buffers from slot offset (default 0) with bufs and tags.
Note
Function arguments position has changed for C origin for better usability.
Linux 5.13+. Returns the number of buffers updated. A slot may be empty (from io_uring_register_buffers_sparse) or already in use: a replaced buffer stays valid until the requests using it complete, then its tag's CQE (see io_uring_register_buffers) is posted.
Returns 0; OSError: ENXIO when none are registered. Not needed before io_uring_queue_exit, which unregisters them too (the pages may be unpinned a little later then).
'Sqe','put_sqe','io_uring_prep_open','io_uring_prep_open_direct','io_uring_prep_read','io_uring_prep_write','io_uring_prep_statx','Statx','io_uring_register_files','io_uring_register_files_tags',) # fmt: skipdeffile_ring(ns):'''A ring (2 entries for `put_sqe`, to show a full one), an empty `cqe` and `fd` of `./file.txt` (`hi... bye!`).'''make_ring(ns,entries=2if ns['_name'] =='put_sqe'else1024) ns['cqe'] =Cqe()make_file(ns)
Note
Returns 0. Use a registered file with IOSQE_FIXED_FILE in the sqe's flags and its index in files as the sqe's fd. A file table is also what direct descriptors (*_direct requests) live in.
The kernel copies the descriptors at registration: ids can be freed, and the fds closed, afterwards (the ring holds its own reference to each file).
OSError: EBUSY when files are already registered (unregister first, or use io_uring_register_files_update), EMFILE over the fixed file table maximum or, for an unprivileged user, over RLIMIT_NOFILE (a per user limit), EBADF for a descriptor that is not open. Before Linux 5.13 the registration waits for the ring to go idle.
"Registered files have less overhead per operation than normal files. This is due to the kernel grabbing a reference count on a file when an operation begins, and dropping it when it's done. When the process file table is shared, for example if the process has ever created any threads, then this cost goes up even more. Using registered files reduces the overhead of file reference management across requests that operate on a file."
'Sqe','put_sqe','io_uring_prep_open','io_uring_prep_open_direct','io_uring_prep_read','io_uring_prep_write','io_uring_prep_statx','Statx','io_uring_register_files','io_uring_register_files_tags',) # fmt: skipdeffile_ring(ns):'''A ring (2 entries for `put_sqe`, to show a full one), an empty `cqe` and `fd` of `./file.txt` (`hi... bye!`).'''make_ring(ns,entries=2if ns['_name'] =='put_sqe'else1024) ns['cqe'] =Cqe()make_file(ns)
tags needs one item for each file in files. A tag of 0 means none; for any other, once the file is unregistered (or replaced) and no request uses it any more, a CQE is posted with user_data set to the tag and every other field 0.
Update the registered files from slot offset (default 0) with files and tags.
Note
Function arguments position has changed for C origin for better usability.
Linux 5.13+. Returns the number of files updated. In files, -1 empties a slot and IORING_REGISTER_FILES_SKIP leaves one as it is; a replaced file stays open for the requests using it, then its tag's CQE (see io_uring_register_files_tags) is posted.
Update the registered files from slot offset (default 0) with fds.
Note
Function arguments position has changed for C origin for better usability.
Linux 5.5+. Returns the number of files updated. In fds, -1 empties a slot (closing a direct descriptor) and IORING_REGISTER_FILES_SKIP (Linux 5.12+) leaves one as it is. Unlike a registration, an update never waits for the ring to go idle (since Linux 5.13).
Linux 5.6+. Returns a positive id. An sqe with it in personality runs with these credentials: a ring shared between users or processes keeps them separate this way.
Register the restrictions added to res (ValueError if it is empty), for a ring set up with IORING_SETUP_R_DISABLED.
Note
Linux 5.10+. Returns 0. The restrictions are an allowlist (an opcode or flag not added is denied, OSError: EACCES for the request or register call), registered all in one call before io_uring_enable_rings.
OSError: EBADFD when the ring is not disabled, EBUSY when restrictions are already registered, EINVAL for an unknown restriction opcode or too many restrictions.
Linux 5.18+. Worth it in a threaded process (its file table is shared, so the lookup costs more).
The registration is per thread: a ring shared between threads (one submits, another reaps) can not use it, as each thread would have its own index. At most 16 ring fds per thread.
Linux 5.19+. Returns 0. reg.ring_entries must be a power of 2, at most 32768; reg.ring at least ring_entries * 16 bytes, page aligned. reg.flags: IOU_PBUF_RING_INC (Linux 6.12+) lets a buffer be consumed over several completions (IORING_CQE_F_BUF_MORE while more are to come), reg.min_left the least left for it to be handed out again (1 byte if 0).
A request with IOSQE_BUFFER_SELECT and buf_group = reg.bgid takes a buffer when its data arrives; its CQE has IORING_CQE_F_BUFFER set and the buffer id in flags >> IORING_CQE_BUFFER_SHIFT.
Linux 6.8+ (IORING_REGISTER_PBUF_STATUS). FileNotFoundError (ENOENT) when buf_group has no buffers, OSError: EINVAL when it has legacy provided buffers (io_uring_prep_provide_buffers) and not a ring.
Cancel requests as described by reg and wait (up to reg.timeout) for it.
Note
Linux 6.0+. Like io_uring_prep_cancel, but the result comes back here instead of in a CQE: 0 when the request was found, or with a flag matching several (IORING_ASYNC_CANCEL_ALL, ...) how many. The cancelled requests still post their own CQEs (their res raises OSError: ECANCELED).
OSError: ENOENT (FileNotFoundError) when nothing matches (completed already, or a wrong addr), EALREADY when the request is too far along to cancel (it completes shortly on its own), ETIME when reg.timeout ran out first.
reg.addr is the user_data you gave the request (the kernel's is its in-flight entry, looked up here).
Send the IORING_OP_MSG_RING request prepared in sqe to another ring at once, needs no ring of its own.
Note
Linux 6.13+. Returns 0 (the result comes back here, not in a CQE); the target ring, which may be any the caller can reach (even its own), gets the CQE as for a submitted io_uring_prep_msg_ring.
Limit the direct descriptor slots the kernel allocates (IORING_FILE_INDEX_ALLOC) to len slots from off.
Note
Linux 6.0+. Returns 0. Needs a registered file table (io_uring_register_files, io_uring_register_files_sparse) first; by default the kernel allocates from the whole table. The slots outside the range stay for explicit indexes.
Register the busy polling settings napi; the old settings are written back into it.
Note
Linux 6.9+. Returns 0; OSError on a kernel without CONFIG_NET_RX_BUSY_POLL. Busy polling the network device (as the SO_PREFER_BUSY_POLL socket option) lowers the round trip time.
Control a zero-copy receive instance as described by ctrl.
Note
Returns 0. ZCRX_CTRL_FLUSH_RQ flushes the buffers pending in the refill queue; ZCRX_CTRL_EXPORT sets ctrl.zcrx_fd, a file descriptor another ring can take the instance over with.
Not tested with a real device. Refer to help(ZcrxCtrl).
Returns 0. The filter runs for each request of bpf.opcode submitted through this ring: a non-zero result allows it, 0 denies it (the request fails with EACCES). Several filters for one opcode stack: all must allow. IO_URING_BPF_FILTER_DENY_REST in bpf.flags denies every opcode without a filter (an allowlist).
OSError: EACCES unless the caller has CAP_SYS_ADMIN or no_new_privs (prctlPR_SET_NO_NEW_PRIVS, as for seccomp); EMSGSIZE when bpf.pdu_size is over the kernel's size for the opcode, or differs from it with IO_URING_BPF_FILTER_SZ_STRICT (the kernel writes its size into bpf.pdu_size then: read it and retry); EINVAL for a bad filter, opcode or flags.
Applies to every ring the task creates afterwards (not to existing ones) and is inherited by fork children, who may add filters but not loosen these; it can not be removed.
A request with IOSQE_BUFFER_SELECT and buf_group = bgid takes a buffer when its data arrives (its res raises OSError: ENOBUFS when none is left); its CQE has IORING_CQE_F_BUFFER set and the id in flags >> IORING_CQE_BUFFER_SHIFT. With IOU_PBUF_RING_INC, IORING_CQE_F_BUF_MORE says the buffer is not used up yet.
OSError if the kernel refuses (C liburing's err out-parameter is the error number).
Limits the cpus the ring's async workers (io-wq) may run on to those set in mask (bytes / bytearray in the cpu_set_t layout, bit n of byte n // 8 for cpu n).
Linux 5.14+. Returns 0; OSError: EINVAL for an empty mask. By default the workers inherit the cpu mask of the task that made the ring. A worker already running moves at its next schedule point.
Sets the most io-wq workers the ring may run: bounded (regular files) and unbounded (sockets, ...); 0 leaves one as is. Gives the previous (bounded, unbounded).
Linux 5.15+. Defaults: bounded the ring's SQ entries or 4 times the online cpus, whichever is less; unbounded the task limit (RLIMIT_NPROC). The limits are per NUMA node, and maxima: idle workers are reaped. OSError: EINVAL for a value over the kernel's maximum.
bid comes back in the CQE (flags >> IORING_CQE_BUFFER_SHIFT), and may name a different buffer each time it is added: only you track which. A request writes at most INT_MAX bytes into a buffer.
addr must be writable (the kernel fills it); br holds it (locked: no resize) until bid is added again or the buffer ring is freed (the ring holds br until then).
The number of buffers in br the kernel has not used yet.
Note
Linux 6.8+ (asks the kernel for its head, io_uring_buf_ring_head). The count is of that moment: with requests in flight that may take buffers, it is already stale.
Returns a negative error number, not an exception: -ENOENT when bgid has no buffers, -EINVAL when it is not a buffer ring or the kernel lacks the call.