How the Linux Network Transmit Path Works: sk_buffs, qdiscs, and Why send() Returns Before the Packet Leaves

A process that calls send() on a TCP socket expects the bytes to leave the machine eventually. It does not expect the call to wait until the NIC has finished DMA. On a quiet path the call can return while the packet is still sitting in a qdisc, or even while the driver has not yet programmed the device. The kernel has accepted the data into its own buffers; the wire has not seen it.

This article explains that contract: how userspace data becomes an sk_buff, how the protocol stack adds headers, why a large TCP write becomes a GSO super-packet, how the qdisc decides when the packet may leave, and why the hard interrupt that completes transmission is not the same event as the send() return. It is not a second tour of the receive path, of TCP congestion control, or of DMA mapping. Those live in other posts. The single search intent is how the Linux transmit path turns a send() into a packet that the driver can hand to hardware.

The contract

send() copies or references application data into kernel memory and returns when the kernel has taken ownership of those bytes. Ownership means the socket buffer is linked into the protocol's send queue or has been handed to the device layer. It does not mean the NIC has transmitted the frame. TCP reliability is a separate state machine.

The unit that moves through the transmit path is the sk_buff. It carries the payload, headers, checksum offload flags, and GSO segmentation metadata. A single large GSO sk_buff can represent tens of kilobytes that will become many wire frames.

From userspace to the first sk_buff

For a normal TCP send(), tcp_sendmsg walks the user iovec and allocates or appends to sk_buffs. The copy happens into linear data or page fragments. Once in the sk_buff, the userspace buffer can be reused. The sk_buff is linked onto the socket's write queue. If the write queue is full relative to the congestion window or sndbuf, the call blocks or returns EAGAIN.

Protocol headers and GSO

tcp_write_xmit builds the TCP header. If the payload is larger than one MSS and TSO/GSO is available, the stack records gso_size and gso_type instead of cutting the sk_buff. The large sk_buff continues. The IP layer adds its header, runs netfilter hooks, and neighbor resolution fills the Ethernet header. Segmentation happens near the driver or in the NIC.

The qdisc decides when the packet may leave

dev_queue_xmit enters the device layer. The root qdisc (pfifo_fast or mq + per-queue) enqueues or drops the sk_buff. Shaping qdiscs may delay; work-conserving ones dequeue if the driver is free. The driver's ndo_start_xmit programs the hardware. Completion is asynchronous and frees the sk_buff later from the TX interrupt or NET_TX_SOFTIRQ.

Misconceptions

• send() does not wait for the NIC to transmit.
• Not every sk_buff is transmitted immediately; qdiscs can shape or drop.
• GSO is metadata for deferred segmentation; TSO lets hardware do it.

Takeaways

The transmit path accepts data into sk_buffs, adds headers, applies qdisc policy, and hands off to the driver. send() returns on kernel ownership, not on wire transmission.

Related: How the Linux Network Receive Path Works, How TCP Works, How DMA and the IOMMU Work.