Windowless Cumulative ACK Extension for RDMA Retransmission

Existing reliable transport retransmission schemes have two major categories with inherent defects: TCP SACK and Go-Back-N: Performance heavily relies on sliding window size and RTO timeout configuration. Small window generates excessive reverse-path ACK overhead; oversized window triggers massive redundant retransmissions once packet loss occurs. Standard RDMA AETH SACK: Only supports a single continuous receive segment, cannot compact multiple discontinuous received PSN blocks into one ACK frame. For high-throughput RDMA workloads including distributed AI training, distributed storage, and cross-DC data replication, minor mismatch between tuning parameters and actual network conditions significantly degrades link utilization. This draft introduces two core innovations: Dual-trigger ACK reporting: The receiver sends extended SACK once either cumulative receive time or cumulative received packet count reaches preconfigured threshold; per-packet ACK is eliminated. Extended T/C AETH fragment inserted into RDMA ETH header: Carries multiple discontinuous receive segments, enabling precise selective retransmission without receiver-side sliding window maintenance. The proposed mechanism fully removes sliding window constraints. The sender transmits data at a fixed preset line rate continuously, while the receiver batches loss feedback via extended ACK to minimize control-plane overhead on reverse path. This specification extends the RDMA ETH header defined in [RFC5040] by introducing a new Target Fragment (T/C AETH). It defines end-to-end operation rules for RC QP pairs, applicable to RoCEv1, RoCEv2 and InfiniBand RC transport. This draft does not modify RDMA data payload format or existing congestion control algorithms. Currently, the application of RDMA for data transmission in wide area networks encounters the following issues: Go-Back-N Limitation: If a single PSN is lost, the sender retransmits all outstanding packets after the last acknowledged PSN, resulting in severe bandwidth waste under non-zero packet loss rate. Sliding Window Overhead: Credit-based window requires frequent ACK feedback to advance transmission window. Small window causes ACK flooding on reverse path; large window leads to bulk redundant retransmission upon loss. RTO Sensitivity: Underestimated RTO induces spurious retransmissions; overestimated RTO stalls throughput for long-latency cross-DC links. Single Segment AETH Restriction: Legacy RDMA AETH only reports one continuous receive range, cannot aggregate multiple discontinuous received blocks into a single ACK frame. This draft resolves all above limitations via windowless transmission combined with multi-segment batch SACK reporting.

The complete end-to-end workflow contains two symmetric operation planes: Receiver Plane (Section 5): Accumulate received PSNs, monitor dual time/count thresholds, construct T/C AETH extended ACK and transmit to sender when any threshold is hit. Sender Plane (Section 6): Transmit packets at fixed preset rate without sliding window limitation; parse multi-segment blocks in T/C AETH to extract missing PSN ranges, perform selective retransmission only for identified lost segments. Core technical advantages: Windowless sender transmission: No per-flow credit window tracking required on sender side. Batch ACK reporting: Dual-threshold batching drastically reduces reverse-path ACK traffic volume. Multi-segment compact SACK: Single ACK frame carries multiple discontinuous received PSN ranges to enable precise loss recovery. High-Level End-to-End Flow: Sender continuously transmits RDMA data packets at negotiated fixed preset rate. Receiver accumulates received data packets, tracks cumulative receive time and cumulative packet count. If cumulative time >= T_thresh OR cumulative count >= C_thresh: a. Receiver constructs RDMA ACK frame with embedded T/C AETH target fragment. b. Transmit extended ACK (First Message) back to sender QP. Sender parses T/C AETH segments, calculates PSN gaps between all confirmed receive blocks. Sender selectively retransmits only PSN ranges identified as lost. Fallback timeout retransmission activates if sender receives no extended ACK after fixed timeout window expires.

For threshold configuration, during RC QP setup handshake: Sender advertises its preset packet send rate to receiver via CM REQ/REP. Receiver computes and configures T_thresh (cumulative time threshold) and C_thresh (cumulative count threshold) proportional to sender transmission rate. Receiver maintains per-QP batch state variables: BatchStartTime: Timestamp of the first packet received in current batch cycle. - BatchPacketCount: Total valid unique PSNs received in current batch. SegmentList: Ordered list of continuous received PSN blocks, sorted by ascending PSN value. For each incoming valid data packet: Record PSN if packet is valid and not duplicate. Merge new PSN into existing continuous segment in SegmentList, or create a new standalone segment block. Increment BatchPacketCount counter by one. Calculate current cumulative receive time = CurrentTime - BatchStartTime. After batch receive state tracking, it transmits T/C AETH extended ACK when either condition evaluates to true: Condition A: Cumulative receive time >= T_thresh Condition B: Cumulative receive count >= C_thresh After sending extended ACK frame: Reset BatchStartTime to current system timestamp. Clear BatchPacketCount to zero. Flush SegmentList to empty for new batch collection cycle. To complete the extended ACK transmission, the sender needs to construct an extended ACK frame. Base frame: Standard RDMA ACK ETH header sequence defined in RFC5040. Insert new T/C AETH Target Fragment between standard AETH and Atomic AETH fields inside ETH header. Populate all T/C AETH fields with data extracted from SegmentList, detailed format specified in Section 7. Encapsulate complete ACK frame and transmit to corresponding sender QP.

In this draft, we design a windowless continuous transmission. After QP establishment handshake completes: Sender transmits data packets at pre-negotiated fixed preset rate continuously. No sliding window credit check is executed before sending new outbound packets. Cache all transmitted PSNs within send buffer for potential selective retransmission processing. Upon receiving T/C AETH extended ACK frame: Parse SegmentNum field to read total number of valid receive confirmation segments stored inside Target Fragment. Extract Last Finished PSN, representing end PSN of first continuous fully received block. Extract each subsequent SegX Left / SegX Right pair representing discontinuous received PSN blocks. Compute all PSN gaps between adjacent confirmed segments; these gaps are defined as lost packet ranges. Trigger selective retransmission for all PSNs inside identified gap ranges only. The designed multi-segment loss calculation logic is as follows: Case 1: SegmentNum = 1 (single continuous received block) All PSNs smaller than Last Finished PSN are fully received without loss. Lost packets = all PSNs greater than Last Finished PSN up to latest transmitted PSN. Case 2: SegmentNum > 1 (multiple discontinuous receive segments) Lost range 1: (Last Finished PSN, Seg1 Left PSN) Lost range N: (SegN Right PSN, SegN+1 Left PSN) All PSNs inside above intervals shall be retransmitted by sender.

The T/C AETH fragment occupies fixed 30 Bytes inside RDMA ETH header, inserted between standard AETH and Atomic AETH subheaders. All fields use big-endian byte order, aligned to 24-bit unless otherwise specified.

0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | SegmentNum(24b)| Last Finished PSN (24bit) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seg1 Left(24b)| Seg1 Right(24b) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seg2 Left(24b)| Seg2 Right(24b) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ | Seg3 Left(24b)| Seg3 Right(24b) Reserved (24 bits) | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ SegmentNum (24 bits): Count of valid receive confirmation segments. Valid range: 1 ≤ SegmentNum ≤ 4. Values exceeding 4 are treated as reserved. Last Finished PSN (24 bits): Maximum PSN of the first continuous receive block (continuous segment without prior packet loss). SegX Left (24 bits): Minimum PSN of discontinuous received segment X. SegX Right (24 bits): Maximum PSN of discontinuous received segment X. Reserved (24 bits): Future extension field, set to all zero bits on transmission, ignored entirely on receiver processing.

T/C AETH fragment encodes receive state in following logical sequence: [Continuous Segment: 0 ~ Last Finished PSN] | Lost Gap 1 | [Seg1 Left ~ Seg1 Right] | Lost Gap2 | [Seg2 Left ~ Seg2 Right] ...

Sender strictly retransmits only PSNs within gap ranges calculated from T/C AETH segment fields. No extra packets outside lost intervals are retransmitted, eliminating redundant retransmission overhead existing in Go-Back-N and oversized sliding window implementations. For example, sender transmits PSN range 100 ~ 599 at fixed preset rate. Receiver receives continuous block 100~199, loses 200~299, receives 300~399, loses 400~499, receives 500~599. Corresponding T/C AETH field values: SegmentNum = 3 Seg1 Left=300, Seg1 Right=399 Seg2 Left=500, Seg2 Right=599 Sender identifies lost ranges: 200~299, 400~499; retransmits only these two PSN intervals. To handle potential loss of extended ACK frames on reverse data path: Sender maintains per-QP ACK timeout timer configured during QP setup. If sender transmits PSN range [X, Y] and receives no T/C AETH frame after timeout expires: a. Retransmit the latest transmitted sequential PSN block (highest PSNs sent). b. Restart ACK timeout timer immediately after retransmission. Fallback timeout mechanism serves only as safety backup; primary loss recovery relies on extended T/C AETH SACK reports.

This document has no requests to IANA.