Minimum RTT Estimation Under Low ACK Frequency

Abstract

In traditional acknowledgment mechanisms, the sender frequently "pulls" ACK packets, resulting in significant protocol control overhead. This leads to wasted CPU and I/O resources, contention for packet spectrum on half-duplex links (e.g., WLAN), and reverse-path congestion in asymmetric links (e.g., satellite network). Reducing the number of ACKs is essential in scenarios where ACK overhead is non-negligible. However, a lower ACK frequency can introduce biases in delay estimation, such as overestimating the minimum round-trip time (minRTT). This document proposes how to calibrate the estimation of the minRTT under low ACK frequency conditions.¶

1. Requirements Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119].¶

2. Overview of Standards on ACK Mechanism

[RFC9000] specifies a simple delayed ACK mechanism that a receiver can send an ACK for every other packet, and for every packet when reordering is observed, or when the max_ack_delay timer expires. However, this ACK mechanism may not match the number of ACKs to the transport's required intensity under different network conditions. For example, when the data throughput of a WLAN transport is extremely high, QUIC will generate a large number of ACKs. In this case, minimizing the ACK intensity of QUIC is not only a win for data throughput improvement but also a win for energy and CPU efficiency.¶

[RFC1122] and [RFC5681] were two core functionality standards that introduced delayed ACK, which was the default acknowledgment mechanism in most Linux distributions. [RFC4341] and [RFC5690] described an acknowledgment congestion control mechanism in which the minimum ACK frequency allowed is twice per send window. [RFC3449] discussed the imperfection and variability of TCP's acknowledgment mechanism because of asymmetric effects and recommended scaling ACK frequency as a mitigation to these effects. These RFCs reveal that the dependence on frequent ACKs is an artifact of current transport protocol designs rather than a fundamental requirement. Based on this insight, some work-in-progress IETF drafts have paid great attention to ACK scaling technologies in both TCP and QUIC working groups.¶

First of all, [ACK-PULL] proposed the TCP ACK pull mechanism, which allows a sender to request the ACK for a data segment to be sent without additional delay by the receiver. This helps in some cases when the delayed ACKs degrade transport performance.¶

Instead of pulling more ACKs, [QUIC-SCALING] recommended reducing the ACK frequency by sending an ACK for at least every 10 received packets and [QUIC-SATCOM] recommended an ACK frequency of four ACKs every round-trip time (RTT), aiming to reduce link transmission costs for asymmetric paths.¶

Different from using an empirical value of ACK frequency, instead, Li tried to improve the scalability by proposing a novel ACK mechanism named Tame ACK (TACK), whose frequency is a function of the bandwidth-delay product of network connections. The detailed TCP-based implementation (i.e., TCP-TACK) details and evaluation results in industrial applications have been shown in [Tong].¶

The recent work [IYENGAR-ACK] has been discussed in QUIC working group, it specifies an extension to QUIC that enables an endpoint to request its peer change its behavior when acknowledging at a low ACK frequency.¶

Based on the observations, low ACK frequency not only addresses an industry-wide need but also demonstrates broad applicability across diverse operational environments.¶

3. Problem Statement

In this document, the minRTT is the minimum RTT samples observed at the sender for a given network path during a period of time. The minRTT is a fundamental state variable in transmission control, and its accuracy is critical for achieving both high throughput and low latency. For example, some congestion controllers depend on minRTT to estimate the congestion window [Neal]. minRTT is also used by QUIC loss detection to reject implausibly small RTT samples [RFC9002]. minRTT is further used to update the ACK frequency in [Tong].¶

An RTT estimation system contains a sender and a receiver. Ideally, when sending an ACK for every data packet, the minimum RTT sample can be computed by monitoring the per-packet RTT samples. However, the sender can hardly generate per-packet RTT samples in the case of sending fewer ACKs, which is the root cause of the minimum RTT estimation biases. When multiple packets carrying departure timestamps are transported between endpoints via the same path, an RTT of this path can be sampled at the sender upon receiving an ACK frame. However, when sending fewer ACK frames, more data packets might be received during the ACK interval, generating only one RTT sample among multiple packets is likely to result in biases. For example, a larger minimum RTT estimate. In general, the higher the throughput, the larger the biases.¶

One alternative way to reduce biases can be that, each ACK frame carries multiple timestamps (as well as ACK delays in [RFC9002]) for the sender to generate more RTT samples. However, (1) the overhead is high, which is unacceptable, especially for high-bandwidth transport. Also, (2) the number of data packets might be far more than the maximum number of timestamps that an ACK frame is capable of carrying.¶

4. minRTT Estimation Under Low ACK Frequency

Since the receiver is capable of monitoring per-packet state, the one-way delay (OWD) of each packet can be easily computed according to the departure timestamps (carried in the TIMESTAMP frame) and the arrival timestamps of each packet. In this case, QUIC SHOULD adopt the OWD-based minRTT estimation.¶

The rationale is that the variation of OWD reflects the variation of RTT over near-symmetric links. The OWD-based minRTT estimation requires the sender to record the departure timestamp in each ack-eliciting packet. Meanwhile, at the receiver, the per-packet OWD samples SHOULD be computed upon packet arrivals and a function of computing the minimum OWD (minOWD) SHOULD be newly added. In this document, minOWD is the minimum OWD samples observed on the same network path during a period of time. The receiver then generates an ACK frame for the sender, in which the ACK delay and departure timestamp for the packet that achieves the minimum OWD is reported. The ACK delay is defined as the delay incurred between when the packet is received and when the ACK frame is sent. Based on the information reported by the incoming ACK frames and the ACK arrival timestamps, the sender can generate RTT samples and then compute minRTT accordingly.¶

5. Sender-Side Operation

Generally, the minRTT is calculated at the sender.¶

Before estimating the minRTT, the RTT samples should be computed based on the ACK frames collected during a period. Assume that a packet is sent by the sender at time t_1 and arrives at time t_3, and the ACK frame is sent at time t_4. The ACK delay can be computed at the receiver. For example, the receiver computes the ACK delay delta_t = t_4 - t_3, and syncs the ACK delay to the sender via an ACK frame. The ACK delay can also be computed at the sender. For example, the receiver directly syncs an ACK frame carrying t_4 and t_3 to the sender, the sender then computes the ACK delay delta_t = t_4 - t_3.¶

The sender therefore computes an RTT sample according to delta_t, t_1, and the arrival time (t_2) of the ACK frame, i.e., RTT_sample = t_2 − t_1 − delta_t. Measuring delta_t at the receiver assures an explicit correction for a more accurate RTT estimate. RTT samples SHOULD be smoothed using exponentially weighted moving average (EWMA) as specified in [RFC6298]. The sender then computes the minRTT according to these RTT samples during a period.¶

The sender SHOULD insert a TIMESTAMP frame into each packet for measuring OWD at the receiver.¶

6. Receiver-side Operation

Currently, the QUIC receiver reports ACK delays for only the largest acknowledged packet in an ACK frame, hence an RTT sample is generated using only the largest acknowledged packet in the received ACK frame. For a more accurate minRTT estimate when sending fewer ACK frames, QUIC SHOULD adopt the OWD-based minRTT estimation. The OWD-based minRTT estimation requires the QUIC receiver to filter the departure timestamp for the packet that achieves the minOWD during the interval between two ACK frames and report the ACK delay of this packet. Whether redefining the meaning of ACK delay or not, depends on the negotiation between endpoints of the QUIC connection.¶

Upon packet arrivals, the receiver is capable of generating per-packet OWD samples according to the difference between the packet departure timestamp and packet arrival timestamp. The receiver then computes the minOWD by comparing the per-packet OWD samples. The OWD estimation does not require clock synchronization here because the relative values are adopted.¶

Afterwards, based on the ACK delay and the departure timestamp corresponding to the packet that achieves the minOWD, the sender calculates the RTT of this packet as a minimum RTT sample. Ultimately, the minRTT is computed according to these minimum RTT samples.¶

The ACK Delay field SHOULD be carried in the ACK Frame. Other fields carried in the ACK frame have the same meaning as defined in [RFC9002].¶

7. Modification to QUIC Protocol

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                             0x XX (i)                            ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                       Departure Timestamp (i)                   ...
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

8. Security Considerations

TBD¶

9. IANA Considerations

The value for the timestamp-support transport parameter and TIMESTAMP frame should be allocated.¶

11. References