|
|---|
|
|---|
Too Small TCP Window SizeTCP window size is usually equal to the TCP buffer space and should usually be the same for both sides of a connection (otherwise, the minimum of the two will often be in effect). In traditional TCP, the window size cannot be larger than 65,535 bytes (because the unsigned integer field that holds it is only 16 bits wide). You will probably need a bigger window than 64KB. This is achieved by turning on TCP extensions specified in RFC1323, `TCP Extensions for High Performance', published in 1992 and now supported by most operating systems. The TCP window scaling option allows one to use TCP window sizes of up to 1,073,741,823 bytes, which should be good until speeds that approach 1Tb/s (with RTT of 100ms). Turn on this option. (It might already be on. No operating-system-specific instructions are provided in this document, but there are some links at the end that might help. If you have or know of more publicly available documents describing operating-systems-specific steps to set TCP options and window sizes, propose them to be included.) Once you figure out how to do this, you should check if your TCP stack supports selective acknowledgment (SACK) or explicit congestion notification (ECN) options; if it does, turn these on, too (SACK and ECN and not strictly speaking necessary to get good performance but they might help in certain environments). The sure way to learn the TCP window sizes in question is to perform a packet capture of your connection (e.g., with tcpdump, snoop, Etherpeek, or equivalent tool for your OS) and examine it. You will need a few packets from the beginning of the connection, including the SYN packets. (If you don't know what is a SYN packet or cannot do packet capture yourself, find someone local who has some networking expertise -- such as your network support technician. Otherwise, prepare to spend some time figuring this stuff out, but you must be the type that likes to fix car engines by yourself.) For each direction, the window scaling option will be in the SYN packet (if it is not, it is implicitly 0). For each direction, take the actual window size number from the first data or pure ACK packet and multiply it by 2^{window scaling}; this is the actual window size at the start of the connection. (For example, if window scale in the SYN packet is 3 and the window size in the first data or pure ACK packet is specified as 32768, the actual window size for that direction is 32768*2^3 = 262144; this figure is in bytes.) Make note of the window size numbers in both directions. To make matters more confusing, some packet sniffers will already have done the multiplication for you; it happened if and only if the window size advertised in the first data or pure ACK packet is already presented as being larger than 65535; if this is the case, you do not need to multiply the number by the exponentiated window scale (it is already usable as is). Having established the window sizes, make note of the minimum of the two numbers for each direction and use it in subsequent paragraphs. Measure the round-trip time. The simplest way is to use the Make it send 60 packets slowly (e.g., 1 packet/second) while the network does not carry your traffic and take the minimum and the average of the 60 numbers (these are normally printed at the bottom along with some other numbers). On a properly functioning high-speed network, the numbers should both resemble within a factor of two your propagation delay (to be estimated in seconds as distance between hosts in kilometers divided by 200,000km/s, which is approximately equal to the speed of light in fiber; the delay should be fudged 50% up for network path not being a part of a great circle of the planet). Take the minimum as your RTT, to be used subsequently. The theoretical maximum TCP throughput that you can obtain with your tuning parameters is window/RTT. Rule of thumb: For a realistic number, divide the theoretic hard limit by two. With 70ms RTT (typical cross-country delay), you might see the following throughputs for different window sizes (assumed equal at both sides):
(I.e., for what is currently considered large values of window size, perhaps a factor of two of safety margin is required; for small window sizes, the safety margin is largely determined simply by the remainder of the division of allocated buffer space by TCP Maximum Segment Size (MSS); MSS is simply the maximum number of payload bytes that can be sent inside a single TCP packet on a given link.) The theoretical maximum is a hard limit of sustained performance with arbitrary TCP options, only a small number of non-consecutive drops per RTT, and a perfect network. The realistic values are what the author would personally expect in a typical case, based on experience, from hosts, TCP implementation, and networks from the real world (in particular, it assumes no SACK, no ECN, standard aggressiveness, 1500-byte MTU, and no RED at the bottleneck). For example, if a 64KB window size is used on both sides and one sees 3Mb/s actual throughput with 70ms RTT, this is nothing out of the ordinary: it's only slightly less than twice the hard limit one would see in a perfect world. To make the long story short: On a high-speed wide-area network, hosts should have RFC1323 extensions turned on and default TCP send and receive buffers that are equal to each other and are at least 1 megabyte.
|
|
About NPI |
Contact Us |
Services | Tools |
Site Map |
Reseller Programs
Professional Ethics |
Privacy
Copyright 1993-2006 Network Partners, Inc. All rights reserved
