Last week I ran into a nasty little problem while implementing an application with soft real-time requirements. I was aiming at 1 ms or less for a TCP-based request-response roundtrip on a local network. Should be trivial, but why did my tests indicate that I wasn’t even getting close? The scenario was simple: A server (my part) gets a request from a client. Before it can answer, it has to ask a backend system for some information. The backend system listens on a TCP port and answers in a query-response fashion (processing time is far below 1 ms). Both query and response typically fit in a single TCP segment. The response may sometimes be larger, which is one reason why TCP was an adequate choice for the backend system. As a first optimization I used a TCP connection pool to get around TCP’s three-way handshake (SYN, SYN ACK, ACK) and thus cutting down latency. However, tracing showed response times around 40 ms on the local network. Actually worse than without the connection pool! Admittedly, the conventional use for connection pools is to circumvent TCP’s slow start mechanism. But still, something was very wrong. Read more: Unmaintainable