tcp: don't abort splice() after small transfers
authorWilly Tarreau <w@1wt.eu>
Sun, 2 Dec 2012 11:49:27 +0000 (11:49 +0000)
committerGreg Kroah-Hartman <gregkh@linuxfoundation.org>
Thu, 14 Feb 2013 18:48:36 +0000 (10:48 -0800)
commit301008f5b5a734d5d1ddd5c4a95d69812ae4c7e7
treee9e1640b528b18c611e75e5f29f2186281a05258
parentfa237a5c85320c6597caaea5dfeff5e2d995e666
tcp: don't abort splice() after small transfers

[ Upstream commit 02275a2ee7c0ea475b6f4a6428f5df592bc9d30b ]

TCP coalescing added a regression in splice(socket->pipe) performance,
for some workloads because of the way tcp_read_sock() is implemented.

The reason for this is the break when (offset + 1 != skb->len).

As we released the socket lock, this condition is possible if TCP stack
added a fragment to the skb, which can happen with TCP coalescing.

So let's go back to the beginning of the loop when this happens,
to give a chance to splice more frags per system call.

Doing so fixes the issue and makes GRO 10% faster than LRO
on CPU-bound splice() workloads instead of the opposite.

Signed-off-by: Willy Tarreau <w@1wt.eu>
Signed-off-by: Eric Dumazet <edumazet@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
net/ipv4/tcp.c