c - Why kernel forcefully sends TCP RST from client after certain amount of bytes send?

Question

I have simple "echo" client-server code and with watching tcp stream with tcpdump, The client always sends RST after it sends 274. Always. But I have no idea how to track down the issue. The client:

//FILE to read from (stdin), sockfd from connect()
void str_cli(FILE *fp, int sockfd)
{
    int maxfd;
    fd_set rset;
    //BSIZE == 4096
    char recvline[BSIZE], sendline[BSIZE];
    int streamfd = fileno(fp);

    //clear the read set
    FD_ZERO(&rset);
    while (1)
    {
        FD_SET(streamfd, &rset);
        FD_SET(sockfd, &rset);
        maxfd = max(streamfd, sockfd) + 1;

        //select blocks until one of the fds are readable
        if (select(maxfd, &rset, NULL, NULL, NULL) < 0)
        {
            die("select");
        }

        if (FD_ISSET(sockfd, &rset))
        {
            //socket is readable
            if (Readline(sockfd, recvline, BSIZE) == 0)
            {
                die("str_cli: server terminated prematurely");
            }
            Fputs(recvline, stdout);
        }

        if (FD_ISSET(streamfd, &rset))
        {
            //got input and can read from streamfd
            if (fgets(sendline, BSIZE, fp) == NULL)
            {
                //EOF == all done
                perror("fgets in select");
                return;
            }
            Writen(sockfd, sendline, strlen(sendline));
        }
    }
}

server:

//sockfd is socket returned from accept()
void str_echo(int sockfd)
{
    ssize_t len;
    //BSIZE == 4096
    char buf[BSIZE];

again:
    while ((len = read(sockfd, buf, BSIZE)) > 0)
    {
        Writen(sockfd, buf, len);
    }
    if (len < 0 && errno == EINTR)
    {
        goto again;
    }
    else if (len < 0)
    {
        perror("str_echo::read");
    }
}

I had the BSIZE to 256, and I thought the kernel is killing the client because of buffer overflow, so I have changed it to 4096. But the problem remains, after client sends 274 bytes, after it it sends RST and I have no idea why. The last dump

127.0.0.1:9877 -> server 127.0.0.1:46790 -> client

tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
IP 127.0.0.1.46790 > 127.0.0.1.9877: Flags [S], seq 1941257529, win 65495, options [mss 65495,sackOK,TS val 2375820429 ecr 0,nop,wscale 7], length 0
IP 127.0.0.1.9877 > 127.0.0.1.46790: Flags [S.], seq 4099864764, ack 1941257530, win 65483, options [mss 65495,sackOK,TS val 2375820429 ecr 2375820429,nop,wscale 7], length 0
IP 127.0.0.1.46790 > 127.0.0.1.9877: Flags [.], ack 1, win 512, options [nop,nop,TS val 2375820429 ecr 2375820429], length 0
IP 127.0.0.1.46790 > 127.0.0.1.9877: Flags [P.], seq 1:37, ack 1, win 512, options [nop,nop,TS val 2375820430 ecr 2375820429], length 36
IP 127.0.0.1.9877 > 127.0.0.1.46790: Flags [.], ack 37, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 0
IP 127.0.0.1.46790 > 127.0.0.1.9877: Flags [P.], seq 37:81, ack 1, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 44
IP 127.0.0.1.9877 > 127.0.0.1.46790: Flags [.], ack 81, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 0
IP 127.0.0.1.46790 > 127.0.0.1.9877: Flags [P.], seq 81:82, ack 1, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 1
IP 127.0.0.1.9877 > 127.0.0.1.46790: Flags [.], ack 82, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 0
IP 127.0.0.1.46790 > 127.0.0.1.9877: Flags [P.], seq 82:92, ack 1, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 10
IP 127.0.0.1.9877 > 127.0.0.1.46790: Flags [.], ack 92, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 0
IP 127.0.0.1.9877 > 127.0.0.1.46790: Flags [P.], seq 1:82, ack 92, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 81
IP 127.0.0.1.9877 > 127.0.0.1.46790: Flags [.], ack 194, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 0
IP 127.0.0.1.46790 > 127.0.0.1.9877: Flags [P.], seq 194:210, ack 82, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 16
IP 127.0.0.1.46790 > 127.0.0.1.9877: Flags [P.], seq 210:226, ack 82, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 16
IP 127.0.0.1.46790 > 127.0.0.1.9877: Flags [P.], seq 226:236, ack 82, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 10
IP 127.0.0.1.9877 > 127.0.0.1.46790: Flags [.], ack 266, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 0
IP 127.0.0.1.9877 > 127.0.0.1.46790: Flags [P.], seq 82:274, ack 274, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 192
IP 127.0.0.1.46790 > 127.0.0.1.9877: Flags [R.], seq 274, ack 274, win 512, options [nop,nop,TS val 2375820430 ecr 2375820430], length 0
19 packets captured
56 packets received by filter
16 packets dropped by kernel

Does anyone know, why kernel forcefully terminate the client? (by sending RST)?

EDIT: if I use netcat instead of client nc 127.0.0.1 9877 < somefile, the server echoes it correctly, so I suspect the client have a issue somewhere. But still the kernel forcefully makes client to send RST, but why?

Answer 1

The problem was, After client has sent all its bytes, it does not wait for server reply and closed all its sockets (well the process exit(0), but when process exits, kernel closes all its opened file descriptors). So the data were send in one end, but the socket was closed before the other end could echoed it back. So because the buffer of receiving socket was not-empty, the reply was RST. I have make a dummy solution of sleep(1), after sending the bytes from client. After which it worked.