[c] How large should my recv buffer be when calling recv in the socket library

I have a few questions about the socket library in C. Here is a snippet of code I'll refer to in my questions.

char recv_buffer[3000];
recv(socket, recv_buffer, 3000, 0);

1. How do I decide how big to make recv_buffer? I'm using 3000, but it's arbitrary.
2. what happens if recv() receives a packet bigger than my buffer?
3. how can I know if I have received the entire message without calling recv again and have it wait forever when there is nothing to be received?
4. is there a way I can make a buffer not have a fixed amount of space, so that I can keep adding to it without fear of running out of space? maybe using strcat to concatenate the latest recv() response to the buffer?

I know it's a lot of questions in one, but I would greatly appreciate any responses.

There is no absolute answer to your question, because technology is always bound to be implementation-specific. I am assuming you are communicating in UDP because incoming buffer size does not bring problem to TCP communication.

According to RFC 768, the packet size (header-inclusive) for UDP can range from 8 to 65 515 bytes. So the fail-proof size for incoming buffer is 65 507 bytes (~64KB)

However, not all large packets can be properly routed by network devices, refer to existing discussion for more information:

What is the optimal size of a UDP packet for maximum throughput?
What is the largest Safe UDP Packet Size on the Internet

For SOCK_STREAM socket, the buffer size does not really matter, because you are just pulling some of the waiting bytes and you can retrieve more in a next call. Just pick whatever buffer size you can afford.

For SOCK_DGRAM socket, you will get the fitting part of the waiting message and the rest will be discarded. You can get the waiting datagram size with the following ioctl:

#include <sys/ioctl.h>
int size;
ioctl(sockfd, FIONREAD, &size);

Alternatively you can use MSG_PEEK and MSG_TRUNC flags of the recv() call to obtain the waiting datagram size.

ssize_t size = recv(sockfd, buf, len, MSG_PEEK | MSG_TRUNC);

You need MSG_PEEK to peek (not receive) the waiting message - recv returns the real, not truncated size; and you need MSG_TRUNC to not overflow your current buffer.

Then you can just malloc(size) the real buffer and recv() datagram.

For streaming protocols such as TCP, you can pretty much set your buffer to any size. That said, common values that are powers of 2 such as 4096 or 8192 are recommended.

If there is more data then what your buffer, it will simply be saved in the kernel for your next call to recv.

Yes, you can keep growing your buffer. You can do a recv into the middle of the buffer starting at offset idx, you would do:

recv(socket, recv_buffer + idx, recv_buffer_size - idx, 0);

16kb is about right; if you're using gigabit ethernet, each packet could be 9kb in size.

If you have a SOCK_STREAM socket, recv just gets "up to the first 3000 bytes" from the stream. There is no clear guidance on how big to make the buffer: the only time you know how big a stream is, is when it's all done;-).

If you have a SOCK_DGRAM socket, and the datagram is larger than the buffer, recv fills the buffer with the first part of the datagram, returns -1, and sets errno to EMSGSIZE. Unfortunately, if the protocol is UDP, this means the rest of the datagram is lost -- part of why UDP is called an unreliable protocol (I know that there are reliable datagram protocols but they aren't very popular -- I couldn't name one in the TCP/IP family, despite knowing the latter pretty well;-).

To grow a buffer dynamically, allocate it initially with malloc and use realloc as needed. But that won't help you with recv from a UDP source, alas.