I am working on a (personal) light weight distributed project for the Rpi platform. Making it distributed requires P2P communication between the nodes, and I thought it would be a good time to learn sockets and UDP.

The end goal is to get to a point where nodes can do things like leader election, and distribute tasks between them through messages.

The below code uses pthreads to open two sockets; one sending and one receiving. The receiving socket just prints the message it receives for now.

I compiled and ran this code on two docker containers and it works. I am wondering about the following:

Since I can get the Hostname/IP of the sender, I can have a simple switch/if-else and take action based on the sender.

What message size is small enough so that packets of multiples senders are not mixed?

Also about the following sentence

the networking stack will buffer up (a finite number of) incoming UDP packets for you, so that (assuming you call recv() in a relatively timely manner), no incoming packets should get lost

Does the code below satisfy this (assuming of course I've added a check on the Hostname/IP)?

If not, how should it be modified? Would using select/poll inside the thread be better?

#include <netdb.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <pthread.h>
#include <sys/types.h>
#include <sys/socket.h>

#define BUF_SIZE 500

struct host
    char *node;
    char *port;
    char *msg;

create_socket(char *host, char *port)
    struct addrinfo hints;
    struct addrinfo *result, *rp;
    int sfd;

    memset(&hints, 0, sizeof(struct addrinfo));
    hints.ai_family = AF_UNSPEC;    /* Allow IPv4 or IPv6 */
    hints.ai_socktype = SOCK_DGRAM; /* Datagram socket */
    hints.ai_protocol = 0;          /* Any protocol */

    if (host != NULL)
        hints.ai_flags = AI_PASSIVE;    /* For wildcard IP address */    
    } else {
        hints.ai_flags = AI_PASSIVE;    /* For wildcard IP address */
        hints.ai_protocol = 0;          /* Any protocol */
        hints.ai_canonname = NULL;
        hints.ai_addr = NULL;
        hints.ai_next = NULL;

    int s = getaddrinfo(host, port, &hints, &result);
    if (s != 0) {
        fprintf(stderr, "getaddrinfo: %s\n", gai_strerror(s));

    /* getaddrinfo() returns a list of address structures.
        Try each address until we successfully connect(2).
        If socket(2) (or connect(2)/bind(2)) fails, we (close the socket
        and) try the next address. */
    for (rp = result; rp != NULL; rp = rp->ai_next) {
        // printf("this is the socketaddr data %s\n", rp->ai_addr->sa_data[]);
        sfd = socket(rp->ai_family, rp->ai_socktype,
        if (sfd == -1)

        if (host == NULL)
            printf("%s\n", "host is null");
            if (bind(sfd, rp->ai_addr, rp->ai_addrlen) == 0) /* server socket */
               break;                  /* Success */
            if (connect(sfd, rp->ai_addr, rp->ai_addrlen) != -1) /* client socket */
                break;                  /* Success */


    if (rp == NULL) {               /* No address succeeded */
        fprintf(stderr, "Could not bind/connect\n");

    freeaddrinfo(result);           /* No longer needed */

    return sfd;

send_msg(void *arg)
    size_t len;
    struct host *receiver = (struct host *) arg;

    int sfd = create_socket(receiver->node, receiver->port);
    printf("sending sfd = %d\n", sfd);

    len = strlen(receiver->msg) + 1;

    /* +1 for terminating null byte */
    if (len + 1 > BUF_SIZE) {
            "Ignoring long message\n");

        printf("sending message %s\n", receiver->msg);
        if (write(sfd, receiver->msg, len) != len) {
            fprintf(stderr, "partial/failed write\n");
            sfd = create_socket(receiver->node, receiver->port);


recv_msg(void *arg)
    int s;
    ssize_t nread;
    char buf[BUF_SIZE];

    socklen_t peer_addr_len;
    struct sockaddr_storage peer_addr;
    struct host *me = (struct host *) arg;

    int sfd = create_socket(me->node, me->port);

    printf("receive sfd = %d\n", sfd);
    for (;;) {
        printf("ready to receive message\n");

        peer_addr_len = sizeof(struct sockaddr_storage);

        nread = recvfrom(sfd, buf, BUF_SIZE, 0,
               (struct sockaddr *) &peer_addr, &peer_addr_len);

        printf(">>> %d\n", nread);
        if (nread == -1)
            continue;               /* Ignore failed request */

        char host[NI_MAXHOST], service[NI_MAXSERV];

        s = getnameinfo((struct sockaddr *) &peer_addr,
                       peer_addr_len, host, NI_MAXHOST,
                       service, NI_MAXSERV, NI_NUMERICSERV);
        if (s == 0)
            printf("Received %zd bytes from %s:%s\n",
                nread, host, service);
           fprintf(stderr, "getnameinfo: %s\n", gai_strerror(s));


create_threads(struct host *me,struct host *receiver)
    pthread_t tids[2];
    pthread_attr_t attr;

    pthread_create(&tids[0], &attr, send_msg, receiver);
    pthread_create(&tids[1], &attr, recv_msg, me);

    pthread_join(tids[0], NULL);
    pthread_join(tids[1], NULL);

main(int argc, char *argv[])

    struct host me;
    struct host receiver;

    if (argc < 3) {
        fprintf(stderr, "Usage: %s host port msg...\n", argv[0]);

    receiver.node = argv[1];
    receiver.port = argv[2];
    receiver.msg  = argv[3];

    me.node = NULL;
    me.port = argv[2];

    create_threads(&me, &receiver);

Does the code below satisfy this (?)

Small review: minor issues.

  1. if (len + 1 > BUF_SIZE) { looks off-by-1, in a bad direction. I'd expect if (len > BUF_SIZE) { as len already includes space for the null character.

  2. Code assumes ssize_t is int when printing. Alternative:

    ssize_t nread;
    nread = recvfrom(sfd, buf, BUF_SIZE, 0,
           (struct sockaddr *) &peer_addr, &peer_addr_len);
    // printf(">>> %d\n", nread);
    printf(">>> %lld\n", (long long) nread);
    // or
    printf(">>> %jd\n", (intmax_t) nread);
  • \$\begingroup\$ I mean: in the scenario where one node is managing the cluster, and is receiving more messages than the others, such that it is receiving many, many messages. Will the above code handle a fast message rate? what happens when the rate is too fast? what should be changed to handle that? (other than use TCP to guarantee delivery :) ) \$\endgroup\$ Jun 5 '18 at 16:23
  • 1
    \$\begingroup\$ @SamHammamy IDK about comment. Yet these are things about your code I could recognize that may fail and so not satisfy overall functional goal. Good luck. \$\endgroup\$ Jun 5 '18 at 16:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.