Deserialize bytes from the network into a struct

Question

I want to deserialize bytes coming from the network to a struct. The bytes come in big-endian order (BE) and my computer is using little-endian order (LE).

I'd like your advice concerning in order:

1. The absence/presence of Undefined Behavior.
2. The style used. Is it idiomatic modern C++ ?
3. What would be the best way to deserialize bytes from the network into a struct. Assign the struct members one at a time and avoid copying memory directly into the struct (through recvfrom() or memcpy()) ?

The example used receives a 16-bit integer, then 16 bits of padding and then a 32-bit float. It is done on purpose to force the use of padding to align the float on an address divisible by 4 (I know that I should swap them to avoid unneccessary padding). There are only two members in the struct but there might be dozens of them.

struct mystruct_t {
  uint16_t i;
  unsigned int : 16;
  float f;
};

The file mwe.cpp below is a MWE and has been compiled with GCC 13 (trunk) in C++23 mode using the equivalent of:

g++-13 -std=c++23 -fmodules-ts -pedantic -Wall -Wextra -o mwe mwe.cpp

I have tested it by sending three UDP datagrams under Linux (Debian 11.6) using:

for i in 1 2 3; do echo '1234 0000 4203 C28F' | xxd -r -p | nc -uq 0 localhost 8888 ; done

1234 (in hexadecimal) in BE represents the 16-bit integer 4660 (in decimal). 4203C28F in BE represents the 32-bit float 32.9399986 (approximately).

The code does not need to be portable. I have full control over the environment of execution.

I have used four different methods:

• Method 1 directly receives the network data into the struct. Thus I need to know that the data is in BE and my system is in LE and as such have to swap bytes after reception. I am dependent on the endianness of my system which is not good but ok since I control everything.

   memset(&mystruct, 0, sizeof(mystruct_t));
   memset(&cliaddr, 0, sizeof(cliaddr));
   len = sizeof(cliaddr);
   n = recvfrom(sockfd, &mystruct, sizeof(mystruct_t), MSG_WAITALL, (struct sockaddr *) &cliaddr, &len);
   cout << n << " bytes received\n";
   mystruct.i = byteswap(mystruct.i);
   mystruct.f = byteswap(mystruct.f);
  

• Method 2 uses an std:array of std::byte. I only need to know that data is coming in BE order. The order of my system does not matter. It is ok regarding the "The byte order fallacy" by Rob Pike. The commented lines are another way of filling the struct compared to the two lines below.

   memset(&mystruct, 0, sizeof(mystruct_t));
   std::array<std::byte, buffer_size> network_buffer_1;
   memset(&cliaddr, 0, sizeof(cliaddr));
   len = sizeof(cliaddr);
   n = recvfrom(sockfd, (void*)network_buffer_1.data(), buffer_size, MSG_WAITALL, (struct sockaddr *) &cliaddr, &len);
   cout << n << " bytes received\n";
   // reverse(network_buffer_1, 0, 2);
   // reverse(network_buffer_1, 4, 4);
   // memcpy(&mystruct, (void*)network_buffer_1.data(), sizeof(mystruct_t));
   mystruct.i = be2uint16(network_buffer_1, 0);
   mystruct.f = be2float(network_buffer_1, 4);
  

• Method 3 uses a traditional char array. I only need to know that data is coming in BE order. The order of my system does not matter. It is ok regarding the "The byte order fallacy" by Rob Pike.

   memset(&mystruct, 0, sizeof(mystruct_t));
   char network_buffer_2[buffer_size];
   memset(&cliaddr, 0, sizeof(cliaddr));
   len = sizeof(cliaddr);
   n = recvfrom(sockfd, network_buffer_2, buffer_size, MSG_WAITALL, (struct sockaddr *) &cliaddr, &len);
   cout << n << " bytes received\n";
   // reverse(network_buffer_2, 0, 2);
   // reverse(network_buffer_2, 4, 4);
   // memcpy(&mystruct, network_buffer_2, sizeof(mystruct_t));
   mystruct.i = be2uint16(network_buffer_2, 0);
   mystruct.f = be2float(network_buffer_2, 4);
  

• Method 4 uses reinterpret_cast() to directly use the buffer.

    memset(&cliaddr, 0, sizeof(cliaddr));
    len = sizeof(cliaddr);
    n = recvfrom(sockfd, network_buffer_2, buffer_size, MSG_WAITALL, (struct sockaddr *) &cliaddr, &len);
    cout << n << " bytes received\n";
    reverse(network_buffer_2, 0, 2);
    reverse(network_buffer_2, 4, 4);
    mystruct_t* ptr = reinterpret_cast<mystruct_t*>(network_buffer_2);
  

EDIT: I eventually used method 3 to let the struct be independent of the format of the input data.

#include <cstdint>
#include <cstdlib> // ok
#include <unistd.h>
#include <cstring> // memcpy
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <algorithm>
#include <iostream>
#include <iomanip>
#include <bit>
#include <iomanip>
#include <array>
#include <cstddef>

constexpr int port = 8888;
constexpr int buffer_size = 8192;

using std::cout;
using std::cerr;
using std::to_integer;

struct mystruct_t {
  uint16_t i;
    unsigned int : 16;
    float f;
};
using mystruct_t = struct mystruct_t;

std::ostream& operator<<(std::ostream& os, const mystruct_t& s) {
  os << "i: " << s.i << std::setprecision(12) << ", f: " << s.f;
  return os;
}

// no std::integral concept
template<typename T> constexpr T byteswap(T& value) noexcept {
  auto value_representation = std::bit_cast<std::array<std::byte, sizeof(T)>>(value);
  std::ranges::reverse(value_representation);
  return std::bit_cast<T>(value_representation);
}

void reverse(std::array<std::byte, buffer_size>& array, std::size_t pos, std::size_t n) {
  for (std::size_t i = 0; i < n / 2; i++)
    std::swap(array[pos+i], array[pos+n-1-i]);
}

void reverse(char * array, std::size_t pos, std::size_t n) {
  for (std::size_t i = 0; i < n / 2; i++)
    std::swap(array[pos+i], array[pos+n-1-i]);
}

uint16_t be2uint16(
  std::array<std::byte, buffer_size> array,
  std::array<std::byte, buffer_size>::size_type pos
) {
  return (to_integer<uint16_t>(array[pos+1])<<0) | (to_integer<uint16_t>(array[pos])<<8);
}
uint16_t be2uint16(const char * array, std::size_t pos) {
  return (static_cast<uint16_t>(array[pos+1])<<0) | (static_cast<uint16_t>(array[pos])<<8);
}

float be2float(
  std::array<std::byte, buffer_size> array,
  std::array<std::byte, buffer_size>::size_type pos
) {
  std::uint32_t i = (to_integer<uint32_t>(array[pos+3])<<0) | (to_integer<uint32_t>(array[pos+2])<<8) | (to_integer<uint32_t>(array[pos+1])<<16) | (to_integer<uint32_t>(array[pos])<<24);
    cout << "be2float 1 i: " << i << "\n";
  return std::bit_cast<float>(i);
}

float be2float(const char * array, std::size_t pos) {
  std::uint32_t i = (static_cast<uint32_t>(static_cast<unsigned char>(array[pos+3]))<<0) | (static_cast<uint32_t>(static_cast<unsigned char>(array[pos+2]))<<8) | (static_cast<uint32_t>(static_cast<unsigned char>(array[pos+1]))<<16) | (static_cast<uint32_t>(static_cast<unsigned char>(array[pos]))<<24);
    cout << "be2float 1 i: " << 2 << "\n";
  return std::bit_cast<float>(i);
}

int main() {
  if constexpr (std::endian::native == std::endian::big) std::cout << "big-endian\n";
  else if constexpr (std::endian::native == std::endian::little) std::cout << "little-endian\n";
  else std::cout << "mixed-endian\n";

  int sockfd;
  if ((sockfd = socket(AF_INET, SOCK_DGRAM, 0)) < 0) { perror("socket creation failed"); exit(EXIT_FAILURE); }

  struct sockaddr_in servaddr;
  memset(&servaddr, 0, sizeof(servaddr));
  servaddr.sin_family      = AF_INET;
  servaddr.sin_addr.s_addr = INADDR_ANY;
  servaddr.sin_port        = htons(port);

  if (bind(sockfd, (const struct sockaddr *)&servaddr, sizeof(servaddr)) < 0) { perror("bind failed"); exit(EXIT_FAILURE); }

  struct sockaddr_in cliaddr;
  socklen_t len;
    int n;

    struct mystruct_t mystruct;

  // Method 1
  memset(&mystruct, 0, sizeof(mystruct_t));
  memset(&cliaddr, 0, sizeof(cliaddr));
  len = sizeof(cliaddr);
    n = recvfrom(sockfd, &mystruct, sizeof(mystruct_t), MSG_WAITALL, (struct sockaddr *) &cliaddr, &len);
  cout << n << " bytes received\n";
    mystruct.i = byteswap(mystruct.i);
    mystruct.f = byteswap(mystruct.f);
    cout << "Method 1: " << mystruct << "\n";

  // Method 2
  memset(&mystruct, 0, sizeof(mystruct_t));
  std::array<std::byte, buffer_size> network_buffer_1;
  memset(&cliaddr, 0, sizeof(cliaddr));
  len = sizeof(cliaddr);
    n = recvfrom(sockfd, (void*)network_buffer_1.data(), buffer_size, MSG_WAITALL, (struct sockaddr *) &cliaddr, &len);
  cout << n << " bytes received\n";
    // reverse(network_buffer_1, 0, 2);
    // reverse(network_buffer_1, 4, 4);
    // memcpy(&mystruct, (void*)network_buffer_1.data(), sizeof(mystruct_t));
    mystruct.i = be2uint16(network_buffer_1, 0);
    mystruct.f = be2float(network_buffer_1, 4);
    cout << "Method 2: " << mystruct << "\n";

  // Method 3
  memset(&mystruct, 0, sizeof(mystruct_t));
  char network_buffer_2[buffer_size];
  memset(&cliaddr, 0, sizeof(cliaddr));
  len = sizeof(cliaddr);
    n = recvfrom(sockfd, network_buffer_2, buffer_size, MSG_WAITALL, (struct sockaddr *) &cliaddr, &len);
  cout << n << " bytes received\n";
    // reverse(network_buffer_2, 0, 2);
    // reverse(network_buffer_2, 4, 4);
    // memcpy(&mystruct, network_buffer_2, sizeof(mystruct_t));
    mystruct.i = be2uint16(network_buffer_2, 0);
    mystruct.f = be2float(network_buffer_2, 4);
    cout << "Method 3: " << mystruct << "\n";

  // Method 4
  memset(&cliaddr, 0, sizeof(cliaddr));
  len = sizeof(cliaddr);
    n = recvfrom(sockfd, network_buffer_2, buffer_size, MSG_WAITALL, (struct sockaddr *) &cliaddr, &len);
  cout << n << " bytes received\n";
    reverse(network_buffer_2, 0, 2);
    reverse(network_buffer_2, 4, 4);
    mystruct_t* ptr = reinterpret_cast<mystruct_t*>(network_buffer_2);
    cout << "Method 4: " << *ptr << "\n";

  return 0;
}

Answer 1

It is good that you use big-endian in the network. Most of the binary protocols use big-endian. So it is also called: 'network byte order'.

Instead of own functions like be2uint16(), be2float(), reverse(), try to use some commonly used existing one. Such like: ntohl() and htonl() (see this)

When the protocol is getting more complex, then your offset counting method will be painful. For example 0 and 4 in the following lines:

mystruct.i = be2uint16(network_buffer_1, 0);
mystruct.f = be2float(network_buffer_1, 4);

With C++ you can create nice class for making it easier. For example java have class ByteBuffer (see API). It could look something like this:

mystruct.i = bb.getShort();
bb.getShort(); // skip 2 bytes of padding
mystruct.f = bb.getFloat();

Answer 2

Method 1 directly receives the network data into the struct ... I am dependent on the endianness of my system which is not good but ok since I control everything.

You are also assuming that the packing of the struct is non-existent.

This is a common error - many years ago I helped a co-worker because his programming assignment crashed on our computer. The lecturer had given them a framework making the same mistake (it was a fill-in-the-blank assignment).

Depending on the compiler type, target processor, OS and even the settings for a given build, there may be different alignment of fields within the struct, with dead space between them.

Great explanation on GeeksForGeeks

Another example of how 64bit processors may complicate alignment even if you think you're hacking it into the right spacing - this IBM page

    struct lii{
              long la;
              int ia;
              int ib;
              } lii;
    struct ili{
              int ia;
              long la;
              int ib;
              } ili;

The struct lii and the struct ili have the same members, but in a different member order. ... Because of the padding differences in each environment:

• Under ILP32:
  • The size of struct lii is 12 bytes (4-byte long + 4-byte int + 4-byte int)
  • The size of struct ili is 12 bytes (4-byte int + 4-byte long + 4-byte int)
• Under LP64:
  • The size of struct lii is 16 bytes (8-byte long + 4-byte int + 4-byte int)
  • The size of struct ili is 24 bytes (4-byte int + 4-byte pad + 8-byte long + 4-byte int + 4-byte pad)