Motivation
When working with storage or wire protocols, we often read or write structures containing integers with specific byte-ordering (e.g. big-endian for Internet Protocol, or little-endian for USB).
It's common to use functions such as the htons() family to convert values, but it's easy to miss a conversion (particularly if the protocol is the same endianness as the development system). Instead, I prefer to use the type system to distinguish host integers from their representations' protocol byte-order.
Known limitations (non-goals):
- I haven't had a need to handle floating-point values, so only integers are handled here.
- It's not suitable for protocols such as IIOP or X11 where one of the hosts chooses at run-time which endianness to use.
Example usage
I'm currently using this when sending values over the wire. A simplified example, with all the error handling removed, looks something like:
struct Response { BigEndian<std::uint16_t> seq_no; BigEndian<std::uint16_t> sample_value; };
We send by assigning (which implicitly converts our integer value to big-endian byte sequence) and then writing the structure:
void send_result(std::uint16_t value) { Response r; r.seq_no = counter++; r.sample_value = value; write(fd, &r, sizeof r); }
On the receive side, we read the wire representation into the same structure (so it's bitwise identical to the sending side) and then use the conversion operator to access the data in native form:
std::uint16_t recv_result() { Response r; read(fd, &r, sizeof r); // ignore seq_no, for now return r.sample_value; }
Changes since previous version
Since version 1, I've changed the following:
- Renamed the views which give us access to the storage in the desired order, and added comments to make it clearer. This was previously giving the impression that round-trip would byte-reverse the value, which is not the case.
- Uniformly scatter/gather to native
charsize (in units ofCHAR_BIT). - Added warning for non-8-bit platforms to encourage audit of octet-orientated code.
- Optimised for reading/writing native-endian values directly with no wrapper class (will do the Right Thing for unicorn/dinosaur platforms such as mixed-endian or no-endian).
- Split out the default constructor, since serialising zero is a no-op.
- Added
constexprto conversion operations. - Demonstrate plain old round-trip in the tests, rather than modifying the storage. I think this makes the tests clearer.
Implementation
#ifndef ENDIAN_HPP
#define ENDIAN_HPP
#include <array>
#include <bit>
#include <climits>
#include <concepts>
#include <ranges>
#include <type_traits>
#ifndef ENDIAN_SUPPORT_NON_8BIT
static_assert(CHAR_BIT == 8,
"This header splits into chars, not octets. "
"Define ENDIAN_SUPPORT_NON_8BIT to enable.");
#endif
namespace endian
{
namespace detail {
template<std::integral T, // type to represent
auto BigFirst, // view that presents MSB first
auto LittleFirst> // view that presents LSB first
struct Endian
{
// We use unsigned T for bitwise operations
using U = std::make_unsigned_t<T>;
// The underlying storage
std::array<unsigned char, sizeof (T)> data = {};
constexpr Endian() = default;
// implicit conversion from T
constexpr Endian(T value)
{
// unpack value starting with the least-significant bits
auto uval = static_cast<U>(value);
for (auto& c: data | LittleFirst) {
c = static_cast<unsigned char>(uval);
uval >>= CHAR_BIT;
}
}
// implicit conversion to T
constexpr operator T() const
{
// compose value starting with most-significant bits
U value = 0;
for (auto c: data | BigFirst) {
value <<= CHAR_BIT;
value |= c;
}
return static_cast<T>(value);
}
};
}
template<std::integral T>
using BigEndian =
std::conditional_t<std::endian::native == std::endian::big,
T, // no conversion needed
detail::Endian<T, std::views::all, std::views::reverse>>;
template<std::integral T>
using LittleEndian =
std::conditional_t<std::endian::native == std::endian::little,
T, // no conversion needed
detail::Endian<T, std::views::reverse, std::views::all>>;
}
#endif // ENDIAN_HPP
Unit Tests
using endian::BigEndian;
using endian::LittleEndian;
#include <gtest/gtest.h>
#include <cstdint>
#include <cstring>
// Ensure there's no padding
static_assert(sizeof (BigEndian<int>) == sizeof (int));
static_assert(sizeof (LittleEndian<int>) == sizeof (int));
// Helper function to inspect representation
template<typename T>
auto byte_array(const T& t)
{
std::array<unsigned char, sizeof t> bytes;
std::memcpy(bytes.data(), &t, sizeof t);
return bytes;
}
// Now the tests themselves
TEST(big_endian, uint8)
{
const std::uint8_t x = 2;
auto be = BigEndian<std::uint8_t>{x};
std::array<unsigned char, 1> expected{{2}};
EXPECT_EQ(byte_array(be), expected);
// round trip back to native
std::uint8_t y = be;
EXPECT_EQ(y, x);
}
TEST(little_endian, uint8)
{
const std::uint8_t x = 2;
auto le = LittleEndian<std::uint8_t>{x};
std::array<unsigned char, 1> expected{{2}};
EXPECT_EQ(byte_array(le), expected);
std::uint8_t y = le;
EXPECT_EQ(y, x);
}
TEST(big_endian, uint16)
{
const std::uint16_t x = 0x1234;
BigEndian<std::uint16_t> be = x;
std::array<unsigned char, 2> expected{{0x12, 0x34}};
EXPECT_EQ(byte_array(be), expected);
std::uint16_t y = be;
EXPECT_EQ(y, x);
}
TEST(little_endian, uint16)
{
const std::uint16_t x = 0x1234;
auto le = LittleEndian<std::uint16_t>{x};
std::array<unsigned char, 2> expected{{0x34, 0x12}};
EXPECT_EQ(byte_array(le), expected);
std::uint16_t y = le;
EXPECT_EQ(y, x);
}
TEST(big_endian, uint32)
{
const std::uint32_t x = 0x12345678;
auto be = BigEndian<std::uint32_t>{x};
std::array<unsigned char, 4> expected{{ 0x12, 0x34, 0x56, 0x78 }};
EXPECT_EQ(byte_array(be), expected);
std::uint32_t y = be;
EXPECT_EQ(y, x);
}
TEST(little_endian, uint32)
{
const std::uint32_t x = 0x12345678;
auto le = LittleEndian<std::uint32_t>{x};
std::array<unsigned char, 4> expected{{ 0x78, 0x56, 0x34, 0x12 }};
EXPECT_EQ(byte_array(le), expected);
std::uint32_t y = le;
EXPECT_EQ(y, x);
}
