5
\$\begingroup\$

I've tried to make a portable way of ensuring endian-specific code gets generated at compile time using C++11, however I only have a computer with Windows on it to test at the moment. Because of this, I'm a bit limited in the amount of places where I can test my code. Also, would anyone be able to offer some best-practices or tips different ways to improve this? My intention is to use this in a small math library where serialization is a pretty high priority.

The function itself is fairly simple. It checks the values of an array for whichever byte comes first. It then returns a constant value, representing the target machine's endianness, through an enumeration. If everything works correctly, then this code can replace any runtime checks that either I or anyone else uses for endian checking.

/* 
 * A simple compile-time endian test
 * g++ -std=c++11 -Wall -Werror -Wextra -pedantic -pedantic-errors endian.cpp -o endian
 *
 * This can be used with specialized template functions, classes, and class
 * methods in order better tailor code and reduce reliance on runtime
 * checking systems.
 */

#include <cstdint>
#include <iostream>

/**
 * hl_endianness
 *
 * This enumeration can be placed into templated objects in order to generate
 * compile-time code based on a program's target endianness.
 *
 * The values placed in this enum are used just in case the need arises in
 * order to manually compare them against the number order in the
 * endianValues[] array.
 */
enum hl_endianness : uint32_t {
    HL_LITTLE_ENDIAN   = 0x03020100,
    HL_BIG_ENDIAN      = 0x00010203,
    HL_PDP_ENDIAN      = 0x01000302,
    HL_UNKNOWN_ENDIAN  = 0xFFFFFFFF
};

/**
 * A constant array used to determine a program's target endianness. The
 * values
 *  in this array can be compared against the values placed in the
 * hl_endianness enumeration.
 */
static constexpr uint8_t endianValues[4] = {0, 1, 2, 3};

/**
 * A simple function that can be used to help determine a program's endianness
 * at compile-time.
 */
constexpr hl_endianness getEndianOrder() {
    return
        (0x00 == endianValues[0])           // If Little Endian Byte Order,
            ? HL_LITTLE_ENDIAN              // return 0 for little endian.
            : (0x03 == endianValues[0])     // Else if Big Endian Byte Order,
                ? HL_BIG_ENDIAN             // return 1 for big endian.
                : (0x02 == endianValues[0]) // Else if PDP Endian Byte Order,
                    ? HL_PDP_ENDIAN         // return 2 for pdp endian.
                    : HL_UNKNOWN_ENDIAN;    // Else return -1 for wtf endian.
}

#define HL_ENDIANNESS getEndianOrder()

/*
 * Test program
 */
int main() {
    #if defined _WIN32 || defined _WIN64
        static_assert(
            HL_ENDIANNESS == HL_LITTLE_ENDIAN,
            "Aren't Windows programs Little-Endian?"
        );
    #endif

    constexpr hl_endianness endianness = HL_ENDIANNESS;

    std::cout << "This machine is: ";

    switch (endianness) {
        case HL_LITTLE_ENDIAN:
            std::cout << "LITTLE";
            break;
        case HL_BIG_ENDIAN:
            std::cout << "BIG";
            break;
        case HL_PDP_ENDIAN:
            std::cout << "PDP";
            break;
        case HL_UNKNOWN_ENDIAN:
        default:
            std::cout << "UNKNOWN";
    }

    std::cout << " endian" << std::endl;
}

Edit:

I can try dereferencing the pointer offset to the endianValues array but I'm still not sure if it will end up defaulting to 0 (the first explicitly set value in the array).

constexpr hl_endianness getEndianOrder() {
    return
        (0x00 == *endianValues)           // If Little Endian Byte Order,
            ? HL_LITTLE_ENDIAN              // return 0 for little endian.
            : (0x03 == *endianValues)     // Else if Big Endian Byte Order,
                ? HL_BIG_ENDIAN             // return 1 for big endian.
                : (0x02 == *endianValues) // Else if PDP Endian Byte Order,
                    ? HL_PDP_ENDIAN         // return 2 for pdp endian.
                    : HL_UNKNOWN_ENDIAN;    // Else return -1 for wtf endian.
}

Edit #2

So after looking into how the bits might be stored on different systems, I finally realized that I might be able to just use a single bit to test for endianness. It looks to be much less error-prone than using an array and I still get the correct answer on my Windows box.

enum hl_endianness : uint32_t {
    HL_LITTLE_ENDIAN   = 0x00000001,
    HL_BIG_ENDIAN      = 0x01000000,
    HL_PDP_ENDIAN      = 0x00010000,
    HL_UNKNOWN_ENDIAN  = 0xFFFFFFFF
};

/**
 * A simple function that can be used to help determine a program's endianness
 * at compile-time.
 */
constexpr hl_endianness getEndianOrder() {
    return
        ((1 & 0xFFFFFFFF) == HL_LITTLE_ENDIAN)
            ? HL_LITTLE_ENDIAN
            : ((1 & 0xFFFFFFFF) == HL_BIG_ENDIAN)
                ? HL_BIG_ENDIAN
                : ((1 & 0xFFFFFFFF) == HL_PDP_ENDIAN)
                    ? HL_PDP_ENDIAN
                    : HL_UNKNOWN_ENDIAN;
}

#define HL_ENDIANNESS getEndianOrder()
\$\endgroup\$
3
  • \$\begingroup\$ Or #include <boost/predef/detail/endian_compat.h> and then test for BOOST_LITTLE_ENDIAN or BOOST_BIG_ENDIAN or BOOST_PDP_ENDIAN \$\endgroup\$ Commented Mar 29, 2014 at 17:56
  • 1
    \$\begingroup\$ @Loki I would rather not use Boost since this is also a personal exercise in learning how different computers process their data. Although if it was for production code then Boost would definitely not be a bad idea. \$\endgroup\$
    – icdae
    Commented Mar 29, 2014 at 18:06
  • \$\begingroup\$ In C++20, you can use std::endian: wandbox.org/permlink/3FkeNpZx5Ix4PbdP. \$\endgroup\$ Commented Aug 21, 2018 at 8:04

3 Answers 3

4
\$\begingroup\$

Just a couple points:

  1. Endian-ness is not generally based on the Operating System but on the processor. For example, Intel x86 processors are little-endian regardless of it running Windows or Linux.

  2. Your code will always return HL_LITTLE_ENDIAN. Why? Because if

    static constexpr uint8_t endianValues[4] = {0, 1, 2, 3};
    

    then endianValues[0] == 0 will always be true! Suppose you had

    char x[4] = {'c','o','d', 'e'};
    

    Don't you think it would be shocking if x[0] == 'e' instead of x[0] == 'c'?

    The standard way is to use a union. Something like this:

    union endian_tester {
        uint32_t   n;
        uint8_t    p[4];
    };
    
    const endian_tester sample = {0x01020304}; // this initializes .n
    
    constexpr hl_endianness getEndianOrder() {
        return
            (0x04 == sample.p[0])               // If Little Endian Byte Order,
                ? HL_LITTLE_ENDIAN              
                : (0x01 == sample.p[0])         // Else if Big Endian Byte Order,
                    ? HL_BIG_ENDIAN             
                    : (0x02 == sample.p[0])     // Else if PDP Endian Byte Order,
                           ...(etc)...
    

    Be aware that constexpr isn't fully supported in my version of Visual Studio 2013 Express.

  3. Not clear to me why you need to use fancy values for HL_LITTLE_ENDIAN, HL_BIG_ENDIAN, etc. You can use 1, 2, etc instead of 0x03020100, 0x00010203, etc.

  4. A related question answered on Stack Overflow (Detecting endianness programmatically in a C++ program)

\$\endgroup\$
6
  • \$\begingroup\$ Some processors can run in either big- or little-endian mode, leaving the choice up to the operating system. However, Windows will probably only ever run in little-endian mode. \$\endgroup\$ Commented Mar 29, 2014 at 8:05
  • \$\begingroup\$ I originally tried a union but got errors from G++ for undefined behavior. After setting the data for one member, then trying to access the other, I'll get errors that prevent the code from compiling. I figured the current method would be a safe bet, but completely overlooked how the data would be accessed through an array. The problem is that using behavior defined by the standard seems to limit how byte-level data can be access during compilation. Neither casting to another data type or bit-leven operations seem to work. \$\endgroup\$
    – icdae
    Commented Mar 29, 2014 at 8:23
  • \$\begingroup\$ @icdae you're right; I think someone mentioned that the standard technically prevents setting a union with one field but accessing it with a different one. But many compilers allow it. In that Stack Overflow link, there is one answer that uses the idea of casting int to char which might work better (but you should try uint32_t instead of int). \$\endgroup\$ Commented Mar 29, 2014 at 16:37
  • \$\begingroup\$ Your union "standard way" has undefined behavior according to the Standard. \$\endgroup\$
    – Ruslan
    Commented Nov 15, 2015 at 10:36
  • 3
    \$\begingroup\$ -1. Using unions for type punning in C++ is undefined behavior. \$\endgroup\$
    – user109835
    Commented Jul 1, 2016 at 8:43
2
\$\begingroup\$

I would change this:

static constexpr uint8_t endianValues[4] = {0, 1, 2, 3};

To this:

static const uint32_t value = HL_LITTLE_ENDIAN; // 0x03020100
static const uint8_t* endianValues = (uint8_t*)&value;

Alternatively, you can change the getEndianOrder function to read the endianValues array as a uint32, but you will have to add a preprocessor directive (#pragma) to make sure that it is placed in a memory address aligned to 4 bytes (and that by itself might yield some platform-dependency issues, which is pretty much in contrast with your goal here to begin with):

constexpr hl_endianness getEndianOrder()
{
    switch (*(uint32_t*)endianValues))
    {
        case HL_LITTLE_ENDIAN: return HL_LITTLE_ENDIAN;
        case HL_BIG_ENDIAN:    return HL_BIG_ENDIAN;
        case HL_PDP_ENDIAN:    return HL_PDP_ENDIAN;
    }
    return HL_UNKNOWN_ENDIAN;
}
\$\endgroup\$
6
  • \$\begingroup\$ I tried your solution (had to remove the switch statement to make it a constexpr function) but I believe this will always return as little endian. Since the value stored in HL_LITTLE_ENDIAN is how it will actually be stored in memory on all computers, retrieving an 8-bit pointer to value is always going to grab the same value of 0x03. I tried another solution though end edited it into the original question. \$\endgroup\$
    – icdae
    Commented Mar 29, 2014 at 18:17
  • \$\begingroup\$ @icdae: It's not an 8-bit pointer, it's a 32-bit pointer (look inside the switch statement that you removed). Though I have to admit, I'm not really sure how it is applied into a constexpr during compile-time. \$\endgroup\$ Commented Mar 29, 2014 at 18:21
  • \$\begingroup\$ Oops, I misspoke (just woke up, everything's a bit fuzzy); being a pointer to a constant value won't change how the value itself is stored in memory. The function will always return the same result since value = HL_LITTLE_ENDIAN. As for the switch statement, one of the restrictions for a constexpr statement is that the body of the function can only consist of a single return statement. In this case the switch can be converted to a simple series of ternary operators. \$\endgroup\$
    – icdae
    Commented Mar 29, 2014 at 18:37
  • \$\begingroup\$ @icdae: Please note the emphasis on the word Alternatively, which means that you should leave endianValues[4] = {0, 1, 2, 3} as is, but read it using *(uint32_t*)endianValues). The value variable is meant to be used only in the first option suggested. \$\endgroup\$ Commented Mar 29, 2014 at 19:17
  • \$\begingroup\$ -1, this violates strict aliasing rules. \$\endgroup\$
    – user109835
    Commented Jul 1, 2016 at 8:44
2
\$\begingroup\$

You have to use predefined compiler macros (\__BIG_ENDIAN__ or \__LITTLE_ENDIAN__ with clang, or \__BYTE_ORDER__ with gcc).

The other compiler macro tricks mentioned will only detect the endianness of the architecture you are compiling on, not the endianness of the architecture you are compiling for, so something like this is wrong:

\#define IS_BIG_ENDIAN ('\x01\x02\x03\x04' == 0x01020304)
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.