4
\$\begingroup\$

I've tried to make a portable way of ensuring endian-specific code gets generated at compile time using C++11, however I only have a computer with Windows on it to test at the moment. Because of this, I'm a bit limited in the amount of places where I can test my code. Also, would anyone be able to offer some best-practices or tips different ways to improve this? My intention is to use this in a small math library where serialization is a pretty high priority.

The function itself is fairly simple. It checks the values of an array for whichever byte comes first. It then returns a constant value, representing the target machine's endianness, through an enumeration. If everything works correctly, then this code can replace any runtime checks that either I or anyone else uses for endian checking.

/* 
 * A simple compile-time endian test
 * g++ -std=c++11 -Wall -Werror -Wextra -pedantic -pedantic-errors endian.cpp -o endian
 *
 * This can be used with specialized template functions, classes, and class
 * methods in order better tailor code and reduce reliance on runtime
 * checking systems.
 */

#include <cstdint>
#include <iostream>

/**
 * hl_endianness
 *
 * This enumeration can be placed into templated objects in order to generate
 * compile-time code based on a program's target endianness.
 *
 * The values placed in this enum are used just in case the need arises in
 * order to manually compare them against the number order in the
 * endianValues[] array.
 */
enum hl_endianness : uint32_t {
    HL_LITTLE_ENDIAN   = 0x03020100,
    HL_BIG_ENDIAN      = 0x00010203,
    HL_PDP_ENDIAN      = 0x01000302,
    HL_UNKNOWN_ENDIAN  = 0xFFFFFFFF
};

/**
 * A constant array used to determine a program's target endianness. The
 * values
 *  in this array can be compared against the values placed in the
 * hl_endianness enumeration.
 */
static constexpr uint8_t endianValues[4] = {0, 1, 2, 3};

/**
 * A simple function that can be used to help determine a program's endianness
 * at compile-time.
 */
constexpr hl_endianness getEndianOrder() {
    return
        (0x00 == endianValues[0])           // If Little Endian Byte Order,
            ? HL_LITTLE_ENDIAN              // return 0 for little endian.
            : (0x03 == endianValues[0])     // Else if Big Endian Byte Order,
                ? HL_BIG_ENDIAN             // return 1 for big endian.
                : (0x02 == endianValues[0]) // Else if PDP Endian Byte Order,
                    ? HL_PDP_ENDIAN         // return 2 for pdp endian.
                    : HL_UNKNOWN_ENDIAN;    // Else return -1 for wtf endian.
}

#define HL_ENDIANNESS getEndianOrder()

/*
 * Test program
 */
int main() {
    #if defined _WIN32 || defined _WIN64
        static_assert(
            HL_ENDIANNESS == HL_LITTLE_ENDIAN,
            "Aren't Windows programs Little-Endian?"
        );
    #endif

    constexpr hl_endianness endianness = HL_ENDIANNESS;

    std::cout << "This machine is: ";

    switch (endianness) {
        case HL_LITTLE_ENDIAN:
            std::cout << "LITTLE";
            break;
        case HL_BIG_ENDIAN:
            std::cout << "BIG";
            break;
        case HL_PDP_ENDIAN:
            std::cout << "PDP";
            break;
        case HL_UNKNOWN_ENDIAN:
        default:
            std::cout << "UNKNOWN";
    }

    std::cout << " endian" << std::endl;
}

Edit:

I can try dereferencing the pointer offset to the endianValues array but I'm still not sure if it will end up defaulting to 0 (the first explicitly set value in the array).

constexpr hl_endianness getEndianOrder() {
    return
        (0x00 == *endianValues)           // If Little Endian Byte Order,
            ? HL_LITTLE_ENDIAN              // return 0 for little endian.
            : (0x03 == *endianValues)     // Else if Big Endian Byte Order,
                ? HL_BIG_ENDIAN             // return 1 for big endian.
                : (0x02 == *endianValues) // Else if PDP Endian Byte Order,
                    ? HL_PDP_ENDIAN         // return 2 for pdp endian.
                    : HL_UNKNOWN_ENDIAN;    // Else return -1 for wtf endian.
}

Edit #2

So after looking into how the bits might be stored on different systems, I finally realized that I might be able to just use a single bit to test for endianness. It looks to be much less error-prone than using an array and I still get the correct answer on my Windows box.

enum hl_endianness : uint32_t {
    HL_LITTLE_ENDIAN   = 0x00000001,
    HL_BIG_ENDIAN      = 0x01000000,
    HL_PDP_ENDIAN      = 0x00010000,
    HL_UNKNOWN_ENDIAN  = 0xFFFFFFFF
};

/**
 * A simple function that can be used to help determine a program's endianness
 * at compile-time.
 */
constexpr hl_endianness getEndianOrder() {
    return
        ((1 & 0xFFFFFFFF) == HL_LITTLE_ENDIAN)
            ? HL_LITTLE_ENDIAN
            : ((1 & 0xFFFFFFFF) == HL_BIG_ENDIAN)
                ? HL_BIG_ENDIAN
                : ((1 & 0xFFFFFFFF) == HL_PDP_ENDIAN)
                    ? HL_PDP_ENDIAN
                    : HL_UNKNOWN_ENDIAN;
}

#define HL_ENDIANNESS getEndianOrder()
\$\endgroup\$
  • \$\begingroup\$ Or #include <boost/predef/detail/endian_compat.h> and then test for BOOST_LITTLE_ENDIAN or BOOST_BIG_ENDIAN or BOOST_PDP_ENDIAN \$\endgroup\$ – Martin York Mar 29 '14 at 17:56
  • 1
    \$\begingroup\$ @Loki I would rather not use Boost since this is also a personal exercise in learning how different computers process their data. Although if it was for production code then Boost would definitely not be a bad idea. \$\endgroup\$ – icdae Mar 29 '14 at 18:06
  • \$\begingroup\$ In C++20, you can use std::endian: wandbox.org/permlink/3FkeNpZx5Ix4PbdP. \$\endgroup\$ – Daniel Langr Aug 21 '18 at 8:04
3
\$\begingroup\$

Just a couple points:

  1. Endian-ness is not generally based on the Operating System but on the processor. For example, Intel x86 processors are little-endian regardless of it running Windows or Linux.

  2. Your code will always return HL_LITTLE_ENDIAN. Why? Because if

    static constexpr uint8_t endianValues[4] = {0, 1, 2, 3};
    

    then endianValues[0] == 0 will always be true! Suppose you had

    char x[4] = {'c','o','d', 'e'};
    

    Don't you think it would be shocking if x[0] == 'e' instead of x[0] == 'c'?

    The standard way is to use a union. Something like this:

    union endian_tester {
        uint32_t   n;
        uint8_t    p[4];
    };
    
    const endian_tester sample = {0x01020304}; // this initializes .n
    
    constexpr hl_endianness getEndianOrder() {
        return
            (0x04 == sample.p[0])               // If Little Endian Byte Order,
                ? HL_LITTLE_ENDIAN              
                : (0x01 == sample.p[0])         // Else if Big Endian Byte Order,
                    ? HL_BIG_ENDIAN             
                    : (0x02 == sample.p[0])     // Else if PDP Endian Byte Order,
                           ...(etc)...
    

    Be aware that constexpr isn't fully supported in my version of Visual Studio 2013 Express.

  3. Not clear to me why you need to use fancy values for HL_LITTLE_ENDIAN, HL_BIG_ENDIAN, etc. You can use 1, 2, etc instead of 0x03020100, 0x00010203, etc.

  4. A related question answered on Stack Overflow (Detecting endianness programmatically in a C++ program)

|improve this answer|||||
\$\endgroup\$
  • \$\begingroup\$ Some processors can run in either big- or little-endian mode, leaving the choice up to the operating system. However, Windows will probably only ever run in little-endian mode. \$\endgroup\$ – 200_success Mar 29 '14 at 8:05
  • \$\begingroup\$ I originally tried a union but got errors from G++ for undefined behavior. After setting the data for one member, then trying to access the other, I'll get errors that prevent the code from compiling. I figured the current method would be a safe bet, but completely overlooked how the data would be accessed through an array. The problem is that using behavior defined by the standard seems to limit how byte-level data can be access during compilation. Neither casting to another data type or bit-leven operations seem to work. \$\endgroup\$ – icdae Mar 29 '14 at 8:23
  • \$\begingroup\$ @icdae you're right; I think someone mentioned that the standard technically prevents setting a union with one field but accessing it with a different one. But many compilers allow it. In that Stack Overflow link, there is one answer that uses the idea of casting int to char which might work better (but you should try uint32_t instead of int). \$\endgroup\$ – Apprentice Queue Mar 29 '14 at 16:37
  • \$\begingroup\$ Your union "standard way" has undefined behavior according to the Standard. \$\endgroup\$ – Ruslan Nov 15 '15 at 10:36
  • 3
    \$\begingroup\$ -1. Using unions for type punning in C++ is undefined behavior. \$\endgroup\$ – Lyberta Jul 1 '16 at 8:43
2
\$\begingroup\$

You have to use predefined compiler macros (\__BIG_ENDIAN__ or \__LITTLE_ENDIAN__ with clang, or \__BYTE_ORDER__ with gcc).

The other compiler macro tricks mentioned will only detect the endianness of the architecture you are compiling on, not the endianness of the architecture you are compiling for, so something like this is wrong:

\#define IS_BIG_ENDIAN ('\x01\x02\x03\x04' == 0x01020304)
|improve this answer|||||
\$\endgroup\$
0
\$\begingroup\$

I would change this:

static constexpr uint8_t endianValues[4] = {0, 1, 2, 3};

To this:

static const uint32_t value = HL_LITTLE_ENDIAN; // 0x03020100
static const uint8_t* endianValues = (uint8_t*)&value;

Alternatively, you can change the getEndianOrder function to read the endianValues array as a uint32, but you will have to add a preprocessor directive (#pragma) to make sure that it is placed in a memory address aligned to 4 bytes (and that by itself might yield some platform-dependency issues, which is pretty much in contrast with your goal here to begin with):

constexpr hl_endianness getEndianOrder()
{
    switch (*(uint32_t*)endianValues))
    {
        case HL_LITTLE_ENDIAN: return HL_LITTLE_ENDIAN;
        case HL_BIG_ENDIAN:    return HL_BIG_ENDIAN;
        case HL_PDP_ENDIAN:    return HL_PDP_ENDIAN;
    }
    return HL_UNKNOWN_ENDIAN;
}
|improve this answer|||||
\$\endgroup\$
  • \$\begingroup\$ I tried your solution (had to remove the switch statement to make it a constexpr function) but I believe this will always return as little endian. Since the value stored in HL_LITTLE_ENDIAN is how it will actually be stored in memory on all computers, retrieving an 8-bit pointer to value is always going to grab the same value of 0x03. I tried another solution though end edited it into the original question. \$\endgroup\$ – icdae Mar 29 '14 at 18:17
  • \$\begingroup\$ @icdae: It's not an 8-bit pointer, it's a 32-bit pointer (look inside the switch statement that you removed). Though I have to admit, I'm not really sure how it is applied into a constexpr during compile-time. \$\endgroup\$ – barak manos Mar 29 '14 at 18:21
  • \$\begingroup\$ Oops, I misspoke (just woke up, everything's a bit fuzzy); being a pointer to a constant value won't change how the value itself is stored in memory. The function will always return the same result since value = HL_LITTLE_ENDIAN. As for the switch statement, one of the restrictions for a constexpr statement is that the body of the function can only consist of a single return statement. In this case the switch can be converted to a simple series of ternary operators. \$\endgroup\$ – icdae Mar 29 '14 at 18:37
  • \$\begingroup\$ @icdae: Please note the emphasis on the word Alternatively, which means that you should leave endianValues[4] = {0, 1, 2, 3} as is, but read it using *(uint32_t*)endianValues). The value variable is meant to be used only in the first option suggested. \$\endgroup\$ – barak manos Mar 29 '14 at 19:17
  • \$\begingroup\$ -1, this violates strict aliasing rules. \$\endgroup\$ – Lyberta Jul 1 '16 at 8:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.