# Converting Array of Float32 (float) to Array of UINT8 (unsigned char) Using AVX2 [closed]

Given input array of Float32 (float) with numElements how could one efficiently convert it into array of UINT8 (unsigned char)?
The tricky part here is to apply Unsigned Saturation in the conversion

For instance, here is a vanilla code for it (Pay attention there is a scaling operation):

void ConvertToUint8(unsigned char* mO, float* mI, int numElements, float scalingFctr)
{
int ii;
for (ii = 0; ii < numElements; ii++) {
mO[ii] = (unsigned char)(fmin(fmax(mI[ii] * scalingFctr, 0.0), 255.0));
}
}


Where mO is the output array.

I'm looking for a way to optimize (Performance wise) this code on AVX2 enabled CPU's. Any idea, Intrinsics included, is welcome.

Pay attention the above code apply unsigned saturation manually (Is there a function for unsigned saturation based casting in C?). I think in practice SSE and AVX have it built in (See _mm_packus_epi16() for SSE).

The objective is to yield faster code than the vanilla example as in Compiler Explorer - ConvertToUint8.

For simplicity one could assume the arrays are aligned.

• Can the scale factor put the resulting float outside of the range of an int as well? That would be more annoying Commented Apr 26, 2019 at 11:21
• Yes. You should assume mI[ii] * scalingFctr can have any legit Float32 value (But not NAN or INF). I think in SSE the intrinsic _mm_packus_epi16 does the trick
– Royi
Commented Apr 26, 2019 at 11:22
• I don't think it works out so nicely, vcvttps2dq produces INT_MIN for out of range floats (including large positive), and then a pack-with-unsigned-saturation still interprets that as INT_MIN so it would result in zero, but maybe we wanted 0xFF. So it gets trickier Commented Apr 26, 2019 at 11:29
• Harold, I tried to define the code more accurately. I mean that values after scaling which are lower than 0 will be clipped into zero and values above 255 will be clipped into 255.
– Royi
Commented Apr 26, 2019 at 11:34
• C doesn't have a saturation cast operator built-in. Commented Apr 27, 2019 at 19:31

Harold's comment is correct.

Consider what happens for float inputs like 5000000000 * 1.0. Conversion to int32_t with cvtps2dq will give you -2147483648 from that out-of-range positive float. (2's complement integer bit-pattern 0x80000000 is the "indefinite integer value" described by Intel's documentation for this case.)

In that case, your vectorized version that clamps via integer saturation will start with a negative (and ultimately do unsigned saturation to 0), not matching your fmin which clamps before even converting to integer, resulting in 255.

So you have to be able to rule out such inputs if you want to vectorize without clamping in the FP domain before conversion to integer.

Remember that IEEE754 binary32 float can represent values outside the range of int32_t or int64_t, and what x86 FP->int conversions do in that case.

• I see. In my case you can assume the range of values is within the range of INT32.
– Royi
Commented Apr 27, 2019 at 19:27

This is a tryout I cam up with - ConvertToUint8():

#include <immintrin.h> // AVX & AVX2

#define AVX_STRIDE          8
#define AVX_STRIDE_DOUBLE   16
#define AVX_STRIDE_TRIPLE   24

void ConvertToUint8(unsigned char* mO, float* mI, int numElements, float scalingFctr)
{
int ii;
float *ptrInputImage;
int *ptrOutputImage;

__m256  floatPx1, floatPx2, floatPx3, floatPx4;
__m256  scalingFactor;
__m256i int32Px1, int32Px2, int32Px3, int32Px4;
__m256i uint8Px1, uint8Px2;
__m256i *ptrOutputImageAvx;

for (ii = 0; ii < numElements; ii += AVX_STRIDE_QUAD) {
ptrInputImage       = mI;
ptrOutputImageAvx   = (__m256i*)(mO);
// AVX Pack is 8 Floats (8 * 32 Bit) -> 32 UINT8 (32 * 8 Bit)
// Hence loading 4 * 8 Floats which will be converted into 32 UINT8

// See https://stackoverflow.com/questions/51778721
int32Px1 = _mm256_cvtps_epi32(_mm256_mul_ps(floatPx1, scalingFactor)); // Converts the 8 SP FP values of a to 8 Signed Integers (32 Bit).
int32Px2 = _mm256_cvtps_epi32(_mm256_mul_ps(floatPx2, scalingFactor));
int32Px3 = _mm256_cvtps_epi32(_mm256_mul_ps(floatPx3, scalingFactor));
int32Px4 = _mm256_cvtps_epi32(_mm256_mul_ps(floatPx4, scalingFactor));
uint8Px1 = _mm256_packs_epi32(uint16Px1, uint16Px2); // Saturating and packing 2 of 8 Integers into 16 of INT16
uint8Px2 = _mm256_packs_epi32(uint16Px3, uint16Px4); // Saturating and packing 2 of 8 Integers into 16 of INT16
uint8Px1 = _mm256_packus_epi16(uint8Px1, uint8Px2); // Saturating and packing 2 of 16 INT16 into 32 of UINT8
uint8Px1 = _mm256_permutevar8x32_epi32(uint8Px1, _mm256_setr_epi32(0, 4, 1, 5, 2, 6, 3, 7)); // Permitation for Linear Orderinmg
_mm256_storeu_si256(ptrOutputImageAvx++, uint8Px1); // Storing 32 UINT8, Promoting the pointer

}
}


The code is based on answer of Peter Cordes - How to Convert 32 [Bit] Float to 8 [Bit] Signed char?
I'd love to hear thoughts about it.

• This looks like the code you actually want a review of. You should put this in your question. Your current codereview question is written like an SO question, asking for someone to vectorize it. It should be written to ask for a review of your vectorization of it, with the scalar version working as documentation of what the vectorized code does. Commented Apr 27, 2019 at 16:49
• I revised my question objective to match the site rules. So I'm not explicitly asks for intrinsics but open to any code that improve the performance of the vanilla code on AVX2 enabled CPU's.
– Royi
Commented Apr 27, 2019 at 19:29
• You seem to be missing the point of code review, and still trying to use it like Stack Overflow without even posting your own attempt at an optimized implementation. There's very little to review in the code you posted as the question, as far as style or performance, because you're not even claiming it's an actual finished attempt at a good implementation. Commented Apr 27, 2019 at 19:33
• @PeterCordes, Let's play a game. I'm a student on a university. You are my University Research Software Engineer. I tell you I have a code which runs not fast enough on my research computer which has AVX2. I show you the code above and ask you to review it. What would you suggest me doing? On top of that, if you show me where my question doesn't fulfill the community guidelines I will vote closing it as well. On top of that, I don't understand why people even bother with it. Someone asks for assistance, can you assist?
– Royi
Commented Apr 27, 2019 at 22:31
• It's not a codereview question because the answer you're really hoping/asking for is a complete rewrite using intrinsics. You know the implementation in the question isn't what you want or close to efficient on any compiler. Everything about how you present the problem is written as a StackOverflow question (which is a duplicate of How to convert 32-bit float to 8-bit signed char? because there isn't a faster way to multiply). Unless your inputs and scaling factor are both exact integers, in which case you could use 16-bit integer multiply. Commented Apr 27, 2019 at 23:29