Timeline for C/C++ hash looking in flat database compared to PHP
Current License: CC BY-SA 4.0
7 events
| when toggle format | what | by | license | comment | |
|---|---|---|---|---|---|
| Nov 22, 2022 at 21:00 | comment | added | James | I tried to use fread with chunks of 8k/16k in C, it did improve a bit performance, I'm now at 1.4/1.5s went from 1.7/1.8s. On par with PHP perf, but PHP is simpler to code for me. I guess bottleneck is file IO now. | |
| Nov 22, 2022 at 20:02 | comment | added | Toby Speight |
Small factual error: fgets() and std::getline() don't necessarily make system calls every time they are called - implementations generally buffer standard input and make larger reads than they will immediately use.
|
|
| Nov 22, 2022 at 19:32 | comment | added | James | Well I was talking about an array for PHP, but maybe there's a way to store every couple of hexa 00 to FF as constants hard coded in the C code to avoid this line : decimal = (unsigned char)strtoul(offsetHexa,NULL,16); Also I tried to read file with chunks of 4096 and 8192 bytes, the PHP code went from 2.5s average to 1.6s which is a pretty good improvement :) I will try for the C/C++ | |
| Nov 22, 2022 at 18:49 | comment | added | pacmaninbw♦ |
@James In some cases in C and C++ the optimizing compiler might inline functions. Not sure what the array will do for you.
|
|
| Nov 22, 2022 at 18:26 | comment | added | James | Second comment sorry. I tried one thing in the PHP code. Since there's only 256 possible values for the hexadecimal couple $hash[4].$hash[5], I did an array with key being 00 to FF and value being hexdec($hexa) * 8 to avoid computing this each time, but for a reason I don't know this seems to be slower than hexdec call each time. May that be a good improvement for C/C++ versions or it doesn't matter in terms of performance ? | |
| Nov 22, 2022 at 18:22 | comment | added | James | Thanks a lot for this very detailed answer ! I will try to get chunks of data instead of one at a time. I did try to improve the algorithm by replacing the call to fseek/fread by file() call in the PHP version to load the entire file into memory, it's actually equally fast but I think it won't be with larger files. I didn't make any function because function calls are expensive in PHP, so I decided to do the same for C/C++ but I guess the rules may not apply the same. I miss a lot of best practice that I will try to add to my habits. I'll try to read chunks of data and will return with results | |
| Nov 22, 2022 at 17:47 | history | answered | pacmaninbw♦ | CC BY-SA 4.0 |