Skip to main content
added 546 characters in body
Source Link
Your Common Sense
  • 8.8k
  • 1
  • 21
  • 51

Initially I wrote an answer related to PHP part, suggesting the usage of dedicated functions for reading and writing csv, fgetcsv() and fputcsv() respectively, but I didn't test the code. Thanks to @akki who pointed out to some errors and the profiling results, I learned that these functions are dramatically slower, as explained in this answer. A code using then run 20It looks like fgetcsv() is 40 times slower than fread/explode. However, to parse the code fromproper csv, with field delimiters and escaping, you have to use the OPproper function anyway.

So we can conclude that the PHP partHere is already optimized and hardly can be improved performance-wisethe code

<?php

$t = microtime(1);
$i_fp = fopen("inFile.csv","r");
while(!feof($i_fp)) {
    $line = fgets($i_fp);
    $split = explode(';',$line);
}
echo "fgets: ".round(microtime(1)-$t,2)."\n";    
$t = microtime(1);
$i_fp = fopen("inFile.csv","r");
while (($split = fgetcsv($i_fp, 512, ';')) !== FALSE)
{
}
echo "fgetcsv: ".round(microtime(1)-$t,2)."\n";    

outputs for me

fgets: 2.1
fgetcsv: 84.45

on a file with 1 mil rows

Initially I wrote an answer related to PHP part, suggesting the usage of dedicated functions for reading and writing csv, fgetcsv() and fputcsv() respectively, but I didn't test the code. Thanks to @akki who pointed out to some errors and the profiling results, I learned that these functions are dramatically slower, as explained in this answer. A code using then run 20 times slower than the code from the OP.

So we can conclude that the PHP part is already optimized and hardly can be improved performance-wise

Initially I wrote an answer related to PHP part, suggesting the usage of dedicated functions for reading and writing csv, fgetcsv() and fputcsv() respectively, but I didn't test the code. Thanks to @akki who pointed out to some errors and the profiling results, I learned that these functions are dramatically slower, as explained in this answer. It looks like fgetcsv() is 40 times slower than fread/explode. However, to parse the proper csv, with field delimiters and escaping, you have to use the proper function anyway.

Here is the code

<?php

$t = microtime(1);
$i_fp = fopen("inFile.csv","r");
while(!feof($i_fp)) {
    $line = fgets($i_fp);
    $split = explode(';',$line);
}
echo "fgets: ".round(microtime(1)-$t,2)."\n";    
$t = microtime(1);
$i_fp = fopen("inFile.csv","r");
while (($split = fgetcsv($i_fp, 512, ';')) !== FALSE)
{
}
echo "fgetcsv: ".round(microtime(1)-$t,2)."\n";    

outputs for me

fgets: 2.1
fgetcsv: 84.45

on a file with 1 mil rows

Post Undeleted by Your Common Sense
deleted 458 characters in body
Source Link
Your Common Sense
  • 8.8k
  • 1
  • 21
  • 51

As this is a codereview, let's review the PHP part as well.

Initially I doubt these improvements will add a lot of speed but they will definitely addwrote an answer related to the main benefits of PHP code - the clarity and conciseness. And the time and effort spent for writing the code to dopart, suggesting the job

<?php
$i_fp = fopen("inFile.csv","r");
$o_fp = fopen("outFile.csv","w");

while (($split = fgetcsv($i_fp, 0, ';')) !== FALSE)
{
    if(isset($split[3][0]) && ($split[3][0] == 'E' || $split[3][0] == 'T'))
    {
        fputcsv($o_fp, [
            $split[0], $split[1], $split[3], $split[4], $split[5], $split[6],
            $split[10], $split[9],$split[11],$split[7],$split[32]
        ]);
    }
}

Here we are just usingusage of dedicated functions for reading and writing csv, fgetcsv() and also omitting the fclose() calls because PHP will happily close all the handlers for usfputcsv() respectively, but I didn't test the code. Thanks to @akki who pointed out to some errors and also the closing tagprofiling results, I learned that these functions are dramatically slower, as it's usage is discouragedexplained in this answer. A code using then run 20 times slower than the code from the OP.

Edit: thanks to @akki, several errors have been fixedSo we can conclude that the PHP part is already optimized and hardly can be improved performance-wise

As this is a codereview, let's review the PHP part as well.

I doubt these improvements will add a lot of speed but they will definitely add to the main benefits of PHP code - the clarity and conciseness. And the time and effort spent for writing the code to do the job

<?php
$i_fp = fopen("inFile.csv","r");
$o_fp = fopen("outFile.csv","w");

while (($split = fgetcsv($i_fp, 0, ';')) !== FALSE)
{
    if(isset($split[3][0]) && ($split[3][0] == 'E' || $split[3][0] == 'T'))
    {
        fputcsv($o_fp, [
            $split[0], $split[1], $split[3], $split[4], $split[5], $split[6],
            $split[10], $split[9],$split[11],$split[7],$split[32]
        ]);
    }
}

Here we are just using dedicated functions for reading and writing csv, and also omitting the fclose() calls because PHP will happily close all the handlers for us, and also the closing tag as it's usage is discouraged.

Edit: thanks to @akki, several errors have been fixed

Initially I wrote an answer related to PHP part, suggesting the usage of dedicated functions for reading and writing csv, fgetcsv() and fputcsv() respectively, but I didn't test the code. Thanks to @akki who pointed out to some errors and the profiling results, I learned that these functions are dramatically slower, as explained in this answer. A code using then run 20 times slower than the code from the OP.

So we can conclude that the PHP part is already optimized and hardly can be improved performance-wise

Post Deleted by Your Common Sense
added 86 characters in body
Source Link
Your Common Sense
  • 8.8k
  • 1
  • 21
  • 51

As this is a codereview, let's review the PHP part as well.

I doubt these improvements will add a lot of speed but they will definitely add to the main benefits of PHP code - the clarity and conciseness. And the time and effort spent for writing the code to do the job

<?php
$i_fp = fopen("inFile.csv","r");
$o_fp = fopen("outFile.csv","w");

while (($split = fgetcsv($i_fp, 0, ';')) !== FALSE) {
{
    if($split[3]isset($split[3][0]) && ($split[3][0] == 'E' || $split[3]$split[3][0] == 'T'))
    {
        fputcsv($o_fp, [
            $split[0], $split[1], $split[3], $split[4], $split[5], $split[6],
            $split[10], $split[9],$split[11],$split[7],$split[32]
        ]);
    }
}

Here we are just using dedicated functions for reading and writing csv, and also omitting the fclose() calls because PHP will happily close all the handlers for us, and also the closing tag as it's usage is discouraged.

Edit: thanks to @akki, several errors have been fixed

As this is a codereview, let's review the PHP part as well.

I doubt these improvements will add a lot of speed but they will definitely add to the main benefits of PHP code - the clarity and conciseness. And the time and effort spent for writing the code to do the job

<?php
$i_fp = fopen("inFile.csv","r");
$o_fp = fopen("outFile.csv","w");

while (($split = fgetcsv($i_fp, 0, ';')) !== FALSE) {
{
    if($split[3] == 'E' || $split[3] == 'T')
    {
        fputcsv($o_fp, [
            $split[0], $split[1], $split[3], $split[4], $split[5], $split[6],
            $split[10], $split[9],$split[11],$split[7],$split[32]
        ]);
    }
}

Here we are just using dedicated functions for reading and writing csv, and also omitting the fclose() calls because PHP will happily close all the handlers for us, and also the closing tag as it's usage is discouraged.

As this is a codereview, let's review the PHP part as well.

I doubt these improvements will add a lot of speed but they will definitely add to the main benefits of PHP code - the clarity and conciseness. And the time and effort spent for writing the code to do the job

<?php
$i_fp = fopen("inFile.csv","r");
$o_fp = fopen("outFile.csv","w");

while (($split = fgetcsv($i_fp, 0, ';')) !== FALSE)
{
    if(isset($split[3][0]) && ($split[3][0] == 'E' || $split[3][0] == 'T'))
    {
        fputcsv($o_fp, [
            $split[0], $split[1], $split[3], $split[4], $split[5], $split[6],
            $split[10], $split[9],$split[11],$split[7],$split[32]
        ]);
    }
}

Here we are just using dedicated functions for reading and writing csv, and also omitting the fclose() calls because PHP will happily close all the handlers for us, and also the closing tag as it's usage is discouraged.

Edit: thanks to @akki, several errors have been fixed

added 221 characters in body
Source Link
Your Common Sense
  • 8.8k
  • 1
  • 21
  • 51
Loading
Source Link
Your Common Sense
  • 8.8k
  • 1
  • 21
  • 51
Loading