Skip to main content
Corrected assertion that fgetln is startard C
Source Link
William Morris
  • 9.2k
  • 18
  • 42

For completeness, I tried a few alternative ways of doing the file access. I tried your fgets and the suggested alternatives from other answers.

On my system, using fgets, which you used, is about the same speed as using getline. Using fscanf is about 3x slower than these solutions. The fastest method I tested was to use fgetln (standardavailable on a Mac from the BSD C library), which was 10-15% faster than fgets/getline.

char *line = NULL;
size_t len = 0;

while ((line = fgetln(f, &len)) != NULL) {
    if (line[len - 1] == '\n') {
        line[len - 1] = '\0';
    }
    //printf("\"%s\"\n", line);
}
if (ferror(f)) {
    perror("fgetln");
}

Another alternative is to use mmap to map the file (only real files though, not pipes, etc) into memory and use string handling directly (strchr etc) to look for line endings. This might be the fastest of all but I didn't test it here.

Generally it is best not to get too hung up about speed unless you know that a piece of code is critical (and normally one doesn't know this). So think more about the advantages and disadvantages of the various solutions:

  • fgets is quick but uses a fixed buffer length and you have to handle overflow as a special case.

  • getline and fgetln are quick, elegant and capture complete lines. These are attractive if we do not need to worry about attacks by huge strings. fgetln is not in the Cany standard so is slightly morerather less portable than getline (which seems to be POSIX). I tried fgetln with a file containing one line > INT_MAX chars long (INT_MAX is the limit that fgetln handles) and although it did slow down the system considerably, it failed cleanly with a memory allocation failure:

      fgetln: Cannot allocate memory
    

    getline has a limit of SSIZE_MAX which is much bigger.

  • fscanf - I can't think of a reason to use this for the simple example you pose, but it has its uses if you want to control the input format.

  • mmap is (probably) quick and elegant but only handles real files, which restricts the application a bit (eg you can't just pipe input into the application if it wants to map a file).

For completeness, I tried a few alternative ways of doing the file access. I tried your fgets and the suggested alternatives from other answers.

On my system, using fgets, which you used, is about the same speed as using getline. Using fscanf is about 3x slower than these solutions. The fastest method I tested was to use fgetln (standard C library), which was 10-15% faster than fgets/getline.

char *line = NULL;
size_t len = 0;

while ((line = fgetln(f, &len)) != NULL) {
    if (line[len - 1] == '\n') {
        line[len - 1] = '\0';
    }
    //printf("\"%s\"\n", line);
}
if (ferror(f)) {
    perror("fgetln");
}

Another alternative is to use mmap to map the file (only real files though, not pipes, etc) into memory and use string handling directly (strchr etc) to look for line endings. This might be the fastest of all but I didn't test it here.

Generally it is best not to get too hung up about speed unless you know that a piece of code is critical (and normally one doesn't know this). So think more about the advantages and disadvantages of the various solutions:

  • fgets is quick but uses a fixed buffer length and you have to handle overflow as a special case.

  • getline and fgetln are quick, elegant and capture complete lines. These are attractive if we do not need to worry about attacks by huge strings. fgetln is in the C standard so is slightly more portable than getline. I tried fgetln with a file containing one line > INT_MAX chars long (INT_MAX is the limit that fgetln handles) and although it did slow down the system considerably, it failed cleanly with a memory allocation failure:

      fgetln: Cannot allocate memory
    

    getline has a limit of SSIZE_MAX which is much bigger.

  • fscanf - I can't think of a reason to use this for the simple example you pose, but it has its uses if you want to control the input format.

  • mmap is (probably) quick and elegant but only handles real files, which restricts the application a bit (eg you can't just pipe input into the application if it wants to map a file).

For completeness, I tried a few alternative ways of doing the file access. I tried your fgets and the suggested alternatives from other answers.

On my system, using fgets, which you used, is about the same speed as using getline. Using fscanf is about 3x slower than these solutions. The fastest method I tested was to use fgetln (available on a Mac from the BSD C library), which was 10-15% faster than fgets/getline.

char *line = NULL;
size_t len = 0;

while ((line = fgetln(f, &len)) != NULL) {
    if (line[len - 1] == '\n') {
        line[len - 1] = '\0';
    }
    //printf("\"%s\"\n", line);
}
if (ferror(f)) {
    perror("fgetln");
}

Another alternative is to use mmap to map the file (only real files though, not pipes, etc) into memory and use string handling directly (strchr etc) to look for line endings. This might be the fastest of all but I didn't test it here.

Generally it is best not to get too hung up about speed unless you know that a piece of code is critical (and normally one doesn't know this). So think more about the advantages and disadvantages of the various solutions:

  • fgets is quick but uses a fixed buffer length and you have to handle overflow as a special case.

  • getline and fgetln are quick, elegant and capture complete lines. These are attractive if we do not need to worry about attacks by huge strings. fgetln is not in any standard so is rather less portable than getline (which seems to be POSIX). I tried fgetln with a file containing one line > INT_MAX chars long (INT_MAX is the limit that fgetln handles) and although it did slow down the system considerably, it failed cleanly with a memory allocation failure:

      fgetln: Cannot allocate memory
    

    getline has a limit of SSIZE_MAX which is much bigger.

  • fscanf - I can't think of a reason to use this for the simple example you pose, but it has its uses if you want to control the input format.

  • mmap is (probably) quick and elegant but only handles real files, which restricts the application a bit (eg you can't just pipe input into the application if it wants to map a file).

Source Link
William Morris
  • 9.2k
  • 18
  • 42

For completeness, I tried a few alternative ways of doing the file access. I tried your fgets and the suggested alternatives from other answers.

On my system, using fgets, which you used, is about the same speed as using getline. Using fscanf is about 3x slower than these solutions. The fastest method I tested was to use fgetln (standard C library), which was 10-15% faster than fgets/getline.

char *line = NULL;
size_t len = 0;

while ((line = fgetln(f, &len)) != NULL) {
    if (line[len - 1] == '\n') {
        line[len - 1] = '\0';
    }
    //printf("\"%s\"\n", line);
}
if (ferror(f)) {
    perror("fgetln");
}

Another alternative is to use mmap to map the file (only real files though, not pipes, etc) into memory and use string handling directly (strchr etc) to look for line endings. This might be the fastest of all but I didn't test it here.

Generally it is best not to get too hung up about speed unless you know that a piece of code is critical (and normally one doesn't know this). So think more about the advantages and disadvantages of the various solutions:

  • fgets is quick but uses a fixed buffer length and you have to handle overflow as a special case.

  • getline and fgetln are quick, elegant and capture complete lines. These are attractive if we do not need to worry about attacks by huge strings. fgetln is in the C standard so is slightly more portable than getline. I tried fgetln with a file containing one line > INT_MAX chars long (INT_MAX is the limit that fgetln handles) and although it did slow down the system considerably, it failed cleanly with a memory allocation failure:

      fgetln: Cannot allocate memory
    

    getline has a limit of SSIZE_MAX which is much bigger.

  • fscanf - I can't think of a reason to use this for the simple example you pose, but it has its uses if you want to control the input format.

  • mmap is (probably) quick and elegant but only handles real files, which restricts the application a bit (eg you can't just pipe input into the application if it wants to map a file).