4
\$\begingroup\$

This is a follow up to Find the Size of a File in a Portable Manner.

What's new:

  • The fast version (POSIX stat()) does not change the position indicator of the stream at start of file.
  • The portable version (fread()) does not leave the position indicator of the stream at the end of the file. (It saves the original position indicator and restores it before returning).
  • The fast version now checks for different types of files (failing at times and setting errno accordingly).
  • New macros (S_ISREG, S_ISDIR et cetera) because Microsoft does not define them.
  • The read buffer size has been reduced to 16K (as the minimum bit size of size_t may only be 16 bits).
  • The fast version now checks for _POSIX_C_SOURCE instead of trying to detect most UNIX-like operating systems with the respective macros they claim to define.

Code:

This goes in the header:

#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>

/* `size` should be a non-null pointer. On success, the function assigns `size`
 * with the number of bytes read and returns `true`, or returns `false` elsewise.
 *
 * If an overflow will occur, returns `false` and sets `errno` to `EFBIG` 
 * (if defined).
 * If an error occurs on `stream`, returns `false`. `ferror()` can be used to
 * check for this.
 * If `stream` refers to a directory, returns `false` and sets `errno` to 
 * `EISDIR` (if defined).
 * For other file types (besides symbolic links), returns `false` and sets 
 * `errno` to `EPERM` (if defined).
 *
 * Note: The size of a file is a transient thing - by the time the information 
 *       is used, it may well be out of date. For instance, it can grow between
 *       `io_fsize()` and a subsequent read.  */
bool io_fsize(FILE *stream, uintmax_t *size) 
    ATTRIB_NONNULL(1, 2) ATTRIB_WARN_UNUSED_RESULT;

where ATTRIB_NONNULL and ATTRIB_WARN_UNUSED_RESULT expand to GNU attributes of the same name (if available), or nothing.

And this goes in the corresponding source file:

#include <errno.h>
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>

#ifdef _POSIX_C_SOURCE
    #define HAVE_STAT 1
    #include <sys/stat.h>
#endif  /* _POSIX_C_SOURCE */

/* Reasons to not use `fseek()` and `ftell()` to compute the size of the file:
 * 
 * Subclause 7.12.9.2 of the C Standard [ISO/IEC 9899:2011] specifies the
 * following behavior when opening a binary file in binary mode:
 * 
 * >> A binary stream need not meaningfully support fseek calls with a whence 
 * >> value of SEEK_END.
 *
 * In addition, footnote 268 of subclause 7.21.3 says:
 *
 * >> Setting the file position indicator to end-of-file, as with 
 * >> fseek(file, 0, SEEK_END) has undefined behavior for a binary stream.
 *
 * For regular files, the file position indicator returned by `ftell()` is 
 * useful only in calls to `fseek()`. As such, the value returned may not
 * reflect the physical byte offset. */
bool io_fsize(FILE *stream, uintmax_t *size)
{
/*   Windows supports `fileno()`, `struct stat`, and `fstat()` as `_fileno()`,
 *   `_fstat()`, and `struct _stat`.
 *
 *   See: https://learn.microsoft.com/en-us/cpp/c-runtime-library/reference/fstat-fstat32-fstat64-fstati64-fstat32i64-fstat64i32?view=msvc-170 */
#ifdef _WIN32
    #define HAVE_STAT 1
    
    /* Windows does not define S_ISREG, S_ISLNK, and S_ISDIR macros in stat.h,
     * so we do. We have to define _CRT_INTERNAL_NONSTDC_NAMES before including
     * sys/stat.h in order for Microsoft's stat.h to define names like S_IFMT,
     * S_IFDIR, and S_IFREG et cetera as it normally does. */
    #define _CRT_INTERNAL_NONSTDC_NAMES 1
    #include <sys/stat.h>

    #ifndef S_ISDIR
        #define S_ISDIR(m)  (((m) & S_IFMT) == S_IFDIR) 
    #endif  /* S_ISDIR */

    #ifndef S_ISREG
        #define S_ISREG(m)  (((m) & S_IFMT) == S_IFREG)
    #endif  /* S_ISREG */
    
    #ifdef S_IFLNK
        #ifndef S_ISLNK
            #define S_ISLNK(m)  (((m) & S_IFMT) == S_IFLNK)
        #endif  /* S_ISLINK */
    #else
        #define S_ISLNK(m)  (0)
    #endif  /* S_IFLNK */
    
    #define fileno _fileno

    #ifdef _WIN64
        #define fstat  _fstat64
        #define stat   __stat64
    #else
        /* Does this suffice for a 32-bit system? */
        #define fstat  _fstat
        #define stat   _stat
    #endif    /* _WIN64 */
#endif    /* _WIN32 */

#ifdef HAVE_STAT
    struct stat st;

    if (fstat(fileno(stream), &st) == 0) {
        if (S_ISDIR(st.st_mode)) {
            #ifdef EISDIR
            errno = EISDIR;
            #endif
            return false;
        } else if (!S_ISREG(st.st_mode) || !S_ISLNK(st.st_mode)) {
            #ifdef EPERM
            errno = EPERM;
            #endif
            return false;
        }

        *size = (uintmax_t) st.st_size;
        return true;
    }
    return false;
#else
    /* Fall back to the default and read it in chunks. */
    uintmax_t rcount = 0;

    /* 16K has the advantage of working on systems where `size_t` is 16-bit
     * with only nominal performance loss. */
    char chunk[16 * 1024];  

    /* Save the original file position indicator. We'd restore this before 
     * returning. */
    const long orig_pos = ftell(stream);

    do {
        rcount = fread(chunk, 1, sizeof chunk, stream);

        if ((*size + rcount) < *size) {
            /* Overflow. */
            #ifdef EFBIG
            errno = EFBIG;
            #endif
            return false;
        }
        *size += rcount;
    } while (rcount == sizeof chunk);

    fseek(stream, -orig_pos, SEEK_CUR);
    return !ferror(stream);
#endif      /* HAVE_STAT */ 
#undef fstat
#undef stat
#undef fileno
#undef HAVE_STAT
}

I see there being a problem with symbolic links if S_IFLNK is not defined in Windows.

Blame Microsoft for this mess.

\$\endgroup\$
4
  • \$\begingroup\$ A bunch of #endifs is missing. \$\endgroup\$
    – vnp
    May 14 at 23:54
  • \$\begingroup\$ @vnp I've added them. Thank you. \$\endgroup\$
    – Harith
    May 15 at 3:21
  • \$\begingroup\$ And sorry for not catching it right away, IO_CHUNK_SIZE is still undefined. \$\endgroup\$
    – vnp
    May 15 at 3:35
  • \$\begingroup\$ @vnp Oops, that must be sizeof chunk, it is fixed now. Thank you. \$\endgroup\$
    – Harith
    2 days ago

2 Answers 2

1
\$\begingroup\$

Incorrect count

When file is in text_mode and uses the default approach, the file size can readily be wrong due to \n --> \r\n conversion and other reasons. Consider when default mode is used with a text stream, re-open in binary.

Return inconsistency

Comments have "the function assigns size with the number of bytes read and returns true, or returns false elsewise."

Code ends with return !ferror(stream);

So if the stream error flag was set before the beginning of the function call, function can return false even if the operation was successful. Alternative: return false if feof() not true.

If ftell() or fseek() fail, code can return true as their results are not checked. Better to test them.

Testable?

if ((*size + rcount) < *size) { is a check for 64-bit (or wider math) overflow. How do you plan to test this? Untestable code is of reduced value.

Range?

st_size is type off_t.

off_t is a "signed integer type."

(uintmax_t) st.st_size; returns a huge incorrect number when the file size is negative. Perhaps this is not possible? Better to error out when .st_size < 0.

Wrong way to restore position

In text mode, fseek(stream, -orig_pos, SEEK_CUR); is not reliable.

Use fseek(stream, orig_pos, SEEK_SET)

Small systems

When SIZE_MAX is small, even a 16K buffer is sizeable. Consider BUFSIZ from <stdio.h>.

\$\endgroup\$
5
  • \$\begingroup\$ "if ((*size + rcount) < *size) { is a check for 64-bit (or wider math) overflow. How do you plan to test this? Untestable code is of reduced value." ==> Am I to remove the check then? \$\endgroup\$
    – Harith
    yesterday
  • \$\begingroup\$ "Consider when default mode is used with a text stream, re-open in binary." ==> How do I determine which mode stream was opened with? Or do I unconditionally freopen() in binary mode? \$\endgroup\$
    – Harith
    yesterday
  • \$\begingroup\$ cppreference.com says that if the first argument to freopen() is a nullptr, Windows treats it as an error. But it provides _set_mode(). Would this: #if _WIN32 _setmode(_fileno(stream), _O_BINARY); #endif do? \$\endgroup\$
    – Harith
    yesterday
  • \$\begingroup\$ @Harith, C spec has "If filename is a null pointer, the freopen function attempts to change the mode of the stream to that specified by mode, as if the name of the file currently associated with the stream had been used." "cppreference.com says that if the first argument to freopen() is a nullptr, Windows treats it as an error" is unclear as cppreference.com says the same as the C spec. \$\endgroup\$ yesterday
  • \$\begingroup\$ It is mentioned in the "Notes" section: "Microsoft CRT version of freopen does not support any mode changes when filename is a null pointer and treats this as an error (see documentation. A possible workaround is the non-standard function _setmode()." \$\endgroup\$
    – Harith
    yesterday
2
\$\begingroup\$
  • There is no need to #undef everything. You are not offering to #include this file.

  • In case (*size + rcount) < *size the original file position is not restored.

  • It is very unclear what the expected result should be if fd corresponds to a tty, to a socket, or to some other serial device. On a HAVE_STAT platform the function happily returns 0 (I didn't test it on Windows though). On a stat-less platform it is an obvious disaster. I'd be tempted to test that fseek doesn't fail with EBADF beforehand, but it needs more investigation.

\$\endgroup\$
3
  • \$\begingroup\$ Great points, thank you. EBADF seems to be documented by pubs.opengroup.org/onlinepubs/9699919799/functions/fseek.html as "The file descriptor underlying the stream file is not open for writing or the stream's buffer needed to be flushed and the file is not open." \$\endgroup\$
    – Harith
    yesterday
  • \$\begingroup\$ ESPIPE is documented as "The file descriptor underlying stream is associated with a pipe, FIFO, or socket." Linux's man page includes "is not seekable". \$\endgroup\$
    – Harith
    yesterday
  • \$\begingroup\$ Speaking of portability, MacOS (and perhaps all FreeBSD based platforms) does set errno to EBADF on a non-seekable streams. \$\endgroup\$
    – vnp
    yesterday

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.