4
\$\begingroup\$

I wrote a program to copy directories in Windows 10. I have tested copying a directory (696MB) in 3 different ways:

  • xcopy: time taken - 52 seconds
  • ctrl+c: time taken - 78 seconds
  • My program: time taken - 193 seconds

What can I do to improve the performance of my program to be as fast as xcopy?

#include <stdio.h>
#include <string.h>
#include <time.h>
#include <io.h>
#include <Windows.h>

int copyfile(const char *inputfile, const char *outputfile);
void CopyDirectory(const char *src_dir, const char *dst_dir);

int main(void)
{
    clock_t t1 = clock();

    // src directory must end with \\*
    /* dst directory must end with \\ */
    CopyDirectory("C:\\Users\\green\\Downloads\\Folder1\\*",
                  "C:\\users\\green\\desktop\\Folder1\\");

    clock_t t2 = clock();
    printf("time elapsed : %.4f secs\n", (double)(t2 - t1) / CLOCKS_PER_SEC);

    puts("Press any key to continue...");
    getchar();
    return 0;
}

void CopyDirectory(const char *src_dir, const char *dst_dir)
{
    intptr_t hFile;
    struct _finddata_t c_file;

    char *ptr;
    char buf[MAX_PATH + 20];
    char tmp_dir[MAX_PATH + 20];
    char f_name[MAX_PATH + 20];

    strcpy(buf, src_dir);
    ptr = buf + strlen(buf) - 1;

    if ((hFile = _findfirst(src_dir, &c_file)) == -1L) {
        fprintf(stderr, "Failed to open directory %s\n", src_dir);
        return;
    }
    CreateDirectoryA(dst_dir, NULL);
    while (_findnext(hFile, &c_file) == 0) {
        if (c_file.attrib & _A_SUBDIR) {
            if (strcmp(c_file.name, "..")) {
                /*......*/
                strcpy(tmp_dir, dst_dir);
                strcat(tmp_dir, c_file.name);
                strcat(tmp_dir, "\\");
                /*......*/
                *ptr = '\0';
                strcat(buf, c_file.name);
                strcat(buf, "\\*");
                CopyDirectory(buf, tmp_dir);
            }
        }
        else {
            strcpy(tmp_dir, dst_dir);
            strcat(tmp_dir, c_file.name);

            *ptr = '\0';
            strcpy(f_name, buf);
            strcat(f_name, "\\");
            strcat(f_name, c_file.name);
            copyfile(f_name, tmp_dir);
            strcpy(tmp_dir, dst_dir);
        }
    }
    _findclose(hFile);
}

int copyfile(const char *inputfile, const char *outputfile)
{
    static FILE *fp_I;
    static FILE *fp_O;

    fp_I = fopen(inputfile, "rb");
    if (!fp_I) {
        perror(inputfile);
        return -1;
    }

    fp_O = fopen(outputfile, "wb");
    if (!fp_O) {
        fclose(fp_I);
        perror(outputfile);
        return -1;
    }

    static size_t nr;
    const static size_t bufsize = 512 * 512;
    static char buf[bufsize];

    while ((nr = fread(buf, 1, bufsize, fp_I)) > 0) {
        fwrite(buf, 1, nr, fp_O);
    }

    fclose(fp_I);
    fclose(fp_O);
    return 0;
}
\$\endgroup\$
17
  • \$\begingroup\$ And you do not even copy ADS and ACLs!!! The truth is that OS has access to lower level structures and buffer size is optimized and crefully chosen (maybe even dynamically according to sector size and media type) and probably bigger than your buffer \$\endgroup\$ – Adriano Repetti Sep 2 '16 at 9:33
  • 1
    \$\begingroup\$ Have you tried to use CopyFile instead of rolling your own? \$\endgroup\$ – ChrisWue Sep 2 '16 at 9:40
  • 1
    \$\begingroup\$ Alternate Data Streams and Access Control List. Also note that overlapped I/O may greatly increase performance (and, well, multithreading...God knows) \$\endgroup\$ – Adriano Repetti Sep 2 '16 at 10:29
  • 1
    \$\begingroup\$ Don't call ASCII versions of Windows APIs if you need performance. They incur the extra overhead of transforming the strings to Unicode before calling the Unicode version of the API. Letting the kernel handle copying the content of the file should be much faster as it doesn't have to context switch and copy buffers in process memory just to have it copied again into kernel memory. \$\endgroup\$ – D. Jurcau Sep 4 '16 at 10:15
  • 1
    \$\begingroup\$ @RubberDuck I am not reinventing the wheel. But sometimes I can't find what i want to perform, and sometimes I find it but I need more functionality so, I do it myself. \$\endgroup\$ – machine_1 Sep 4 '16 at 16:10
2
+50
\$\begingroup\$

Re-architect to do lots of reading, then lots of writing - then loop.

OP's code presently reads file A, writes file A, reads file B, writes file B, ...

Reading file A, file B,.... partial file Z and then writing those into a large buffer is certainly faster. Of course it is more complicated as code needs to pick-up where it left off in the middle of file Z.

I first noticed this huge saving BITD when XCOPY *.* came out as it read many files first versus the old COPY *.*. This paradigm has been consistently true with my subsequent coding experiences.

A central issue, that unless you get deeper into the DISK I/O, code is limited to trying to do what it thinks the file system wants best in order to perform fast.

Using multiple threads (one for reading, one for writing) may help too.

\$\endgroup\$
4
  • \$\begingroup\$ What is BITD ?? \$\endgroup\$ – machine_1 Sep 9 '16 at 17:38
  • \$\begingroup\$ @machine_1 BITD A time when computers were made of stone knives and bearskins. \$\endgroup\$ – chux - Reinstate Monica Sep 9 '16 at 17:42
  • \$\begingroup\$ It would be worthy of the bounty to provide some code, you know! I didn't offer the bounty for a textual answer only :)). Answer's prize is +15... \$\endgroup\$ – machine_1 Sep 10 '16 at 13:40
  • \$\begingroup\$ @machine_1 The amount of quality coding to re-architect this is significant, far more than 100 points, but a lot of work. \$\endgroup\$ – chux - Reinstate Monica Sep 10 '16 at 13:50
0
\$\begingroup\$

Ignore all the C functions and using the windows API calls: CreateFile, ReadFile, WriteFile, etc. Then you can specify unbuffered file reads (for larger files), and use asynchronous IO calls to read and write simultaneously, which is particularly handy when copying between two different physical disks.

The hardware solution would be to upgrade to a decent Solid State Drive.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.