#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Oct 2012
    Posts
    2
    Rep Power
    0

    Linux and Windows write timing difference


    Hi guys,

    This is not related to syntax or runtime problem. What I am going ask is more about how Linux and Windows handle writing data from buffer to a file. I have this code here, wrapped around a timing block, to write a buffer to a file.

    Code:
    StartCounter();
       if(rows != fwrite(image, cols, rows, fp)){
          fprintf(stderr, "Error writing the image data in write_pgm_image().\n");
          if(fp != stdout) fclose(fp);
          return(0);
       }
    
       test = GetCounter();
    Code:
    #include <shrUtils.h>
    
    #ifdef _WIN32
    double PCFreq = 0.0;
    __int64 CounterStart = 0;
    #endif
    
    #ifdef __linux__
    struct timeval ts_start,ts_end;
    #endif
    
    void StartCounter()
    {
    #ifdef _WIN32
    	LARGE_INTEGER li;
        if(QueryPerformanceFrequency(&li) == 0)
    		printf("QueryPerformanceFrequency failed!\n");
    
        PCFreq = (float)((li.QuadPart)/1000.0);
    
        QueryPerformanceCounter(&li);
        CounterStart = li.QuadPart;
    #endif
    
    #ifdef __linux__
    	gettimeofday(&ts_start, NULL);
    #endif
    }
    double GetCounter()
    {
    #ifdef _WIN32
    	LARGE_INTEGER li;
        QueryPerformanceCounter(&li);
        return (float)((li.QuadPart-CounterStart)/PCFreq);
    #endif
    	
    #ifdef __linux__
    	gettimeofday(&ts_end, NULL);
    	//time = timespec_sub(ts_end, ts_start);
    	return (float)((ts_end.tv_sec - ts_start.tv_sec + 1e-6 * (ts_end.tv_usec - ts_start.tv_usec))*1000.0);
    #endif
    }


    When I measure time to write data, I found out that:
    1 - in Linux, the time to write data is linearly proportional to the data size.
    2 - in Windows, the time to write data is quadratically proportional to the data size.

    I think the fwrite function writes the data line by line to the file, therefore the linear relationship in Linux. But seems like Windows behaves differently. Do you think of any explanation for this?

    Any help is greatly appreciate
  2. #2
  3. I'm Baaaaaaack!
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Jul 2003
    Location
    Maryland
    Posts
    5,538
    Rep Power
    244
    Measuring performance is tricky and more art than science so the differences you are measuring might be artifacts of the way you designed the measurement rather than anything underlying the OS. Also, the seemingly simple act of trying to measure time is actually a massive problem, most particularly on multi-core/CPU systems, and one that makes measuring small events a nightmare. You don't say if you are measuring with the exact same hardware and if not, then your results are largely meaningless.

    As an FYI, fwrite() is implemented using underlying OS-specific API calls and different implementations of the library interface behave differently on different OSs. Meaning, Visual Studio should be your IDE of choice when attempting to optimize on Windows; the use of gcc on Windows, while it _may_ give you high performance, is a more generic compiler. Of course, gcc on Windows has the added complexity of cygwin, though I believe there is a supported port of gcc that works direct on Windows.

    How the data is buffered is highly dependent on hardware (i.e., disk) settings and different OSs can 'tweak' drive settings in firmware as they boot, further confounding the issue even when running on identical hardware. There are several layers between your C code call of fwrite() and the actually placement of bytes on the disk (presuming you aren't using a SSD; you are aware of details at that level, I hope) and most of those layers are totally outside your control. It is also possible to tweak OS settings during install and even boot time to modify the way data is cached. Without taking the time to understand all these nitty gritty little details your attempt at measuring write performance is pretty much a waste of time. You could, though, use this as an incentive to learn those types of details, possibly becoming a superlative programmer as a consequence, or you could just choose to ignore them and not worry about things that are difficult (and sometimes impossible) to change. Though I am aware of these issues, I choose to ignore them except at the highest level (meaning if I am doing lots of writes I will get a RAID SSD for maximal throughput, then ignore). Awareness can make you a better programmer when performance is important, but generally speaking, these sorts of issues are almost never an issue in real-world programming.

    BTW, I have a writeup on performance programming you can access via my sig if you are interested. It might provide some insights for you.

    My blog, The Fount of Useless Information http://sol-biotech.com/wordpress/
    Free code: http://sol-biotech.com/code/.
    Secure Programming: http://sol-biotech.com/code/SecProgFAQ.html.
    Performance Programming: http://sol-biotech.com/code/PerformanceProgramming.html.
    LinkedIn Profile: http://www.linkedin.com/in/keithoxenrider

    It is not that old programmers are any smarter or code better, it is just that they have made the same stupid mistake so many times that it is second nature to fix it.
    --Me, I just made it up

    The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.
    --George Bernard Shaw
  4. #3
  5. Banned ;)
    Devshed Supreme Being (6500+ posts)

    Join Date
    Nov 2001
    Location
    Woodland Hills, Los Angeles County, California, USA
    Posts
    9,640
    Rep Power
    4247
    Originally Posted by chipbu
    I think the fwrite function writes the data line by line to the file, therefore the linear relationship in Linux. But seems like Windows behaves differently. Do you think of any explanation for this?
    This is because of the I/O buffering done by the OS. In Linux, you could use setvbuf() to change the file handle from line buffering to full buffering (or unbuffered, if you want it that way).
    Up the Irons
    What Would Jimi Do? Smash amps. Burn guitar. Take the groupies home.
    "Death Before Dishonour, my Friends!!" - Bruce D ickinson, Iron Maiden Aug 20, 2005 @ OzzFest
    Down with Sharon Osbourne

    "I wouldn't hire a butcher to fix my car. I also wouldn't hire a marketing firm to build my website." - Nilpo
  6. #4
  7. Transforming Moderator
    Devshed Supreme Being (6500+ posts)

    Join Date
    Mar 2007
    Location
    Washington, USA
    Posts
    14,126
    Rep Power
    9398
    Originally Posted by mitakeet
    though I believe there is a supported port of gcc that works direct on Windows.
    I think you're remembering MinGW.

IMN logo majestic logo threadwatch logo seochat tools logo