#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Oct 2003
    Location
    Canada
    Posts
    185
    Rep Power
    0

    file locking absolute on a website?


    i've created a cgi prgrm with python and it allows users to input info on a regular basis to a .csv file and .txt. will the file lock up if two ppl try to access it simultaneously or if it is not finished closing? or does it matter that it is on a website ( a client/server architecture?) is there a way to circumvent this if only one person can access it a time - other than using a relational database.
    ex: if one person is submit information can another be submiting at the same time and another be browsing the file - likethe administrator?

    Thank you in advance
    "In theory, there is no difference between theory and practice.
    But, in practice, there is."

  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Oct 2003
    Location
    Canada
    Posts
    185
    Rep Power
    0
    Anyone have this knowledge?
    "In theory, there is no difference between theory and practice.
    But, in practice, there is."

  4. #3
  5. Mini me.
    Devshed Novice (500 - 999 posts)

    Join Date
    Nov 2003
    Location
    Cambridge, UK
    Posts
    783
    Rep Power
    14
    You will have a problem.

    Each new client accessing your website will get it's own thread/process whatever. This means you cannot guarantee the order in which data is written and occasionally attempts to open the file will fail as two or more client sessions write to the file.

    Also the write is unlikely to be atomic and could be corrupted if the actual write is broken into smaller segments by the OS.


    That is one reason for using a database - only one application actually writes the data to file. The write requests from the web site are queued by the database engine.

    Just an idea
    If your site is not busy then perhaps you could use a simple filelock . Where the filelock is a file just containing the process id or something eqaully small and unique with a simple checksum.

    You want to write data:

    Check if filelock exists. if it does, wait some time and check again.
    Keep checking upto a check limit - if reached advise the user that site is busy or email the data to admin. Check the content of the file lock (and calc the checksum).

    If the file lock does not exist, create it. Then read it back to make sure it is your filelock and not corrupted. If it isn't your lock wait some time again for the lock to disappear as above.

    If it is your file lock then write data to your log file.
    Then delete the filelock file.

    I have no idea if this would work in practice or what the timing would be like.
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Intermediate (1500 - 1999 posts)

    Join Date
    Feb 2004
    Location
    London, England
    Posts
    1,585
    Rep Power
    1373
    The file locking mechanism that Grim describes is quite common in older revision control systems such as PVCS and RCS (and maybe CVS, I am not sure of its internals).

    It works quite well, but if anything goes wrong, such as the server dying in the middle of a transaction, then it will leave the lock file behind. This will prevent anyone writing to the files until it is manually deleted. You could get round this by checking the 'last modified' timestamp on the file and deleting it if it is significantly older than maximum duration of your CGI script.

    Another common way to do it would be to have a separate logging program running that you could connect to and send it your logging messages. The program would queue up the messages and write them to the log file one at a time. I believe that both NT and UNIX systems have a system logging process that does this, but they only write to the system log, not to any file.

    Or you could use a DB, as you said. It would not need to be a full blown relational database - the BerkleyDB (bsddb) can support transactions if compiled with the right options. See http://pybsddb.sourceforge.net/ for the Python bindings to it.


    Dave - The Developers' Coach
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Oct 2003
    Location
    Canada
    Posts
    185
    Rep Power
    0
    Thanks for the reply guys. sorry about the double post. I'll check out what you were all talking about.
    [confusion] i noticed if i open a text file and then go online and submit to that same file, the info gets written. however if i do the same with a csv file, cgitb tells me it can't do that. obviously because the file is open. is this because its not plain text and csv is an actually program or something[/confusion]
    Last edited by caroundw5h; April 23rd, 2004 at 05:33 PM.
    "In theory, there is no difference between theory and practice.
    But, in practice, there is."


IMN logo majestic logo threadwatch logo seochat tools logo