Newbie: problems deleting lines from text file (large)

hello,

i'm a newbie programmer (and to python, double trouble :) and running into issues reading and writing files. i'm successfully running the code below to manipulate and remove the first few lines of a text file. The text file (file1.txt) is approx. 329kb but when i run the code it only writes a new 5kb file (file2.txt)... so the other 300 something kb below that is missing. how can i get the full file (to file2.txt) with only the first 4 lines removed???

so cofused... please help!! :)

lines = file('./simple-retrieval/file1.txt', 'r').readlines()
del lines[0:4]
file('./simpleunwrapped/file2.txt', 'w').writelines(lines)

Thanks in advance!!!

Write a slice of variable lines to disk.
Example:

thx for the reply...

i am able to delete the 4 lines.. its rewriting the complete file that i'm having issues with.

here's another thing i should throw in... if i comment out the "del lines" line the results are the same... the whole file is still not copying to the new location.

it looks like the whole file never really opens...

As far as opening the file... am i doing it correctly?? i had an impression that readlines() as opposed to readline() is suppose to read the whole file into memory... is this true?

I think you have to take another look at exactly the code you are using. I tried the following in the interactive shell (I should point out that the file is a lot smaller than the ones you mentioned though):



This seems to work, i.e it loads all the lines from the file into a list (your right that is what the readlines() function does, readline only reads in one line at a time), removes the first 4 lines (although I wouldent say this is the best way to do it), and then writes the lines to a new file.

The only thing I can think of is that the file is too large to be read in all at once, perhaps you should move to a solution using readline() and write the files one at a time to the new location, this will take more time, but I'm not sure what option you have.

Nope, just had a go at it with a much larger file than you are using (around 4Mb) and it works as expected. I don't know what else to suggest.

When you write to a file, there needs to be some action that flushes the the output buffers. Create a file object, write to it, then close it.

Don't the buffers flush automatically when they become full? I also have a feeling (although I may be wrong), that writelines() flushes the buffer itself after each list item.

You should really call close() though, just make sure you don't do it before you need to read/write to the file in question.

Yes, the buffers flush when full. I don't think any file method flushes the buffers automatically except for f.flush() and f.close(). You can control the buffering behavior of the file object with the optional bufsize parameter: