Python Programming
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsProgramming LanguagesPython Programming

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
Get inside! Sample the range of functionality easily built with JMSL Library for Time Series Data Analysis, Heat Maps, Portfolio Optimization, Monte Carlo Simulation, Stock Price Charting and more. Download Now!
  #1  
Old February 5th, 2004, 01:59 PM
rickt's Avatar
rickt rickt is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Feb 2004
Posts: 23 rickt User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Question shutil copyfile: how come it truncates data?

I'm not sure what's going on but the copy I'm using below doesn't really copy. It only copies up to 196 lines the rest it leaves off.

Copied file:
Lines: 196
File size: 12483 bytes
Size on disk: 16,384 bytes

Original file:
Lines: 247
File size: 13499 bytes
Size on disk: 16,384 bytes

Code:
shutil.copyfile('file1.html', 'file2.html')


I'm using python 2.3 on win2k. Did I miss something here?

Any help would be greatly appreciated!

-Rick

EDIT:
Code:
infile = open('file1.html', "rb")
    outfile = open('file2.html', "wb")
    outfile.write(infile.read())


The above code snippet yeilds the same truncated results... what gives?

Last edited by rickt : February 5th, 2004 at 02:23 PM.

Reply With Quote
  #2  
Old February 5th, 2004, 04:31 PM
netytan's Avatar
netytan netytan is offline
Hello World :)
Dev Shed Frequenter (2500 - 2999 posts)
 
Join Date: Mar 2003
Location: Hull, UK
Posts: 2,529 netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 17 h 19 m 5 sec
Reputation Power: 63
Send a message via ICQ to netytan Send a message via AIM to netytan Send a message via MSN to netytan Send a message via Yahoo to netytan
If you can attach the file for testing perposes it would probably help alot , of even if you provide a URL or emailed it to me...

netytan at hormail dot come

as far as i can see theres no reason why this should be happening so hopefully we can figure this out.

Note: you should really close() file instances or use the

var = file('name', 'mode').method()

format; where the mode is r, w, a etc and the method is read() or write() i.e. your code in a simple line (doesnt require close())

Code:
>>> file('file2.txt', 'w').write(file('file1.txt', 'r').read())


Mark.
__________________
programming language development: www.netytan.com Hula


Reply With Quote
  #3  
Old February 5th, 2004, 06:02 PM
Strike Strike is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2001
Location: Houston, TX
Posts: 383 Strike User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 41 m 27 sec
Reputation Power: 7
Send a message via ICQ to Strike Send a message via AIM to Strike Send a message via Yahoo to Strike
netytan: creating a file object and using methods on it in one line is no different than creating it and then using its methods on a separate line. Closing it is unimportant too. As long as all references to it go away, it will be closed and summarily gc'ed. Writing big-*** one-liners like that just detract from readability.

Reply With Quote
  #4  
Old February 5th, 2004, 06:14 PM
netytan's Avatar
netytan netytan is offline
Hello World :)
Dev Shed Frequenter (2500 - 2999 posts)
 
Join Date: Mar 2003
Location: Hull, UK
Posts: 2,529 netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 17 h 19 m 5 sec
Reputation Power: 63
Send a message via ICQ to netytan Send a message via AIM to netytan Send a message via MSN to netytan Send a message via Yahoo to netytan
Although contrary to what i have read/heard! Infact i believe this (closing file instances) came form Learning Python which admitadly this is pretty old

And yes i am aware that the garbage colector will remove unneeded objects

Finally , i dont see anything wrong with one liners in certain cases...

>>> file('file.txt', 'r').read()

looks cleaner to me than

>>> name = file('file.txt', 'r')
>>> name.read()
>>> name.close()

Mark.

Last edited by netytan : February 5th, 2004 at 06:16 PM.

Reply With Quote
  #5  
Old February 5th, 2004, 06:21 PM
Strike Strike is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2001
Location: Houston, TX
Posts: 383 Strike User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 41 m 27 sec
Reputation Power: 7
Send a message via ICQ to Strike Send a message via AIM to Strike Send a message via Yahoo to Strike
I agree that that one-liner is fine, but nesting that inside another similar one is probably too much.

A 59-character line of python with only a few characters devoted to literals and with no function name longer than 5 letters is generally too long.

Reply With Quote
  #6  
Old February 5th, 2004, 07:37 PM
lazy_yogi lazy_yogi is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Mar 2003
Posts: 325 lazy_yogi User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 7 h 58 m 36 sec
Reputation Power: 6
It's just bad practice not to close a file when you're finished using it. You will be using other languages at some stage, and this bad habbit will likely spill over to there.

I'd probably go with this. It's readable and fairly short and closes all handles.
Not that I've checked that it works :P

Code:
infile = file('file1.txt', 'r').read()
outfile = file('file2.txt', 'w').write(infile)
infile.close()
outfile.close()



[EDIT] Ok .. that didnt work =P
hmmm ... I'd possibly go with the first 2 lines due to consiceness and readability. Tho then Iv'e contradicted myself completely from before ;P[/EDIT]

Last edited by lazy_yogi : February 5th, 2004 at 07:42 PM.

Reply With Quote
  #7  
Old February 5th, 2004, 11:36 PM
Strike Strike is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2001
Location: Houston, TX
Posts: 383 Strike User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 41 m 27 sec
Reputation Power: 7
Send a message via ICQ to Strike Send a message via AIM to Strike Send a message via Yahoo to Strike
Well, in python I believe you pretty much never see people explicitly closing file handles (or del'ing them) because it's understood that they will be closed when they are gc'ed. That's part of the beauty of high-level languages, you don't have to worry about tasks like that. Much like you don't have to worry about freeing memory for every time you allocate it, you don't have to worry about closing a file handle every time you open one.

Reply With Quote
  #8  
Old February 5th, 2004, 11:49 PM
lazy_yogi lazy_yogi is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Mar 2003
Posts: 325 lazy_yogi User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 7 h 58 m 36 sec
Reputation Power: 6
You have to explicitely close a connection to the database every time or mysql locks up.

And this would be a serious problem with commercial software.

Taking care of memory is a well konwn issue with high level languages, but you can't assume freeing all types of resources isn't automatically handled.

Tho it's nice that python handles this.

Reply With Quote
  #9  
Old February 6th, 2004, 05:30 AM
netytan's Avatar
netytan netytan is offline
Hello World :)
Dev Shed Frequenter (2500 - 2999 posts)
 
Join Date: Mar 2003
Location: Hull, UK
Posts: 2,529 netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 17 h 19 m 5 sec
Reputation Power: 63
Send a message via ICQ to netytan Send a message via AIM to netytan Send a message via MSN to netytan Send a message via Yahoo to netytan
Now that i think about it - that being it isnt 1am - surly lines like this one...

>>> file('file.txt', 'r').read()

are caught by the grabage collector nie imediatly because there is no referance to the file anywhere. Its simply created, used and lost. where in this example the referance is there longer (and if you dont close() it untill the grabage collector removes it).

>>> name = file('file.txt', 'r')
>>> name.read()
>>> name.close()

seems logical?

Mark.

Reply With Quote
  #10  
Old February 6th, 2004, 07:35 AM
Strike Strike is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2001
Location: Houston, TX
Posts: 383 Strike User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 41 m 27 sec
Reputation Power: 7
Send a message via ICQ to Strike Send a message via AIM to Strike Send a message via Yahoo to Strike
Yeah, if all you are going to do to the file is just read the data in, then doing:
Code:
data = file('foo', 'r').read()

is fine. I just wouldn't try to accomplish more than one task on the same line like was being done with the line where the data was read from one file and written to another file at once.

Reply With Quote
  #11  
Old February 6th, 2004, 07:46 AM
netytan's Avatar
netytan netytan is offline
Hello World :)
Dev Shed Frequenter (2500 - 2999 posts)
 
Join Date: Mar 2003
Location: Hull, UK
Posts: 2,529 netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 17 h 19 m 5 sec
Reputation Power: 63
Send a message via ICQ to netytan Send a message via AIM to netytan Send a message via MSN to netytan Send a message via Yahoo to netytan
Got ya

Reply With Quote
  #12  
Old February 6th, 2004, 07:58 AM
rickt's Avatar
rickt rickt is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Feb 2004
Posts: 23 rickt User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Post Python Script

Quote:
Originally posted by netytan
If you can attach the file for testing perposes it would probably help alot


First off, thanks for the suggestions. Second, I'm going to go out on a limb pull a little SYA and past my code here. Try not to poke too much fun. I'm new to this.

Code:
import urllib
import httplib
import filecmp
import os
import shutil

params = urllib.urlencode({"R":"yes","D":"MH"})
h = httplib.HTTP("securityresponse.symantec.com:80")
h.putrequest("POST", "/avcenter/cgi-bin/updates_msa.cgi")
h.putheader("Content-length", "%d" % len(params))
h.putheader('Accept', 'text/plain')
h.putheader('Host', 'securityresponse.symantec.com')
h.endheaders()
h.send(params)
reply, msg, hdrs = h.getreply()
print reply 
data = h.getfile().read()
file_name = "NIDS.html"
the_file = open(file_name,'w')
the_file.write(data)
print "Grabbed the HTML and wrote the file."

old_file_name = "NIDS_Original.html"
return_info = filecmp.cmp (file_name, old_file_name)
# print return_info

if return_info == False:
    desktop_file_name = 'c:\\documents and settings\\all users\\desktop\\ALERT Symantec Page Changed.html'
    the_file1 = open(desktop_file_name, 'w')
    the_file1.write(data)
    shutil.copyfile('NIDS.html', 'c:\\documents and settings\\all users\\desktop\\ALERT Symantec Page Changed.html')
    infile = open('NIDS.html', "rb")
    outfile = open('c:\\documents and settings\\all users\\desktop\\ALERT Symantec Page Changed.html', "wb")
    shutil.copy('NIDS.html', 'NIDS_Original.html')
    print 'FALSE:  Copied NIDS.html to ALERT on the desktop.'
print 'TRUE:  Deleted the temp file NIDS.html.'
os.remove('NIDS.html')


Ok, one thing I know I need to change is I need to get into the habbit of throwing in some comments. I'll be sure to touch up my code right now. Let me know if I should post the commented version. You guys will probably all think this is really basic stuff though. <shrug>

Reply With Quote
  #13  
Old February 6th, 2004, 09:35 AM
netytan's Avatar
netytan netytan is offline
Hello World :)
Dev Shed Frequenter (2500 - 2999 posts)
 
Join Date: Mar 2003
Location: Hull, UK
Posts: 2,529 netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level)netytan User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 17 h 19 m 5 sec
Reputation Power: 63
Send a message via ICQ to netytan Send a message via AIM to netytan Send a message via MSN to netytan Send a message via Yahoo to netytan
Naw i've seen worse believe me , its actually pretty neat! But really needing a sample file to test it with. are you just using the symantec website?

Edit: Never mind i got it, just renamed the file created by you're program, since NIDS_Original.html sisnt exist. I'll give it a go now.

Mark.

Last edited by netytan : February 6th, 2004 at 10:10 AM.

Reply With Quote
  #14  
Old February 6th, 2004, 11:02 AM
rickt's Avatar
rickt rickt is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Feb 2004
Posts: 23 rickt User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
More info

Thanks for looking at it. The bottom line that I'm looking for is:

Go to that web site and check if there's a new update for the NIDS. Then, if there is do something. The somethi