#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Dec 2012
    Posts
    76
    Rep Power
    2

    New to pickling, need help with function.


    So, I usually use Python to crunch (what I consider to be) large quantities of data and would like to be able to pull/retrieve the results of calculations now and again. So I've created this little program designed to pickle my data to a specific location. I hope to then write an unpickler function to restore it at will from that same preset location. Here's what I got:

    Code:
    import os,errno
    
    class glob():
    	dcache="F:\\Data\\"
    
    def makedir(path):
            try:
                    os.makedirs(path)
            except OSError as exc: # Python >2.5
                    if exc.errno == errno.EEXIST and os.path.isdir(path):
                            pass
                    else: raise
    
    def garbage_name(clength,nlength):
    	vldchars='ABCDEFGHIJKLMNOPQRSTUVWXYZ'
    	vldnums='0123456789'
    	from random import choice
    	outstr=''
    	while clength > 0:
    		outstr += choice(vldchars)
    		clength -= 1
    	while nlength > 0:
    		outstr += choice(vldnums)
    		nlength -= 1
    	return outstr
    
    def store(objs,filename=None,ext='pck',loc=glob.dcache):
    	if filename == None:
    		filename=garbage_name(4,4)
    	outfile = open(str(glob.dcache)+filename+'.'+ext,'w')
    	import pickle
    	pickle.dump(objs,outfile)
    	outfile.close()
    	return 'Complete'
    
    
    makedir(glob.dcache)
    tlist=[2,4,5,6,7,8]
    store(tlist)
    I found the makedir thing on StackOverflow and have no idea if it works, it doesn't seem to be raising errors and the directory already exists. It's writing my file, and I found it at the target location. But it doesn't write anything to it and it returns this error:

    Code:
    Traceback (most recent call last):
      File "C:\Documents and Settings\*****\Desktop\catchfile.py", line 39, in <module>
        store(tlist)
      File "C:\Documents and Settings\*****\Desktop\catchfile.py", line 32, in store
        pickle.dump(objs,outfile)
    TypeError: must be str, not bytes
    I'm guessing I have one more conversion step before pickling is going to work.

    Note: If you're going to try to run the code, you may need to swap the directory around to your desktop or something. It's designed to write to a flash drive.
  2. #2
  3. Contributing User
    Devshed Demi-God (4500 - 4999 posts)

    Join Date
    Aug 2011
    Posts
    4,706
    Rep Power
    480
    In python3 open the pickle file in binary mode.
    Code:
    def store(objs,filename=None,ext='pck',loc=glob.dcache):
        if filename == None:
            filename=garbage_name(4,4)
        outfile = open(str(glob.dcache)+filename+'.'+ext,'wb')##############
        pickle.dump(objs,outfile)
        outfile.close()
        return 'Complete'
    If you want old versions of python to read the file you'll need to also use the protocol parameter. Or you could write a program to translate between protocols.
    [code]Code tags[/code] are essential for python code and Makefiles!
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Dec 2012
    Posts
    76
    Rep Power
    2
    Thank you very much. Works like a charm.

IMN logo majestic logo threadwatch logo seochat tools logo