Creating large arrays

I wanted to create an array to store the data I have from an image file. Being that they are medical images they are quite large (anywhere from 35mb to 600 mb)

I tried doing the following but it just crashes

for i in range(0, arraySize):
myArray.append(0)

I need to create an array just filled with zeros the proper size so that I can run the array through a filter that will put in the actual values.

Doing the above crashes if I make the arraySize too large. (ie. over 1,000,000) I need arrays at least 18 million large.

The strange thing is this worked when I did myArray.fromfile() and just streamed the entire binary into the array.

Is this a limit of python and I need to figure out how to do it some other way or am I programming it wrong? Thank you for any help.

(PS: I have 2 gigs of memory so it's not filling that up)

Seems to work for me, I'm running the test now and its running through nicely, its very slow but then it is a very high number! Anyway, just wondering why if it worked with fromfile() why your not using it? It seems pretty pointless to rewrite code, plus fromfile is probably faster .

Have fun,
Mark.

Oh ok, test just finished at 199999! I dont think it's an array size problem, if you can store 18000000 entries with the fromfile() function then it is most likly to do with the loop. Maybe you should try a while loop, i'm not sure whether there is a max number of iterations on this but it's worth a try, or maybe you could try several loops... 18 in fact but thats a really bad way of doing things!

Hope this helps,
Mark.

I can't use fromfile because I create the values on the fly. Thus I want to be able to insert them into the array at different indices. Does anyone know of any other way to initialize an array other than appending items?

(fromfile does work much faster but that's not an option)

** This is how to do it for anyone that might have to do this in the future

you want to do the following

a = array('h', myArray = [0] * arraySize)


Where of course the 0 is whatever you want to initialize the array to.

here h is a signed int (2 bytes)

that will give you an array of 18,000,000 elements (just numbers) that you can iterate through, etc.