August 1st, 2003, 07:06 AM
How many inserts one by one?
I've got a problem. I've got a program in C that inserts some data into my database. It should insert about 100000 records to one table and about 90000 to another.
I'm reloading transaction each 1000 rows (commit and begin).
The program should insert the filesystem structure into the database, and it stops always at the same file, but it's not special file, just a normal one.
The program prints no errors, except the one saying that program failed (no sql error, no dir reading error).
So the result is I've got only 27000 records in one and about 26000 in another table in my database
Why does this happen, are there any limits in postgresql, or maybe sth else?
I've got slackware 9.0 and postgreql 7.3.3 if it can help you helping me
Thank for advice
August 1st, 2003, 06:08 PM
Hmm... sounds like you are running into a configuration limit of some sort. Check out the possible settings for postgresql.conf at http://www.postgresql.org/docs/7.3/i...me-config.html
"shared_buffers" might be the setting you want to change.
The other possibility is that you are running into some sort of operating system limit, such as max_filehandles, shared memory, or something like that. Try running your whole program in smaller sets, such as 10,000 files at a time, and see what happens.
August 2nd, 2003, 03:03 PM
But I'm not even opening the files, I'm only reading its name (as I said, I'm inserting filesystem *structure*).
I'll try to change the pg configuration.