#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2003
    Location
    Poland
    Posts
    57
    Rep Power
    12

    How many inserts one by one?


    Hi,
    I've got a problem. I've got a program in C that inserts some data into my database. It should insert about 100000 records to one table and about 90000 to another.
    I'm reloading transaction each 1000 rows (commit and begin).
    The program should insert the filesystem structure into the database, and it stops always at the same file, but it's not special file, just a normal one.
    The program prints no errors, except the one saying that program failed (no sql error, no dir reading error).
    So the result is I've got only 27000 records in one and about 26000 in another table in my database
    Why does this happen, are there any limits in postgresql, or maybe sth else?
    I've got slackware 9.0 and postgreql 7.3.3 if it can help you helping me

    Thank for advice
  2. #2
  3. No Profile Picture
    Gödelian monster
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Jul 1999
    Location
    Central Florida, USA
    Posts
    2,307
    Rep Power
    61
    Hmm... sounds like you are running into a configuration limit of some sort. Check out the possible settings for postgresql.conf at http://www.postgresql.org/docs/7.3/i...me-config.html

    "shared_buffers" might be the setting you want to change.

    The other possibility is that you are running into some sort of operating system limit, such as max_filehandles, shared memory, or something like that. Try running your whole program in smaller sets, such as 10,000 files at a time, and see what happens.
    The real n-tier system:

    FreeBSD -> PostgreSQL -> [any_language] -> Apache -> Mozilla/XUL

    Amazon wishlist -- rycamor (at) gmail.com
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2003
    Location
    Poland
    Posts
    57
    Rep Power
    12
    But I'm not even opening the files, I'm only reading its name (as I said, I'm inserting filesystem *structure*).
    I'll try to change the pg configuration.

IMN logo majestic logo threadwatch logo seochat tools logo