#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2012
    Posts
    3
    Rep Power
    0

    Manage big data properly with libpq, c++


    Hello
    I am Alin and I am programmer using C++, libpq, PostgreSQL,PL/SQL.
    I have only 2 months experience with Postgresql but I still learn.
    I have worked at GIS project for a company.
    The database is big having many tables. Some tables are 4 GB almost. The tables contains data from all Europe, that's why are so big. My problem is how to manage the big tables properly.
    I made tunning for server Postgresql, I indexed some tables
    but I am disappointed because of low performance.
    Obviously the queries are slow.
    I have a computer with raid HDD and 8 GB RAM.
    I tried to read a table using cursor and fetch command but
    it does not work ( it takes very long time).
    I would like to read some advices regarding this issue.
    The data provider do not want to split data per country, for example, to make me life easier. I do not know why but for me it is a stupid thing to have big tables with data from all Europe.
    Please tell me about some technics to speed up the processing data.
    Thank you
    Best regards,
    Alin
  2. #2
  3. I'm Baaaaaaack!
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Jul 2003
    Location
    Maryland
    Posts
    5,538
    Rep Power
    243
    Why are you asking a database question in a C/C++ forum? Is it because you program in C/C++? If so, clearly you don't know anything about databases because the language you use to interface with the program is totally irrelevant to the performance of the database.

    My blog, The Fount of Useless Information http://sol-biotech.com/wordpress/
    Free code: http://sol-biotech.com/code/.
    Secure Programming: http://sol-biotech.com/code/SecProgFAQ.html.
    Performance Programming: http://sol-biotech.com/code/PerformanceProgramming.html.
    LinkedIn Profile: http://www.linkedin.com/in/keithoxenrider

    It is not that old programmers are any smarter or code better, it is just that they have made the same stupid mistake so many times that it is second nature to fix it.
    --Me, I just made it up

    The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.
    --George Bernard Shaw
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2012
    Posts
    3
    Rep Power
    0
    Originally Posted by mitakeet
    Why are you asking a database question in a C/C++ forum? Is it because you program in C/C++? If so, clearly you don't know anything about databases because the language you use to interface with the program is totally irrelevant to the performance of the database.
    Yes, I use C++ and libpq.
    I do not know what section is more suitable for my thread.
    I was in hury.
  6. #4
  7. I'm Baaaaaaack!
    Devshed God 1st Plane (5500 - 5999 posts)

    Join Date
    Jul 2003
    Location
    Maryland
    Posts
    5,538
    Rep Power
    243
    How about a forum devoted to Postgresql?

    My blog, The Fount of Useless Information http://sol-biotech.com/wordpress/
    Free code: http://sol-biotech.com/code/.
    Secure Programming: http://sol-biotech.com/code/SecProgFAQ.html.
    Performance Programming: http://sol-biotech.com/code/PerformanceProgramming.html.
    LinkedIn Profile: http://www.linkedin.com/in/keithoxenrider

    It is not that old programmers are any smarter or code better, it is just that they have made the same stupid mistake so many times that it is second nature to fix it.
    --Me, I just made it up

    The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.
    --George Bernard Shaw

IMN logo majestic logo threadwatch logo seochat tools logo