Manage big data properly with libpq, c++

Dev Shed Forums Sponsor:

August 25th, 2012, 09:32 AM

aliosa_sbbv

Registered User

Join Date: Aug 2012

Posts: 3

Time spent in forums: 44 m 34 sec
Reputation Power: 0

Manage big data properly with libpq, c++

Hello
I am Alin and I am programmer using C++, libpq, PostgreSQL,PL/SQL.
I have only 2 months experience with Postgresql but I still learn.
I have worked at GIS project for a company.
The database is big having many tables. Some tables are 4 GB almost. The tables contains data from all Europe, that's why are so big. My problem is how to manage the big tables properly.
I made tunning for server Postgresql, I indexed some tables
but I am disappointed because of low performance.
Obviously the queries are slow.
I have a computer with raid HDD and 8 GB RAM.
I tried to read a table using cursor and fetch command but
it does not work ( it takes very long time).
I would like to read some advices regarding this issue.
The data provider do not want to split data per country, for example, to make me life easier. I do not know why but for me it is a stupid thing to have big tables with data from all Europe.
Please tell me about some technics to speed up the processing data.
Thank you
Best regards,
Alin

August 26th, 2012, 05:27 PM

mitakeet

I'm Baaaaaaack!

Dev Shed God 1st Plane (5500 - 5999 posts)

Join Date: Jul 2003

Location: Maryland

Posts: 5,538

Time spent in forums: 2 Weeks 4 Days 2 h 38 m 46 sec
Reputation Power: 242

Why are you asking a database question in a C/C++ forum? Is it because you program in C/C++? If so, clearly you don't know anything about databases because the language you use to interface with the program is totally irrelevant to the performance of the database.

__________________

My blog, The Fount of Useless Information http://sol-biotech.com/wordpress/
Free code: http://sol-biotech.com/code/.
Secure Programming: http://sol-biotech.com/code/SecProgFAQ.html.
Performance Programming: http://sol-biotech.com/code/PerformanceProgramming.html.
LinkedIn Profile: http://www.linkedin.com/in/keithoxenrider

It is not that old programmers are any smarter or code better, it is just that they have made the same stupid mistake so many times that it is second nature to fix it.
--Me, I just made it up

The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.
--George Bernard Shaw