December 19th, 2012, 12:37 PM
Big Data and PostgreSQL
Hi. After I told a friend at another company about the results we are seeing in PostgreSQl, he asked these questions:
Are you processing any “Big Data”?
Can you run complex analytics against “Big Data”? That’s the issue we’re having here….
He means "in PostgreSQL".
Can the community respond at a high level to these questions? He would appreciate any links as well but first he would like to hear answers that would justify him taking time to read docs at those links.
By complex analytics he means "...reading every row in a multi-trillion row table, and doing sums, averaging, linear and non-linear regression, etc….".
I've asked him about pre aggregation and will post his answer here once I get a better understanding of why he doesnt do that.
Update: Preaggregation is a challenge for him because his users ask for different things all of the time not just common industry calculations. For now, they load their new data into SAS from a db and ask it to do the heavy lifting.