Oracle Development
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsDatabasesOracle Development

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
Get inside! Sample the range of functionality easily built with JMSL Library for Time Series Data Analysis, Heat Maps, Portfolio Optimization, Monte Carlo Simulation, Stock Price Charting and more. Download Now!
  #1  
Old June 2nd, 2004, 02:09 AM
ProggerPete ProggerPete is offline
Contributing User
Dev Shed Beginner (1000 - 1499 posts)
 
Join Date: May 2003
Location: Brisbane, Australia
Posts: 1,431 ProggerPete User rank is Sergeant (500 - 2000 Reputation Level)ProggerPete User rank is Sergeant (500 - 2000 Reputation Level)ProggerPete User rank is Sergeant (500 - 2000 Reputation Level)ProggerPete User rank is Sergeant (500 - 2000 Reputation Level)ProggerPete User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 1 Day 22 m 9 sec
Reputation Power: 23
Best way to build a very large table

I'm wondering the best way to build a table that will end up containing a very large number of rows. Probably at least 10 million and could easily end up even larger. It's actually going to contain individual call details from a number of different phonelines.

Because it will be so large I'm want to make querying it as efficient as possible. A given client may have any number of phonelines. If I want to query on all calls made for client X will it improve the speed of the query significantly if I include in my calldetails table the client_id and search on that. Or would querying a seperate table to get a list of phonelines and then searching for the phonelines in the call_details table give comparable performance.

Any ideas on improving performance will be welcome.

Reply With Quote
  #2  
Old June 3rd, 2004, 10:44 AM
Kraeg Kraeg is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: May 2004
Location: Schaumburg, IL
Posts: 20 Kraeg User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Check out "Oracle Performance Tuning" ISBN 1565922379 by Gurry and Corrigan and/or any information you can find regarding partitioned views and parallel queries. Recommends creating several smaller tables seperated by date and creating partitioned view across the tables. This should also help you with a purging strategy as a table with an out-of-date range can be dropped easily and a current table created with only minimal recoding. There is no in-depth discussion on that particular topic in this book, but it is a very good resource for performance tuning at the database, pl/sql and form level in general.

Reply With Quote
  #3  
Old June 6th, 2004, 06:02 PM
ProggerPete ProggerPete is offline
Contributing User
Dev Shed Beginner (1000 - 1499 posts)
 
Join Date: May 2003
Location: Brisbane, Australia
Posts: 1,431 ProggerPete User rank is Sergeant (500 - 2000 Reputation Level)ProggerPete User rank is Sergeant (500 - 2000 Reputation Level)ProggerPete User rank is Sergeant (500 - 2000 Reputation Level)ProggerPete User rank is Sergeant (500 - 2000 Reputation Level)ProggerPete User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 1 Day 22 m 9 sec
Reputation Power: 23
Perhaps you can answer a more specific question. Thinking through things logically it seems to me that if I put a customer id into my calls table I would be able to retreive all calls for a given customer faster than if I had to check if the service number of each call matched a given customer.

i.e.
WHERE cust_id = 'xxxx'

should run faster than

WHERE service_number IN (SELECT service_number FROM services WHERE cust_id = 'xxxx')

In cases where a customer has 20 numbers I'd expect the 1st query to run about 20 times faster than the 2nd. Is this correct?

Reply With Quote
  #4  
Old June 7th, 2004, 03:53 AM
pabloj's Avatar
pabloj pabloj is offline
Modding: Oracle MsSQL Firebird
Dev Shed God 6th Plane (7500 - 7999 posts)
 
Join Date: Jun 2001
Location: Outside US
Posts: 7,711 pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 2 Months 2 Weeks 6 Days 4 h 53 m 59 sec
Reputation Power: 259
You should also look at partitioned tables and maybe bitmap join indexes.

A little explaination of the suggestion above:

If you have a big table with a structure like:

CustID, DateID, ServID, ....

You can partition it by date range, or customer id, or service or whatever and even have subpartition, something like main partitioning by date (i.e. monthly partitions) and subpartitions by customer.
With such a table (and updated statistics of course) Oracle's optimizer will do "partition pruning", this means that it will not consider partitions that will not be affected by your query, scanning only indexes or rows that matter.
This gives you a much better performance on very big tables.
Partitioned tables can also be managed (loaded, unloaded ...) by partition, which gives also better manageability.

Last edited by pabloj : June 7th, 2004 at 06:23 AM.

Reply With Quote
  #5  
Old June 16th, 2004, 11:21 AM
astrocanis astrocanis is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jun 2004
Posts: 14 astrocanis User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Send a message via AIM to astrocanis
If you can add the id to the table, you should. It is absolutely more efficient to eliminate a subquery where you can. But, in order to make it perform, you must index on the ID that you have added. Otherwise, you will still be doing full table scans across that column.

You might consider index-organized tables or partitioning by ID range (if the ID query is a driving consideration).

Reply With Quote
Reply

Viewing: Dev Shed ForumsDatabasesOracle Development > Best way to build a very large table


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 1 hosted by Hostway