Firebird SQL Development
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsDatabasesFirebird SQL Development

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old February 20th, 2007, 06:35 AM
miztaken miztaken is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Feb 2007
Posts: 24 miztaken User rank is Private First Class (20 - 50 Reputation Level)miztaken User rank is Private First Class (20 - 50 Reputation Level) 
Time spent in forums: 13 h 7 m 22 sec
Reputation Power: 0
Indexing Problem...

Okai,
Like Transactions, Indexing is too acting like necessary evil for me.
I just created a table and indexed (ascending by default) its one of its column.
Then i inserted data in the table, and it took some time, Time which was much less than time taken to insert when i tested with indexing three of its column (to ascending and one descending).
I think this is normal..
But, it is taking me very much of time.
I indexed the columns to make search query fast. But due to this insertion process slow.
Note: I have to insert millions of records and it keeps on incresing.
I also tried doing like:
ALTER INDEX ind_name INACTIVE;
Insert data in table
ALTER INDEX ind_name ACTIVE;
Then i updated index statistics.

For small lump of data in table this works fine. But when data is huge then.. i guess
"ALTER INDEX ind_name ACTIVE" tries to re create index. So i noticed FB creating temp files (under my profiles \Local Settings\temp) and as i dont have enough primary space so it just threw exception.. and that was the end of my hopes.

I dont know where i went wrong..
Please point me the right way to do this things..
I want to make both insertion and search fast. and also support TB of data.

Pre-Thanks for any suggestions

Reply With Quote
  #2  
Old February 20th, 2007, 09:03 AM
pabloj's Avatar
pabloj pabloj is offline
Modding: Oracle MsSQL Firebird
Dev Shed God 7th Plane (8000 - 8499 posts)
 
Join Date: Jun 2001
Location: Outside US
Posts: 8,073 pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 2 Months 4 Weeks 4 h 19 m 37 sec
Reputation Power: 281
Try dropping and recreating indexes, disabling and reenabling is the common way, but ...
Note that anyone claiming
Quote:
support TB of data
can NOT say
Quote:
i dont have enough primary space
. Get your technical infrastructure up to the task.
Indexing slowing down inserts is a common problem in datawarehouses, solved as I said before.

Reply With Quote
  #3  
Old February 21st, 2007, 05:17 AM
miztaken miztaken is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Feb 2007
Posts: 24 miztaken User rank is Private First Class (20 - 50 Reputation Level)miztaken User rank is Private First Class (20 - 50 Reputation Level) 
Time spent in forums: 13 h 7 m 22 sec
Reputation Power: 0
Quote:
Originally Posted by pabloj
Try dropping and recreating indexes, disabling and reenabling is the common way, but ...
Note that anyone claiming can NOT say . Get your technical infrastructure up to the task.
Indexing slowing down inserts is a common problem in datawarehouses, solved as I said before.

Cant we just give FB another path for doing those stuffs.. i mean other than primary drive and also why is it taking such a huge space i my temp folder just to reindex.

Building up infrastructure isnt something some one should suggest me.
I am out here asking for some help..
Thanks, for all the support

Reply With Quote
  #4  
Old February 21st, 2007, 06:27 AM
pabloj's Avatar
pabloj pabloj is offline
Modding: Oracle MsSQL Firebird
Dev Shed God 7th Plane (8000 - 8499 posts)
 
Join Date: Jun 2001
Location: Outside US
Posts: 8,073 pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 2 Months 4 Weeks 4 h 19 m 37 sec
Reputation Power: 281
Quote:
Originally Posted by miztaken
Cant we just give FB another path for doing those stuffs.. i mean other than primary drive and also why is it taking such a huge space i my temp folder just to reindex.
Yes you can, look at the env variable FIREBIRD_TMP or the config param TempDirectories.
Quote:
Building up infrastructure isnt something some one should suggest me
I'm not saying that you shouldn't tune your db and simply beef up the hardware, but, in case of Tb databases you can't skip on hardware.

Reply With Quote
Reply

Viewing: Dev Shed ForumsDatabasesFirebird SQL Development > Indexing Problem...


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump

 Free IT White Papers!
 
How to Present Effectively Online
This white paper offers practical and actionable advice on the key steps that any presenter should consider as they plan and execute a Webinar or online meeting.

 
Open Source Security Myths
Open Source Software (OSS) is computer software whose source code is available to the general public with relaxed or non-existent intellectual property restrictions (or arrangement such as the public domain), and is usually developed with the input of many contributors.

 
Power and Cooling Capacity Management for Data Centers
This paper describes the principles for achieving power and cooling capacity management.

 
Scalable, Fault-Tolerant NAS for Oracle - The Next Generation
For several years NAS has been evolving as a storage alternative for Oracle databases, and for good reason: NAS is quite often the simplest, most cost-effective storage approach for Oracle. Learn about the benefits that HP's approach to scalable NAS brings to Oracle environments in this comprehensive white paper.

 
Understanding Web Application Security Challenges
This white paper discusses many common threats and preventive measures for Web application security, and explains what you can do to help protect your organization.

 

Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 3 hosted by Hostway
Stay green...Green IT