Firebird SQL Development
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsDatabasesFirebird SQL Development

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old December 18th, 2006, 04:27 AM
jesper jesper is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2004
Posts: 10 jesper User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 35 m
Reputation Power: 0
Database File Fragmentation

I'm having trouble with fragmentation of the database file.
I have a firebird database that grows to several gigabytes.
I found out that it's heavily fragmented, more than 7000 fragments for a 5GB file, and performance suffers a lot from this.
A partial solution I found was preallocating the file by setting the length to f.ex. 6 BG after database creation.
The problem of course is that I don't know how much to allocate, and whenever the database file needs more space then the fragmenting behaviour returns.
Is there any way to force FB to grow the file in larger increments?

Reply With Quote
  #2  
Old December 18th, 2006, 05:20 AM
pabloj's Avatar
pabloj pabloj is offline
Modding: Oracle MsSQL Firebird
Dev Shed God 6th Plane (7500 - 7999 posts)
 
Join Date: Jun 2001
Location: Outside US
Posts: 7,917 pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 2 Months 3 Weeks 3 Days 14 h 31 m 53 sec
Reputation Power: 279
Use a larger pagesize, but I don't think it's what you are looking for.
Do you know that a Firebird database can span more files?
Simply build more than one datafile.
On what OS are you on? File fragmentation is at filesystem level and not at database level, you'd get it with other DBMS also.

Reply With Quote
  #3  
Old December 18th, 2006, 05:45 AM
jesper jesper is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2004
Posts: 10 jesper User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 35 m
Reputation Power: 0
Quote:
Originally Posted by pabloj
Use a larger pagesize, but I don't think it's what you are looking for.
Do you know that a Firebird database can span more files?
Simply build more than one datafile.
On what OS are you on? File fragmentation is at filesystem level and not at database level, you'd get it with other DBMS also.

This is actually the opposite of what I want.
Multiple files would be just as bad or worse than a big fragmented file - both cases force the disk to seek a lot, especially for sequential access.
I do realize that file fragmentation is an OS issue, but if FB could be configured to grow a file in larger increments typically fewer fragments would be created by the OS.

Reply With Quote
  #4  
Old December 18th, 2006, 06:05 AM
shammat shammat is offline
Contributing User
Dev Shed Beginner (1000 - 1499 posts)
 
Join Date: Oct 2003
Location: Munich, Bavaria
Posts: 1,007 shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 3 Days 13 h 7 m 13 sec
Reputation Power: 67
Quote:
Originally Posted by jesper
This is actually the opposite of what I want.
Multiple files would be just as bad or worse than a big fragmented file - both cases force the disk to seek a lot, especially for sequential access.

Not necessarily. If you distribute the files over several physical harddisks you can actually speed up the read time as the seeking can be done in parallel by each harddisk.

Btw: how do you know that fragmentation is your problem?
I'm quite curious how you can pinpoint fragmentation as the root cause of your problem.

Reply With Quote
  #5  
Old December 18th, 2006, 06:32 AM
jesper jesper is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2004
Posts: 10 jesper User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 35 m
Reputation Power: 0
Because after I defragmented the file (By copying to another file, deleting old file, renaming new file to old name) I got a serious performance increase.
(I also made sure that it wasn't just because a large part of the file was being cached by Windows because of the previous copy).

Reply With Quote
  #6  
Old December 18th, 2006, 09:21 AM
shammat shammat is offline
Contributing User
Dev Shed Beginner (1000 - 1499 posts)
 
Join Date: Oct 2003
Location: Munich, Bavaria
Posts: 1,007 shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level)shammat User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 1 Week 3 Days 13 h 7 m 13 sec
Reputation Power: 67
Quote:
Originally Posted by jesper
Because after I defragmented the file (By copying to another file, deleting old file, renaming new file to old name) I got a serious performance increase
Ah, makes sense

Reply With Quote
  #7  
Old December 18th, 2006, 09:30 AM
pabloj's Avatar
pabloj pabloj is offline
Modding: Oracle MsSQL Firebird
Dev Shed God 6th Plane (7500 - 7999 posts)
 
Join Date: Jun 2001
Location: Outside US
Posts: 7,917 pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 2 Months 3 Weeks 3 Days 14 h 31 m 53 sec
Reputation Power: 279
I meant don't let it grow by itself, add yourself a new datafile of enough size.
After a check it seems that unlike oracle, in firebird you can't control growth parameters

Reply With Quote
  #8  
Old December 19th, 2006, 12:21 AM
jesper jesper is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2004
Posts: 10 jesper User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 35 m
Reputation Power: 0
Quote:
Originally Posted by pabloj
I meant don't let it grow by itself, add yourself a new datafile of enough size.
After a check it seems that unlike oracle, in firebird you can't control growth parameters

Well problem is:
1. I would have to know beforehand that the database needed to grow.
2. Unless I can put the new file on another physical disk then the end result is worse than a single fragmented file.

Reply With Quote
  #9  
Old December 19th, 2006, 02:23 AM
pabloj's Avatar
pabloj pabloj is offline
Modding: Oracle MsSQL Firebird
Dev Shed God 6th Plane (7500 - 7999 posts)
 
Join Date: Jun 2001
Location: Outside US
Posts: 7,917 pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 2 Months 3 Weeks 3 Days 14 h 31 m 53 sec
Reputation Power: 279
Quote:
Originally Posted by jesper
Well problem is:
1. I would have to know beforehand that the database needed to grow.
But I think you can make a reasonable estimate
Quote:
Originally Posted by jesper
2. Unless I can put the new file on another physical disk then the end result is worse than a single fragmented file.
Shure you can.
But I think that two non fragmented files are better than one fragmented

Reply With Quote
  #10  
Old December 19th, 2006, 08:05 AM
jesper jesper is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2004
Posts: 10 jesper User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 35 m
Reputation Power: 0
Quote:
Originally Posted by pabloj
But I think you can make a reasonable estimate
Shure you can.
But I think that two non fragmented files are better than one fragmented


To be able to make an estimate I would have to know how much of a given file the database is currently using. Is this possible?

1 file with 2 fragments is definitely better than 2 unfragmented files If I know/suspect that the file is going to grow then I might as well just grow it a couple of gigs myself.

As for files on different disks - it's not really a solution as I don't have an unlimited supply of extra physical disks connected to the computer;p

When creating a database and setting the file length to 6GB both DatabaseInfo.DatabaseSizeInPages and FBDatabaseInfo.AllocationPages return the same number: 6 GB / PageSize, not the amount of pages actually being used.

PS: The above method names are from the :Net connector, it's using the isc_database_info() function on client dll.

Reply With Quote
Reply

Viewing: Dev Shed ForumsDatabasesFirebird SQL Development > Database File Fragmentation


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 3 hosted by Hostway
Stay green...Green IT