Database Management
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsDatabasesDatabase Management

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
  #1  
Old December 30th, 2003, 02:43 PM
pcervelli pcervelli is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2003
Posts: 1 pcervelli User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Question Database design -- storing sparse matrices

I would like to store data and associated data covariance in an SQL database. Every day we generate a list of positions for a network of GPS receivers. Each position consists of a coordinate triple corresponding to a unique station at a unique time.

Suppose we have 20 stations, this entails 60 data points per day. No big deal, but we also must store the 60x60 covariance matrix which contains crucial information about the uncertainties of the data.

Years of data will then have a giant (mostly sparse, since individual days are indepedent) covariance matrix associated with it.

I realize there is no native support for a matrix data type in SQL. I've come up with several ways of storing the covariance matrix, but none is efficient (i.e., certain important queries take too long to execute).

I'd appreciate any relevant thoughts.

Reply With Quote
  #2  
Old December 30th, 2003, 06:39 PM
vanekl vanekl is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Dec 2003
Posts: 229 vanekl User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 5
Well, it depends primarily on the 'certain important queries'. Perhaps you could
explain them.

Solution 1: the obvious solution:
GPS_ID INT
COVARIANCE_VALUE DOUBLE
INDEX_X TINYINT /* 0 - 59 */
INDEX_Y TINYINT /* 0 - 59 */

Solution 2: store entire covariance matrix on one long, fixed-width text string.
You could make each field fixed width, so if you wanted 10 digits of precision for each
field the text string would be exactly 10 * 60 * 60 characters long. This would make searching
on a specific field easy. You could remove all decimal points and just use an implied
decimal point. Or you could store every field in scientific notation.

Solution 3: use RLE, Run-Length Encoding. Compress the matrix into one text string
by encoding the portions of the matrix that are the same.

Example: If the first 5 cells in the matrix are zeros, and the next cell is 1.23,
and the following 6 cells are zero, the encoding could be something like:

R5V0 1.23 R6V0

It all depends on what type of queries on this data you plan on doing.

Reply With Quote
Reply

Viewing: Dev Shed ForumsDatabasesDatabase Management > Database design -- storing sparse matrices


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 3 hosted by Hostway