PostgreSQL Help
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsDatabasesPostgreSQL Help

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
AT&T devCentral & BlackBerry(r) Webcast Series: BlackBerry and GPS -Build Location Awareness into your BlackBerry Applications, July 10th-1:00PM EST. Register Today!
  #1  
Old March 31st, 2008, 09:22 AM
baxteras baxteras is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2004
Posts: 41 baxteras User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 11 h 31 m 8 sec
Reputation Power: 5
Search for a string in a text column (1 million rows+)

hi,
i have a table where a store data. there is also a text column where there is an average of 1000 chars in each entry.
this table has more that 1 million rows.

what i want, is to do a search to the text column for some text.
how can i do this to speed up things?
doing it with LIKE is not a solution (too much time)
is there some way to get some results in milliseconds?


question 2: count() on this table for all rows is also time-expensive.
keeping the count of all rows in another table and updating it with every insert and delete is a solution?
what would you suggest?

thanx in advance

Reply With Quote
  #2  
Old March 31st, 2008, 10:16 AM
pabloj's Avatar
pabloj pabloj is online now
Modding: Oracle MsSQL Firebird
Dev Shed God 6th Plane (7500 - 7999 posts)
 
Join Date: Jun 2001
Location: Outside US
Posts: 7,687 pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level)pabloj User rank is Captain (20000 - 30000 Reputation Level) 
Time spent in forums: 2 Months 2 Weeks 5 Days 17 h 59 m 31 sec
Reputation Power: 259
Did you look into full text indexing?
Also, if you keep fresh table stats you should easily have an approximate record count for that table (here) .

Last edited by pabloj : March 31st, 2008 at 10:20 AM.

Reply With Quote
  #3  
Old March 31st, 2008, 02:26 PM
pgFrank's Avatar
pgFrank pgFrank is offline
So help me Codd
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Feb 2008
Location: Brazil
Posts: 129 pgFrank User rank is Sergeant (500 - 2000 Reputation Level)pgFrank User rank is Sergeant (500 - 2000 Reputation Level)pgFrank User rank is Sergeant (500 - 2000 Reputation Level)pgFrank User rank is Sergeant (500 - 2000 Reputation Level)pgFrank User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Day 17 h 45 m 18 sec
Reputation Power: 18
See the Tsearch2-functions in PostgreSQL 8.3 (latest version) or install Tsearch2 yourself in an older version of PostgreSQL (only takes a minute).

Reply With Quote
  #4  
Old March 31st, 2008, 02:49 PM
baxteras baxteras is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jul 2004
Posts: 41 baxteras User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 11 h 31 m 8 sec
Reputation Power: 5
Quote:
Originally Posted by pabloj
Did you look into full text indexing?
Also, if you keep fresh table stats you should easily have an approximate record count for that table (here) .


i found tsearch2.. seems pretty good , ill give it a try..

for my 2nd question,
the problem is that i need a count(*) of a subset of rows, not all rows.. from 1.000.000 the matching may be up to 50.000..
how can i count these rows fast?

Reply With Quote
  #5  
Old April 1st, 2008, 04:42 AM
pgFrank's Avatar
pgFrank pgFrank is offline
So help me Codd
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Feb 2008
Location: Brazil
Posts: 129 pgFrank User rank is Sergeant (500 - 2000 Reputation Level)pgFrank User rank is Sergeant (500 - 2000 Reputation Level)pgFrank User rank is Sergeant (500 - 2000 Reputation Level)pgFrank User rank is Sergeant (500 - 2000 Reputation Level)pgFrank User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Day 17 h 45 m 18 sec
Reputation Power: 18
SELECT COUNT(*) FROM yourtable WHERE columnname = 'your value';

You do need an index on the columnname in the WHERE, otherwise it will slow down the query.

Reply With Quote
Reply

Viewing: Dev Shed ForumsDatabasesPostgreSQL Help > Search for a string in a text column (1 million rows+)


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 3 hosted by Hostway