UNIX Help
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsOperating SystemsUNIX Help

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
You eat, breathe and sleep innovation. Build your mobile intelligence with BlackBerry® experts this July. Register Today!
  #1  
Old April 9th, 2005, 07:46 AM
chrisjj chrisjj is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Apr 2005
Posts: 13 chrisjj User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 18 m 51 sec
Reputation Power: 0
Talking Awk reading CSV

What's the simplest way to get Awk to parse CSV input records such as

col1,"col2","col""3""","col,4"

I.e. such that

print $1"|"$2"|"$3"|"$4

yields

col1|col2|col"3"|col,4

This one has had me stumped for ages! Any help greatly appreciated. Thanks.

Reply With Quote
  #2  
Old April 12th, 2005, 08:38 AM
zby zby is offline
Contributing User
Dev Shed Novice (500 - 999 posts)
 
Join Date: Jul 2003
Location: Prague, Czech Republic
Posts: 825 zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 6 Days 16 h 4 m 55 sec
Reputation Power: 65
Quote:
col1,"col2","col""3""","col,4"


You definetly have to agree with the data supplier on the fixed structure of the CSV. You may find out yourself wasting your time trying to make a script work with data that come in everytime with a new hickup.

try this:
Code:
[zby@montana c1]$ echo "col1,\"col2\",\"col\"\"3\"\"\",\"col,4\"" | awk -F ",\"" '{print $1"|"$2"|"$3"|"$4}'|sed -e s/\"//g
col1|col2|col3|col,4
[zby@montana c1]$

Reply With Quote
  #3  
Old April 14th, 2005, 03:56 PM
frood frood is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Apr 2005
Posts: 4 frood User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 1 h 59 m 5 sec
Reputation Power: 0
The field seperator variable

Set the FS to a comma, or whatever else you want to use, then deal with all the quoting headaches as patiently as possible.

$ cat test.data
fred,george,sarah
marcy,wallace,larry

$awk '{print $3,$2,$1}' FS="," < test.data
sarah george fred
larry wallace marcy

Reply With Quote
  #4  
Old April 15th, 2005, 03:36 AM
zby zby is offline
Contributing User
Dev Shed Novice (500 - 999 posts)
 
Join Date: Jul 2003
Location: Prague, Czech Republic
Posts: 825 zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level)zby User rank is Second Lieutenant (5000 - 10000 Reputation Level) 
Time spent in forums: 6 Days 16 h 4 m 55 sec
Reputation Power: 65
What frood posts is true, of course, but if the data come in in sort of a "loose" format as chrisjj posted than all you can do is set the awk FS to: ," and hope it'll parse. Get rid of the quotes once sepparated.
There is also a recommendation how an average CSV file should look like.
You may find below links helpful. Ifound them via google just to be sure before I start off running my mouth too much.

http://www.creativyst.com/Doc/Articles/CSV/CSV01.htm
http://www.hi-tier.de/Entwicklung/Konzept/HITP/csvformat.html

Anyways, this may be helpfull:
Code:
[zby@montana c1]$ echo "col1,\"col2\",\"col\"\"3\"\"\",\"col,4\"" | awk -F ",\"" '{print $1"|"$2"|"$3"|"$4}'|sed -e s/\"//g
col1|col2|col3|col,4
[zby@montana c1]$

Reply With Quote
Reply

Viewing: Dev Shed ForumsOperating SystemsUNIX Help > Awk reading CSV


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 3 hosted by Hostway