SunQuest
           UNIX Help
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsOperating SystemsUNIX Help

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
Get inside! Sample the range of functionality easily built with JMSL Library for Time Series Data Analysis, Heat Maps, Portfolio Optimization, Monte Carlo Simulation, Stock Price Charting and more. Download Now!
  #1  
Old February 1st, 2005, 02:02 PM
dsshed dsshed is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Feb 2005
Posts: 10 dsshed User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 3 h 8 m 13 sec
Reputation Power: 0
Post sed usage

Hi

I have a file with data similar to the line below

1'~'Rowone#@#@#2'~'Rowtwo#@#@#3'~'Rowthree#@#@#4'~'Rowfour#@#@#5'~'Rowfive#@#@#


all the data is in one single stream, i mean to say it's one
single line of data.

when i try to vi the file i cant because the file itself is huge
and the data embedded in the file is in one single big line.

i need a script or command that can help me to create a file
with data as below from the file shown above

1|Rowone
2|Rowtwo
3|Rowthree
4|Rowfour
5|Rowfive


as you see in the source file the column separator is '~' and the row terminator is #@#@#
this is want to change to ,| as the column separator and
unix carriage return as the row terminator

any help would be appreciated

thanks
dsshed

Reply With Quote
  #2  
Old February 1st, 2005, 02:38 PM
vgersh99 vgersh99 is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jan 2005
Posts: 47 vgersh99 User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 Days 5 h 36 m 43 sec
Reputation Power: 4
Send a message via AIM to vgersh99 Send a message via MSN to vgersh99 Send a message via Yahoo to vgersh99
one way:

sed -e "s#'~'#|#g;s/#@#@#/#/g" file | tr '#' '\n'

Reply With Quote
  #3  
Old February 1st, 2005, 03:29 PM
dsshed dsshed is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Feb 2005
Posts: 10 dsshed User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 3 h 8 m 13 sec
Reputation Power: 0
Post sed usage

vgresh99

your reply is very apt and it works , and thank you for that
but here is the problem, the command you send doesnt
work on the file i am operating on.

but when i cat the file and copy half of its content and paste
it to a new file and run your command on this new file it works like a charm.

but when i run this against my original file it does nothing.
as i said i can't even open the file with vi as it comes up
with a message as below
Line too long

is there a way to get around this limit of size on the line
i am reading from my file, whic seems to be an extremely
lengthy line and sed command is not able to process it

any advise or help is very much appreciated

dsshed

Reply With Quote
  #4  
Old February 1st, 2005, 03:36 PM
vgersh99 vgersh99 is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jan 2005
Posts: 47 vgersh99 User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 Days 5 h 36 m 43 sec
Reputation Power: 4
Send a message via AIM to vgersh99 Send a message via MSN to vgersh99 Send a message via Yahoo to vgersh99
what does 'wc -c file' return?

If on Solaris, try /usr/xpg4/bin/sed instead of plain old sed.

Reply With Quote
  #5  
Old February 1st, 2005, 04:06 PM
dsshed dsshed is offline
Registered User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Feb 2005
Posts: 10 dsshed User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 3 h 8 m 13 sec
Reputation Power: 0
yes, that did it
xpg4 version of sed works fine

wc -c file output is 4143
i tried on that file and it worked very well

i need to see how to fine tune it to work on a file of size
of 1434194
i ran the command on this its taking some time to
process, which is abvious but needs to be tuned


one other way i am trying is this
cat CS.txt | awk 'BEGIN { RS="#@#@#"} {print $1}'

in those lines


thank you

Reply With Quote
  #6  
Old February 1st, 2005, 04:20 PM
vgersh99 vgersh99 is offline
Contributing User
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Jan 2005
Posts: 47 vgersh99 User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: 5 Days 5 h 36 m 43 sec
Reputation Power: 4
Send a message via AIM to vgersh99 Send a message via MSN to vgersh99 Send a message via Yahoo to vgersh99
Quote:
Originally Posted by dsshed
yes, that did it
xpg4 version of sed works fine

wc -c file output is 4143
i tried on that file and it worked very well

i need to see how to fine tune it to work on a file of size
of 1434194
i ran the command on this its taking some time to
process, which is abvious but needs to be tuned


one other way i am trying is this
cat CS.txt | awk 'BEGIN { RS="#@#@#"} {print $1}'

in those lines


thank you



unfortunately any of the Sun-supplied awk-s don't allow specifying RS longer than 1 character - gawk does though.

Here's something to try with /usr/xpg4/bin/awk:

dsshed.awk:
Code:
BEGIN {
  FS="('~')|(#@#@#)"
  OFS="|"
}
{ for(i=1; i <= NF; i=i+2) print $i,$(i+1) }


/usr/xpg4/bin/awk -f dsshed.awk CS.txt

(no UUOC, pls)

Reply With Quote
Reply

Viewing: Dev Shed ForumsOperating SystemsUNIX Help > sed usage


Thread Tools  Search this Thread 
Search this Thread:

Advanced Search
Display Modes  Rate This Thread 
Rate This Thread:


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
View Your Warnings | New Posts | Latest News | Latest Threads | Shoutbox
Forum Jump


Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
  
 





© 2003-2008 by Developer Shed. All rights reserved. DS Cluster 5 hosted by Hostway