#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2013
    Posts
    23
    Rep Power
    0

    Sed command for fast access


    Guys,
    I am using sed command to replace a particular char in a file which consist of huge data around 8.5 lakhs but its taking a lot time if I use that can any one suggest me with performed command with sed or

    Command using:
    Sed -e 's/,/ /g;s/*;/,g' <output.txt> output.temp
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2006
    Posts
    2,448
    Rep Power
    1751
    Are you missing a delimiter on the final substitution?
    Also I am not convinced you can group two commands together like that, each should be specified with it's own -e:

    Code:
    sed -e 's/,/ /g' -e 's/*;/,/g' ...
    The first will change all commas to space, what do you intend with the second?
    The moon on the one hand, the dawn on the other:
    The moon is my sister, the dawn is my brother.
    The moon on my left and the dawn on my right.
    My brother, good morning: my sister, good night.
    -- Hilaire Belloc
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2013
    Posts
    23
    Rep Power
    0
    once the , is replaced with space , then space will be replaced with *, which need to be accessed fast
  6. #4
  7. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jul 2013
    Posts
    23
    Rep Power
    0
    sorry you are correct
    , replaced to space
    and *, replaces ,


    sed -e 's/,/ /g' -e 's/*;/,/g'

    but its taking lot of time for 8lakh records which should be optimized
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Regular (2000 - 2499 posts)

    Join Date
    Mar 2006
    Posts
    2,448
    Rep Power
    1751
    Ok, so we convert all commas found to the space character and then we replace ...? What your sed says to do and what you say it want to do are the reverse of one another! Beware of using * as it has special meaning in regexes. If you want to replace asterisk followed by semi-colon with a comma (which is what your sed says) then use:
    Code:
    -e 's/\*;/,/g'
    where the \ quotes the asterisk character forcing sed to treat it as the 'raw' character, not the special use regex has for it.

    <edit>
    Forgot to mention: processing 800,000 records will take as long as it needs to take! Whatever you do it will end up meaning that all records will need to be read, and that an equal number will need to be written. Even if you could pre-cache the input file you'll still have the over-head of writing 800,000 records, so it'll never be devastatingly quick.
    </edit>
    Last edited by SimonJM; July 23rd, 2013 at 06:51 AM.
    The moon on the one hand, the dawn on the other:
    The moon is my sister, the dawn is my brother.
    The moon on my left and the dawn on my right.
    My brother, good morning: my sister, good night.
    -- Hilaire Belloc

IMN logo majestic logo threadwatch logo seochat tools logo