#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2013
    Posts
    10
    Rep Power
    0

    Regexp log pattern


    #!/usr/bin/perl -w
    use strict;
    use warnings;

    my $log_pattern =q{(.*) \- (.*) \- \[(.*)\] \"(.*) (/[^/]+))};#(.*)\?(.*)} HTTP\/(.*)\" ([0-9]*) ([0-9]*) \"(.*)\" \"(.*)\" \"(.*)\"};
    my $entry ='213.4.28.244 - 95.123.101.114 - [21/May/2013:15:54:32] "GET /V/0/11573/display.cgi?2643943|3334115 HTTP/1.1" 200 55 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-us) AppleWebKit/523.10.6 (KHTML, like Gecko)" "67.60.185.31"';


    $entry =~ /$log_pattern/;
    print $1, "|";
    print $2, "|";
    print $3, "|";
    print $4, "|";
    #print $5, "|";
    #print $6, "|";
    #print $7, "|";
    #print $8, "|";
    #print $9, "|";
    #print $10, "|";
    #print $11, "\n";

    Can you plz check and give exact log patttern for above log record
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    /dev/null
    Posts
    164
    Rep Power
    19
    Originally Posted by ssrao
    my $log_pattern =q{(.*) \- (.*) \- \[(.*)\] \"(.*) (/[^/]+))};
    This appears to be working. Just continue writing the pattern in a similar fashion and you would have matched the entire log line. By the way, which part of the log entry do you exactly want to match. Is it the whole line or just a part of it?
  4. #3
  5. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2013
    Posts
    10
    Rep Power
    0
    #!/usr/bin/perl -w
    use strict;
    use warnings;

    my $log_pattern =q{(.*) \- (.*) \- \[(.*)\] \"(.*) (/[^/]+))};#(.*)\?(.*)} HTTP\/(.*)\" ([0-9]*) ([0-9]*) \"(.*)\" \"(.*)\" \"(.*)\"};
    my $entry ='13.4.28.244 - 95.123.101.114 - - [21/May/2013:15:58:24] "GET /V/0/11573/Granturismo1.mp4?start=0 HTTP/1.1 Ref.URL" 200 11125709 111257
    09 10 0 "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)"';


    $entry =~ /$log_pattern/;
    print $1, "|";
    print $2, "|";
    print $3, "|";
    print $4, "|";
    print $5, "|";
    print $6, "|";
    print $7, "|";
    print $8, "|";
    print $9, "|";
    print $10, "|";
    print $11, "|";
    print $12, "|";
    print $13, "|";
    print $14, "|";
    print $15, "|";
    print $16, "|";
    print $17, "|";
    print $18, "\n";

    Can you plz check and give exact log patttern for above log record

    Output should below format
    13.4.28.244|95.123.101.114|21/May/2013:15:58:24|GET|V|0|11573|Granturismo1.mp4?start=0|Granturismo1.mp4|1.1|Ref.URL|200|11125709|11125 7|09|10|0 |Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)

    I am not find right log pattern for above log and need to process all log file at one time.
    if there is any value in the respective field,need to replace with null value and process that record.

    Refer.URL is not mentioned as per forum rules
    Can any one help me in this regard.

    Thanks in advance.
  6. #4
  7. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2013
    Posts
    10
    Rep Power
    0
    Thanks nobbie..

    but it was not working and giving error as Unmatched ) in regex; marked by <-- HERE in m/(.*) \- (.*) \- \[(.*)\] \"(.*) (/[^/]+)) <-- HERE / at parse.log.pl line 9.
  8. #5
  9. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    /dev/null
    Posts
    164
    Rep Power
    19
    Remove the extra braces:
    Code:
    q{(.*) \- (.*) \- \[(.*)\] \"(.*) (/[^/]+)}
  10. #6
  11. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2013
    Posts
    10
    Rep Power
    0
    Hi nobbie,

    output is coming like this way
    213.4.28.244|95.123.101.114|21/May/2013:15:54:32|GET|/V|

    but I do not want '/'

    output should be
    213.4.28.244|95.123.101.114|21/May/2013:15:54:32|GET|/V|

    final output will be
    .4.28.244|95.123.101.114|21/May/2013:15:58:24|GET|V|0|11573|Granturismo1.mp4?start=0|Granturismo1.mp4|1.1|Ref.URL|200|11125709|11125 7|09|10|0 |Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    /dev/null
    Posts
    164
    Rep Power
    19
    Originally Posted by noobie1000
    This appears to be working. Just continue writing the pattern in a similar fashion and you would have matched the entire log line. By the way, which part of the log entry do you exactly want to match. Is it the whole line or just a part of it?
    I had suggested that you continue writing the regex until it matched the whole string.

    Originally Posted by ssrao
    Code:
    my $log_pattern =q{(.*) \- (.*) \- \[(.*)\] \"(.*) (/[^/]+))};#(.*)\?(.*)} HTTP\/(.*)\" ([0-9]*) ([0-9]*) \"(.*)\" \"(.*)\" \"(.*)\"};
    Did you write this regex yourself?
  14. #8
  15. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2013
    Posts
    10
    Rep Power
    0
    No..I have taken this example from google...that is adapting my requirement.

    Could you plz give correct log pattern
  16. #9
  17. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2013
    Posts
    10
    Rep Power
    0
    Hi nobbie,

    Could you able to give regex log pattern or you want me to try in some other forum?

    Thanks in advance
  18. #10
  19. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    /dev/null
    Posts
    164
    Rep Power
    19
    Originally Posted by ssrao
    Could you able to give regex log pattern
    Try the below. But I must say, I just expanded the regex you provided in your previous posts. This is very inefficient and this may not match if you changed $entry.

    Code:
    my $log_pattern =q{(.*) \- (.*) \- \[(.*)\] \"(.*) /([^/]+)/([^/]+)/([^/]+)/([^?]+)?([^\s]+) HTTP\/([^\s]+) (Ref\.URL)\" (\d+) (\d+) (\d+) (\d+) (\d+) \"(.*)\"};
    my $entry ='13.4.28.244 - 95.123.101.114 - [21/May/2013:15:58:24] "GET /V/0/11573/Granturismo1.mp4?start=0 HTTP/1.1 Ref.URL" 200 11125709 11125709 10 0 "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)"';
    Originally Posted by ssrao
    or you want me to try in some other forum?
    I don't suppose you're under any obligation to use only forums.devshed.com , are you?
  20. #11
  21. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jun 2013
    Posts
    10
    Rep Power
    0
    No nobbie...I did not intended like that...

    Anyway it is working now.......thanks a lot

IMN logo majestic logo threadwatch logo seochat tools logo