#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2005
    Posts
    19
    Rep Power
    0

    Strings From os.system


    I am new to Python (which will be obvious very soon), and I am having trouble conceptualizing something rather simple.

    I am trying to do some simple shell stuff using Python (even though I know I can do it with Bash). My command thus far is:

    Code:
    include system
    include os
    
    os.system ("lsof -i | grep -i est")
    This gives me the output that I would have got from running this as a command, obviously, but the problem is that I want to work on each of the returned lines further.

    I'd like to be able to do things like trim the hostname/ip out of the lines using regex (easy enough), and do things with those addresses. The problem is I need to be able to get them into variables in the first place. The problem is, I can't figure out how to get these lines into Python to work with them. Using os.system just returns it to stdout immediately, and I don't see how to stop that.

    Anyway, hopefully someone can see what I'm talking about and help the noob.

    Thanks.
    Last edited by danielrm26; February 15th, 2005 at 06:21 PM. Reason: Name change
  2. #2
  3. Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2005
    Posts
    174
    Rep Power
    11
    hi

    can you give us some example lines of output and what kind of output you expect to produce? that will help a lot

    i'm sorry I'm not familiar with the unix commands (other than grep). Basically if I got you right, I'd dump the output into a file, and then loop through it's lines to produce my variables for each loop of each line returned until I run out of variables.

    os.walk is a recursive method that goes down each directory branch and then each succeeding branch, so take a look at that one. But since I don't know what you're looking for in particular I'm not sure if this is relevant here.
  4. #3
  5. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Jan 2005
    Posts
    37
    Rep Power
    10
    I don't have a unix box here to test this out, but I bet using the popen method would work, i.e.

    >>> import os
    >>> text = os.popen("lsof -i | grep -i est").read()

    now you should be able to capture all the output from stdout into text, and process at will.

    hope this works for you...
  6. #4
  7. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    os.popen() like other file() objects also has a readlines() method in case you need to do something for each line in the output i.e. from ls -l command.

    You should also look at the subprocess module in 2.4 if you need to do anything more complex, in fact you should probably check it out even if you don’t.

    Mark .
    programming language development: www.netytan.com Hula

  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2005
    Posts
    19
    Rep Power
    0
    Excellent.

    So here's a sample line of text I'd like to mess with:

    Code:
    firefox-b 8681 foouser   22u  IPv4 0x02c53824      0t0  TCP 192.168.0.1:64679->44.223.49.120:http (ESTABLISHED)
    How can I then perform text functions on these lines, i.e. go from hostname to IP, test ping time to, etc. I don't want to know how to do those things, but how to start processing the lines.

    For example, I know I want to capture everything from between "TCP" and the next ":"; that'll represent the IP. I want to then capture those into variables to work them.

    Hmm, better yet, where's a good tutorial on text processing with Python? I'll RTFM from there. I know of the re method (regex) to some degree, but what I need is a solid tutorial with some examples. RTFM does wonders once pointed in the right direction.



    This forum rocks, btw.
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed Novice (500 - 999 posts)

    Join Date
    Nov 2003
    Posts
    624
    Rep Power
    34
    Python has good string manipulation even before you get to the re module.

    e.g. string.find and string slicing:

    Code:
    >>> txt = "firefox-b 8681 foouser   22u  IPv4 0x02c53824      0t0  TCP 192.168.0.1:64679->44.223.49.120:http (ESTABLISHED)"
    >>> txt
    'firefox-b 8681 foouser   22u  IPv4 0x02c53824      0t0  TCP 192.168.0.1:64679->44.223.49.120:http (ESTABLISHED)'
    >>> txt.find("TCP ")
    56
    >>> txt.find(":")
    71
    >>> txt[56:71]
    'TCP 192.168.0.1'
    >>> txt[60:71]
    '192.168.0.1'
    There is not just a tutorial, but a whole book on text processing with Python:

    http://gnosis.cx/TPiP/

    Though at a glance it does look a little... comprehensive ... to be an introduction.

    Here is a more introductory and likely more immediately helpful guide to text processing:

    http://www-106.ibm.com/developerwork...l-python5.html

    But it is from 2000 and a bit out of date now - a lot of the examples he uses the string module for have been moved to methods of the string type, so:

    Code:
    >>> s1 = "text"
    >>> s2 = "ex"
    >>> import string
    >>> string.find(s1, s2)
    
    becomes
    
    >>> s1 = "text"
    >>> s1.find(s2)
    Last edited by sfb; February 17th, 2005 at 05:12 PM.
  12. #7
  13. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2005
    Posts
    19
    Rep Power
    0
    Hmm, ok, I've made a bit of progress, i.e. I'm now able to isolate the hostname from a single line of output. Now, I'm trying to figure out how to iterate over each line in the lsof output. If I try "for line in est", it prints out vertically, with one character per line.

    Here's what I have so far:

    Code:
    import os
    
    est = os.popen ("lsof -i | grep -i est").read()
    
    front = est.find ('->')
    back = est.find ('(')
    
    host_service = est[front+2:back]
    host_delimiter = host_service.find(':')
    host = host_service[:host_delimiter]
    
    print host
    Ultimately, my goal is to collect these hostnames, resolve them and get some info via whois/dig, and then show some information on the hosts in a neat fashion. The issue is, I need to be able to get *all* lines from lsof, not just the first one -- which is what I'm getting now.

    Thanks for all your help, guys...
  14. #8
  15. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    You can get each of the lines returned from the command by using the readlines() method. I don't have access to Python right now but here's an example:

    Code:
    import os
    
    ests = os.popen ("lsof -i | grep -i est").readlines()
    
    for est in ests:
        front = est.find ('->')
        back = est.find ('(')
    
        host_service = est[front+2:back]
        host_delimiter = host_service.find(':')
        host = host_service[:host_delimiter]
    
        print host
    Note: this code is untested. If you have any problems with it please let me know.

    The reason that the results were being printed in this way is because you were iterating over a string; more specifically each character in the string .

    Hope this helps,

    Mark.
    programming language development: www.netytan.com Hula

  16. #9
  17. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2005
    Posts
    19
    Rep Power
    0
    Originally Posted by netytan
    Note: this code is untested. If you have any problems with it please let me know.

    The reason that the results were being printed in this way is because you were iterating over a string; more specifically each character in the string .
    Ah, yes. Thanks.

    Ok, one more question on this -- could I use some sort of iterator like i++ for each line to capture each hostname as a variable? In other words, I want to be able to capture each hostname into a variable (like h1, h2, h3, etc.), so I can do stuff to them later.

    Any idea how one would go about that?

    Thanks...
  18. #10
  19. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Feb 2005
    Posts
    78
    Rep Power
    10
    Originally Posted by danielrm26
    Ah, yes. Thanks.

    Ok, one more question on this -- could I use some sort of iterator like i++ for each line to capture each hostname as a variable? In other words, I want to be able to capture each hostname into a variable (like h1, h2, h3, etc.), so I can do stuff to them later.

    Any idea how one would go about that?

    Thanks...
    It really depends what you want to do with it and how you want to access it.
    You could use a dictionary:
    Code:
    hostdata = {}
    for est in ests:
         ...
         hostdata[host] = hostservice
    
    hserv1 = hostdata["foo"]
    If order is important, store tuples of (hostname, hostdata) in a list:
    Code:
    hostdata = []
    for est in ests:
        ...
        hostdata.append((host, hostservices))
    
    host1, hserv1 = hostdata[0]
    Alternatively, you could use objects:
    Code:
    class Host(object):
        def __init__(self, host, hostservice):
           self.host = host
           ...
    
    hostdata = {}
    for est in ests:
        ...
       curr = Host(host, hostservice)
       hostdata[host] = curr
    
    hserv1 = hostdata["foo"].hostservice
    I would tend to use the object version (since when you're adding features, it can make organisation easier). YMMV.

    --OH.
  20. #11
  21. Hello World :)
    Devshed Frequenter (2500 - 2999 posts)

    Join Date
    Mar 2003
    Location
    Hull, UK
    Posts
    2,537
    Rep Power
    69
    Python doesn’t include the Increment and Decrement operators, and may never. It also doesn’t allow you to create a numbered variable like you can in Perl and shell scripts i.e. $Variable$i;

    Not strictly true, it can be done but, it's a bit of a hack: by interacting with the different namespaces (global & local) you can create numbered variables. This works because the namespaces are Dictionaries so you have as much control over the variables as you do Strings .

    To do this you would simply create a string with the variable name you want to use it as the Dictionaries key; namespaceName[key] = value.

    If you want to do something like this then I would suggest that you use a Dictionary or List as Hydroxide showed in his post since IMO numbered variables decrease program clarity .

    You can use a variable inside your loop, but this will be overwritten each time the loop repeats. If you want to do it lke this then you need to do all your work inside the loop, which doesn't seem to be what you want .

    Hope this helps,

    Mark.
    programming language development: www.netytan.com Hula

  22. #12
  23. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2003
    Posts
    74
    Rep Power
    12

    find a process


    Here is the script I came up with. Quite simple. I just thought there might have been a function built into python which may have done it for me.

    Code:
    # This script searches to see if an instance of "PROGRAM_NAME" is running.
    # If it is not, then start it.
    # It could be run from cron at whatever desired interval.
    
    import os;
    
    found=0;
    
    text=os.popen("ps");
    for item in text:
        if(item.find('PROGRAM_NAME')>-1):
            found=1;
    
    if found!=1:
        os.system("PROGRAM_NAME.py");

IMN logo majestic logo threadwatch logo seochat tools logo