#1
  1. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2013
    Location
    Burlinton, VT
    Posts
    36
    Rep Power
    2

    HELP converting str to float from a list


    Hi, I wrote this code on MatLab to storage data from certain file, organize it into matrices and then plot WindRose graphs. I am new to Python and I am trying to translate the MatLab code into python. Here's what I got so far:

    Code:
    f = open ('//home//enrique//Desktop//WindCube//WLS7-94_merged_sta_files//WLS7-94_2012_01_02__00_10_00.sta')
    data = [] # Define list
    from numpy import * # Open numpy for later data handling
    x = True 
    while x == True: # Read file until you reach 'Altitudes' (get rid of header')
        line = f.readline()
        Altitudes = line
        if str.find(line,'Altitudes') == 0:
            x = False
    header = f.readline() # Get rid of last line of header
    while line: #data starts, read data until the file is over
        line = f.readline() #read line by line
        delimitador = str.find(line,':') # delimiter to get rid of date when data was collected
        data.append(line[delimitador+4:]) #storage each line of data into a list
    f.close()
    As you can see, I got what matters from the data into a list of objects (basically lines of strings). what I did in MatLab was creating a loop that converts each line to double (str2num) and storages into a matrix, basically transform each line into a vector keeping the same format. I have no idea how to do this in matlab; the approach I am trying to take is to create a cellarray and storage each line as a row. The problem is, I have to convert the str into float. I tried the following:

    Code:
    [float(i) for i in data[1]]
    Tried picking object 1 from the list, but it doesn't work, it says it cant convert str into float. Some of the values into the data are empty or 'NaN', since it is formatted to be read into matlab. So I guess the problem is the 'NaN', is there any way I could get read of the NaN??? I did somthing similar in matlab once using string functions, but I have no clue of how to do it in python :cry: .

    Help please!!!

    Thank you.
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2013
    Location
    Burlinton, VT
    Posts
    36
    Rep Power
    2
    Code:
    f = open ('//home//enrique//Desktop//WindCube//WLS7-94_merged_sta_files//WLS7-94_2012_01_02__00_10_00.sta')
    data = [] # Define list
    from numpy import * # Open numpy for later data handling
    x = True 
    while x == True: # Read file until you reach 'Altitudes' (get rid of header')
        line = f.readline()
        Altitudes = line
        if str.find(line,'Altitudes') == 0:
            x = False
    header = f.readline() # Get rid of last line of header
    while line: #data starts, read data until the file is over
        line = f.readline() #read line by line
        delimitador = str.find(line,':') # delimiter to get rid of date when data was collected
        data.append(line[delimitador+4:]) #storage each line of data into a list
    f.close() 
    
    print data[1]
    a = [float(i) for i in data[1]]
    Code:
    >>> 
    24.3	NaN	NaN	NaN	0	3.66	0.93	6.08	167.9	1.72	1.72	-21.2	-22.9	2.32	23		3.89	0.94	5.95	169.5	1.72	1.72	-21.1	-23	2.17	16		4.34	0.73	6.22	170.4	1.49	1.49	-20.9	-23	2.25	11		4.46	0.91	6.16	166.3	0.63	0.63	-20.6	-22.8	2.13	8		5.72	1.42	7.9	170.6	1.22	1.22	-20.8	-23	2.22	5		6.66	0.47	7.32	177.5	1.49	1.14	-20.7	-22.9	2.28	2		7.45	0.88	9.98	185.5	1.14	0.6	-20.9	-22.9	1.88	2		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
    
    Traceback (most recent call last):
      File "/home/enrique/Documents/Python/scan file.py", line 18, in <module>
        a = [float(i) for i in data[1]]
    ValueError: could not convert string to float: .
    Aparently the dots are the problem. From my lack of knowledge this does't make any sense to me, why are dots issues since I am converting to float not to integer???? is it character by character??? I think I know what the issue is, but I have no clue how to solve this!
  4. #3
  5. Contributing User
    Devshed Demi-God (4500 - 4999 posts)

    Join Date
    Aug 2011
    Posts
    4,963
    Rep Power
    481
    You need to work with words, not characters.

    You need str.split


    a = [float(i) for i in data[1].split()]

    You've done well.
    Code:
    good = True
    try:
        import numpy
    except:
        print('install numpy!')
        good = False
    
    filename = '/home/enrique/Desktop/WindCube/WLS7-94_merged_sta_files/WLS7-94_2012_01_02__00_10_00.sta'
    
    
    filename = 'example.dat'
    
    
    with open(filename) as f:
        for line in f:
            if line.find('Altitudes') == 0:
                break
        header = f.readline() # Get rid of last line of header
        data = [line[line.find(':')+4:] for line in f]
    
    a = [[float(word) for word in line.split()] for line in data]
    
    
    print(a)
    
    if good:
        A = numpy.array(a)    # or something like that
        A = numpy.asarray(a)  # or something like that
    with exampe.dat as
    Code:
    Altitudes
    Altitudes
    :xxx24.3	NaN	NaN	NaN	0	3.66	0.93	6.08	167.9	1.72	1.72	-21.2	-22.9	2.32	23		3.89	0.94	5.95	169.5	1.72	1.72	-21.1	-23	2.17	16		4.34	0.73	6.22	170.4	1.49	1.49	-20.9	-23	2.25	11		4.46	0.91	6.16	166.3	0.63	0.63	-20.6	-22.8	2.13	8		5.72	1.42	7.9	170.6	1.22	1.22	-20.8	-23	2.22	5		6.66	0.47	7.32	177.5	1.49	1.14	-20.7	-22.9	2.28	2		7.45	0.88	9.98	185.5	1.14	0.6	-20.9	-22.9	1.88	2		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
    :xxx24.3	NaN	NaN	NaN	0	3.66	0.93	6.08	167.9	1.72	1.72	-21.2	-22.9	2.32	23		3.89	0.94	5.95	169.5	1.72	1.72	-21.1	-23	2.17	16		4.34	0.73	6.22	170.4	1.49	1.49	-20.9	-23	2.25	11		4.46	0.91	6.16	166.3	0.63	0.63	-20.6	-22.8	2.13	8		5.72	1.42	7.9	170.6	1.22	1.22	-20.8	-23	2.22	5		6.66	0.47	7.32	177.5	1.49	1.14	-20.7	-22.9	2.28	2		7.45	0.88	9.98	185.5	1.14	0.6	-20.9	-22.9	1.88	2		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

    Side note: where you wrote str.find(string, target) as an unbound method call, you can also write the shorter
    string.find(target)

    Code:
    TheType.theMethod(object_of_TheType, other, arguments)  # unbound method.
    
    object_of_TheType.theMethod(other,arguments) # bound method

    Comments on this post

    • eangola agrees : Pretty helpful
    Last edited by b49P23TIvg; August 15th, 2013 at 09:37 PM. Reason: tab nightmare repaired, code valid in python 3
    [code]Code tags[/code] are essential for python code and Makefiles!
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2013
    Location
    Burlinton, VT
    Posts
    36
    Rep Power
    2
    wow thank you,

    That's pretty neat, I'll try to keep going from here. It is my second day in python so I am a bit confused.

    I have to stop thinking Matlab and start thinking Python.

    I love the if good:

    boom that's it, python is so smart.
  8. #5
  9. Contributing User
    Devshed Demi-God (4500 - 4999 posts)

    Join Date
    Aug 2011
    Posts
    4,963
    Rep Power
    481
    I haven't yet installed numpy/scipy on this computer.
    [code]Code tags[/code] are essential for python code and Makefiles!
  10. #6
  11. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2013
    Location
    Burlinton, VT
    Posts
    36
    Rep Power
    2
    yea that's the thing

    the variable 'a' is a list right? with a certain number of objectrs (the lines). And then when trying to put that into an array each line has to be a row, and the number of columns? number of element of each line object in 'a' right??? so I would have to preallocaty the array? or can I just push the values into it?
  12. #7
  13. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2013
    Location
    Burlinton, VT
    Posts
    36
    Rep Power
    2
    the code has certain errors though,

    can't use f.readline because it complaints it is mixing reading methods .

    and a = [[float(word)...]

    ValueError: could not convert string to float: stamp
  14. #8
  15. Contributing User
    Devshed Demi-God (4500 - 4999 posts)

    Join Date
    Aug 2011
    Posts
    4,963
    Rep Power
    481

    version for python 2 or 3


    Code:
    import numpy
    
    filename = '/home/enrique/Desktop/WindCube/WLS7-94_merged_sta_files/WLS7-94_2012_01_02__00_10_00.sta'
    
    
    filename = 'example.dat'
    
    
    with open(filename) as f:
        all_the_lines = f.readlines()
    
    for (i, line) in enumerate(all_the_lines):
        if line.find('Altitudes') == 0:
            break
    
    data = []
    for line in all_the_lines[i+2:]:
        data.append([])
        append = data[-1].append
        for word in line[4:].split():
            append(float(word))
    
    # data is a list of lists.
    
    print(numpy.array(data))
    print(numpy.asarray(data))
    Concerning "stamp", I'm not responsible for your poor description of the data.

    If "stamp" signals end of data then use
    float(word)
    in a try: except: statement to break from the loop.

    Comments on this post

    • eangola agrees
    [code]Code tags[/code] are essential for python code and Makefiles!
  16. #9
  17. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2013
    Location
    Burlinton, VT
    Posts
    36
    Rep Power
    2
    Thanks a lot.

    I'll try this, if I run into a problem I'll try to solve it, and if I can't i'll post a chunk example of the data, but it is essentially pretty similar to your data example.

    Thnx
  18. #10
  19. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Aug 2013
    Location
    Burlinton, VT
    Posts
    36
    Rep Power
    2
    Originally Posted by b49P23TIvg
    Code:
    import numpy
    
    filename = '/home/enrique/Desktop/WindCube/WLS7-94_merged_sta_files/WLS7-94_2012_01_02__00_10_00.sta'
    
    
    filename = 'example.dat'
    
    
    with open(filename) as f:
        all_the_lines = f.readlines()
    
    for (i, line) in enumerate(all_the_lines):
        if line.find('Altitudes') == 0:
            break
    
    data = []
    for line in all_the_lines[i+2:]:
        data.append([])
        append = data[-1].append
        for word in line[4:].split():
            append(float(word))
    
    # data is a list of lists.
    
    print(numpy.array(data))
    print(numpy.asarray(data))
    Concerning "stamp", I'm not responsible for your poor description of the data.

    If "stamp" signals end of data then use
    float(word)
    in a try: except: statement to break from the loop.
    The data is nicely formatter into a cell array, now I can proceed to manipulate it as I want.

    Thanks a lot, I learnt a lot doing this, and the file scanning is
    definetely much more efficient than Matlab.

IMN logo majestic logo threadwatch logo seochat tools logo