Forums: » Register « |  Free Tools |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support |

New Free Tools on Dev Shed!

#1
August 15th, 2013, 01:31 PM
 eangola
Registered User

Join Date: Aug 2013
Location: Burlinton, VT
Posts: 18
Time spent in forums: 3 h 52 m 53 sec
Reputation Power: 0
HELP converting str to float from a list

Hi, I wrote this code on MatLab to storage data from certain file, organize it into matrices and then plot WindRose graphs. I am new to Python and I am trying to translate the MatLab code into python. Here's what I got so far:

Code:
```f = open ('//home//enrique//Desktop//WindCube//WLS7-94_merged_sta_files//WLS7-94_2012_01_02__00_10_00.sta')
data = [] # Define list
from numpy import * # Open numpy for later data handling
x = True
while x == True: # Read file until you reach 'Altitudes' (get rid of header')
Altitudes = line
if str.find(line,'Altitudes') == 0:
x = False
while line: #data starts, read data until the file is over
delimitador = str.find(line,':') # delimiter to get rid of date when data was collected
data.append(line[delimitador+4:]) #storage each line of data into a list
f.close() ```

As you can see, I got what matters from the data into a list of objects (basically lines of strings). what I did in MatLab was creating a loop that converts each line to double (str2num) and storages into a matrix, basically transform each line into a vector keeping the same format. I have no idea how to do this in matlab; the approach I am trying to take is to create a cellarray and storage each line as a row. The problem is, I have to convert the str into float. I tried the following:

Code:
`[float(i) for i in data[1]]`

Tried picking object 1 from the list, but it doesn't work, it says it cant convert str into float. Some of the values into the data are empty or 'NaN', since it is formatted to be read into matlab. So I guess the problem is the 'NaN', is there any way I could get read of the NaN??? I did somthing similar in matlab once using string functions, but I have no clue of how to do it in python :cry: .

Thank you.

#2
August 15th, 2013, 02:22 PM
 eangola
Registered User

Join Date: Aug 2013
Location: Burlinton, VT
Posts: 18
Time spent in forums: 3 h 52 m 53 sec
Reputation Power: 0
Code:
```f = open ('//home//enrique//Desktop//WindCube//WLS7-94_merged_sta_files//WLS7-94_2012_01_02__00_10_00.sta')
data = [] # Define list
from numpy import * # Open numpy for later data handling
x = True
while x == True: # Read file until you reach 'Altitudes' (get rid of header')
Altitudes = line
if str.find(line,'Altitudes') == 0:
x = False
while line: #data starts, read data until the file is over
delimitador = str.find(line,':') # delimiter to get rid of date when data was collected
data.append(line[delimitador+4:]) #storage each line of data into a list
f.close()

print data[1]
a = [float(i) for i in data[1]]
```

Code:
```>>>
24.3	NaN	NaN	NaN	0	3.66	0.93	6.08	167.9	1.72	1.72	-21.2	-22.9	2.32	23		3.89	0.94	5.95	169.5	1.72	1.72	-21.1	-23	2.17	16		4.34	0.73	6.22	170.4	1.49	1.49	-20.9	-23	2.25	11		4.46	0.91	6.16	166.3	0.63	0.63	-20.6	-22.8	2.13	8		5.72	1.42	7.9	170.6	1.22	1.22	-20.8	-23	2.22	5		6.66	0.47	7.32	177.5	1.49	1.14	-20.7	-22.9	2.28	2		7.45	0.88	9.98	185.5	1.14	0.6	-20.9	-22.9	1.88	2		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN

Traceback (most recent call last):
File "/home/enrique/Documents/Python/scan file.py", line 18, in <module>
a = [float(i) for i in data[1]]
ValueError: could not convert string to float: .```

Aparently the dots are the problem. From my lack of knowledge this does't make any sense to me, why are dots issues since I am converting to float not to integer???? is it character by character??? I think I know what the issue is, but I have no clue how to solve this!

#3
August 15th, 2013, 02:57 PM
 b49P23TIvg
Contributing User

Join Date: Aug 2011
Posts: 4,211
Time spent in forums: 1 Month 3 Weeks 2 Days 17 h 32 m 42 sec
Reputation Power: 455
You need to work with words, not characters.

You need str.split

a = [float(i) for i in data[1].split()]

You've done well.
Code:
```good = True
try:
import numpy
except:
print('install numpy!')
good = False

filename = '/home/enrique/Desktop/WindCube/WLS7-94_merged_sta_files/WLS7-94_2012_01_02__00_10_00.sta'

filename = 'example.dat'

with open(filename) as f:
for line in f:
if line.find('Altitudes') == 0:
break
data = [line[line.find(':')+4:] for line in f]

a = [[float(word) for word in line.split()] for line in data]

print(a)

if good:
A = numpy.array(a)    # or something like that
A = numpy.asarray(a)  # or something like that```
with exampe.dat as
Code:
```Altitudes
Altitudes
:xxx24.3	NaN	NaN	NaN	0	3.66	0.93	6.08	167.9	1.72	1.72	-21.2	-22.9	2.32	23		3.89	0.94	5.95	169.5	1.72	1.72	-21.1	-23	2.17	16		4.34	0.73	6.22	170.4	1.49	1.49	-20.9	-23	2.25	11		4.46	0.91	6.16	166.3	0.63	0.63	-20.6	-22.8	2.13	8		5.72	1.42	7.9	170.6	1.22	1.22	-20.8	-23	2.22	5		6.66	0.47	7.32	177.5	1.49	1.14	-20.7	-22.9	2.28	2		7.45	0.88	9.98	185.5	1.14	0.6	-20.9	-22.9	1.88	2		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN
:xxx24.3	NaN	NaN	NaN	0	3.66	0.93	6.08	167.9	1.72	1.72	-21.2	-22.9	2.32	23		3.89	0.94	5.95	169.5	1.72	1.72	-21.1	-23	2.17	16		4.34	0.73	6.22	170.4	1.49	1.49	-20.9	-23	2.25	11		4.46	0.91	6.16	166.3	0.63	0.63	-20.6	-22.8	2.13	8		5.72	1.42	7.9	170.6	1.22	1.22	-20.8	-23	2.22	5		6.66	0.47	7.32	177.5	1.49	1.14	-20.7	-22.9	2.28	2		7.45	0.88	9.98	185.5	1.14	0.6	-20.9	-22.9	1.88	2		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN		NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN	NaN```

Side note: where you wrote str.find(string, target) as an unbound method call, you can also write the shorter
string.find(target)

Code:
```TheType.theMethod(object_of_TheType, other, arguments)  # unbound method.

object_of_TheType.theMethod(other,arguments) # bound method```
__________________
[code]Code tags[/code] are essential for python code!

Last edited by b49P23TIvg : August 15th, 2013 at 09:37 PM. Reason: tab nightmare repaired, code valid in python 3

#4
August 15th, 2013, 03:21 PM
 eangola
Registered User

Join Date: Aug 2013
Location: Burlinton, VT
Posts: 18
Time spent in forums: 3 h 52 m 53 sec
Reputation Power: 0
wow thank you,

That's pretty neat, I'll try to keep going from here. It is my second day in python so I am a bit confused.

I have to stop thinking Matlab and start thinking Python.

I love the if good:

boom that's it, python is so smart.

#5
August 15th, 2013, 03:39 PM
 b49P23TIvg
Contributing User

Join Date: Aug 2011
Posts: 4,211
Time spent in forums: 1 Month 3 Weeks 2 Days 17 h 32 m 42 sec
Reputation Power: 455
I haven't yet installed numpy/scipy on this computer.

#6
August 15th, 2013, 03:45 PM
 eangola
Registered User

Join Date: Aug 2013
Location: Burlinton, VT
Posts: 18
Time spent in forums: 3 h 52 m 53 sec
Reputation Power: 0
yea that's the thing

the variable 'a' is a list right? with a certain number of objectrs (the lines). And then when trying to put that into an array each line has to be a row, and the number of columns? number of element of each line object in 'a' right??? so I would have to preallocaty the array? or can I just push the values into it?

#7
August 15th, 2013, 04:16 PM
 eangola
Registered User

Join Date: Aug 2013
Location: Burlinton, VT
Posts: 18
Time spent in forums: 3 h 52 m 53 sec
Reputation Power: 0
the code has certain errors though,

can't use f.readline because it complaints it is mixing reading methods .

and a = [[float(word)...]

ValueError: could not convert string to float: stamp

#8
August 15th, 2013, 09:47 PM
 b49P23TIvg
Contributing User

Join Date: Aug 2011
Posts: 4,211
Time spent in forums: 1 Month 3 Weeks 2 Days 17 h 32 m 42 sec
Reputation Power: 455
version for python 2 or 3

Code:
```import numpy

filename = '/home/enrique/Desktop/WindCube/WLS7-94_merged_sta_files/WLS7-94_2012_01_02__00_10_00.sta'

filename = 'example.dat'

with open(filename) as f:

for (i, line) in enumerate(all_the_lines):
if line.find('Altitudes') == 0:
break

data = []
for line in all_the_lines[i+2:]:
data.append([])
append = data[-1].append
for word in line[4:].split():
append(float(word))

# data is a list of lists.

print(numpy.array(data))
print(numpy.asarray(data))```

Concerning "stamp", I'm not responsible for your poor description of the data.

If "stamp" signals end of data then use
float(word)
in a try: except: statement to break from the loop.
eangola agrees!

#9
August 16th, 2013, 09:26 AM
 eangola
Registered User

Join Date: Aug 2013
Location: Burlinton, VT
Posts: 18
Time spent in forums: 3 h 52 m 53 sec
Reputation Power: 0
Thanks a lot.

I'll try this, if I run into a problem I'll try to solve it, and if I can't i'll post a chunk example of the data, but it is essentially pretty similar to your data example.

Thnx

#10
August 16th, 2013, 02:17 PM
 eangola
Registered User

Join Date: Aug 2013
Location: Burlinton, VT
Posts: 18
Time spent in forums: 3 h 52 m 53 sec
Reputation Power: 0
Quote:
 Originally Posted by b49P23TIvg Code: ```import numpy filename = '/home/enrique/Desktop/WindCube/WLS7-94_merged_sta_files/WLS7-94_2012_01_02__00_10_00.sta' filename = 'example.dat' with open(filename) as f: all_the_lines = f.readlines() for (i, line) in enumerate(all_the_lines): if line.find('Altitudes') == 0: break data = [] for line in all_the_lines[i+2:]: data.append([]) append = data[-1].append for word in line[4:].split(): append(float(word)) # data is a list of lists. print(numpy.array(data)) print(numpy.asarray(data))``` Concerning "stamp", I'm not responsible for your poor description of the data. If "stamp" signals end of data then use float(word) in a try: except: statement to break from the loop.

The data is nicely formatter into a cell array, now I can proceed to manipulate it as I want.

Thanks a lot, I learnt a lot doing this, and the file scanning is
definetely much more efficient than Matlab.

 Viewing: Dev Shed Forums > Programming Languages > Python Programming > HELP converting str to float from a list