New Free Tools on Dev Shed!
We're Excited to announce that Dev Shed now has 70 free tools on the site. To learn more, click here!
|
 |
|
Dev Shed Forums
> Programming Languages
> Perl Programming
|
CSV Parsing
Discuss CSV Parsing in the Perl Programming forum on Dev Shed.
CSV Parsing Perl Programming forum discussing coding in Perl, utilizing Perl modules, and other Perl-related topics. Perl, the Practical Extraction and Reporting Language, is the choice for many for parsing textual information.
|
|
 |
|
|
|
|
Dev Shed Forums Sponsor:
|
|
|
July 28th, 2013, 11:35 AM
|
|
Registered User
|
|
Join Date: Jul 2013
Posts: 5
Time spent in forums: 2 h 5 m 31 sec
Reputation Power: 0
|
|
|
CSV Parsing
Hi.
I'm trying to parse a .csv file which contains the following data
2, 0, Pitch_bend_c, 0, 8242
2, 0, Control_c, 0, 11, 38
2, 0, Note_on_c, 0, 36, 71
What i would like to do is convert some events like notes and duration to a new csv but my main problem is searching through the csv file for specific numbers to apply the conversion.
Here's what i mean, given this line
2, 0, Note_on_c, 0, 36, 71
Get the 36 and convert it to C2, get the 37 and convert it to C# and so on.
If somebody knows of a sample to help me it would be appreciated.
|
July 28th, 2013, 11:52 AM
|
|
|
|
The line you reference doesn't contain a 37 in any of the fields so no conversion.
What are the conversion rules? Why would 36 be converted to C2 and 37 be converted to C#?
What does your current code look like and how does it fail to do what you want?
|
July 28th, 2013, 12:04 PM
|
|
Registered User
|
|
Join Date: Jul 2013
Posts: 5
Time spent in forums: 2 h 5 m 31 sec
Reputation Power: 0
|
|
The 37 was another example that could could occur on the next line.
The conversion rules would be based on the MIDI CC notes table that i want to incorporate on the script.
I still haven't got any code written just a sample of a transposition script that looks like this
Code:
$offset = -12;
$percussion = 9;
while ($a = <>) {
# Recognise Note_on_c and Note_off_c records and crack into:
# $1 Start of record
# $2 Channel number
# $3 Note number
# $a Balance of record
if ($a =~ s/(\d+,\s*\d+,\s*Note_\w+,\s*(\d+),\s*)(\d+)//) {
$n = $3;
if ($2 != $percussion) {
$n += $offset;
}
if ($n < 0) {
next;
}
$a = "$1$n$a";
}
print($a);
}
|
July 28th, 2013, 12:21 PM
|
|
|
Since you haven't included any info on the "MIDI CC notes table", I can't help you with coding the conversion.
Don't use a regex for parsing csv data. Use the Text::CSV module.
$a is one of perl's built-in global vars used in sort routines (i.e., the sort function). It is best not to use it outside of the sort routine.
Your script should start out with these 2 pragmas:
Code:
use strict;
use warnings;
The strict pragma will require you to declare your vars, which is done with the 'my' keyword.
Code:
my $offset = -12;
my $percussion = 9;
|
July 28th, 2013, 12:33 PM
|
|
Registered User
|
|
Join Date: Jul 2013
Posts: 5
Time spent in forums: 2 h 5 m 31 sec
Reputation Power: 0
|
|
Where is the MIDI CC table.
Code:
http://www.tonalsoft.com/pub/news/pitch-bend.aspx
The conversion would be from the MIDI note number to the note name and frequency.
Thanks for the help so far.
|
July 28th, 2013, 01:11 PM
|
|
|
Create a hash where the keys are the MIDI note numbers and the values are the corresponding note names.
Example:
Code:
my %num2note = (
0 => 'C',
1 => 'C#',
2 => 'D',
# etc
);
As you parse each line in the csv file (using Text::CSV), you simply lookup the note number in the hash and replace that field with the corresponding note name.
Once each required field in the row has been updated, you output that updated row data to the new csv file.
Last edited by FishMonger : July 28th, 2013 at 01:14 PM.
|
July 28th, 2013, 01:47 PM
|
|
|
|
Once you have done the hash for the first octave (12 semitones), use a loop to fill it for the next octaves.
|
July 28th, 2013, 02:25 PM
|
|
Registered User
|
|
Join Date: Jul 2013
Posts: 5
Time spent in forums: 2 h 5 m 31 sec
Reputation Power: 0
|
|
So i've started to write something.
Here's what i have so far
Code:
#!/usr/bin/perl
use strict;
use warnings;
use Text::CSV;
my %num2note = (
0 => 'C-5',
1 => 'C#-5',
2 => 'D-5',
3 => 'D#-5',
4 => 'E-5',
5 => 'F-5',
6 => 'F#-5',
7 => 'G-5',
8 => 'G#-5',
9 => 'A-5',
10 => 'A#-5',
11 => 'B-5',
12 => 'C-4',
13 => 'C#-4',
14 => 'D-4',
15 => 'D#-4',
16 => 'E-4',
17 => 'F-4',
18 => 'F#-4',
19 => 'G-4',
20 => 'G#-4',
21 => 'A-4',
22 => 'A#-4',
23 => 'B-4',
24 => 'C-3',
25 => 'C#-3',
26 => 'D-3',
27 => 'D#-3',
28 => 'E-3',
29 => 'F-3',
30 => 'F#-3',
31 => 'G-3',
32 => 'G#-3',
33 => 'A-3',
34 => 'A#-3',
35 => 'B-3',
36 => 'C-2',
37 => 'C#-2',
38 => 'D-2',
39 => 'D#-2',
40 => 'E-2',
41 => 'F-2',
42 => 'F#-2',
43 => 'G-2',
44 => 'G#-2',
45 => 'A-2',
46 => 'A#-2',
47 => 'B-2',
48 => 'C-1',
49 => 'C#-1',
50 => 'D-1',
51 => 'D#-1',
52 => 'E-1',
53 => 'F-1',
54 => 'F#-1',
55 => 'G-1',
56 => 'G#-1',
57 => 'A-1',
58 => 'A#-1',
59 => 'B-1',
60 => 'C0',
61 => 'C#0',
62 => 'D0',
63 => 'D#0',
64 => 'E0',
65 => 'F0',
66 => 'F#0',
67 => 'G0',
68 => 'G#0',
69 => 'A0',
70 => 'A#0',
71 => 'B0',
72 => 'C1',
73 => 'C#1',
74 => 'D1',
75 => 'D#1',
76 => 'E1',
77 => 'F1',
78 => 'F#1',
79 => 'G1',
80 => 'G#1',
81 => 'A1',
82 => 'A#1',
83 => 'B1',
84 => 'C2',
85 => 'C#2',
86 => 'D2',
87 => 'D#2',
88 => 'E2',
89 => 'F2',
90 => 'F#2',
91 => 'G2',
92 => 'G#2',
93 => 'A2',
94 => 'A#2',
95 => 'B2',
96 => 'C3',
97 => 'C#3',
98 => 'D3',
99 => 'D#3',
100 => 'E3',
101 => 'F3',
102 => 'F#3',
103 => 'G3',
104 => 'G#3',
105 => 'A3',
106 => 'A#3',
107 => 'B3',
108 => 'C4',
109 => 'C#4',
110 => 'D4',
111 => 'D#4',
112 => 'E4',
113 => 'F4',
114 => 'F#4',
115 => 'G4',
116 => 'G#4',
117 => 'A4',
118 => 'A#4',
119 => 'B4',
120 => 'C5',
121 => 'C#5',
122 => 'D5',
123 => 'D#5',
124 => 'E5',
125 => 'F5',
126 => 'F#5',
127 => 'G5',
);
my $file = 'out.csv';
my $csv = Text::CSV->new();
open (CSV, "<", $file) or die $!;
while (<CSV>) {
next if ($. == 1);
if ($csv->parse($_)) {
my @columns = $csv->fields();
print "Track: $columns[0]\nTime: $columns[1]\nType: $columns[2]\nChannel: $columns[3]\nNote: $columns[4]\nVelocity: $columns[5]\n";
} else {
my $err = $csv->error_input;
print "Failed to parse line: $err";
}
}
close CSV;
My problem now is to print the correct columns on each line and match them with the note names.
Here's another example:
2, 110, Pitch_bend_c, 0, 7986 <- this line has a only 5 rows
2, 0, Note_on_c, 0, 36, 71 <- this has 6 rows
All parameters are different for each line so i want to for the first line:
Track : 2, Time: 110, Type: Pitch_bend_c, Channel: 0, Value: 7986
and the second one
Track: 2, Time: 0, Type: Note_on_c, Channel: 0, Note: C-2, Velocity: 71.
Sorry for all these questions, i'm new to this.
|
July 28th, 2013, 03:12 PM
|
|
|
You should probably use the Text::CSV module, but I'll give you a simple split/regex solution:
Perl Code:
 Original
- Perl Code |
|
|
|
my ($track, $time, $type, $channel, @rest) = split / [\ s, ]+/, $line; if (@rest == 2) { my ($note, $velocity) = @rest; $note = $num2note{$note}; } else { }
|
July 28th, 2013, 04:08 PM
|
|
|
|
In a properly formatted csv file, each row should have the same number of fields. Some of those fields could be empty.
As I understand it, you only want to modify the rows that contain "Note" in the 3rd field. Is that correct? Will that row always have the same number of fields and is the 5th field the only one that needs to be altered?
Do you want all lines in the new csv file, or just the ones that were updated?
|
July 28th, 2013, 04:22 PM
|
|
|
Code:
#!/usr/bin/perl
use strict;
use warnings;
use Text::CSV;
my %num2note = (
0 => 'C-5',
1 => 'C#-5',
2 => 'D-5',
3 => 'D#-5',
4 => 'E-5',
5 => 'F-5',
6 => 'F#-5',
7 => 'G-5',
8 => 'G#-5',
9 => 'A-5',
10 => 'A#-5',
11 => 'B-5',
12 => 'C-4',
13 => 'C#-4',
14 => 'D-4',
15 => 'D#-4',
16 => 'E-4',
17 => 'F-4',
18 => 'F#-4',
19 => 'G-4',
20 => 'G#-4',
21 => 'A-4',
22 => 'A#-4',
23 => 'B-4',
24 => 'C-3',
25 => 'C#-3',
26 => 'D-3',
27 => 'D#-3',
28 => 'E-3',
29 => 'F-3',
30 => 'F#-3',
31 => 'G-3',
32 => 'G#-3',
33 => 'A-3',
34 => 'A#-3',
35 => 'B-3',
36 => 'C-2',
37 => 'C#-2',
38 => 'D-2',
39 => 'D#-2',
40 => 'E-2',
41 => 'F-2',
42 => 'F#-2',
43 => 'G-2',
44 => 'G#-2',
45 => 'A-2',
46 => 'A#-2',
47 => 'B-2',
48 => 'C-1',
49 => 'C#-1',
50 => 'D-1',
51 => 'D#-1',
52 => 'E-1',
53 => 'F-1',
54 => 'F#-1',
55 => 'G-1',
56 => 'G#-1',
57 => 'A-1',
58 => 'A#-1',
59 => 'B-1',
60 => 'C0',
61 => 'C#0',
62 => 'D0',
63 => 'D#0',
64 => 'E0',
65 => 'F0',
66 => 'F#0',
67 => 'G0',
68 => 'G#0',
69 => 'A0',
70 => 'A#0',
71 => 'B0',
72 => 'C1',
73 => 'C#1',
74 => 'D1',
75 => 'D#1',
76 => 'E1',
77 => 'F1',
78 => 'F#1',
79 => 'G1',
80 => 'G#1',
81 => 'A1',
82 => 'A#1',
83 => 'B1',
84 => 'C2',
85 => 'C#2',
86 => 'D2',
87 => 'D#2',
88 => 'E2',
89 => 'F2',
90 => 'F#2',
91 => 'G2',
92 => 'G#2',
93 => 'A2',
94 => 'A#2',
95 => 'B2',
96 => 'C3',
97 => 'C#3',
98 => 'D3',
99 => 'D#3',
100 => 'E3',
101 => 'F3',
102 => 'F#3',
103 => 'G3',
104 => 'G#3',
105 => 'A3',
106 => 'A#3',
107 => 'B3',
108 => 'C4',
109 => 'C#4',
110 => 'D4',
111 => 'D#4',
112 => 'E4',
113 => 'F4',
114 => 'F#4',
115 => 'G4',
116 => 'G#4',
117 => 'A4',
118 => 'A#4',
119 => 'B4',
120 => 'C5',
121 => 'C#5',
122 => 'D5',
123 => 'D#5',
124 => 'E5',
125 => 'F5',
126 => 'F#5',
127 => 'G5',
);
my $file = 'out.csv';
my $csv = Text::CSV->new( {eol => "\n"} );
#open (CSV, "<", $file) or die $!;
while (<DATA>) {
#next if ($. == 1);
if ($csv->parse($_)) {
my @columns = $csv->fields();
s/^\s+|\s+$//g for @columns;
$columns[4] = $num2note{$columns[4]} if $columns[2] =~ /^Note/;
$csv->print(\*STDOUT, \@columns);
} else {
my $err = $csv->error_input;
print "Failed to parse '$_': $err";
}
}
#close CSV;
__DATA__
2, 0, Pitch_bend_c, 0, 8242
2, 0, Control_c, 0, 11, 38
2, 0, Note_on_c, 0, 36, 71
Output of D:\test> test.pl
2,0,Pitch_bend_c,0,8242
2,0,Control_c,0,11,38
2,0,Note_on_c,0,C-2,71
|
July 28th, 2013, 07:34 PM
|
|
Registered User
|
|
Join Date: Jul 2013
Posts: 5
Time spent in forums: 2 h 5 m 31 sec
Reputation Power: 0
|
|
Thank you.
Using your code i managed to filter some more things, now i'm just trying tweak it a bit and wrap it up.
Here's what i have
Code:
#!/usr/bin/perl
use strict;
use warnings;
use Text::CSV;
my %num2note = (
0 => 'C-5',
1 => 'C#-5',
2 => 'D-5',
3 => 'D#-5',
4 => 'E-5',
5 => 'F-5',
6 => 'F#-5',
7 => 'G-5',
8 => 'G#-5',
9 => 'A-5',
10 => 'A#-5',
11 => 'B-5',
12 => 'C-4',
13 => 'C#-4',
14 => 'D-4',
15 => 'D#-4',
16 => 'E-4',
17 => 'F-4',
18 => 'F#-4',
19 => 'G-4',
20 => 'G#-4',
21 => 'A-4',
22 => 'A#-4',
23 => 'B-4',
24 => 'C-3',
25 => 'C#-3',
26 => 'D-3',
27 => 'D#-3',
28 => 'E-3',
29 => 'F-3',
30 => 'F#-3',
31 => 'G-3',
32 => 'G#-3',
33 => 'A-3',
34 => 'A#-3',
35 => 'B-3',
36 => 'C-2',
37 => 'C#-2',
38 => 'D-2',
39 => 'D#-2',
40 => 'E-2',
41 => 'F-2',
42 => 'F#-2',
43 => 'G-2',
44 => 'G#-2',
45 => 'A-2',
46 => 'A#-2',
47 => 'B-2',
48 => 'C-1',
49 => 'C#-1',
50 => 'D-1',
51 => 'D#-1',
52 => 'E-1',
53 => 'F-1',
54 => 'F#-1',
55 => 'G-1',
56 => 'G#-1',
57 => 'A-1',
58 => 'A#-1',
59 => 'B-1',
60 => 'C0',
61 => 'C#0',
62 => 'D0',
63 => 'D#0',
64 => 'E0',
65 => 'F0',
66 => 'F#0',
67 => 'G0',
68 => 'G#0',
69 => 'A0',
70 => 'A#0',
71 => 'B0',
72 => 'C1',
73 => 'C#1',
74 => 'D1',
75 => 'D#1',
76 => 'E1',
77 => 'F1',
78 => 'F#1',
79 => 'G1',
80 => 'G#1',
81 => 'A1',
82 => 'A#1',
83 => 'B1',
84 => 'C2',
85 => 'C#2',
86 => 'D2',
87 => 'D#2',
88 => 'E2',
89 => 'F2',
90 => 'F#2',
91 => 'G2',
92 => 'G#2',
93 => 'A2',
94 => 'A#2',
95 => 'B2',
96 => 'C3',
97 => 'C#3',
98 => 'D3',
99 => 'D#3',
100 => 'E3',
101 => 'F3',
102 => 'F#3',
103 => 'G3',
104 => 'G#3',
105 => 'A3',
106 => 'A#3',
107 => 'B3',
108 => 'C4',
109 => 'C#4',
110 => 'D4',
111 => 'D#4',
112 => 'E4',
113 => 'F4',
114 => 'F#4',
115 => 'G4',
116 => 'G#4',
117 => 'A4',
118 => 'A#4',
119 => 'B4',
120 => 'C5',
121 => 'C#5',
122 => 'D5',
123 => 'D#5',
124 => 'E5',
125 => 'F5',
126 => 'F#5',
127 => 'G5',
);
my %num2ctl = (
1 => 'Modulation',
7 => 'Volume)',
10 => 'Pan',
11 => 'Expression',
64 => 'Sustain'
);
my %num2div = (
2 => '4',
3 => '8',
4 => '16',
5 => '32');
my $file = $ARGV[0] or die $!;;
my $csv = Text::CSV->new( {eol => "\n"} );
open (CSV, "<", $file) or die $!;
while (<CSV>) {
if ($csv->parse($_)) {
my @columns = $csv->fields();
s/^\s+|\s+$//g for @columns;
#my $freq = 440 * 2 ** (($columns[4]-69)/12);
$columns[4] = $num2div{$columns[4]} if $columns[2] =~ /^Time_signature/;
$columns[4] = $num2note{$columns[4]} if $columns[2] =~ /^Note/;
$columns[4] = $num2ctl{$columns[4]} if $columns[2] =~ /^Control/;
my $pitchbend = $freq + (($columns[4]-8192)/(4096*12));
$columns[4] = $pitchbend if $columns[2] =~ /^Pitch/;
$csv->print(\*STDOUT, \@columns);
} else {
my $err = $csv->error_input;
print "Failed to parse '$_': $err";
}
}
close CSV;
My problem is adding the $freq to columns[4] when Note occurs i've tried to use splice but no luck.
The calculations on the pitchbend column are coming screwed up too.
|
Developer Shed Advertisers and Affiliates
| Thread Tools |
Search this Thread |
|
|
|
| Display Modes |
Rate This Thread |
 Linear Mode
|
|
Posting Rules
|
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts
HTML code is Off
|
|
|
|
|
Del.icio.us
Digg
Google
Spurl
Blink
Furl
Simpy
Y! MyWeb
