#1
  1. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Sep 2012
    Posts
    5
    Rep Power
    0

    Array of hashes... what am i doing wrong?


    so what i'm trying to do is this

    1)read in a file with random header information
    2)once it hits the line "DS,,SNMPKEYINFO," turns into CSV
    3)CSV parse, allow for commas (will always be between qoutes if there are any in the value of the CSV)
    4)store the values in @temparray
    5)after i store the values (array size will always be 11) i then want to put it in a hash %dbstrings ($temparray[0] will always be a number, but don't know how much could be 1 to 100)

    if i refer to the hash in my code in the while loop, success.
    if i refer to the hash after i close <EXPORTINFO>, failure.

    Code:
    while(<EXPORTINFO>)
    {
    	if($startcapture==1)
    	{
    		$string=$_;
    		chomp($string);
    		@letter=split(undef,$string);
    		$inqoutes=0;
    		$index=0;
    		for ($count = 0; $count<= $#letter; $count++)
    		{
    			if($letter[$count] eq ',' && $inqoutes == 0)
    			{
    				$index++;
    				next;
    			}
    			if($letter[$count] eq '"')
    			{ 
    				if ($inqoutes==0){$inqoutes=1;}
    				elsif ($inqoutes==1){$inqoutes=0;}
    			}
    			$temparray[$index]=$temparray[$index].$letter[$count];
    			#at the end of the configurations, pop the DE off and continue on
    			if($temparray[$index] =~ m/^DE$/i) {pop(@temparray);}
    		}
    		$var = $temparray[0];
    		%dbstrings = (	$var => 
    						{
    							ip=> $temparray[1],
    							port=> $temparray[2],
    							v3user=> $temparray[3],
    							contextid=> $temparray[4],
    							contextname=> $temparray[5],
    							seclev=> $temparray[6],
    							authalg=> $temparray[7],
    							privalg=> $temparray[8],
    							authkey=> $temparray[9],
    							privkey=> $temparray[10],
    						}
    					);
    
    	}
    
    	undef @temparray;
    	
    	}
    	
    	if(m/DS,,SnmpKeyInfo,/i)
    	{
    		$startcapture=1;
    	}
    	if (m/^DE/)
    	{
    		$startcapture=0;
    	}
    }
    close(EXPORTINFO);
  2. #2
  3. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Sep 2012
    Posts
    5
    Rep Power
    0
    by the way, my example of printing a hash value is like this:

    Code:
    print %dbstrings->{'1'}->{'ip'};
  4. #3
  5. !~ /m$/
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    May 2004
    Location
    Reno, NV
    Posts
    4,264
    Rep Power
    1810
    There's a few things wrong. First of all, when you have a parsing question you should provide sample data. It's just a waste of time to ask people to guess at what the file looks like, and to make the same assumptions you did. There could be better ways of doing things.

    That's a sort of general comment borne of frustration of people rarely providing a good sample up front. Since you say it is a CSV file, I'll continue by recommending you use Text::CSV available at CPAN. If you install Text::CSV_XS as well, the parsing will be even faster.

    Another thing that seems curious is that you titled your post "Array of hashes", but I don't see an array. It appears that %dbstrings is a hash of hashes. The keys, based on the first field of each row, point to a sub-hash. If that isn't the structure you want, let me know. A hash of hashes is a good structure when you know you your keys will be unique. Since your keys are just numbers, that might not be true. Any duplicates in the file will overwrite earlier entries.

    Code:
    by the way, my example of printing a hash value is like this:
    
    Code:
    print %dbstrings->{'1'}->{'ip'};
    That's no good. When you are trying to reach a specific element, you drop the collection identifier (% for hash, @ for array) and use $ as in a single scalar. It's like the difference between singular and plural.

    Single hash element is $dbstrings{1}{ip}; The quotes around keys are optional.

    You only need an arrow (->) when the first variable you refer to in an expression is a reference. Perl will figure out the things beneath. In other words, you never need arrows between the elements of deeply nested structures, only after the first thing if it is a reference. Since %dbstrings is an actual hash rather than a reference, no arrow needed.
    Last edited by keath; September 22nd, 2012 at 11:18 AM.
  6. #4
  7. !~ /m$/
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    May 2004
    Location
    Reno, NV
    Posts
    4,264
    Rep Power
    1810
    Here's an example of parsing with Text::CSV

    Code:
    #!/usr/bin/perl
    use strict;
    use warnings;
    
    use Text::CSV;
    use Data::Dumper;
    
    my %data;
    my @hashkeys = qw/ip port v3user contextid contextname seclev authalg privalg authkey privkey/;
    
    my $csv = Text::CSV->new ( { binary => 1 } ) or die "Error: ".Text::CSV->error_diag ();
     
    open my $fh, "<", "test.csv" or die "test.csv: $!";
    
    # advance through the file until you reach your starting point
    while (<$fh>) {
        last if substr($_,0,16) eq 'DS,,SNMPKEYINFO,';
    }
    
    # begin processing CSV rows
    while ( my $row = $csv->getline( $fh ) ) {
         my %rowhash;
         @rowhash{@hashkeys} = @$row[1..10];
         $data{$row->[0]} = \%rowhash;
    }
    
    $csv->eof or $csv->error_diag();
    close $fh;
    
    print Dumper \%data;
    Rather than assigning keys to %data, you could push each parsed row into an array @data. It depends on what you intend to do later.

    This is the test file I used:

    Code:
    "REVIEW_DATE","AUTHOR","ISBN","DISCOUNTED_PRICE"
    "REVIEW_DATE","AUTHOR","ISBN","DISCOUNTED_PRICE"
    "REVIEW_DATE","AUTHOR","ISBN","DISCOUNTED_PRICE"
    "REVIEW_DATE","AUTHOR","ISBN","DISCOUNTED_PRICE"
    "REVIEW_DATE","AUTHOR","ISBN","DISCOUNTED_PRICE"
    DS,,SNMPKEYINFO,AND,SOME,OTHER,STUFF
    "1985/01/21","Douglas Adams",0345391802,5.95,a,b,c,d,e,f,g
    "1990/01/12","Douglas Hofstadter",0465026567,9.95,a,b,c,d,e,f,g
    "1998/07/15","Timothy ""The Parser"" Campbell",0968411304,18.99,a,b,c,d,e,f,g
    "1999/12/03","Richard Friedman",0060630353,5.95,a,b,c,d,e,f,g
    "2001/09/19","Karen Armstrong",0345384563,9.95,a,b,c,d,e,f,g
    "2002/06/23","David Jones",0198504691,9.95,a,b,c,d,e,f,g
    "2002/06/23","Julian Jaynes",0618057072,12.50,a,b,c,d,e,f,g
    "2003/09/30","Scott Adams",0740721909,4.95,a,b,c,d,e,f,g
    "2004/10/04","Benjamin Radcliff",0804818088,4.95,a,b,c,d,e,f,g
    "2004/10/04","Randel Helms",0879755725,4.50,a,b,c,d,e,f,g
  8. #5
  9. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Sep 2012
    Posts
    5
    Rep Power
    0
    thanks for the reply,

    the version of perl i'm using is a custom lightweight version that i can't modify or add packages to. those are just the rules where i am.

    The issue isn't parsing. i've verified the parsing is working (by printing the value of @temparray)

    the file i'm parsing is basically a database for snmpv3 devices so the file isn't entirely a CSV, there is some random header information it needs (to load into the database) here is an example of the csv portion:
    note: there is potential for the username/passwords to contain commas thats why i was parsing with my method.

    1,10.1.1.1,snmpv3username,"","",authPriv,SHA,AES,"authpassword","privpassword"
    2, etc
    3, etc will always be in increasing order i'll never know how much

    when the array fills it fills correctly (verified via a printout) and i assign it to the hash it doesn't persist through out the program.

    it prints out correctly in the "while loop" (via print %dbstrings->{'1'}->{'ip'};) but after i close the file it won't print out. it must be that i'm using it incorrectly. its been a while since i've used hashes in perl and refreshing the syntax is killling me. my new weekend goal is to read the cook book.

    i used your suggestion of "$dbstrings{1}{ip}" for calling the value of the hash. stil works if i place it in the while loop. as soon as i exit and try to print it, it fails. just doesn't seem like it holds the value.. how would you go about iterating through the hash? here is what i'm using and it isn't printing out anything.

    Code:
    foreach $key (keys %dbstrings)
    {
    print $key." ";
    foreach $entry (keys %{$dbstrings{$key}})
    {
    print "  $dbstrings{$key}{$entry}\n";
    }
    }
  10. #6
  11. !~ /m$/
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    May 2004
    Location
    Reno, NV
    Posts
    4,264
    Rep Power
    1810
    Use strict mode as I have done. When you don't use it, everything becomes global.

    I think the big error for the lack of data persistence is this:

    Code:
    %dbstrings = (	$var =>
    The assignment operator is going to erase everything held before, and so when the loop ends you will have no more than a single row's data.

    A hash is like a little database. You only need to declare it once at the top of your script.

    Code:
    my %dbstrings;
    From then on, only assign to a specific key. Don't write over the whole storage area.

    Code:
    $data{$key} = \%hash;
  12. #7
  13. Guru Meditation
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jan 2004
    Location
    Amsterdam
    Posts
    1,304
    Rep Power
    382
    There are a lot of Perl modules on CPAN that are PurePerl - you can basically just copy the files and add the correct folders to your @INC path.

    http://search.cpan.org/~makamaka/Tex...ib/Text/CSV.pm

    You want the pure Perl (XS = in C and needs to be compiled).

    Anyway ... this should probably look something like:
    Code:
    foreach my $key (keys %dbstrings){
        print $key." ";
        my $key_string = $dbstrings{$key};
        foreach my $entry (keys %{$key_string}){
            print "  " . $key_string->{$entry}. "\n";
        }
    }
    You are mixing up plain hashes (%hash and $hash{key}) and hash-references ($hash and $hash->{key}).

    And if you just want to check if you've got correct data read and in correct structure/format - just use Data:umper.

    Code:
    use Data::Dumper;
    
    ...
    
    print Dumper( \%dbstrings );

    Comments on this post

    • keath agrees
    www.booking.com is hiring Perl developers!
    Work along some of the biggest names in Perl community. Live in Amsterdam - relocation assistance is provided (paperwork/visa and financial) for you and your family members - for details send me an message

  14. #8
  15. !~ /m$/
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    May 2004
    Location
    Reno, NV
    Posts
    4,264
    Rep Power
    1810
    the version of perl i'm using is a custom lightweight version that i can't modify or add packages to. those are just the rules where i am.
    This is almost never true. Many perl modules are pure perl. In other words, they are just text files. Text::CSV is such a module. You can see part of the code right here:
    Code:
    http://cpansearch.perl.org/src/MAKAMAKA/Text-CSV-1.21/lib/Text/CSV_PP.pm
    Text::CSV_XS on the other hand needs to be compiled, so requiring that module could be difficult. But Text::CSV will fall back on the pure perl version when XS is not present.

    You can view the manifest for any module at CPAN, and review its dependencies, and whether it contains any xs files. When they don't, you can just copy and paste the code to your own local library. You can tell perl to use any local directory you want with the 'use lib' command at the top of your script.

    Consider all the abilities that are present in the CPAN libraries, and all the pre-tested code. With all you'll gain, it's worth figuring out how to move some of these files to your own library.
  16. #9
  17. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Sep 2012
    Posts
    5
    Rep Power
    0
    for instance the data i'm going through-

    1,10.1.1.1,snmpv3username,"","",authPriv,SHA,AES,"authpassword","privpassword"
    2,10.1.1.2,snmpv3username,"","",authPriv,SHA,AES,"authpassword","privpassword"
    3,10.1.1.3,snmpv3username,"","",authPriv,SHA,AES,"authpassword","privpassword"

    if i run through my loop (shown below)
    iteration 1: $dbstrings{1}{ip}
    $var =1
    ip=10.1.1.1
    ..etc

    iteration 2: $dbstrings{2}{ip}
    $var =2
    ip=10.1.1.2
    ..etc

    iteration 3: $dbstrings{3}{ip}
    $var =3
    ip=10.1.1.3
    ..etc

    so your saying the way i have it written i'm just reassigning the same "key" in the hash multiple times? end goal is to get this data into some usable format where i can call it and check for repeats, add , remove, modify

    Code:
    $var = $temparray[0];
    %dbstrings = (	$var => 
    					{
    						ip=> $temparray[1],
    						port=> $temparray[2],
    						v3user=> $temparray[3],
    						contextid=> $temparray[4],
    						contextname=> temparray[5],
    						seclev=> $temparray[6],
    						authalg=> $temparray[7],
    						privalg=> $temparray[8],
    						authkey=> $temparray[9],
    						privkey=> $temparray[10],
    					}
    		);
  18. #10
  19. !~ /m$/
    Devshed Specialist (4000 - 4499 posts)

    Join Date
    May 2004
    Location
    Reno, NV
    Posts
    4,264
    Rep Power
    1810
    Originally Posted by elogateumsato
    so your saying the way i have it written i'm just reassigning the same "key" in the hash multiple times?
    No, I'm saying you are destroying anything stored on the previous loop, and starting over.

    Code:
    %dbstrings =
    That right there will remove anything already stored, and cause %dbstrings to contain something entirely new.

    If instead you want to add just one element, you would do:

    Code:
    $dbstrings{$var} = 
    					{
    						ip=> $temparray[1],
    						port=> $temparray[2],
    						v3user=> $temparray[3],
    						contextid=> $temparray[4],
    						contextname=> temparray[5],
    						seclev=> $temparray[6],
    						authalg=> $temparray[7],
    						privalg=> $temparray[8],
    						authkey=> $temparray[9],
    						privkey=> $temparray[10],
    					};
    But if there was already a key of the same value, that one would be overwritten and you would only get the newest row. So if you are concerned there could be duplicate keys in your file, you would need to test before assigning.

    Code:
    unless (exists $dbstrings{$var}) {
       $dbstrings{$var} = { #hash data };   #safe to assign
    }

    Comments on this post

    • elogateumsato agrees : Very helpful
  20. #11
  21. No Profile Picture
    Registered User
    Devshed Newbie (0 - 499 posts)

    Join Date
    Sep 2012
    Posts
    5
    Rep Power
    0
    thanks that makes a lot of sense.

    there will never be duplicate keys in the file

    - it is exported from a working database, if there was duplicates i'd be receiving "snmp communication errors" from the device because only one of the $var will exist.

    never hurts to throw in a check though.

    this script is going to save me a lot of time versus manually added stuff to the db.

IMN logo majestic logo threadwatch logo seochat tools logo