Database Management
 
Forums: » Register « |  User CP |  Games |  Calendar |  Members |  FAQs |  Sitemap |  Support | 
User Name:
Password:
Remember me
Go Back   Dev Shed ForumsDatabasesDatabase Management

Reply
Add This Thread To:
  Del.icio.us   Digg   Google   Spurl   Blink   Furl   Simpy   Y! MyWeb 
Thread Tools Search this Thread Rate Thread Display Modes
 
Unread Dev Shed Forums Sponsor:
AT&T devCentral & BlackBerry(r) Webcast Series: BlackBerry and GPS -Build Location Awareness into your BlackBerry Applications, July 10th-1:00PM EST. Register Today!
  #1  
Old August 8th, 2003, 03:37 PM
sbrock sbrock is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2003
Posts: 14 sbrock User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Question ascii to Access

Can a ascii file that doesn't have a fixed length and no delimiters in it be imported to Access?

Thanks

Reply With Quote
  #2  
Old August 8th, 2003, 05:07 PM
Doug G Doug G is offline
Grumpier Old Moderator
Dev Shed God 12th Plane (10500 - 10999 posts)
 
Join Date: Jun 2003
Posts: 10,703 Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level)Doug G User rank is Brigadier General (60000 - 70000 Reputation Level) 
Time spent in forums: 4 Weeks 1 Day 21 h 33 m 53 sec
Reputation Power: 688
Yes, although you may end up with a one-column table

Reply With Quote
  #3  
Old August 8th, 2003, 05:15 PM
sbrock sbrock is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2003
Posts: 14 sbrock User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
One column table

If it be the case of a one column table can that data be parsed off into lets say text boxes in order to change or manipulate some of this data?


thanks

Reply With Quote
  #4  
Old August 13th, 2003, 09:51 AM
mitakeet's Avatar
mitakeet mitakeet is offline
Last Day: May 28, 2005
Dev Shed Demi-God (4500 - 4999 posts)
 
Join Date: Jul 2003
Location: Maryland
Posts: 4,575 mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 8 h 5 m 34 sec
Reputation Power: 20
Your application can chop up the strings to its heart's content. You can also write a preprocessor to put delimiters into your original data, then import it. You can even (painfully in my experience) write SQL to parse the data and store the SQL as a procedure.
__________________

Left DevShed May 28, 2005. Reason: Unresponsive administrators.
Free code: http://sol-biotech.com/code/.
Secure Programming: http://sol-biotech.com/code/SecProgFAQ.html.
Performance Programming: http://sol-biotech.com/code/PerformanceProgramming.html.

It is not that old programmers are any smarter or code better, it is just that they have made the same stupid mistake so many times that it is second nature to fix it.
--Me, I just made it up

The reasonable man adapts himself to the world; the unreasonable one persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man.
--George Bernard Shaw

Reply With Quote
  #5  
Old August 13th, 2003, 09:58 AM
sbrock sbrock is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2003
Posts: 14 sbrock User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Where do I look for this information? What's a good source?

Reply With Quote
  #6  
Old August 13th, 2003, 10:03 AM
mitakeet's Avatar
mitakeet mitakeet is offline
Last Day: May 28, 2005
Dev Shed Demi-God (4500 - 4999 posts)
 
Join Date: Jul 2003
Location: Maryland
Posts: 4,575 mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 8 h 5 m 34 sec
Reputation Power: 20
Your question is too open ended. I use C to pre-parse files all the time, sometimes Perl and once or twice I used VB. There are various built-in SQL commands to parse strings, they vary by database and version. Whatever application that reads the data can do its own parsing, just like the pre-parser. If you only have to do the data input once, you can even do the pre-parsing by hand.

Reply With Quote
  #7  
Old August 13th, 2003, 10:12 AM
sbrock sbrock is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2003
Posts: 14 sbrock User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
I have been looking into perl for the parsing and it's looking like I want to work with it. The problem is that programming is new to me and I am finding information that isn't really related to what I want to do. This is the reason I am trying to get some direction so to not get lost on a tangent.

Parsing by hand isn't a possibility, the file I have to work with is about 17Meg. The problem I have with the data is that it is a combined file with mutilple record types (they are marked by the first three characters of each line). Then each record has a different data layout.

With this in mind, I was thinking of using perl to sort and place each record type into its own file and using that file to create a front-end to work or manipulate the records. This brings us back to Acces to do this build (I think?).

Reply With Quote
  #8  
Old August 13th, 2003, 10:21 AM
mitakeet's Avatar
mitakeet mitakeet is offline
Last Day: May 28, 2005
Dev Shed Demi-God (4500 - 4999 posts)
 
Join Date: Jul 2003
Location: Maryland
Posts: 4,575 mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 8 h 5 m 34 sec
Reputation Power: 20
If each record type has a determined layout, then you could write a program (in Perl, for example) that would treat each record appropriate. If you want each type of data in a separate table, you could create a single file for each type (go ahead and do the parsing in the script, no sense processing each record more than once). If you want to post some example data for each record type and what you want it to look like, I can (probably) create a quick Perl program for you. Perl was designed for this sort of thing and it is often practical to do something quite complex in just a few lines of code. If your intent with Access is to do your data manipulation, you are better off (in my opinion) spending a couple of hours learning to do it with Perl. If Access is to be the final repository of the data, then it may make sense to invest time in learning the arcana of Access (which, by the way, is not very useful in any other database).

Reply With Quote
  #9  
Old August 13th, 2003, 10:31 AM
sbrock sbrock is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2003
Posts: 14 sbrock User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Thanks, I will try to put some data together as a sample. There is approx. 14-16 record types. With what I have read so far, I like perl. I am not set in stone to Access it is just what my employer has. If I am to get another package I have to give the supporting documents to get the package.

Thanks

Reply With Quote
  #10  
Old August 13th, 2003, 11:09 AM
sbrock sbrock is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2003
Posts: 14 sbrock User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
here's a sample:

090071905090405611071905001029842000281253P0223P0504011 80017
090071905090405611071905001029842000291253P0223P0504011 80007

03007190519912660000739900000026500
03007190519912660000839900000011500

040071905453901800xxxxxx x xxxxxxxx M4000
040071905453901806xxxxxx xxxxxxxx F4131

05007190545393434503187100
05007190545393963405187100

06007190545337000380199516129001399002650
06007190545337182980240356129999399013784

Like I said previously the first three characters are the record type. Each one has it's own data type. Since the 050 record is small here is the sample of the parsing needed.
Columns
1-3 code
4-9 id
10-18 id
19-20 info code
21-23 exp
24-26 emp
27-80 filler

This data is coming from a AS400 ascii dump. I am assuming that with one layout I can get the understanding how to put the others together.

thanks

Reply With Quote
  #11  
Old August 13th, 2003, 11:15 AM
mitakeet's Avatar
mitakeet mitakeet is offline
Last Day: May 28, 2005
Dev Shed Demi-God (4500 - 4999 posts)
 
Join Date: Jul 2003
Location: Maryland
Posts: 4,575 mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 8 h 5 m 34 sec
Reputation Power: 20
What is the chance you can get your hands on a C compiler? Now that I look at the data I think it would be easier in C (Perl is great for data with some sort of delimiter, not so great for fixed-width data (at least not to me)). I will try somethings in Perl and see what happens, but in C I can whip something together in no time.

BTW: Is the data for record type 040 correct? It looks like it didn't get copied correctly to me.

Reply With Quote
  #12  
Old August 13th, 2003, 11:28 AM
sbrock sbrock is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2003
Posts: 14 sbrock User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Record 040 has personal info. I made the changes to the info. The layout is such:
columns
1-3 type
4-9 id
10-18 id
19-35 fname
36-49 mname
50-74 lname
75 code
76 code
77 code
78-79 code
80 code

Reply With Quote
  #13  
Old August 13th, 2003, 12:49 PM
mitakeet's Avatar
mitakeet mitakeet is offline
Last Day: May 28, 2005
Dev Shed Demi-God (4500 - 4999 posts)
 
Join Date: Jul 2003
Location: Maryland
Posts: 4,575 mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level)mitakeet User rank is Sergeant (500 - 2000 Reputation Level) 
Time spent in forums: 1 Week 2 Days 8 h 5 m 34 sec
Reputation Power: 20
This code should work, I ran it on a UNIX box. There is a more elegant way of parsing the data but I don't have the energy to work on it now (it has been several months since I spent a lot of time with Perl). This version requires a block of code for each record type and the more elegant way would have it in a single loop.

Hopefully this makes sense to you. First, open an array of files, one for each type, then read through the source data one record at a time and parse it differently based on which type it is, then write the parsed data out to the appropriate file. This technique should work for your 14-16 different types, though it is possible that your OS will not allow that many concurrent file handles to be open. If that is the case, process the same file twice but with different types and don't print out the others.

I may have off-by-one errors on the parsing, but I am sure you are in the best position to figure that out.


Code:
#!/usr/bin/perl

#this program will read from a data file and will parse
#and print the data to individual files

use IO::File;

#this is used to index into file handle array
#add one for each type (make sure there are no duplicates!)
#the order does not matter

@DataTypes = (
    "090",
    "030",
    "040",
    "050",
    "060"
);

@fileHandles = ();
%typeHash = ();

sub openFiles4Reading{
    for ($cnt = 0; $cnt < @DataTypes; $cnt++){
        $typeHash{$DataTypes[$cnt]} = $cnt;
        $fileHandles{$cnt} = new IO::File;
        $filename = ">tmp_$DataTypes[$cnt].dat";
        open($fileHandles{$cnt}, $filename) or die "Can't open file $filename for writing!\n";
    }
    print STDERR "files successfully open for reading\n";
}

sub closeFiles{
    print STDERR "closing files...\n";

    for ($cnt = 0; $cnt < @DataTypes; $cnt++){
        close($fileHandles{$cnt});
    }
}

&openFiles4Reading;

#process input file here:

#dat file is whatever your source input file is
open(FIN, "dat") || die "Can't open dat file!";
open(FOUT, ">datnotfound") || die "Can't open datnotfound file!";

while ($input = <FIN>){
    chomp($input);#remove trailing line feed
    $type = substr($input, 0, 3);
    $index = $typeHash{$type};
    $handle = $fileHandles{$index};
    if (!$handle){#dump anything unknown into a separate file
         print FOUT "$input\n";
         next;
    }
    @parsDat = ();#this clears the parse array
    if ($type eq "040"){
        $parsDat[0] = substr($input, 0, 3);  #type
        $parsDat[1] = substr($input, 3, 6);  #id
        $parsDat[2] = substr($input, 10, 9); #id
        $parsDat[3] = substr($input, 19, 17);#fname
        $parsDat[4] = substr($input, 36, 14);#mname
        $parsDat[5] = substr($input, 50, 25);#lname
        $parsDat[6] = substr($input, 75, 1); #code
        $parsDat[6] = substr($input, 76, 1); #code
        $parsDat[6] = substr($input, 77, 1); #code
        $parsDat[6] = substr($input, 78, 2); #code
        $parsDat[6] = substr($input, 79, 1); #code  
    }elsif ($type eq "050"){
        $parsDat[0] = substr($input, 0, 3);  #code
        $parsDat[1] = substr($input, 3, 6);  #id
        $parsDat[2] = substr($input, 10, 9); #id
        $parsDat[3] = substr($input, 19, 2); #info code
        $parsDat[4] = substr($input, 21, 3); #exp
        $parsDat[5] = substr($input, 24, 3); #emp
        $parsDat[6] = substr($input, 27, 54);#filler    
    }else{#just to grab anything else, but you shouldn't need an else here
         $parsDat[0] = $input;
    }

    #if you want to tab delimit the data use this:
    #join("\t", @parsDat);
    $output = join(",", @parsDat);
    print $handle "$output\n";
    
}

close(FIN);
close(FOUT);
    
&closeFiles;      

Reply With Quote
  #14  
Old August 13th, 2003, 01:00 PM
sbrock sbrock is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2003
Posts: 14 sbrock User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Thank you for all your help. I'm going to try it and see how it goes.

again thanks

Reply With Quote
  #15  
Old August 14th, 2003, 04:52 PM
sbrock sbrock is offline
Junior Member
Dev Shed Newbie (0 - 499 posts)
 
Join Date: Aug 2003
Posts: 14 sbrock User rank is Just a Lowly Private (1 - 20 Reputation Level) 
Time spent in forums: < 1 sec
Reputation Power: 0
Mita,

I had another ? if you don't mind. After adjusting the program a bit. I was asked how was I to remove the delimiters after the data had been modified? My first reaction was to think that I could use the same program but to use maybe a splice or pop function where the join is being used.

Is this right? Beacause after a while thinking about it, that process seems too cumbersome due to the need of appending all that data again into a file and parsing the files seperatly again. There has got to be an easier way.

Reply With Quote
Reply