Importing of data, resulting in duplicates, HELP!

hi all,

I have a problem with SQL 2000 here...
I juz imported the same set of data to my database table and it gave me duplicated records as all the data is imported although there is same existing data.

Can anyone help me with this?
How can i import the data such that the data which already exists in the database will not be imported in again?

Thanks in advance.

well, if there are dup records now you can eliminate 'em through querying on dup fields and the primary key.

Else, when I do an import on data that has subsets of data that are already in the DB, I do an insert on join conditions to eliminate inserting the data already in there.

Can you elaborate on this with an example? I'm working through a problem right now that could potentially be solved by this.

Sorry, but I don't have too much time right now to show you all of the 2 ways I mentioned. You should note before reading any further that you can avoid this problem almost entirely by re-thinking you're primary key field(s). So instead of having to continuously go through the trouble, think about how your key is defined. Mainly I've found that using Identity fields as a primary key will result in dup records @ some time or another.

The simplest way to explain is deleting them from the DB once they're in there, but I think a more solid method is to avoid them getting in there in the 1st place. When they're already in there, you first have to be able to grab the dup records, from there you can join back on the table to do your delete. Let's assume this table w/the following data...

Surely you see that Andy, Kelly, and Tim have 'duplicate' rows...duplicate being defined as every field having the same value as another record w/the exception of the primary key. Now you can query on a count for the primary key to grab the dups...

You can do this much, ya? I'll try to post more later, but from those records you can join back on the table to do your delete. Since you still want one record left, from the above records select the name, address, phone, and the max(ID) from each of the dup groups and delete all their other records that match on everything BUT the max ID. I can explain more later. Btw, my use of Distinct in the count(distinct ID) isn't totally necessary, if ID is the PK then by definition it will always be distinct.

I agree that the best way to not have to do this is to keep them from happening. That is what I am trying to work towards right now. I have primary keys identified as a date and want to import records for dates that are not already in the table. Each time I run just a regular DTS package, though, it errors out and doesn't add any records. I'm sure it is something simple to fix but I can't figure it out.