[ldaneil]

Hi, everyone

I am working on a project about data mining. my company has given me 6 million dummy customers info of facebook. I was assigned to find out the similarity between any two users. Thanks in advance :)

Problem : I use the status info & hashtag info(hashtags are those words highlighted by user) as the two criteria to measure the similarity between two different users. Since the large number of users, and especially there may be millions of hastags & statuses of each user. Can anyone tell me a good way to fast calculate the similarity between two users? I have tried to use FT-IDF to calculate the similarity between two different users, but it seems infeasible. can anyone have a very super algorithm or good ideas which could make me fast find all the similarities between users? because the number is too large, the time complexity and space complexity is crucial for the project.

[mitakeet]

You are at the wrong forum (unless you get lucky). There is an algorithm forum here you might be better at checking out.

BTW, this is not considered a 'simple' problem and is an area of active research, so you are likely better off digging into primary literature.