#1
  1. Contributing User
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jun 2003
    Location
    Thessaloniki
    Posts
    1,285
    Rep Power
    13

    Pinpoinitng visitors _exact_ city location


    Hello guys!

    Do you know when i go to maps.google.com its always find my exact city of location and not just say Europe/Athens.

    and twitter and facebook too both of them pinpoint my _exact_ location.

    How are they able to do it?
    I need the same way.

    Currently i have this that retrieves the ISP's city of origin and not the user's ip of origin

    Code:
    try:
    	gi = pygeoip.GeoIP('/usr/local/share/GeoLiteCity.dat')
    	city = gi.time_zone_by_addr( os.environ['HTTP_CF_CONNECTING_IP'] )
    	host = socket.gethostbyaddr( os.environ['HTTP_CF_CONNECTING_IP'] )[0]
    except Exception as e:
    	host = repr(e)
    Can you help please?
    What is now proved was once only imagined!
  2. #2
  3. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    Usually Japan when not on contract
    Posts
    240
    Rep Power
    12
    Google, Facebook, Microsoft, Amazon and most other gigantic companies with lots of money track you in several different ways, not just by the IP. They compare several categories of tracking to generate a list of possible locations for you and then pick the one with the highest confidence.

    For example, I have an AU phone. If I register with AU Cloud that also registers me with Google, and then my AU tower, IP and GPS location all get reported to Google. When I login later on a desktop to the same GoogleID account, they only have my IP and tracking cookies to look at, but they already know to check the latest location of my phone -- and whether its turned on/permitting GPS updates right then affects the confidence report % of that method of tracking. Recent reservations, dated product/service reviews, driving directions, map inquiries, map bookmarks/pins, etc. all give some confidence for frequented location and movement history each. Any billing relationship you have with them will give them another tracking point based on your billing address, and they can compare the billing address with frequented GPS locs, past shipping information and recent locale-oriented searches. The more recent the data and the more points of data match the same location the more confidence the potential location has.

    ...and so on.

    Its pretty creepy, actually. Anyway, you can't just do this using IP information. To get reliable, live, pinpoint user location data you need to do one of:
    • Convince the user to report/register/pick their location
    • Convince the user to permit you to track their phone
    • Get a contract with Google that buys you their best guess at user location
    • Be like Google and engage in a conspiracy to invade the privacy of millions that dwarfs the resources of most intelligence agencies (and then sell it to intelligence agencies, just like Google does)
    Last edited by zxq9; July 7th, 2013 at 04:54 AM.
  4. #3
  5. Contributing User
    Devshed Beginner (1000 - 1499 posts)

    Join Date
    Jun 2003
    Location
    Thessaloniki
    Posts
    1,285
    Rep Power
    13
    Originally Posted by zxq9
    Google, Facebook, Microsoft, Amazon and most other gigantic companies with lots of money track you in several different ways, not just by the IP. They compare several categories of tracking to generate a list of possible locations for you and then pick the one with the highest confidence.

    For example, I have an AU phone. If I register with AU Cloud that also registers me with Google, and then my AU tower, IP and GPS location all get reported to Google. When I login later on a desktop to the same GoogleID account, they only have my IP and tracking cookies to look at, but they already know to check the latest location of my phone -- and whether its turned on/permitting GPS updates right then affects the confidence report % of that method of tracking. Recent reservations, dated product/service reviews, driving directions, map inquiries, map bookmarks/pins, etc. all give some confidence for frequented location and movement history each. Any billing relationship you have with them will give them another tracking point based on your billing address, and they can compare the billing address with frequented GPS locs, past shipping information and recent locale-oriented searches. The more recent the data and the more points of data match the same location the more confidence the potential location has.

    ...and so on.

    Its pretty creepy, actually. Anyway, you can't just do this using IP information. To get reliable, live, pinpoint user location data you need to do one of:
    • Convince the user to report/register/pick their location
    • Convince the user to permit you to track their phone
    • Get a contract with Google that buys you their best guess at user location
    • Be like Google and engage in a conspiracy to invade the privacy of millions that dwarfs the resources of most intelligence agencies (and then sell it to intelligence agencies, just like Google does)
    I see, very interesting ideas.
    Of course i cannot be part of Google's conspiracy or pay them just to retrive the city of my visitors :-)

    You said though:
    [*]Convince the user to permit you to track their phone

    So there is really no way to somehow retrive not the ISP's Geo location but the visitro's instead?

    Sinst there spome python code we can use?
    What is now proved was once only imagined!
  6. #4
  7. No Profile Picture
    Contributing User
    Devshed Newbie (0 - 499 posts)

    Join Date
    May 2013
    Location
    Usually Japan when not on contract
    Posts
    240
    Rep Power
    12
    Originally Posted by Nik
    So there is really no way to somehow retrive not the ISP's Geo location but the visitro's instead?
    That's an ISP issue, made marginally better by ISPs that do in-depth IPv6 stuff (which is really rare). Most customers at most ISPs (probably about 95% of the consumer market) have rotating IPv4 addresses assigned dynamically from a pool. With always-on broadband being pretty normal today the reconnect rate is really low compared to 20 years ago, but this still makes everyone's IP address dynamic within a period of about a month or so (depending on ISP and how often they shut their router/modem off).

    Many ISPs use a nation-sized pool because most countries just aren't all that large (that is, exceptions to this are the USA, Russia and China), or at best use a region-sized pool ("region" being loosely defined as "closest major city we have a backbone in"). Most backbones are owned by governments, not private industry (and this difference is why IP address resolution is way more detailed in the US vs anywhere else), and so backbone regions are quite large, usually.

    So... all that said, the best you can get from an IP address in most cases is the closest large city, or sometimes the country.

    I live in Japan where we all have insanely fast broadband and other tech gadgets galore. But NTT (the government telecom monopoly) owns the lines, so the IP assignment blocks go by NTT region, of which there are only two. That's right, just two assignment regions in a country larger than Germany. So usually my IP geotrace says I'm in Osaka, Tokyo or Naha, depending on which IP I happen to have the day I check. But its almost never any other city. They are all wrong. I live in Okinawa -- a small island that is about as geographically (and ad-market) relevant to Japan as Hawaii is to the US. Naha is actually in Okinawa Prefecture, but its nowhere near the part of the archipelago that I live in. So one day my IP says I'm a few hundred km from where I really am, the next several thousand km away.

    My point is that the degree of wrongness is the only thing that changes.

    The ISP knows. The government knows if they want to (government telecom == epic backdoorness). Amazon and Google obviously know if I have JavaScript turned on, and AU knows regardless because they run my phone. Unless you can ask one of those 5 organizations to verify my location or get me to tell you then you're just never going to have an accurate geoloc on my user account.

    Unless you ask me to tell you, either via GPS on Android or by just putting it in my profile. So the trick there is to make it not hard to tell you (shouldn't require extra clicks/lots of registration screens or installation of anything) and make it worth my time to let you know where I am (better service, i.e. your service will call me the cheapest cab in the area if I click "OK" when I use my phone breathalizer, etc.).

IMN logo majestic logo threadwatch logo seochat tools logo