GeoIP database comparison

[Karsten Loesing]

On 4/29/10 6:08 PM, Andrew Lewman wrote:
> On Thursday April 29 2010 04:58:27 Karsten Loesing wrote:
>> - Interestingly, trusted resolves 7368 requests to Nigeria with the
>> Maxmind database, which were only 8 with ip-to-country. I wonder if this
>> can be correct. Is Tor this well-known in Nigeria?
> 
> That seems high to me.  What about other countries like Iran and China?  When 
> China blocked the public list of relays in September 2009, we saw jumps of 
> bridge users in Australia and Japan the same day we saw the jumps from China.  
> This seems like imprecise assignment to me.

China had 3520 requests (2.37% of all requests on that day) with the
Maxmind database and 2248 (2.28%) with ip-to-country, and Iran had 2168
(1.46%) with Maxmind and 1568 (1.59%) with ip-to-country. No changes to
worry about here.

> Iran's connections to the outside world go through Turkey, Kuwait, and 
> Azerbaijan, etc.  See, 
> https://svn.torproject.org/svn/projects/presentations/images/iran-internet-
> connectivity-TAE-14.gif
> 
> I wonder if we're over-counting other countries, even with the new database.

I also had a closer look at the reported numbers for Nigeria. There were
7368 requests from Nigeria (with the Maxmind database), but these
requests came from only 328 unique IP addresses. That's a ratio of 22.5
requests per IP address, with a ratio of 1.6 on average for other
countries. I think we're seeing something unusual going on here, but
false resolution of IP addresses is probably not the problem.

While looking at the numbers, I found more African countries in addition
to Nigeria and Tunisia that suddenly have Tor users. Here are the
request numbers for some of these countries: Nigeria 7368 (from 328
IPs), Tunisia 320, Côte d'Ivoire 272, Morocco 272, Senegal 240, South
Africa 216, Algeria 184, Ghana 104. All of these countries were reported
to have 0 requests with the old database, except for Nigeria which had
8. There are no similar increases for countries in other continents.

Bill's point about being able to replace the free Maxmind with the
commercial Maxmind is also a good one. I'm considering to use the
commercial database on gabelmoo to get somewhat more precise (or more
up-to-date) results. Still, these results will be comparable to what
other relays with the free database report.

Best,
--Karsten