Add support for IPinfo's IP to Country ASN database

[abdullahdevrel]

I am requesting to add support for IPinfo's IP to Country database to the project. The database has the following features:

• It includes country and ASN information in the same database.
• It is updated daily, with zero compromise to accuracy. There is no range clustering, and the database provides full accuracy.
• The data granularity reaches individual IP level.
• The database comes in MMDB database format.
• It is licensed under CC-BY-SA 4.0, permitting commercial usage.
• Available file formats include: CSV, MMDB, JSON
• The data is tabular and unnested, making it very easy to use. The dataset includes both IPv4 and IPv6 in a single file.

Database schema

Field Name	Example	Data Type	Description
start_ip	1.0.16.0	TEXT	Starting IP address of an IP address range
end_ip	1.0.31.255	TEXT	Ending IP address of an IP address range
country	JP	TEXT	ISO 3166 country code of the location
country_name	Japan	TEXT	Name of the country
continent	AS	TEXT	Continent code of the country
continent_name	Asia	TEXT	Name of the continent
asn	AS2519	TEXT	Autonomous System Number
as_name	ARTERIA Networks Corporation	TEXT	Name of the AS (Autonomous System) organization
as_domain	arteria-net.com	TEXT	Official domain or website of the AS organization

Documentation: https://ipinfo.io/developers/ip-to-country-asn-database

Samples are available here: https://github.com/ipinfo/sample-database/tree/main/IP%20to%20Country%20ASN

The database can be downloaded simply by accessing the storage URI with an access token.

curl -L https://ipinfo.io/data/free/country_asn.mmdb?token=<YOUR_TOKEN> -o country_asn.mmdb

Let me know what you think. Thanks!

[maddyhof]

Thanks for reaching out! Can you share more about what capabilities you're looking for that are not already supported? For example, does this IPinfo database provide information that we don't currently offer, or is there some other reason you're interested in an alternative data source?

[abdullahdevrel]

Hey MaddyHof,

I work for IPinfo by the way. I really appreciate hearing from you.

The intention is to replace the current database here:

outline-server/src/shadowbox/scripts/update_mmdb.sh

Lines 30 to 39 in e416938

	download_ip_mmdb() {
	db="$1"
	for monthdelta in $(seq 0 9); do
	newdate="$(date --date="-${monthdelta} months" +%Y-%m)"
	address="https://download.db-ip.com/free/db${db}-lite-${newdate}.mmdb.gz"
	curl --fail --silent "${address}" -o "${TMPDIR}/${db}.mmdb.gz" > /dev/null && return 0
	done
	return 1
	}

You are downloading a free IP database that is of reduced quality and accuracy. We can systematically prove that the IPinfo's free database is significantly more reliable and accurate than your current database provider. Our data is backed by active measurement providing the highest possible geolocation accuracy in the industry. The current database you are using is updated monthly while ours is updated daily.

The project already uses our API service, which we are very glad to see, and we thought the project deserves more better things like a free IP to Country database. It provides full accuracy, daily updates, and absolute reliability. On the licensing side, it is CC-BY-SA 4.0 (the same license as Wikipedia and Stack Overflow), which even permits commercial distribution by only providing an attribution.

Considering I am pitching a drop-in replacement here, I requested to use the IP to Country database, but I also noticed that you are using an ASN database:

outline-server/src/shadowbox/server/main.ts

Line 46 in e416938

const MMDB_LOCATION_ASN = '/var/lib/libmaxminddb/ip-asn.mmdb';

In that case, you should use our free IP to Country ASN database. It includes both country and ASN data in a single database, so you do not need to maintain 2 separate databases for country and ASN. You can just use the one IP to Country ASN database and retrieve both country and ASN information from there: https://ipinfo.io/developers/ip-to-country-asn-database

The promise of full accuracy and daily updates remains the same as well.

Let me know what you think. I highly recommend downloading the database and exploring it a bit and sharing your thoughts with me. Looking forward to hearing from you.

[fortuna]

@abdullahdevrel it seems one needs an API TOKEN to fetch the db, which makes the deployment flow way more complex, since one would have to create an ipinfo account first.

Is that not the case?

[fortuna]

BTW, with outline-ss-server one can use whatever mmdb file they want.

We could migrate from dbip to ipinfo by default, but that requires someone spending time evaluating the change, which we unfortunately don't have, since this is not a priority compared to other tasks.

[abdullahdevrel]

@fortuna

it seems one needs an API TOKEN to fetch the db, which makes the deployment flow way more complex, since one would have to create an ipinfo account first.

You only need to create a single IPinfo account and ship the access token with the codebase, making it a repo-specific IPinfo token. Projects like Akvorado and Librespeed do this, allowing users to utilize the repo's access token without needing their account.

If you're wondering why we make this data so accessible, we’re dedicated to supporting Open-Source projects like Outline. We want to ensure that adopting our database is as easy as possible, even allowing for commercial distribution.

---

We could migrate from dbip to ipinfo by default, but that requires someone to evaluate the change, which we unfortunately don't have, since this is not a priority compared to other tasks.

I understand that implementing this is a request, not a priority, and I truly appreciate your consideration. We're confident that our database offers top-notch accuracy, data versatility (country + ASN) and reliability, making it a perfect fit for Outline.

I hope you’ll consider the migration whenever the timing works. In any case, I’m glad to have introduced our database as an option for future integration!

[fortuna]

Allowing use to have the api token be public is helpful, but we will probably have to have a legal review of the terms of service to ensure we are not violating them by doing so.

[abdullahdevrel]

@fortuna

Allowing use to have the api token be public is helpful, but we will probably have to have a legal review of the terms of service to ensure we are not violating them by doing so.

That is a really good point.

So, many providers actually have something called an EULA (End User License Agreement) that is added on top of the CC-BY-SA 4.0 license. Essentially, it nullifies the free distribution aspects of the free database. These providers actually charge money for free data and database distribution as part of their commercial distribution license that costs thousands.

But we are not like them!

We do not have an EULA. It is purely CC-BY-SA 4.0! Just give us an attribution of using our data that's all.

We recommend adding the attribution to the repo README.md or the website. Simply adding the following should be enough:

IP address data powered by IPinfo

---

Let me know what you think.