Fairywren is a private BitTorrent tracker written around Postgresql and Eventlet.
- Upload torrents through web interface
- Search for torrents by title
- View number of seeders and leechers in real time
- HTML5 & Javascript interface presents a single page application
- Backend of web interface is entirely RESTful
To run fairywren you'll need the following
- Stackless
- Eventlet
- Psycopg2
- Multipart
- ZeroMq
- pyzmq
- GDBM
- Postgresql
- A web server that supports proxying. I use lighttpd.
The unit tests also require
Two seperate Python instances are launched. Each hosts a single HTTP server. One instance is the tracker, which is used by BitTorrent clients to exchange peers. The second is the web interface, which is a RESTful API for interacting with the private tracker. The HTML5 & JavaScript web interface is best served by a traditional web server.
Each instance is ran behind a HTTPS server(lighttpd in my case) which proxies requests to them.
In order to display the seeders and leechers count on each torrent, the web interface needs to get those counts from the tracker. This is done by having the tracker listen on a ZeroMQ PubSub connection. The web interface connects to this as a subscriber. Each time the peer count changes on a torrent, the tracker publishes an update to the web interface. The web interface maintains a list of counts in memory in order to serve them with each request for torrent listings.
The PostgreSQL server is used by both server instances.
The tracker uses the database to authorize specific torrents and users. There is no writing to the database by the tracker. Peers are stored only in memory. At first this seems silly, but given that there is rarely a reason to restart the tracker it works well. If the tracker is restarted, it only takes until all peers have announce'd to rebuild the complete list of peers. If someone comes up with a use case where the tracker is consuming too much memory, the intent will be to move the peer lists into a Redis instance.
The web server uses it to allow users to login and upload new torrents. Torrents themselves are not completely stored in the database, just the info hash and some other information is stored in the torrents table. The actual uploaded BitTorrent files are pickled and stored in the gdbm databse. Any extended information for a torrent is stored at a separate key in the same database. For now this is an appropriate solution. If scalability becomes an issue, I will move to implementing a LRU type cache in the application.
The tables needed are specified in fairywren.sql. The roles needed are in roles.sql. The permissions for the roles are granted in permissions.sql.
Two users are used in my configuration, a read only user for the tracker and a read-write user for the webapi. The example roles and permissions are shown in roles.sql and permissions.sql. Obviously, a single user with global permissions could be substituted.
The configuration file is a JSON file which configures both the tracker and the API. The basic outline is shown in example.conf.json. The JSON file is a dictionary. The keys are
- trackerUrl
- The external URL that the web server proxies to the tracker
- pathDepth
- An integer specifying the depth at which the API and the tracker are proxied from. This is used to allow the code to be independent of the website it is hosted on. For example the URL http://a.com/b/d/f/api/torrents with this configuration value set to 4 causes the first four parts of the path to be ignored and just 'torrents' to be matched against when processing the request.
- salt
- A string used to salt users password before storing them in the database. This value should be random, long, and guarded as secret. Changing this value after adding users is equivalent to setting all users passwords to random values.
- webapi
- Configuration values specific to the API. See webapi.
- tracker
- Configuration values specific to the tracker. See the tracker.
- postgresql
- A dictionary of values. These are passed to the constructor of psycopg2.connect verbatim
- postgresql
- A dictionary of values. These are passed to the constructor of psycopg2.connect verbatim
- torrentPath
- A string which is the path GDBM file. Fairywren stores uploaded BitTorrent files in this database. If the file does not exist it will be created.
- secure
- A boolean indicating if sesssion cookies issued should be flagged with the 'Secure' option. Used when running behind an HTTPS proxy.
The script adduser.py takes a single argument which is the same JSON configuration file as used by the HTTP servers. It prompts for the username and password to add. All users have the same permissions presently.
Presently, users cannot be added via the web interface. The actual REST interface for it exists, but I haven't had time to write the javascript to do it.