The Databox platform is an open-source personal networked device, augmented by cloud-hosted services, that collates, curates, and mediates access to an individual’s personal data by verified and audited third party applications and services. The Databox will form the heart of an individual’s personal data processing ecosystem, providing a platform for managing secure access to data and enabling authorised third parties to provide the owner with authenticated services, including services that may be accessed while roaming outside the home environment. Databox project is led by Dr Hamed Haddadi (Imperial College) in collaboration with Dr Richard Mortier (University of Cambridge) and Professors Derek McAuley, Tom Rodden, Chris Greenhalgh, and Andy Crabtree (University of Nottingham) and funded by EPSRC. See http://www.databoxproject.uk/ for more information.
These instructions will get you a copy of the project up and running on your local machine. For development and testing purposes, see Development section below.
- Requires Docker. Read here for docker installation.
- Once docker is installed and running, install docker-compose. Read here for installation.
- Requires Git (if it is not already on your machine). Read here for git installation.
Note: currently supported platforms are Linux and MacOS. Running on other platforms is passable using a virtual machine running Linux with bridge mode networking.
- Clone Databox Git repo.
git clone https://github.com/me-box/databox.git
Make sure Docker is installed and running before starting Databox. Run the following to get your databox up and running.
cd databox
./databox-start
The above script pulls Databox pre-build images published on Docker hub and run Databox in your local machine.
Once it's started, point a web browser at http://127.0.0.1:8448 and follow the instructions to configure your HTTPS certificates to access Databox UI securely (using a web browser http://127.0.0.1:8989, or the IOS and android app).
To stop databox and clean up,
./databox-stop
It is recommended to remove the ./certs directory before upgrading. Use the following commands in the root databox directory:
./databox-stop
rm -rf ./certs
git pull
./databox-start
The graphical SDK will allow you to quickly build and test simple databox apps. To start the sdk run:
./databox-start sdk
The SDK web UI is available at http://127.0.0.1:8086
To stop the SDK run:
./databox-stop sdk
It is possible to develop Databox apps and driver without the SDK. Currently, Python, Golang and NodeJs all have support libraries. Building outside the SDK allows you to make smaller more efficient containers and use more third-party libraries.
Developing apps and drivers 'does not' require data box to be started in dev mode.
To get started all you need is a Dockerfile and a databox-manifest.json examples can be found in the libraries '/samples' directories. To make you app available to install locally on your databox you will need to upload the manifest to http://127.0.0.1:8181 and use docker build -t [your-app-name] .. Once the manifest is uploaded and the image has built then you should be up to install the app on you local Databox.
If you would like to modify one of the currently available actual drivers you can do so by doing the following:
./databox-install-component driver-os-monitor
This will download and build the code on your machine and upload the Databox manifest to your local app store. You can also use this with your repositories and forks using:
./databox-install-component [GITHUB_USERNAME]/[GITHUB_REPONAME]
To develop on the platform and core components run the data-box start script with 'dev' parameter. See below.
./databox-start dev
Unlike using the pre-build images, this will clone all the relevant source repositories locally, and build them into the required Docker images.
When you start in development mode only the core-components are built from source. If you wish to develop one of the available apps or drivers then you can add them to you local install using:
./databox-install-component driver-os-monitor
This will download and build the code on your machine and upload the Databox manifest to your local app store. You can also use this with your repositories and forks using:
./databox-install-component [GITHUB_USERNAME]/[GITHUB_REPONAME]
Databox has a number of platform components, divided into two parts: Core and Other components. Core components are required for Databox function. Other components of things like apps and drivers to demonstrate Databoxes functionality.
- Databox-container-manager Container manager controls build, installation and running functions of the other databox components.
- databox-arbiter Arbiter manages the flow of data by minting tokens and controlling store discovery.
- databox-export-service This service controls the data to be exported to external URLs.
- databox-store-json This is a data store used by apps and drivers to store and retrieve JSON data.
- databox-store-timeseries This is a data store used by apps and drivers to store and retrieve JSON data or JPEG images.
- databox-app-server This is a Server for storing and serving databox manifests.
- driver-sensingkit This driver provides SensingKit mobile sensor data.
- driver-google-takeout This driver supports bulk import of google takeout data.
- driver-phillips-hue This drivers allows connection to Phillips Hue Platform.
- driver-os-monitor This driver monitors the databox hardware by fetching Memory consumption and CPU load.
- driver-twitter This driver streams data from a twitter account into a datastore.
- driver-tplink-smart-plug This driver collects data from TP-Link smart plugs.
- app-light-graph An app that plots mobile phone light sensor data.
- app-twitter-sentiment An app that used data from driver-twitter to calculate tweet sentiment.
- app-os-monitor An app to plot the output of the data generated by driver-os-monitor.
For writing a new driver or app for Databox, one needs Databox APIs. To make app/driver development easy, we have wrapped Databox APIs in nodejs, python and go. Using any of these libraries, a developer can build their databox app/driver.
- lib-node-databox: Databox Nodejs API library for building databox apps and drivers.
- lib-python-databox: Databox Python API library for building databox apps and drivers.
- lib-go-databox: Databox Go API library for building databox apps and drivers.
Databox System Design document can be find here and general API specifications are here.
./databox-test
For more details, have a look here.
The databox project welcomes contributions via pull requests see CONTRIBUTING.md for more information. Good start is from having a look on the current issues and forking the databox repo and fixing bugs/issues and submitting a pull request. Read more on Fork and Pull here.
This documentation is up-to-date till this commit. The master branches on all components points to the current release and are tagged in git using semver.
The list of contributors who participated in this project.
MIT Licence, See here.
The Databox project welcomes contributions via pull requests see CONTRIBUTING.md for more information.
-
While building the platform and core components you can sometimes get an error:
[some-network-name] failed to createThis is caused when Databox is started before docker has cleaned up the networks.
This issue can be fixed if you run
./databox-start -
In some cases, the time in docker containers on Mac can get out of sync with the system clock.
This stops the HTTPS certs generated by the CM from being valid. See docker/for-mac#17. And results in the following error:
TLS certificates invalidFix this by restarting Docker for Mac.