The MillionTrees benchmark is designed to provide open, reproducible, and rigorous evaluation of tree detection algorithms. This repo is the Python package for rapid data sharing and evaluation.
We have released a beta version of public data, these are datasets that have previously been published and have a DOI. We will followup this release, likely with a 1.0 tag, of the previously unpublished parts of the dataset along with a scientific manuscript.
📊 Current Dataset Status: See our comprehensive Dataset Release Report for up-to-date information on dataset versions, sizes, and download links.
There are three data loaders based on annotation geometry. TreeBoxes are bounding boxes for individual tree detection. TreePoints are centroids for tree counting and detection, and TreePolygons are for finer crown segmentation.
There has been a tremendous number of tree crown detection benchmarks, but a lack of progress towards a single algorithm that can be used globally across aquisition sensors, forest type and annotation geometry. Our view is that the hundreds of tree detection algorithms for RGB data published in the last 10 years are all data starved. There are many good models, but they can only be so useful with the small datasets any research team can collect. The result is years of effort in model development, but ultimately a lacking solution for a large audience. The MillionTrees dataset seeks to collect a million annotations across point, polygon and box geometries at a global scale.
The MillionTrees dataset represents where we are as a community. Many datasets are incompletely annotated, and there is varying degrees of annotation accuracy. This is by design, we aim to reflect the real, not idealized, status of tree detection algorithms and applications. By incluing these data that are normally excluded from benchmarks we can both dramatically increase the diversity of tree presentations and backgrounds, as well as engage the community in solving common computer vision challenges for applied machine learning.
pip install milliontrees
Hugging Face dataset loading and sharing functionality is included in the main package.
To build from the GitHub source and install the required dependencies, follow these instructions:
-
Clone the GitHub repository:
git clone https://github.com/weecology/MillionTrees.git -
Change to the repository directory:
cd MillionTrees -
(Recommended) Create and activate a virtual environment, then install dev extras:
python -m venv .venv && source .venv/bin/activate pip install -e .[dev,docs] -
(Optional) Build distributions:
python -m build
Once the installation is complete, you can use the MillionTrees package in your Python projects.
Datasets are documented on ReadTheDocs with sample images overlayed with annotations. https://milliontrees.idtrees.org/en/latest/datasets.html
Fast sanity-check runs on mini datasets (one image per source):
| Model (script) | Task | Root dir | Key metrics |
|---|---|---|---|
| sam3_points.py (SAM3 native, GPU) | TreePoints | data-mini | KeypointAccuracy: 0.000; Counting MAE: 1164.000 |
| sam3_boxes.py (SAM3 native, GPU) | TreeBoxes | data-mini | Detection Acc: 0.083; Recall: 0.084 |
| baseline_points.py (DeepForest) | TreePoints | /orange/ewhite/web/public/MillionTrees | KeypointAccuracy: 0.000; Counting MAE: 104.250 |
| baseline_boxes.py (DeepForest) | TreeBoxes | /orange/ewhite/web/public/MillionTrees | Detection Acc: 0.559; Recall: 0.794 |
See more in docs/leaderboard.md. To reproduce, use the example commands in that file.
The design of the MillionTrees benchmark was inspired by the WILDS benchmark, and we are grateful to their work, as well as Sara Beery for suggesting the use of this template.