Skip to content

Latest commit

 

History

History
42 lines (27 loc) · 1.94 KB

README.md

File metadata and controls

42 lines (27 loc) · 1.94 KB
Raw

The Archives Unleashed Toolkit

Build Status Contribution Guidelines LICENSE codecov

The Archives Unleashed Toolkit is an open-source platform for analyzing web archives. Tight integration with Hadoop provides powerful tools for analytics and data processing via Apache Spark.

Getting Started

Clone the repo:

$ git clone http://github.com/archivesunleashed/aut.git

You can then build The Archives Unleashed Toolkit.

$ mvn clean install

For the impatient, to skip tests:

$ mvn clean install -DskipTests

The Archives Unleashed Toolkit is built against CDH 5.7.1:

  • Hadoop version: 2.6.0-cdh5.7.1
  • Spark version: 1.6.0-cdh5.7.1

The Hadoop ecosystem is evolving rapidly, so there may be incompatibilities with other versions.

License

Licensed under the Apache License, Version 2.0.

Acknowledgments

This work is primarily supported by the Andrew W. Mellon Foundation. Additional funding for the Toolkit has come from the U.S. National Science Foundation, Columbia University Library's Mellon-funded Web Archiving Incentive Award, the Natural Sciences and Engineering Research Council of Canada, the Social Sciences and Humanities Research Council of Canada, and the Ontario Ministry of Research and Innovation's Early Researcher Award program. Any opinions, findings, and conclusions or recommendations expressed are those of the researchers and do not necessarily reflect the views of the sponsors.