Rename and restructure requirements files

- New/updated requirements files:
  - requirements.txt:
  lists all unpinned immedeate runtime requirements (i.e. combines
  'install_requires' and 'extras_require' from setup.py) and has
  instructions on how to create requirements-pinned.txt
  - requirements-pinned.txt:
  lists all pinned immedeate and transitive runtime requirements,
  based on requirements.txt, including environment markers, and is
  subjected to automatic updates with dependabot
  - requirements-min.txt:
  subset of requirements.txt without 'extras_require' (was
  purepy-requirements.txt). The requirements are not pinned, but
  updates should still trigger tests, if dependabot changes
  requirements-pinned.txt, which lists the same dependabots.
  - requirements-test.txt
  unpinned test runtime dependencies and test tooling. pinning is
  not so important there, because the end-user usually doesn't care
  about those dependencies.
  - requirements-dev.txt
  combines other requirements plus additional tooling and an
  editable install of securesystemslib
- Use requirements-`<suffix>`.txt notation instead of
  `<prefix>`-requirements.txt to group them in file tree view.
- Adopt changes in MANIFEST.in and tox.ini

Author: lukpueh

MANIFEST.in

@@ -4,7 +4,7 @@ include LICENSE
 # Add test config files to show how to run tests
 include tox.ini
 include .travis.yml
-include *requirements.txt
+include requirements*.txt
 
 # Include all files under the tests directory (including test data)
 graft tests

ci-requirements.txt

@@ -0,0 +1,7 @@
+# Install securesystemslib in editable mode with all runtime and test
+# requirements for local testing with tox, and also for the running test suite
+# or individual tests manually
+tox
+-r requirements.txt
+-r requirements-test.txt
+-e .

dev-requirements.txt

@@ -0,0 +1,4 @@
+# Minimal runtime requirements (see 'install_requires' in setup.py)
+six
+python-dateutil
+subprocess32; python_version < '3'

purepy-requirements.txt

@@ -0,0 +1,10 @@
+cffi==1.14.0              # via cryptography, pynacl
+colorama==0.4.3
+cryptography==2.8
+enum34==1.1.6             ; python_version < "3" # via cryptography
+ipaddress==1.0.23         ; python_version < "3" # via cryptography
+pycparser==2.19           # via cffi
+pynacl==1.3.0
+python-dateutil==2.8.1
+six==1.14.0
+subprocess32==3.5.4       ; python_version < "3"

requirements-dev.txt

@@ -0,0 +1,5 @@
+# test runtime dependencies  (see 'tests_require' field in setup.py)
+mock; python_version < "3.3"
+
+# additional test tools
+coverage

requirements-min.txt

@@ -1,6 +1,39 @@
+# All runtime requirements including extras (see 'install_requires' and
+# 'extras_require' in setup.py)
+#
+# This file together with 'pip-compile' is used to generate a pinned
+# requirements file with all immediate and transitive dependencies.
+#
+# 'requirements-pinned.txt' is updated on GitHub with Dependabot, which
+# triggers CI/CD builds to automatically test against updated dependencies.
+#
+# Below instructions can be used to re-generate 'requirements-pinned.txt', e.g.
+# if:
+# - requirements are added or removed from this file
+# - Python version support is changed
+# - CI/CD build breaks due to updates (e.g. transitive dependency conflicts)
+#
+# 1. Use this script to create a pinned requirements file for each Python
+#    version
+# ```
+# for v in 2.7 3.5 3.6 3.7 3.8; do
+#   mkvirtualenv sslib-env-${v} -p python${v};
+#   pip install pip-tools;
+#   pip-compile --no-header -o requirements-${v}.txt requirements.txt;
+#   deactivate;
+#   rmvirtualenv sslib-env-${v};
+# done;
+#
+# ```
+# 2. Use this command to merge per-version files
+#    `sort -o requirements-pinned.txt -u requirements-?.?.txt`
+# 2. Manually add environment markers to requirements-pinned.txt
+# 3. Use this command to remove per-version files
+#    `rm requirements-?.?.txt`
+#
 cryptography
 pynacl
-six
 colorama
+six
 python-dateutil
-subprocess32; python_version < '3'
+subprocess32 ; python_version < '3'

requirements-pinned.txt

@@ -12,22 +12,23 @@ install_command =
     pip install --pre {opts} {packages}
 
 deps =
-    -r{toxinidir}/ci-requirements.txt
+    -r{toxinidir}/requirements-pinned.txt
+    -r{toxinidir}/requirements-test.txt
 
 commands =
     coverage run tests/aggregate_tests.py
     coverage report -m --fail-under 99
 
 [testenv:purepy27]
 deps =
-    -r{toxinidir}/purepy-requirements.txt
+    -r{toxinidir}/requirements-min.txt
 
 commands =
     python -m tests.check_public_interfaces
 
 [testenv:purepy38]
 deps =
-    -r{toxinidir}/purepy-requirements.txt
+    -r{toxinidir}/requirements-min.txt
 
 commands =
     python -m tests.check_public_interfaces