Merge pull request #330 from openlawlibrary/ndusan/remove-hosts

refact: Remove hosts

CHANGELOG.md

@@ -11,10 +11,15 @@ and this project adheres to [Semantic Versioning][semver].
 
 ### Changed
 
+- Remove hosts and hosts.json ([330])
+
 ### Fixed
+
 - Add missing `write` flag to `taf targets sign` ([329])
 
+[330]: https://github.com/openlawlibrary/taf/pull/330
 [329]: https://github.com/openlawlibrary/taf/pull/329
+
 ## [0.26.0] - 07/12/2023
 
 ### Added

docs/developers/repository-classes.md

@@ -111,10 +111,6 @@ Instances of the `AuthenticationRepository` are created by passing the same argu
 the authentication repository that was pulled and validated. Instead of validating the entire commit history when
 re-running the update process, updater starts from `last_validated_commit`.
 - `out_of_band_authentication` - manually specified initial commit, used during the update process to validate the first commit
-- `hosts` -  host data, specified using the `hosts.json` file. Hosts of the current repo can be specified in its
-parent's repo (meaning that this repo is listed in the parent's `dependencies.json`), or it can be specified in hosts.
-json contained by the repo itself. If hosts data is defined in the parent, it can be propagated to the contained
-repos.
 
 While in TAF's `Repository` class target files have no special meaning (it is only important that their actual states
 match the information listed in the corresponding metadata file), `AuthenticationRepository`'s target file consist

docs/setting-up-new-library.md

@@ -196,10 +196,9 @@ in `repositories.json`. E.g. `test` and `repo1` for the first repo, `test` and `
 
 ### `dependencies.json`
 
-This target files is optional, but needs to be defined if the authentication repository references other authentication repositories (to define hierarchies), to make a use of the out-of-band authentication check and/or if the framework
-is to be used to handle information about the hosts.
+This target files is optional, but needs to be defined if the authentication repository references other authentication repositories (to define hierarchies), to make a use of the out-of-band authentication check.
 
-This is an example where there are no hierarchies, but we want to define the current repository's expected commit and want to make use of the hosts handlers.
+This is an example where there are no hierarchies, but we want to define the current repository's expected commit and want to make use of the update handlers.
 
 ```
 {
@@ -226,27 +225,6 @@ This is an example where we defined a hierarchy (define two authentication repos
 }
 ```
 
-### `hosts.json`
-
-This is an optional file used to specify information about the hosts. The framework will only extract this information
-from the file and does not implement anything related configuring the servers. Here is an example of this file:
-
-```
-{
-   "some_domain.org": {
-      "auth_repos": {
-       "test/auth_repo": {}
-      },
-      "custom": {
-         "subdomains": {
-            "development": {},
-            "preview: {},
-         }
-      }
-   }
-}
-```
-
 ### `protected/info.json`
 
 Optional file with authentication repository's metadata. Currently, we support specifying either `library_dir` and
@@ -284,8 +262,6 @@ Commit and push the changes. Having pushed the changes, run local validation to
 taf repo validate auth_path
 ```
 
-If hosts were defined, make sure that there is not message saying that that is not the case - that can suggest that names of the repositories defined in different files do not match.
-
 ## Add targets corresponding to target repositories
 
 Information about target repositories of an authentication repository is listed in  `repositories.json`.

docs/updater/specification.md

@@ -244,8 +244,7 @@ repositories. In addition to them, the framework expects some special target fil
 
 - `repositories.json` (required)
 - `mirrors.json` (recommended, required if URLs are not defined in `repositories.json`, re)
-- `depoendencies.json` (optional)
-- `hosts.json` (optional)
+- `dependencies.json` (optional)
 - scripts (optional)
 
 #### repositories.json
@@ -324,38 +323,6 @@ When updating the repository which contains this target file, the updater will a
 
 `out-of-band-authenticatio` is an optional property which, if defined, it is used to check if the repository's first commit matches this property's value. This is far from perfect security measure, but adds an additional layer or protection. The safest way would still be to contact the repositories' maintainers directly.
 
-### hosts.json
-
-This special target file is used to provide hosting information. It contains mappings of domains and authentication repositories whose content (or more precisely, content of whose target repositories) should be served through them. The framework does not actually
-handle servers configuration - it just extracts host information from the `hosts.json` file and makes it easier to consume this data later on (starting with version `0.9.0`). This is an example of `hosts.json`:
-
-```
-{
-   "some_domain.org": {
-      "auth_repos": {
-         "jurisdiction1/law": {}
-      },
-      "custom": {
-         "subdomains": {
-            "development": {},
-            "preview: {},
-         }
-      }
-   },
-   "another_domain.org": {
-      "auth_repos": {
-      "jurisdiction2/law": {}
-      },
-      "custom": {
-         "subdomains": {
-            "development": {},
-            "preview: {},
-         }
-      }
-   }
-}
-```
-
 ### protected/info.json
 
 Stores current repository metadata. Example:
@@ -371,15 +338,15 @@ Where namespace and name map to `namespace/name` format. This data is expected t
 
 #### Scripts
 
-Every authentication repository can contain target files inside `targets/scripts` folder which are expected to be Python scripts which will be executed after successful/failed update of that repository. Scripts can also be defined on a host level - will be executed after update of all repositories belonging to that host.
+Every authentication repository can contain target files inside `targets/scripts` folder which are expected to be Python scripts which will be executed after successful/failed update of that repository.
 
 If a repository was successfully pulled and updated, `changed`, `succeeded` and
 `completed` handlers will be called. If there were no new changes, `unchanged`,
 `succeeded` and `completed` will be executed. If the update failed, `failed` and
 `completed` handlers will be invoked. Scripts are linked to the mentioned events by being
 put into a folder of the corresponding name in side `targets/scripts`. Each folder can
 contain an arbitrary number of scripts and they will be called in alphabetical order.
-Here is a sketch of the `scriprs` folder:
+Here is a sketch of the `scripts` folder:
 
 ```
 
@@ -390,8 +357,8 @@ Here is a sketch of the `scriprs` folder:
       /unchanged
       /failed - every time a repo is not successfully pulled
       /completed  - like finally (called in both cases)
-    /host
-      /succeeded - once for each host, after host's repositories have been successfully pulled
+    /update
+      /succeeded - once after all authentication's repositories have been successfully pulled
       /changed
       /unchanged
       /failed - if one repository failed

docs/updater/update_process.md

@@ -35,7 +35,7 @@ This flags raises an error if the repository allows unauthenticated commits and
 
 ### Hooks
 
-Every authentication repository can contain target files inside `targets/scripts` folder which are expected to be Python scripts which will be executed after successful/failed update of that repository. Scripts can also be defined on a host level - will be executed after update of all repositories belonging to that host.
+Every authentication repository can contain target files inside `targets/scripts` folder which are expected to be Python scripts which will be executed after successful/failed update of that repository.
 
 If a repository was successfully pulled and updated, `changed`, `succeeded` and
 `completed` handlers will be called. If there were no new changes, `unchanged`,
@@ -52,8 +52,8 @@ Here is a sketch of the `scriprs` folder:
       /unchanged
       /failed - every time a repo is not successfully pulled
       /completed  - like finally (called in both cases)
-    /host
-      /succeeded - once for each host, after host's repositories have been successfully pulled
+    /update
+      /succeeded - once after all authentication's repositories have been successfully pulled
       /changed
       /unchanged
       /failed - if one repository failed

taf/api/roles.py

@@ -6,7 +6,7 @@
 import json
 from pathlib import Path
 from logdecorator import log_on_end, log_on_start
-from taf.hosts import REPOSITORIES_JSON_PATH
+from taf.repositoriesdb import REPOSITORIES_JSON_PATH
 from tuf.repository_tool import TARGETS_DIRECTORY_NAME
 import tuf.roledb
 import taf.repositoriesdb as repositoriesdb

taf/auth_repo.py

@@ -20,9 +20,7 @@ class AuthenticationRepository(GitRepository, TAFRepository):
 
     LAST_VALIDATED_FILENAME = "last_validated_commit"
     TEST_REPO_FLAG_FILE = "test-auth-repo"
-    HOSTS_FILE = "hosts.json"
     SCRIPTS_PATH = "scripts"
-    AUTH_REPOS_HOSTS_KEY = "auth_repos"
 
     _conf_dir = None
     _dependencies = {}
@@ -37,7 +35,6 @@ def __init__(
         allow_unsafe=False,
         conf_directory_root=None,
         out_of_band_authentication=None,
-        hosts=None,
         path=None,
         *args,
         **kwargs,
@@ -54,11 +51,6 @@ def __init__(
           custom (dict): a dictionary containing other data
           default_branch (str): repository's default branch ("main" if not defined)
           out_of_band_authentication (str): manually specified initial commit
-          hosts (dict): host data is specified using the hosts.json file. Hosts of the current repo
-          can be specified in its parent's repo (meaning that this repo is listed in the parent's dependencies.json),
-          or it can be specified in hosts.json contained by the repo itself. If hosts data is defined in the parent,
-          it can be propagated to the contained repos. `load_hosts` function of the `hosts` module sets this
-          attribute.
         """
         super().__init__(
             library_dir,
@@ -76,11 +68,6 @@ def __init__(
             conf_directory_root = Path(self.path).parent
         self.conf_directory_root = Path(conf_directory_root).resolve()
         self.out_of_band_authentication = out_of_band_authentication
-        # host data can be specified in the current authentication repository or in its parent
-        # the input parameter hosts is expected to contain hosts data specified outside of
-        # this repository's hosts file specifying its hosts
-        # in other words, propagate hosts data from parent to the child repository
-        self.hosts = hosts
 
     # TODO rework conf_dir
 
@@ -91,7 +78,6 @@ def to_json_dict(self):
             {
                 "conf_directory_root": str(self.conf_directory_root),
                 "out_of_band_authentication": self.out_of_band_authentication,
-                "hosts": self.hosts,
                 "dependencies": self.dependencies,
             }
         )
@@ -150,14 +136,6 @@ def last_validated_commit(self):
         except FileNotFoundError:
             return None
 
-    _hosts_conf = None
-
-    @property
-    def hosts_conf(self):
-        if self._hosts_conf is None:
-            self._hosts_conf = self.get_target(self.HOSTS_FILE)
-        return self._hosts_conf
-
     @property
     def log_prefix(self):
         return f"Auth repo {self.name}: "

taf/exceptions.py

@@ -52,14 +52,6 @@ def __init__(self, metadata_role):
         )
 
 
-class MissingHostsError(TAFError):
-    pass
-
-
-class InvalidHostsError(TAFError):
-    pass
-
-
 class InvalidOrMissingMetadataError(TAFError):
     pass