Fix first time grant failure bug

README.md

@@ -9,13 +9,22 @@ webhook URL.
 
 The below `sally server start` command shows how to use Sally, once it has been set up (see below for setup instructions).
 
+Make sure to use the `--direct` argument to start the server up in direct agent mode. This means that the agent will listen on the specified, or default 9723, HTTP port
+for direct HTTP requests. The alternate is to use indirect mode by omitting `--direct` which will cause Sally to rely on any configured witnesses as a communication mailbox (relay).
+
 ```bash
 sally server start \
-  --name sally --alias sally \
+  --direct \
+  --http 9723 \
   --salt 0AD45YWdzWSwNREuAoitH_CC \
+  --name sally \
+  --alias sally \
+  --config-dir scripts \
+  --config-file sally.json \
+  --incept-file sally-incept.json \
   --passcode VVmRdBTe5YCyLMmYRqTAi \
   --web-hook http://127.0.0.1:9923 \
-  --auth EMHY2SRWuqcqlKv2tNQ9nBXyZYqhJ-qrDX70faMcGujF
+  --auth EMCRBKH4Kvj03xbEVzKmOIrg0sosqHUF9VG2vzT9ybzv \
   --loglevel INFO
 ```
 
@@ -205,12 +214,34 @@ KERI_SCRIPT_DIR=./scripts KERI_DEMO_SCRIPT_DIR=./scripts/demo ./scripts/demo/vLE
 Now that you have a sample vLEI ecosystem running you will need to configure and run the Sally server.  
 
 In order to start Sally you will need to either:
-1. Use the `kli init` and `kli incept` commands to create an AID for Sally to use.
-2. (Not yet working) Use the `--incept-file` and `--salt` arguments to instruct the `sally server start` command to create a new identifier, or
+1. Use the `--config-file`, `--config-dir`, `--incept-file`, and `--salt` arguments to instruct the `sally server start` command to create a new identifier, or
+2. Use the `kli init` and `kli incept` commands to create and configure an AID for Sally to use.
 
 Both options require the following configuration files:
 
-### Option 1 - `kli` commands
+### Option 1 - `sally server start` command
+
+Example:
+```bash
+sally server start \
+  --direct \
+  --http 9723 \
+  --salt 0AD45YWdzWSwNREuAoitH_CC \
+  --name sally \
+  --alias sally \
+  --config-dir scripts \
+  --config-file sally.json \
+  --incept-file sally-incept.json \
+  --passcode VVmRdBTe5YCyLMmYRqTAi \
+  --web-hook http://127.0.0.1:9923 \
+  --auth EMCRBKH4Kvj03xbEVzKmOIrg0sosqHUF9VG2vzT9ybzv \
+  --loglevel INFO
+```
+
+You must specify both the keystore (Habery) configuration file and the identifier (Hab) inception file. The `--config-dir` argument applies to both the
+keystore and identifier files. For the keystore configuration the directory `keri/cf` is appended to the value of `--config-file` if it is not an absolute path.
+
+### Option 2 - `kli` commands
 
 Creating an identifier with the `kli init` and `kli incept` commands requires the following two commands to be run from an activated 
 Python virtual environment that has `keripy` configured to run so that the `kli` command is available. 
@@ -233,18 +264,14 @@ Finally, you can start (and leave running) the Sally server with:
 
 ```bash
 sally server start --name sally --alias sally --passcode VVmRdBTe5YCyLMmYRqTAi \
+  --http 9723 \
   --web-hook http://127.0.0.1:9923 \
   --auth EHOuGiHMxJShXHgSb6k_9pqxmRb8H-LT0R2hQouHp8pW
 ```
 
 If you require a sample web hook to receive the notifications from the Sally server one is provided in this repo.  You
 can run the sample hook server in a separate terminal with the following command. The above Sally command assumes this 
-server and port by default. 
-
-### Option 2 (not yet working) - `sally server start` command
-
-You must specify both the keystore (Habery) configuration file and the identifier (Hab) inception file. The `--config-dir` argument applies to both the
-keystore and identifier files. For the keystore configuration the directory `keri/cf` is appended to the value of `--config-file` if it is not an absolute path.
+server and port by default.
 
 #### Configuration Files
 
@@ -305,14 +332,17 @@ The following command will start the Sally server with a new identifier and salt
 
 ```bash
 sally server start \
-  --name sally --alias sally \
+  --direct \
+  --http 9723 \
   --salt 0AD45YWdzWSwNREuAoitH_CC \
-  --passcode VVmRdBTe5YCyLMmYRqTAi \
-  --web-hook http://127.0.0.1:9923 \
-  --auth EMHY2SRWuqcqlKv2tNQ9nBXyZYqhJ-qrDX70faMcGujF
+  --name sally \
+  --alias sally \
   --config-dir scripts \
-  --config-file sally-habery.json \
+  --config-file sally.json \
   --incept-file sally-incept.json \
+  --passcode VVmRdBTe5YCyLMmYRqTAi \
+  --web-hook http://127.0.0.1:9923 \
+  --auth EMCRBKH4Kvj03xbEVzKmOIrg0sosqHUF9VG2vzT9ybzv \
   --loglevel INFO
 ```
 

scripts/sally-incept.json

@@ -1,9 +1,7 @@
 {
   "transferable": true,
-  "wits": [
-    "BBilc4-L3tFUnfM_wJr4S4OJanAv_VmF_dJNN6vkf2Ha"
-  ],
-  "toad": 1,
+  "wits": [],
+  "toad": 0,
   "icount": 1,
   "ncount": 1,
   "isith": "1",

setup.py

@@ -11,7 +11,7 @@
 $ pip3 install twine
 
 $ python3 setup.py sdist
-$ twine upload dist/keri-0.0.1.tar.gz
+$ twine upload dist/sally-0.0.1.tar.gz
 
 Create release git:
 $ git tag -a v0.4.2 -m "bump version"
@@ -39,10 +39,12 @@
 
 setup(
     name='sally',
-    version='1.0.0-rc1',  # also change in src/sally/__init__.py
-    license='Apache Software License 2.0',
+    version='1.0.2',  # also change in src/sally/__init__.py
+    license='Apache-2.0',
+    license_files=('LICENSE',),
     description='vLEI Audit Reporting API',
     long_description=long_description,
+    long_description_content_type='text/markdown',
     author='Philip S. Feairheller',
     author_email='pfeairheller@gmail.com',
     url='https://github.com/GLEIF-IT/sally',
@@ -55,7 +57,6 @@
         # complete classifier list: http://pypi.python.org/pypi?%3Aaction=list_classifiers
         'Development Status :: 3 - Alpha',
         'Intended Audience :: Developers',
-        'License :: OSI Approved :: Apache Software License',
         'Operating System :: Unix',
         'Operating System :: POSIX',
         'Operating System :: Microsoft :: Windows',
@@ -78,7 +79,8 @@
     ],
     python_requires='>=3.12.3',
     install_requires=[
-        'cit-keri==1.2.8',
+        # This version of KERIpy uses the GLEIF-IT/keripy v1.2.8 branch as of the exn message fix
+        'keri @ git+https://github.com/GLEIF-IT/keripy.git@e881c9522b9bd38c5d5d5ef04f4e231eb902c36a',
         'hio==0.6.14',
         'multicommand==1.0.0',
         'blake3==0.4.1',
@@ -89,14 +91,6 @@
         'test': ['pytest', 'coverage', 'pytest-mock-server'],
         'docs': ['sphinx', 'sphinx-rtd-theme']
     },
-    tests_require=[
-        'coverage==7.7.1',
-        'pytest==8.3.5',
-        'pytest-mock-server==0.3.2'
-    ],
-    setup_requires=[
-        'setuptools==80.3.1'
-    ],
     entry_points={
         'console_scripts': [
             'sally = sally.app.cli.kli:main',

src/sally/__init__.py

@@ -6,4 +6,30 @@
 
 """
 
-__version__ = '1.0.0-rc1'  # also change in setup.py
+__version__ = '1.0.2'  # also change in setup.py
+
+import logging
+
+from hio.help import ogling
+
+from sally.app.logs import TruncatedFormatter
+
+log_name = 'sally'  # name of this project that shows up in log messages
+log_format_str = f'%(asctime)s [{log_name}] %(levelname)-8s %(module)s.%(funcName)s-%(lineno)s %(message)s'
+
+ogler = ogling.initOgler(prefix=log_name, syslogged=False)
+ogler.level = logging.INFO
+
+formatter = TruncatedFormatter(log_format_str)
+formatter.default_msec_format = None
+logHandler = logging.StreamHandler()
+logHandler.setFormatter(formatter)
+ogler.baseFormatter = formatter
+ogler.baseConsoleHandler = logHandler
+ogler.baseConsoleHandler.setFormatter(formatter)
+ogler.reopen(name=log_name, temp=True, clear=True)
+
+def set_log_level(loglevel, logger):
+    """Set the log level for the logger."""
+    ogler.level = logging.getLevelName(loglevel.upper())
+    logger.setLevel(ogler.level)

src/sally/app/bootstrapping.py

@@ -0,0 +1,82 @@
+from typing import Callable
+
+from hio.base import doing
+from keri.app import habbing, oobiing
+
+from sally import ogler, log_name
+
+logger = ogler.getLogger(log_name)
+
+
+class BootstrapRunner(doing.DoDoer):
+    def __init__(self, hby: habbing.Habery, tymth: Callable):
+        """
+        Parameters:
+            hby (Habery): the hab in which to create the AID, if needed
+            tymth (function): the clock time (scheduler tick rate) to use from the parent Doist
+        """
+        self.hby = hby
+        self.complete = False
+        super(BootstrapRunner, self).__init__(tymth=tymth)
+        self.extend([OobiResolveBootstrapper(hby, self)])
+
+    def recur(self, tyme, deeds=None):
+        if self.complete:
+            return True
+        super(BootstrapRunner, self).recur(tyme, deeds)
+        return False
+
+    def configureAndIncept(self):
+        """Configure a keystore by resolving OOBIs in config"""
+        self.extend(OobiResolveBootstrapper(self.hby, self))
+
+class OobiResolveBootstrapper(doing.Doer):
+    """
+    Bootstraps OOBI Resolutions for a Habery and an AID, resolving all OOBIs in the bootstrap configuration file.
+    """
+
+    def __init__(self, hby: habbing.Habery, parent: BootstrapRunner, tock=0.0, **kwa):
+        self.hby = hby
+        self.parent = parent
+        super(OobiResolveBootstrapper, self).__init__(tock=tock, **kwa)
+
+    def configure_and_incept(self):
+        oobi_count = self.hby.db.oobis.cntAll()
+        if oobi_count:
+            obi = oobiing.Oobiery(hby=self.hby)
+            self.parent.extend(obi.doers)
+
+            while oobi_count > self.hby.db.roobi.cntAll():
+                yield 0.25
+
+            for (oobi,), obr in self.hby.db.roobi.getItemIter():
+                if obr.state in (oobiing.Result.resolved,):
+                    logger.info(f"{oobi} succeeded")
+                if obr in (oobiing.Result.failed,):
+                    logger.error(f"{oobi} failed")
+
+            self.parent.remove(obi.doers)
+
+        wc = [oobi for (oobi,), _ in self.hby.db.woobi.getItemIter()]
+        if len(wc) > 0:
+            logger.info(f"\nAuthenticating Well-Knowns...")
+            authn = oobiing.Authenticator(hby=self.hby)
+            self.parent.extend(authn.doers)
+
+            while True:
+                cap = []
+                for (_,), wk in self.hby.db.wkas.getItemIter(keys=b''):
+                    cap.append(wk.url)
+
+                if set(wc) & set(cap) == set(wc):
+                    break
+
+                yield 0.5
+
+            self.parent.remove(authn.doers)
+
+        self.parent.complete = True
+
+    def recur(self, tock=0.0, **opts):
+        yield from self.configure_and_incept()
+        return True

src/sally/app/cli/commands/hook/demo.py

@@ -12,10 +12,11 @@
 from keri import help
 from keri.app import directing
 
+from sally import log_name, ogler
 from sally.core import handling, httping
 from sally.core.monitoring import HealthEnd
 
-logger = help.ogler.getLogger()
+logger = ogler.getLogger(log_name)
 
 parser = argparse.ArgumentParser(description='Launch SALLY sample web hook server')
 parser.set_defaults(handler=lambda args: launch(args),

src/sally/app/cli/commands/server/start.py

@@ -4,14 +4,15 @@
 
 """
 import argparse
-import logging
 import os
 
-from keri import help
-from keri.app import keeping, habbing, directing, configing, oobiing
+from hio.base import doing
+from keri.app import keeping, habbing, configing
 from keri.app.cli.common import existing
 
 import sally
+from sally import ogler, log_name, set_log_level
+from sally.app.bootstrapping import BootstrapRunner
 from sally.core import serving
 
 parser = argparse.ArgumentParser(description='Launch Sally vLEI credential presentation receiver service.')
@@ -64,19 +65,15 @@
     "-l", "--loglevel", action="store", required=False, default=os.getenv("SALLY_LOG_LEVEL", "INFO"),
     help="Set log level to DEBUG | INFO | WARNING | ERROR | CRITICAL. Default is CRITICAL")
 
-help.ogler.level = logging.getLevelName(logging.INFO)
-logger = help.ogler.getLogger()
+
+logger = ogler.getLogger(log_name)
 
 def launch(args, expire=0.0):
     """Launch Sally vLEI credential presentation receiver service"""
     # Logging config
-    base_formatter = logging.Formatter('%(asctime)s [sally] %(levelname)-8s %(message)s')
-    base_formatter.default_msec_format = None
-    help.ogler.baseConsoleHandler.setFormatter(base_formatter)
-    help.ogler.level = logging.getLevelName(args.loglevel.upper())
-    logger.setLevel(help.ogler.level)
-    help.ogler.reopen(name="sally", temp=True, clear=True)
+    set_log_level(args.loglevel, logger)
 
+    # Parse arguments
     hook = args.web_hook
     name = args.name
     salt = args.salt
@@ -103,6 +100,18 @@ def launch(args, expire=0.0):
         "incept_file": incept_file, "config_dir": config_dir
     }
 
+    hby = init_habery(name=name, base=base, bran=bran, config_file=config_file,
+                      config_dir=config_dir, salt=salt)
+    hab = serving.incept_if_new(hby, alias, incept_args)
+
+    # setup doers
+    doers = serving.setupDoers(hby, hab, alias=alias, http_port=http_port, hook=hook, auth=auth,
+                           timeout=timeout, retry=retry, direct=direct, incept_args=incept_args)
+    logger.info(f"Sally Server v{sally.__version__} listening on {http_port} with DB version {hby.db.version}")
+    serving.run_doers(doers)
+
+def init_habery(name, base, bran, config_file, config_dir, salt):
+    """Initialize or reopen the Habery (keystore)"""
     ks = keeping.Keeper(name=name, base=base, temp=False, reopen=True)
     aeid = ks.gbls.get('aeid')
 
@@ -113,17 +122,13 @@ def launch(args, expire=0.0):
                                     temp=False, reopen=True, clear=False)
         habery_cfg = dict()
         habery_cfg["salt"] = salt if salt else None # When None causes Habery to randomize salt
-        hby = habbing.Habery(name=name, base=base, bran=bran, cf=cf, **habery_cfg)
+        hby = habbing.Habery(name=name, base=base, bran=bran, cf=cf, ks=ks, **habery_cfg)
     else:
+        ks.close() # existing.setupHby reopens, so close here to avoid LMDB conflict.
         hby = existing.setupHby(name=name, base=base, bran=bran)
 
-    # setup doers
-    hbyDoer = habbing.HaberyDoer(habery=hby)
-    obl = oobiing.Oobiery(hby=hby)
-
-    doers = [hbyDoer, *obl.doers]
-    doers += serving.setup(hby, alias=alias, httpPort=http_port, hook=hook, auth=auth,
-                           timeout=timeout, retry=retry, direct=direct, incept_args=incept_args)
-
-    logger.info(f"Sally Server v{sally.__version__} listening on {http_port} with DB version {hby.db.version}")
-    directing.runController(doers=doers, expire=expire)
+    # Resolve OOBIs for the Habery and the new Hab
+    bootstrap_doist = doing.Doist(limit=0.0, tock=0.03125, real=True)
+    runner = BootstrapRunner(hby=hby, tymth=bootstrap_doist.tymen())
+    bootstrap_doist.do(doers=[runner])
+    return hby