Bugfixes, workarounds and Bedework test server

I introduced Bedework test server, and had another multi-day-session of "banging on things until all tests pass" - and while doing so, tons of different commits on different things gets added to the same pull request.  I've asked Claude to rebase it into some few well-organized commits, but i'm not much happy with the result, so I will again make a jumbo-commit.

My own non-AI-assisted summary of the changes introduced in this pull request looks like this:

Test framework:

* Changes in the github workflow and test suite to run tests towards bedework.  The bedework docker container is quite straigth forward, comes up with some default test user and password, no extra dependencies.  But it's very old, not maintained and quite buggy.
* Lots of tweaks in the test code to get it pass on Bedework and all the other servers
* Fix for SOGo - tests didn't want to run with github runners.

Code logic:

* Due to the previous changeset with transparent workarounds for servers not supporting sync-tokens, my compatibility check script started reporting that all servers supported sync-tokens.  To revert to the old behaviour, I needed to add a disable_fallback parameer to the objects_by_sync_token method.
* Continued massaging and bugfixing on the compatibility_hints.py.
  - Added synology compatibility matrix
  - Added bedework compatibility matrix
  - Lots of "unsupported" stuff has been reclassified into "ungraceful". 
  - Some more features added to the definitions
  - Bugfix in FeatureSet constructor.  
* Downgraded an annoying "CRITICAL" to "INFO" (some servers sends unexpected content-type on errors or empty responses)
* Workaround to handle Bedework in search - Bedework does not handle the component filter right - if asking for events, it will send everything, and when askin
g for todos it will send ... nothing!  There is now a workaround in the search algorithm.
* If the server can't do text-searches, then we should use client-side filtering of completed tasks.
* Bugfix to the auto-discover URL code - it did not work very well together with the "read the URL from the compatibility_hints.py" logic

The AI-generated summary of this jumbo-commit will be added as a comment to the github pull request at #584

Author: tobixen

.github/workflows/tests.yaml

@@ -66,6 +66,7 @@ jobs:
           --health-timeout 5s
           --health-retries 20
           --health-start-period 10s
+          --network-alias db
       sogo:
         image: japoch/sogo:latest
         ports:
@@ -75,6 +76,16 @@ jobs:
           sogo_pass: testpass
           sogo_name: Test User
           sogo_fqhn: example.com
+      bedework:
+        image: ioggstream/bedework:latest
+        ports:
+          - 8804:8080
+        options: >-
+          --health-cmd "curl -f http://localhost:8080/bedework/ || exit 1"
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 15
+          --health-start-period 120s
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-python@v5
@@ -209,26 +220,24 @@ jobs:
         run: |
           echo "Configuring SOGo..."
 
-          # Wait for SOGo container to start (it won't be healthy yet due to missing DB config)
+          # Wait for database to be ready
+          echo "Waiting for database..."
           sleep 5
 
-          # Get database container hostname
-          DB_HOST=$(docker inspect ${{ job.services.sogo-db.id }} --format='{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}')
-          echo "Database host: $DB_HOST"
-
-          # Copy and modify configuration to use correct database hostname
-          sed "s/@db:/@$DB_HOST:/g" tests/docker-test-servers/sogo/sogo.conf > /tmp/sogo.conf
-          docker cp /tmp/sogo.conf ${{ job.services.sogo.id }}:/etc/sogo/sogo.conf
-          docker cp tests/docker-test-servers/sogo/init-sogo-users.sql ${{ job.services.sogo-db.id }}:/tmp/init-sogo-users.sql
-
           # Initialize database with test user
+          docker cp tests/docker-test-servers/sogo/init-sogo-users.sql ${{ job.services.sogo-db.id }}:/tmp/init-sogo-users.sql
           docker exec -i ${{ job.services.sogo-db.id }} mariadb -usogo -psogo sogo < tests/docker-test-servers/sogo/init-sogo-users.sql
+          echo "✓ Database initialized with test user"
+
+          # Copy SOGo configuration (database is now reachable via network alias 'db')
+          docker cp tests/docker-test-servers/sogo/sogo.conf ${{ job.services.sogo.id }}:/etc/sogo/sogo.conf
+          echo "✓ SOGo configuration copied"
 
           # Restart SOGo to pick up configuration
           docker restart ${{ job.services.sogo.id }}
 
           # Wait for SOGo to be ready
-          echo "Waiting for SOGo..."
+          echo "Waiting for SOGo to start..."
           if timeout 60 bash -c 'until curl -f http://localhost:8803/SOGo/ 2>/dev/null; do echo -n "."; sleep 2; done'; then
             echo ""
             echo "✓ SOGo is ready"
@@ -246,12 +255,34 @@ jobs:
             echo "✗ Error: SOGo CalDAV access failed"
             exit 1
           fi
+      - name: Configure Bedework
+        run: |
+          echo "Waiting for Bedework..."
+          # Bedework/JBoss takes longer to start up
+          if timeout 180 bash -c 'until curl -f http://localhost:8804/bedework/ 2>/dev/null; do echo -n "."; sleep 5; done'; then
+            echo ""
+            echo "✓ Bedework web interface is ready"
+          else
+            echo ""
+            echo "✗ Error: Bedework did not become ready within 180 seconds"
+            exit 1
+          fi
+
+          # Verify CalDAV access with default user (vbede:bedework)
+          echo "Verifying CalDAV access..."
+          if curl -s -X PROPFIND -H "Depth: 0" -u vbede:bedework http://localhost:8804/ucaldav/user/vbede/ 2>/dev/null | grep -qi "multistatus"; then
+            echo "✓ Bedework CalDAV access verified"
+          else
+            echo "✗ Error: Bedework CalDAV access failed"
+            exit 1
+          fi
       - run: tox -e py
         env:
           NEXTCLOUD_URL: http://localhost:8801
           BAIKAL_URL: http://localhost:8800
           CYRUS_URL: http://localhost:8802
           SOGO_URL: http://localhost:8803
+          BEDEWORK_URL: http://localhost:8804
   docs:
     runs-on: ubuntu-latest
     steps:

caldav/collection.py

@@ -1141,7 +1141,10 @@ def _generate_fake_sync_token(self, objects: List["CalendarObjectResource"]) ->
         return f"fake-{hash_value}"
 
     def objects_by_sync_token(
-        self, sync_token: Optional[Any] = None, load_objects: bool = False
+        self,
+        sync_token: Optional[Any] = None,
+        load_objects: bool = False,
+        disable_fallback: bool = False,
     ) -> "SynchronizableCalendarObjectCollection":
         """objects_by_sync_token aka objects
 
@@ -1164,12 +1167,18 @@ def objects_by_sync_token(
         This method transparently falls back to retrieving all objects if the server
         doesn't support sync tokens. The fallback behavior is identical from the user's
         perspective, but less efficient as it transfers the entire calendar on each sync.
+
+        If disable_fallback is set to True, the method will raise an exception instead
+        of falling back to retrieving all objects. This is useful for testing whether
+        the server truly supports sync tokens.
         """
         ## Check if we should attempt to use sync tokens
         ## (either server supports them, or we haven't checked yet, or this is a fake token)
         use_sync_token = True
         sync_support = self.client.features.is_supported("sync-token", return_type=dict)
         if sync_support.get("support") == "unsupported":
+            if disable_fallback:
+                raise error.ReportError("Sync tokens are not supported by the server")
             use_sync_token = False
         ## If sync_token looks like a fake token, don't try real sync-collection
         if (
@@ -1210,6 +1219,8 @@ def objects_by_sync_token(
                 )
             except (error.ReportError, error.DAVError) as e:
                 ## Server doesn't support sync tokens or the sync-collection REPORT failed
+                if disable_fallback:
+                    raise
                 log.info(
                     f"Sync-collection REPORT failed ({e}), falling back to full retrieval"
                 )