Initial support for Qubes 4.1 #751

eaon · 2022-01-13T17:38:25Z

Status

Ready for review. Towards #600.

Description of changes

Enables installation of SDW on Qubes 4.1.
Updates qrexec policies to use the new R5.0 format where appropriate. We now write some policies to the new location, but continue to maintain others in the legacy location, in order to ensure our policies remain defensive (e.g. qubes.Gpg).
Removes unused duplicate RPM repo config from sys-firewall; closes Repo config for dom0 is duplicated in sys-firewall #702. Technically this change affects 4.0, as well, but was convenient given that sys-firewall is a DispVM in Q4.1.
Liberal test updates throughout. Mostly that's for RPC policies, but also made sure to update the gpg calls for validating key material.

Known problems

Export and print functionality hasn't been implemented for Qubes 4.1 yet. Under 4.1, the sys-usb VM is disposable by default, meaning our custom udev rules do not persist. We should handle that change in a separate PR.

The sd-log setup will complain (via GUI notifications of qrexec denial) about lack of loopback support; see #755. We may be able to leverage notify=false in the R5 policy syntax to resolve.

The dom0 repos currently hardcode the 4.0 locations: https://yum-test.securedrop.org/workstation/dom0/f25/ That's good enough for now, but we'll want to create f32 locations on the yum server(s), and update the URLs here accordingly.

Test plan

We want to verify that the Workstation behaves well under both Qubes 4.0 & Qubes 4.1. To that end, we'll need to test on both platforms. Let's divvy up the testing obligations across the team, to minimize duplication of effort. Tests we'll need to investigate on each platform:

Clean install works: make clone && make clean && make dev and make test has only test_sys_usb related tests failing
Perform full suite of Qubes scenarios
Perform full suite of Client QA checks

See the wiki page for formatting your reports: https://github.com/freedomofpress/securedrop-workstation/wiki/Workstation-test-case-report-template

dom0/sd-sys-vms.sls

conorsch · 2022-01-14T00:24:14Z

Great work here so far, @eaon! Pushed a few fix-ups (which we can squash out later). Please take a closer look at the RPC policy logic. As configured, I'm seeing failures related to the disp-mgmt VMs, because of our disallow declarations in the VMShell config. Perhaps we'll need to loosen those restrictions somewhat for 4.1; my assumption is that in 4.0, the disp-mgmt VMs were exempted from those lines, and in 4.1 they're not.

Also, as a stretch goal, see if you can keep the make clean action supported (via dom0/sd-clean-all.sls); that'll help as we test configs repeatedly.

eaon · 2022-01-18T16:26:49Z

With the recent changes it should now install without salt failures, and make clean should also remove files and text blocks from R4.1 specific files.

There's one remaining failure with regards to the RPC calls: qrexec: securedrop.Log: sd-log -> sd-log: denied: loopback qrexec connection not supported. Very curious to touch on that later in our meeting!

conorsch · 2022-01-18T20:06:19Z

There's one remaining failure with regards to the RPC calls: qrexec: securedrop.Log: sd-log -> sd-log: denied: loopback qrexec connection not supported. Very curious to touch on that later in our meeting!

That's true on 4.0, as well, so not a regression for 4.1. I'll work on verifying the changes here as far as a clean install, then take another look at the disp-mgmt-vm setup, maybe using something like @dispvm:default-mgmt-dvm to prune the allow declarations.

rocodes · 2022-01-20T14:17:06Z

dom0/sd-dom0-files.sls

@@ -60,12 +79,13 @@ dom0-remove-securedrop-workstation-stretch-template:
      - file: dom0-workstation-rpm-repo

 dom0-install-securedrop-workstation-template:


I think that this entire block will need to be in a conditional, since pkg.installed is the required syntax for 4.0, and qvm.template_installted is required for 4.1

(Going to stop before I get too far into all the 4.0/4.1 compatibility notes since we'll have to figure out how we want to organize that)

dom0/sd-dom0-qvm-rpc.sls

rocodes · 2022-01-20T14:32:06Z

Thanks for your work @eaon! I left a couple comments, but stopped before getting too far into the weeds, since a lot of the comments were going to be around branching/dual support for 4.0 vs 4.1, and maybe that kind of feedback is premature rn. The other area of commentary is around RPC policy definition/scoping, but I'm going to dig into the docs before I add any more notes on that as well.

rocodes · 2022-01-20T19:25:37Z

dom0/sd-dom0-files.sls

+    - content: |
+        [securedrop-workstation-templates]
+        gpgcheck=1
+        gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-securedrop-workstation


I noticed that in 4.1, key files are also stored in /etc/qubes/repo-templates/keys/ (I have duplicate RPM-GPG-KEY-xxxx in both locations), but this new location could be preferred to store key files with repo templates.

Agreed 💯 - I assumed the template repo key(s) were moved there and didn't want to duplicate files unnecessarily, but it looks like 4.1 has the same keys in both directories at the moment. I wonder if that's a requirement for qvm-template to work.

nope, currently it's not (from my limited testing anyway)- it works with keys in the /etc/pki directory, as long as that's specified in the .repo file, although long-term I imagine that may change.

rocodes · 2022-01-20T19:31:17Z

dom0/sd-dom0-files.sls

+dom0-workstation-templates-repo:
+  # Using file.blockreplace because /etc/qubes/repo-templates/ is not a .d
+  # style directory, and qvm.template_installed:fromrepo seems to only support
+  # using a repo from this file. Installing manually via a cli-command-instead?


I think you're right that qvm.template_installed does not yet support an alternate repo file location. Another possibility if we want to avoid touching the qubes .repo files directly is using

dom0-install-securedrop-workstation-template: cmd.run: - name: qvm-template install securedrop-workstation-buster --repo-files /etc/qubes/repo-templates/securedrop-workstation-template.repo - require: - file: securedrop-workstation-template-key # either in /etc/pki or in /etc/qubes/repo-templates/keys - file: securedrop-workstation-template-repo # separate template .repo file

conorsch · 2022-02-04T01:21:40Z

So far, so good! I tacked on a few test tweaks, in an attempt to get make test passing under 4.1. Still failing are:

sys-firewall checks (content does not persist, due to DispVM; but see Repo config for dom0 is duplicated in sys-firewall #702 for removing altogether)
gpg checks are failing (same problem affects ./sdw-admin --validate in 4.1: gpg2 doesn't like the cmd we use)
sys-usb lacks the udev rules (content does not persist, due to DispVM; means that export functionality won't work out of the box)

All of those are blockers for releasing 4.1 support, but I'm optimistic we can merge this PR for ongoing support. Would like to spend some more time testing this branch against 4.0 to guard against breakage.

conorsch · 2022-03-16T23:15:44Z

Further work to come:

update tests for RPC policies under 4.1 (current RPC tests assume 4.0)
write detailed test plan (mostly what's in the existing acceptance testing docs
document follow-up tasks as separate tickets referenced from compat epic (e.g. sd-log tries to log to itself, but loopback qrexec connections aren't supported #755)

Today at sprint planning we committed to getting this into a reviewable state this sprint. My goal is to retain 4.0 functionality untouched in this PR, so we can merge these changes and continue to iterate on full 4.1 support.

conorsch · 2022-03-30T19:39:08Z

Updated and liberally squash-rebased. Wrote a fresh test plan, calling out a few areas still to be addressed—notably print and export functionality—but otherwise ready to rock. Let's discuss testing obligations at sprint planning today, to share the load.

eloquence · 2022-03-31T00:11:05Z

dom0/sd-dom0-files.sls

+{% if grains['osrelease'] == '4.1' %}
+  qvm.template_installed:
+    - name: securedrop-workstation-buster
+{% else %}


Nit: Would it maybe be better to use{% elif grains['osrelease'] == '4.0' %} here and in subsequent similar statements? I'm sure we'll be removing this conditional logic pretty soon, but it seems best to be always explicit about the version-based branching.

eloquence · 2022-03-31T01:22:33Z

I've taken this for a first spin on my Qubes 4.0 system. Initial results are looking good:

make clean && make staging completed without errors
make test all passing
Have not noticed any spurious 4.1 files in dom0 yet
Updater run completed successfully
App still appears to work as expected: login, decryption, viewing
USB attach logic still appears to work as expected
Log aggregation still appears to work as expected

I'll start going through the test plans tomorrow. If all looks good there, I'll switch this system to 4.1 via in-place upgrade next.

eloquence · 2022-03-31T22:55:34Z

OS environment: Qubes 4.0
Workstation environment: staging
Server environment: 2.3.0 prod on hardware

SecureDrop Workstation test scenarios

Qubes scenarios

Verify tor connection to Journalist API

Create VM for accessing JI via Tor Browser: qvm-create --template whonix-ws-16 --property netvm=sd-whonix --label orange sd-research. Open Tor Browser in that VM and confirm you can log in to the Journalist Interface. This confirms that sd-whonix is configured correctly (but does not use sd-proxy).
Change the netvm to sys-whonix and confirm you can load the public Source Interface, but not the Journalist Interface. (N.B. you must leave the netvm set to sys-whonix, otherwise make clean and sdw-admin --uninstall will fail.)

Verify default DispVM settings

Open a shell in a non-SDW VM, e.g. sd-dev. Download a PDF file and open it via: qvm-open-in-dvm <pdf_file>. Confirm it opens in a DispVM, and that the DispVM is based on sd-viewer.
Open a shell in sd-app and find an already downloaded submission in ~/.securedrop_client/data/. Run xdg-open <file_path> and confirm it opens in a DispVM, and that the DispVM is based on sd-viewer.

RPC Policies

Open a shell in a non-SDW VM, e.g. sd-dev. Run: QUBES_GPG_DOMAIN=sd-gpg qubes-gpg-client -k. Confirm that the request is denied, i.e. you do NOT see pubkey info for the SecureDrop Submission Key.
Try to copy/paste from the Client to a non-SDW VM, e.g. sd-dev. Confirm you cannot.
Add the clipboard tags to sd-dev as described in the docs, and repeat the copy/paste procedure. Confirm it works.

Logging

Logs are sent to sd-log VM
No warnings are displayed about logging denials (e.g. sd-log to sd-log; see sd-log tries to log to itself, but loopback qrexec connections aren't supported #755)

GUI updater

Reboot the workstation after installing SDW. Confirm that the prelaunch updater window appears automatically after logging, prompting for an update.
Proceed with GUI updater, confirm it runs without errors.

eloquence · 2022-04-01T01:24:32Z

OS environment: Qubes 4.0
Workstation environment: staging
Server environment: 2.3.0 prod on hardware
Client version: securedrop-client_0.6.0-dev-20220331-060621

Client scenarios

Scenario: Online mode

Login

when SecureDrop desktop icon is double-clicked, preflight updater is displayed
After preflight updater runs, when user clicks Continue, login dialog is displayed
In login dialog:
- show/hide password functionality works
- incorrect password cannot log in
- 2FA token reuse cannot log in after password failure
- invalid 2FA token cannot log in
- valid credentials and 2FA can log in

Sources

after valid login:
- the login dialog closes
- source data is downloaded and source list is populated
- user is prompted for GPG key access
- submissions and replies are decrypted
- the source list is displayed but no sources are selected by default
- the conversation view is not populated
when a source is selected in source list:
- conversation view is populated with source conversation
- a source message containing HTML is displayed as unformatted text
- source submissions have an active Download button
- source submission compressed file size is displayed accurately
when the upper right 3-dot button is clicked:
- a menu is displayed with a delete source account option
- when delete source account is selected:
  - the source is deleted from the source list and the conversation view is blanked
  - the source is deleted from the server and not restored on next sync
  - source submissions and messages are removed from the client's data directory
when a source is starred in source list, and the client is closed and reopened in Online mode:
- the source is still starred in the source list

Replies

when a source is selected in the source list:
- the reply panel is available for use and there is no message asking the user to sign in
- a reply can be added to the conversation
- a pending reply can be added to the conversation (for development environments, you can use: wget https://gist.githubusercontent.com/creviera/7f19a7d10334359f40dbdbb2354cd13a/raw/a2ef94913a155aa4019b753cf916f844c9cffa3a/pending-reply && git apply pending-reply then send a reply; alternatively, disconnect the network or sd-whonix after sending a reply)
- a failed reply can be added to the conversation (for development environments, you can use: wget https://gist.githubusercontent.com/creviera/5ba70d50c12b6a6df6f98ed40ad09645/raw/5caef3339ceab1fc997ccb6b9e337bc8828ef12f/failed-reply && git apply failed-reply then send a reply; alternatively, sign out after the previous step to confirm that the reply transitions to "failed" state) (but reported new issue #1457)
- a reply containing HTML is displayed as unformatted text
- ❌ a reply with a single string of characters longer than 100 chars is displayed correctly (see Tracking upstream issue QTBUG-85498 securedrop-client#815 (comment))
- a reply with a line longer than 100 chars is displayed correctly
- two replies added immediately after each other are ordered correctly

eloquence · 2022-04-04T22:13:22Z

OS environment: Qubes 4.0
Workstation environment: staging
Server environment: 2.3.0 prod on hardware
Client version: securedrop-client_0.6.0-dev-20220404-060708

Submissions

Preview

when Download is clicked on a submission:
- the submission is downloaded and decrypted
- the Download button is replaced with Print and Export options
- the submission filename is displayed.
For a DOC submission:
- when the submission filename is clicked, a disposable VM (dispVM) is started.
- after the dispVM starts, the submission is displayed in LibreOffice (with warning, see Opening DOC, DOCX or ODT in LibreOffice triggers "large window" notification #588)
- when LibreOffice is closed, the dispVM shuts down
For a PDF submission:
- when the submission filename is clicked, a dispVM is started.
- after the dispVM starts, the submission is displayed in evince
- when evince is closed, the dispVM shuts down
For a JPEG submission:
- when the submission filename is clicked, a dispVM is started.
- after the dispVM starts, the submission is displayed in Image Viewer
- when Image Viewer is closed, the dispVM shuts down

Export

When Export is first clicked on a submission:
- the "Preparing to export..." message is displayed
- the sd-devices VM is started
- the user is prompted to insert an Export USB
- On clicking Cancel, the prompt closes and the file is not exported
When Export is clicked on the submission again:
- the "Preparing to export..." message is displayed
- the user is prompted to insert an Export USB
- When the user inserts an invalid Export USB, attaches it to the sd-devices VM and clicks OK:
  - a message is displayed indicating that the Export USB is invalid and the user is prompted to insert a valid device
When Export is clicked on the submission again:
- the "Preparing to export..." message is displayed
- the user is prompted to insert an Export USB
- When the user inserts a valid Export USB, attaches it to the sd-devices VM, and clicks OK:
  - the user is prompted for the Export USB's password
- When the user enters an invalid Export USB password and clicks Submit:
  - a failure message is displayed and the user is prompted to enter the password again
- When the user enters an valid Export USB password and clicks Submit:
  - the file is saved to the Export USB
When the user detaches the Export USB and mounts it on another VM or computer:
- the decrypted submission is available in on the Export USB, in a directory sd-export-<timestamp>/export_data

Print

When the user clicks Print on a downloaded submission:
- a "Preparing to print..." message is displayed
- the sd-devicesVM is started
- the user is prompted to connect a supported printer
When the user connects a printer, attaches it to the sd-devices VM, and clicks Continue:
- ❓ a "Printing..." message is displayed [not tested, no supported printer]
- ❓ the X Printer Panel dialog is displayed with the printer selected
When the user clicks Print in the X Printer Panel:
- ❓ the submission is printed successfully. [not tested, no supported printer]

Closing the client

When the user clicks the main window close button:
- the client exits.

eloquence · 2022-04-04T23:12:11Z

OS environment: Qubes 4.0
Workstation environment: staging
Server environment: 2.3.0 prod on hardware
Client version: securedrop-client_0.6.0-dev-20220404-060708

Scenario: Offline mode without existing data

Prerequisites:

server is available and contains source test data

test data includes at least one previously downloaded submission

test data includes at least one undownloaded submission

~/.securedrop_client/data in sd-app is empty, and ~/.securedrop_client/svs.sqlite does not exist (do not delete the entire ~/.securedrop_client directory)

the sd-devices VM is not running (shut down manually if necessary)

a supported printer is available, but not attached.

Offline to Online

Scenario: Offline mode with existing data

Prerequisites:

server is available and contains source test data

test data includes at least one previously downloaded submission

test data includes at least one undownloaded submission

client data directory has been synced with server in a previous login

the sd-devices VM is not running (shut down manually if necessary)

a supported printer is available, but not attached.

Offline to Online

gonzalo-bulnes · 2022-04-27T07:40:02Z

Just for coordination: I'm doing a hardware review on my side in order to jump on the testing train. I'll be testing this branch on R4.1 on a NUC11 machine.

sssoleileraaa · 2022-04-27T17:14:10Z

I just pulled the latest from the 600-qubes-4.1 branch, deleted the securedrop-workstation directory in dom0 (just to be safe), ran make clean && make clone && make dev (on a machine that i previously did a qubes 4.0-4.1 upgrade on), and the results were a provision-all failure point to the mgmt-fedora-34.log error: "Failed to return clean data" (error code 126).

I also noticed one Qubes error popup that said Denied: qubes.VMRootShell Denied qubes.VMRootShell from disp-mgmt-fedora-34 to fedora-34, and one Qubes warning popup that said: Failed: qubes.GetDate Failed to execute qubes.GetDate (from disp-mgmt-fedora-34 to dom0).

I think this highlights even more why we want to recommend to folks the backup-wipe-install method.

conorsch · 2022-04-27T17:28:27Z

@creviera Try running sudo qubesctl --show-output state.highstate first. In the run order for the provisioning logic, the Fedora VMs are configured first, but setting up dom0 state, including the RPC policy changes, should help. In a nutshell, I'm suggesting taking a look at the various commands in https://github.com/freedomofpress/securedrop-workstation/blob/main/scripts/provision-all and running them manually in a different order.

sssoleileraaa · 2022-04-27T17:34:03Z

@conorsch - i was just preparing to test the scenario that we're going to recommend to users (which we still need testers to do, although i believe @gonzalo-bulnes is on this test scenario today), but i can stop that and continue down the upgrade-in-place scenario if you think that would be more useful?

conorsch · 2022-04-27T17:35:34Z

@creviera No, that's fine: go ahead and proceed with the scenario you describe.

rocodes · 2022-04-27T23:02:53Z

Here's my messy-but-potentially-relevant test path (I've already decided to wipe and install again so I get a clean run, but just documenting what happened here for posterity):

I installed Qubes 4.1 (opting for a non disposable sys-usb vm), built an RPM from the tip of this branch, ran make dev, and things looked good. Then realized I didn't want dev, so I ran make clean, make clone, make staging, then make test.

I now have several test failures (log below), as well as an apparent misconfiguration: the client starts, but won't log in, and the error log in sd-app shows ERROR: Configuration file does not exist at /etc/sd-proxy.yaml (sd-proxy) ... which....it shouldn't, because that was the pre-template-consolidation location anyway, right? (The file is present in sd-proxy at .securedrop_proxy/sd-proxy.yaml, and the configuration salt state seems to have run successfully in dom0.

test results here

[lovelace@dom0 securedrop-workstation]$ make test
python3 -m unittest discover -v tests
test_gpg_domain_configured (test_app.SD_App_Tests) ... ok
test_logging_configured (test_app.SD_App_Tests) ... ok
test_mailcap_hardened (test_app.SD_App_Tests) ... ok
test_mimeapps (test_app.SD_App_Tests) ... ok
test_mimeapps_functional (test_app.SD_App_Tests) ... ok
test_open_in_dvm_desktop (test_app.SD_App_Tests) ... ok
test_sd_client_apparmor (test_app.SD_App_Tests) ... ok
test_sd_client_config (test_app.SD_App_Tests) ... ok
test_sd_client_dependencies_installed (test_app.SD_App_Tests) ... ok
test_sd_client_package_installed (test_app.SD_App_Tests) ... ok
test_Templates_cleaned_up (test_dom0_config.SD_Qubes_Dom0_Templates_Tests) ... ok
test_vms_to_update_are_tagged (test_dom0_config.SD_Qubes_Dom0_Templates_Tests) ... ok
test_rpm_repo_config (test_dom0_rpm_repo.SD_Dom0_Rpm_Repo_Tests) ... ok
test_rpm_repo_public_key (test_dom0_rpm_repo.SD_Dom0_Rpm_Repo_Tests) ... ok
test_gpg_domain_configured (test_gpg.SD_GPG_Tests) ... ok
test_logging_disabled (test_gpg.SD_GPG_Tests) ... ok
test_sd_gpg_timeout (test_gpg.SD_GPG_Tests) ... ok
test_we_have_the_key (test_gpg.SD_GPG_Tests) ... ok
test_gpg_domain_configured (test_log_vm.SD_Log_Tests) ... ok
test_log_dirs_properly_named (test_log_vm.SD_Log_Tests) ... ok
test_log_utility_installed (test_log_vm.SD_Log_Tests) ... ok
test_logs_are_flowing (test_log_vm.SD_Log_Tests) ... ok
test_redis_service_running (test_log_vm.SD_Log_Tests) ... ok
test_sd_log_has_no_custom_rsyslog (test_log_vm.SD_Log_Tests) ... ok
test_sd_log_package_installed (test_log_vm.SD_Log_Tests) ... ok
test_sd_log_redis_is_installed (test_log_vm.SD_Log_Tests) ... ok
test_sd_log_service_running (test_log_vm.SD_Log_Tests) ... ok
test_do_not_open_here (test_proxy_vm.SD_Proxy_Tests)
The do-not-open here script has been removed from sd-proxy. ... ok
test_gpg_domain_configured (test_proxy_vm.SD_Proxy_Tests) ... ok
test_logging_configured (test_proxy_vm.SD_Proxy_Tests) ... ok
test_mailcap_hardened (test_proxy_vm.SD_Proxy_Tests) ... ok
test_mime_types (test_proxy_vm.SD_Proxy_Tests) ... ok
test_sd_proxy_package_installed (test_proxy_vm.SD_Proxy_Tests) ... ok
test_sd_proxy_rpc_spec (test_proxy_vm.SD_Proxy_Tests) ... FAIL
test_sd_proxy_writable_config_dir (test_proxy_vm.SD_Proxy_Tests) ... ok
test_sd_proxy_yaml_config (test_proxy_vm.SD_Proxy_Tests) ... ok
test_whonix_ws_repo_absent (test_proxy_vm.SD_Proxy_Tests)
The sd-proxy VM was previously based on Whonix Workstation, ... ok
test_Policies (test_qubes_rpc.SD_Qubes_Rpc_Tests) ... ok
test_current_fedora_for_sys_vms (test_qubes_vms.SD_Qubes_VM_Tests)
Checks that all sys-* VMs are configured to use ... FAIL
test_current_whonix_vms (test_qubes_vms.SD_Qubes_VM_Tests)
Checks that the Qubes-maintained Whonix tooling ... ok
test_files_are_properly_copied (test_sd_devices.SD_Devices_Tests) ... ok
test_gpg_domain_configured (test_sd_devices.SD_Devices_Tests) ... ok
test_logging_configured (test_sd_devices.SD_Devices_Tests) ... ok
test_mailcap_hardened (test_sd_devices.SD_Devices_Tests) ... ok
test_mime_types (test_sd_devices.SD_Devices_Tests) ... ok
test_open_in_dvm_desktop (test_sd_devices.SD_Devices_Tests) ... ok
test_sd_export_package_installed (test_sd_devices.SD_Devices_Tests) ... FAIL
test_accept_sd_xfer_extracted_file (test_sd_whonix.SD_Whonix_Tests) ... ok
test_gpg_domain_configured (test_sd_whonix.SD_Whonix_Tests) ... ok
test_sd_whonix_repo_enabled (test_sd_whonix.SD_Whonix_Tests)
During Whonix 14 -> 15 migration, we removed the apt list file ... ok
test_sd_whonix_verify_tor_config (test_sd_whonix.SD_Whonix_Tests) ... ok
test_v3_auth_private_file (test_sd_whonix.SD_Whonix_Tests) ... ok
test_whonix_torrc (test_sd_whonix.SD_Whonix_Tests)
Ensure Whonix-maintained torrc files don't contain duplicate entries. ... ok
test_files_are_properly_copied (test_sys_usb.SD_SysUSB_Tests) ... ok
test_gpg_domain_configured (test_viewer.SD_Viewer_Tests) ... ok
test_logging_configured (test_viewer.SD_Viewer_Tests) ... ok
test_mailcap_hardened (test_viewer.SD_Viewer_Tests) ... ok
test_mime_types (test_viewer.SD_Viewer_Tests) ... FAIL
test_redis_packages_not_installed (test_viewer.SD_Viewer_Tests)
Only the log collector, i.e. sd-log, needs redis, so redis will be ... ok
test_sd_viewer_evince_installed (test_viewer.SD_Viewer_Tests) ... ok
test_sd_viewer_libreoffice_installed (test_viewer.SD_Viewer_Tests) ... ok
test_sd_viewer_metapackage_installed (test_viewer.SD_Viewer_Tests) ... ok
test_expected (test_vms_exist.SD_VM_Tests) ... ok
test_sd_app_config (test_vms_exist.SD_VM_Tests) ... ok
test_sd_gpg_config (test_vms_exist.SD_VM_Tests) ... ok
test_sd_log_config (test_vms_exist.SD_VM_Tests) ... ok
test_sd_proxy_config (test_vms_exist.SD_VM_Tests) ... ok
test_sd_proxy_template (test_vms_exist.SD_VM_Tests) ... ok
test_sd_viewer_config (test_vms_exist.SD_VM_Tests) ... ok
test_sd_whonix_config (test_vms_exist.SD_VM_Tests) ... ok
test_sd_workstation_template (test_vms_exist.SD_VM_Tests) ... ok
test_all_fedora_vms_uptodate (test_vms_platform.SD_VM_Platform_Tests)
Asserts that all Fedora-based templates, such as sys-net, have all ... ok
test_all_jessie_backports_disabled (test_vms_platform.SD_VM_Platform_Tests)
Asserts that all VMs lack references to Jessie in apt config. ... ok
test_all_sd_vm_apt_sources (test_vms_platform.SD_VM_Platform_Tests)
Test all VMs fpf apt source list iteratively. ... ok
test_all_sd_vm_platforms (test_vms_platform.SD_VM_Platform_Tests)
Test all VM platforms iteratively. ... ok
test_all_sd_vms_uptodate (test_vms_platform.SD_VM_Platform_Tests)
Asserts that all VMs have all available apt packages at the latest ... ok
test_debian_keyring_config (test_vms_platform.SD_VM_Platform_Tests)
Ensure the securedrop keyring package is properly installed and the ... ok
test_dispvm_default_platform (test_vms_platform.SD_VM_Platform_Tests)
Query dom0 Qubes preferences and confirm that new DispVMs ... ok
test_sd_proxy_template (test_vms_platform.SD_VM_Platform_Tests)
Asserts that the 'sd-proxy' VM is using a supported base OS. ... ok

======================================================================
FAIL: test_sd_proxy_rpc_spec (test_proxy_vm.SD_Proxy_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/lovelace/securedrop-workstation/tests/test_proxy_vm.py", line 55, in test_sd_proxy_rpc_spec
    self.assertFileHasLine("/etc/qubes-rpc/securedrop.Proxy", line)
  File "/home/lovelace/securedrop-workstation/tests/base.py", line 126, in assertFileHasLine
    raise AssertionError(msg)
AssertionError: File /etc/qubes-rpc/securedrop.Proxy does not contain expected line /usr/bin/sd-proxy /home/user/.securedrop_proxy/sd-proxy.yaml

======================================================================
FAIL: test_current_fedora_for_sys_vms (test_qubes_vms.SD_Qubes_VM_Tests)
Checks that all sys-* VMs are configured to use
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/lovelace/securedrop-workstation/tests/test_qubes_vms.py", line 32, in test_current_fedora_for_sys_vms
    self.assertEqual(
AssertionError: 'fedora-34' != 'fedora-34-dvm'
- fedora-34
+ fedora-34-dvm
?          ++++
 : Unexpected template for sys-firewall

======================================================================
FAIL: test_sd_export_package_installed (test_sd_devices.SD_Devices_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/lovelace/securedrop-workstation/tests/test_sd_devices.py", line 21, in test_sd_export_package_installed
    self.assertTrue(self._package_is_installed("gnome-disk-utility"))
AssertionError: False is not true

======================================================================
FAIL: test_mime_types (test_viewer.SD_Viewer_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/lovelace/securedrop-workstation/tests/test_viewer.py", line 45, in test_mime_types
    self.assertEqual(actual_app, expected_app)
AssertionError: 'open-in-dvm.desktop' != 'audacious.desktop'
- open-in-dvm.desktop
+ audacious.desktop


----------------------------------------------------------------------
Ran 79 tests in 369.149s

FAILED (failures=4)
make: *** [Makefile:124: test] Error 1

My current plan is to reinstall qubes and sdw again from scratch and go directly to a staging setup. I will stick to the non-disposable sys-USB qube. (we chatted and @gonzalo-bulnes is planning to test the disposable sys-usb qube). So I will also hopefully have some clean findings to document.

marmarek · 2022-04-27T23:25:00Z

The dollar sign was deprecated a while ago due to security concerns. Is it still supported on Qubes 4.1? Regardless of any other factors, it seems to me that we should replace it with @ in that invocation.

New policy format (the one in /etc/qubes/policy.d) no longer supports $, but both old policy (/etc/qubes-rpc/policy) and arguments to qrexec-client-vm should still automatically translate $ to @.

sssoleileraaa · 2022-04-28T00:15:37Z

Test results so far...

Hardware: T490

Clean install scenario:

Downloaded Qubes 4.1 from https://www.qubes-os.org/downloads/
Followed Qubes 4.1 installation guide in https://www.qubes-os.org/doc/installation-guide/

Followed the test plan's instructions to test with all Qubes 4.1 installation defaults
Did not have a sys-usb qube so I created one by running sudo qubesctl state.sls qvm.sys-usb
As per usual, I had to run through https://workstation.securedrop.org/en/stable/admin/hardware.html?highlight=pci#lenovo-thinkpad-t490-with-8th-generation-intel-core-processor to fix the network device configuration issue that prevents sys-net from starting
Saw the expected fedora-34 update, so I ran the Qubes Updater to apply the update

Cloned SDW from this PR branch and installed it
🧇 Saw an sd-log error popup described in sd-log tries to log to itself, but loopback qrexec connections aren't supported #755 (with no errors in the sd-log summary output)
🧇 Saw an sd-whonix warning popup (no errors) that said Failed: qubes.GetDate: Failed to execute qubes.GetDate (from disp-mgmt-sd-whonix to dom0)
Ran make test
❌ 5 failures: test_proxy_vm.SD_Proxy_Tests, test_qubes_rpm.SD_Qubes_Rpc_Tests, test_sd-devices.SD_Devices_Tests, test_sys_usb.SD_SysUSB_Tests, test_viewer.SD_Viewer_Tests (see
make-test-results.txt)
Ran updater
🧇 Saw an sd-log error popup described in sd-log tries to log to itself, but loopback qrexec connections aren't supported #755
Performed full suite of Client QA checks
❌ Could not log into the client, logs say: ERROR: 'Configuration file does not exist at /etc/sd-proxy.yaml' indicating that there was an error during the SDW installation

sssoleileraaa · 2022-04-28T00:41:26Z

I'll be able to debug tomorrow for a bit, but otherwise, might have to pick this up next week. To not block, I think we can rely on @gonzalo-bulnes to continue testing the freshest of fresh install scenario, with help from @rocodes and @eloquence if needed.

rocodes · 2022-04-28T14:21:15Z

Looks like Allie and I ran into the same issue with the location of the sd-proxy yaml file. Wondering if that's something anyone else has seen in the clean install scenario.

eaon · 2022-04-28T14:26:32Z

Yes, that happens for me too - I forgot about running the tests (oof, sorry) until you posted your results - seeing the same (plus some #764 failures (which I'm about to push fixes for)

In 4.1 mouse-action is configurable, which has an effect on the contents of the policy. See: https://github.com/QubesOS/qubes-mgmt-salt-dom0-virtual-machines/blob/78dcb14ef9b41650ec18ec0a40f44005ca1cafcc/pillar/qvm/sys-usb-allow-mouse.sls and https://github.com/QubesOS/qubes-mgmt-salt-dom0-virtual-machines/blob/78dcb14ef9b41650ec18ec0a40f44005ca1cafcc/qvm/sys-usb.sls#L94

eaon · 2022-04-28T19:49:04Z

After a short convo with @rocodes I confirmed a suspicion that prod env installs will end up referencing the correct location for sd-proxy.yaml in /etc/qubes-rpc/securedrop.Proxy (in sd-small-buster-template), but dev installs will reference the outdated /etc/sd-proxy.yaml - I'm not certain why that happens or if it happens on 4.0 as well.

Up until the other day I was using prod exclusively but recently switched to dev, which is why I could test the client before.

eaon · 2022-04-29T19:26:06Z

Realised I forgot to post these before, augmented with a new one using the dev environment after the packages in apt-test were updated to fix the sd-proxy.yaml location that was reported by @rocodes and @creviera (among other problems that would've surfaced I'm sure)

Hardware: X1 Carbon Gen 6
OS environment: Qubes 4.1 (fresh)
Workstation environment: prod
Server environment: 2.3.1 prod in VM

Scenario: Online mode

Login

when SecureDrop desktop icon is double-clicked, preflight updater is displayed
After preflight updater runs, when user clicks Continue, login dialog is displayed
In login dialog:
- show/hide password functionality works
- incorrect password cannot log in
- 2FA token reuse cannot log in after password failure
- invalid 2FA token cannot log in
- valid credentials and 2FA can log in

Sources

after valid login:
- the login dialog closes
- source data is downloaded and source list is populated
- user is prompted for GPG key access
- submissions and replies are decrypted
- the source list is displayed but no sources are selected by default
- the conversation view is not populated
when a source is selected in source list:
- conversation view is populated with source conversation
- a source message containing HTML is displayed as unformatted text
- source submissions have an active Download button
- source submission compressed file size is displayed accurately
when the upper right 3-dot button is clicked:
- a menu is displayed with a delete source account option
- when delete source account is selected:
  - the source is deleted from the source list and the conversation view is blanked
  - the source is deleted from the server and not restored on next sync
  - source submissions and messages are removed from the client's data directory
when a source is starred in source list, and the client is closed and reopened in Online mode:
- the source is still starred in the source list

Replies

when a source is selected in the source list:
- the reply panel is available for use and there is no message asking the user to sign in
- a reply can be added to the conversation
- a pending reply can be added to the conversation (for development environments, you can use: wget https://gist.githubusercontent.com/creviera/7f19a7d10334359f40dbdbb2354cd13a/raw/a2ef94913a155aa4019b753cf916f844c9cffa3a/pending-reply && git apply pending-reply then send a reply; alternatively, disconnect the network or sd-whonix after sending a reply)
- a failed reply can be added to the conversation (for development environments, you can use: wget https://gist.githubusercontent.com/creviera/5ba70d50c12b6a6df6f98ed40ad09645/raw/5caef3339ceab1fc997ccb6b9e337bc8828ef12f/failed-reply && git apply failed-reply then send a reply; alternatively, sign out after the previous step to confirm that the reply transitions to "failed" state)
- a reply containing HTML is displayed as unformatted text
- a reply with a single string of characters longer than 100 chars is displayed, but truncated (Tracking upstream issue QTBUG-85498 securedrop-client#815).
- a reply with a line longer than 100 chars is displayed correctly
- two replies added immediately after each other are ordered correctly

Submissions

Preview

when Download is clicked on a submission:
- the submission is downloaded and decrypted
- the Download button is replaced with Print and Export options
- the submission filename is displayed.
For a DOC submission:
- when the submission filename is clicked, a disposable VM (dispVM) is started.
- after the dispVM starts, the submission is displayed in LibreOffice
- when LibreOffice is closed, the dispVM shuts down
For a PDF submission:
- when the submission filename is clicked, a dispVM is started.
- after the dispVM starts, the submission is displayed in evince
- when evince is closed, the dispVM shuts down
For a JPEG submission:
- when the submission filename is clicked, a dispVM is started.
- after the dispVM starts, the submission is displayed in Image Viewer
- when Image Viewer is closed, the dispVM shuts down

Export

When Export is first clicked on a submission:
- the "Preparing to export..." message is displayed
- the sd-devices VM is started
- the user is prompted to insert an Export USB
- On clicking Cancel, the prompt closes and the file is not exported
When Export is clicked on the submission again:
- the "Preparing to export..." message is displayed
- the user is prompted to insert an Export USB (:warning: at the time the sys-usb did not automatically attach the USB drive, I attached it manually)
- When the user inserts an invalid Export USB, attaches it to the sd-devices VM and clicks OK:
  - a message is displayed indicating that the Export USB is invalid and
    the user is prompted to insert a valid device
When Export is clicked on the submission again:
- the "Preparing to export..." message is displayed
- the user is prompted to insert an Export USB
- When the user inserts a valid Export USB, manually attaches it to the sd-devices VM using the sys-usb menu in the menu bar, and clicks OK:
  - the user is prompted for the Export USB's password
- When the user enters an invalid Export USB password and clicks Submit:
  - a failure message is displayed and the user is prompted to enter the password again
- When the user enters an valid Export USB password and clicks Submit:
  - the file is saved to the Export USB
When the user detaches the Export USB and mounts it on another VM or computer:
- the decrypted submission is available in on the Export USB, in a directory sd-export-<timestamp>/export_data

Print

Skipped for lack of access to a printer

Closing the client

When the user clicks the main window close button:
- the client exits.

Scenario: Offline mode without existing data

Offline to Online

Workstation environment: dev

Scenario: Offline mode with existing data

Offline to Online

rocodes · 2022-05-02T17:13:16Z

@eaon So it looks like right now USB autoattach is not working in 4.1? Do you mind filing an issue with 4.1 label in https://github.com/freedomofpress/securedrop-export so we don't lose track of this (or I can)?

eaon · 2022-05-02T17:20:36Z

@rocodes I already have a draft-PR to fix this as it's just related to sys-usb being a DispVM by default: #764 :)

sssoleileraaa · 2022-05-02T20:12:46Z

Scenario `prod`

Hardware: T490
Clean install + SDW prod scenario:

Installed Qubes 4.1 (kept all default configuration)

As per usual, I had to run through https://workstation.securedrop.org/en/stable/admin/hardware.html?highlight=pci#lenovo-thinkpad-t490-with-8th-generation-intel-core-processor to fix the network device configuration issue that prevents sys-net from starting
Did not have a sys-usb qube so I created one by running sudo qubesctl state.sls qvm.sys-usb
Saw the expected fedora-34 update, so I ran the Qubes Updater to apply the update

Built RPM from this PR branch and installed it in dom0 (copied config.json manually)

❌ Provisioning all the VMs from dom0 completes without error

sd-gpg failed to start during the step to write the updater status flag to sd-app apparently (see updater-logs.txt)

sssoleileraaa · 2022-05-02T21:28:25Z

rebooting got me passed this error, but I still see that the sd-proxy.yaml file is missing in a prod workstation scenario.... let's sync up @eaon so we can debug this

gonzalo-bulnes · 2022-05-03T02:45:35Z

Fresh R4.1 installation on a NUC11:

sys-usb was missing, but after creating it with sudo qubesctl state.sls qvm.sys-usb (:point_up:) ./scripts/provision_all succeeds.

Since @creviera was also missing sys-usb at some point, I'll open an issue to suggest we make sure to assert that the VM exists before attempting to use it.

sssoleileraaa · 2022-05-03T03:16:07Z

Here's another test I did (looks like things work better for me with a dev env so I'll repeat my prod test tomorrow):

Scenario: `dev`

run make clean && make dev without error
run the updater without error
log into the client

❌ run make test without error (almost, one failed test):

======================================================================
FAIL: test_files_are_properly_copied (test_sys_usb.SD_SysUSB_Tests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/user/securedrop-workstation/tests/test_sys_usb.py", line 12, in test_files_are_properly_copied
    self.assertTrue(self._fileExists("/etc/udev/rules.d/99-sd-devices.rules"))
AssertionError: False is not true

----------------------------------------------------------------------
Ran 79 tests in 283.189s

FAILED (failures=1)
make: *** [Makefile:124: test] Error 1

sssoleileraaa · 2022-05-03T20:38:43Z

Scenario: `prod` on clean install

run scripts/provision-all without error
run the updater without error
log into the client
❌ run make test without error
- mostly... one failed test (same test_sys_usb.SD_SysUSB_Tests failure as above)

At this point, I think this test failure is expected, but I'll need confirmation from @eaon (since our custom udev rules do not persist, we'll have to wait until #764 to fix this missing file error, right?)

eaon · 2022-05-03T20:50:52Z

I can confirm, yes! I should probably update the original PR description

sssoleileraaa

Based on #751 (comment) and #751 (comment), I believe we can merge this and refocus our testing efforts on #764 and fixing #766

eaon commented Jan 13, 2022

View reviewed changes

dom0/sd-sys-vms.sls Outdated Show resolved Hide resolved

rocodes reviewed Jan 20, 2022

View reviewed changes

dom0/sd-dom0-qvm-rpc.sls Outdated Show resolved Hide resolved

rocodes reviewed Jan 20, 2022

View reviewed changes

eaon mentioned this pull request Jan 21, 2022

sd-log tries to log to itself, but loopback qrexec connections aren't supported #755

Closed

conorsch force-pushed the 600-qubes-4.1 branch from 5f46546 to 4ba57fa Compare March 30, 2022 19:37

conorsch marked this pull request as ready for review March 30, 2022 19:39

conorsch changed the title ~~First shot at getting SDW to install on R4.1~~ Initial support for Qubes 4.1 Mar 30, 2022

eloquence assigned eloquence and rocodes Mar 30, 2022

eloquence reviewed Mar 31, 2022

View reviewed changes

eloquence mentioned this pull request Mar 31, 2022

Add functional RPC policy tests #759

Closed

This was referenced Apr 1, 2022

"Failed" replies may actually succeed after client exits freedomofpress/securedrop-client#1457

Closed

Tracking upstream issue QTBUG-85498 freedomofpress/securedrop-client#815

Open

eloquence mentioned this pull request Apr 4, 2022

Dialog displays a system error (possibly export-related) freedomofpress/securedrop-client#1368

Open

eloquence mentioned this pull request Apr 27, 2022

Cannot open files in disposable VM using SecureDrop Client #766

Closed

eaon mentioned this pull request Apr 29, 2022

securedrop-proxy package in apt-test repo is fairly outdated freedomofpress/securedrop-builder#311

Closed

gonzalo-bulnes mentioned this pull request May 3, 2022

Qubes 4.1 installer does not create sys-usb #769

Open

sssoleileraaa approved these changes May 3, 2022

View reviewed changes

sssoleileraaa merged commit d18d0c2 into main May 3, 2022

sssoleileraaa deleted the 600-qubes-4.1 branch May 3, 2022 20:59

eloquence mentioned this pull request Jun 29, 2022

Provision-all error when Fedora-35 template already installed #797

Closed

		@@ -60,12 +79,13 @@ dom0-remove-securedrop-workstation-stretch-template:
		- file: dom0-workstation-rpm-repo

		dom0-install-securedrop-workstation-template:

Initial support for Qubes 4.1 #751

Initial support for Qubes 4.1 #751

Conversation

eaon commented Jan 13, 2022 • edited Loading

Status

Description of changes

Known problems

Test plan

conorsch commented Jan 14, 2022

eaon commented Jan 18, 2022

conorsch commented Jan 18, 2022

rocodes Jan 20, 2022

Choose a reason for hiding this comment

rocodes Jan 20, 2022

Choose a reason for hiding this comment

rocodes commented Jan 20, 2022

rocodes Jan 20, 2022

Choose a reason for hiding this comment

eaon Jan 20, 2022

Choose a reason for hiding this comment

rocodes Jan 20, 2022

Choose a reason for hiding this comment

rocodes Jan 20, 2022

Choose a reason for hiding this comment

conorsch commented Feb 4, 2022

conorsch commented Mar 16, 2022

conorsch commented Mar 30, 2022

eloquence Mar 31, 2022 • edited Loading

Choose a reason for hiding this comment

eloquence commented Mar 31, 2022

eloquence commented Mar 31, 2022

SecureDrop Workstation test scenarios

Qubes scenarios

Verify tor connection to Journalist API

Verify default DispVM settings

RPC Policies

Logging

GUI updater

eloquence commented Apr 1, 2022

Client scenarios

Scenario: Online mode

Login

Sources

Replies

eloquence commented Apr 4, 2022

Submissions

Preview

Export

Print

Closing the client

eloquence commented Apr 4, 2022

Scenario: Offline mode without existing data

Offline to Online

Scenario: Offline mode with existing data

Offline to Online

gonzalo-bulnes commented Apr 27, 2022 • edited Loading

sssoleileraaa commented Apr 27, 2022

conorsch commented Apr 27, 2022

sssoleileraaa commented Apr 27, 2022

conorsch commented Apr 27, 2022

rocodes commented Apr 27, 2022 • edited Loading

marmarek commented Apr 27, 2022

sssoleileraaa commented Apr 28, 2022 • edited Loading

Test results so far...

sssoleileraaa commented Apr 28, 2022

rocodes commented Apr 28, 2022

eaon commented Apr 28, 2022

eaon commented Apr 28, 2022

eaon commented Apr 29, 2022

Scenario: Online mode

Login

Sources

Replies

Submissions

Preview

Export

Print

Closing the client

Scenario: Offline mode without existing data

Offline to Online

eaon commented Jan 13, 2022 •

edited

Loading

eloquence Mar 31, 2022 •

edited

Loading

gonzalo-bulnes commented Apr 27, 2022 •

edited

Loading

rocodes commented Apr 27, 2022 •

edited

Loading

sssoleileraaa commented Apr 28, 2022 •

edited

Loading

rocodes commented May 2, 2022 •

edited

Loading

sssoleileraaa commented May 2, 2022 •

edited

Loading

Scenario `prod`

sssoleileraaa commented May 3, 2022 •

edited

Loading

Scenario: `dev`

sssoleileraaa commented May 3, 2022 •

edited

Loading

Scenario: `prod` on clean install