2024-11-04 (Chart Release 5.7.0)

Bug fixes and other updates

• galley: Use bulk query when getting all feature configs for a team user (#4325)

Internal changes

• Block access to assets.*/minio/ path for public access. (#4297)
• galley: Delete unused endpoint for getting feature status for multiple teams (#4326)
• Fix shellcheck problems in all shell scripts (#4220)

2024-10-30 (Chart Release 5.6.0)

Release notes

• To remove phone keys from brig's user_keys table an ad hoc data-migration can be run. See PR #4146 which contains the implementation. (#4130)

• Because the phone column is deleted from Brig's user table in a schema
  migration, temporarily there might be 5xx errors during deployment if Wire
  server 5.4.0 was not deployed previously. To avoid these errors, please deploy
  the Wire server 5.4.0 release first. (#4130)

• With this release it will be possible to invite personal users to teams. In brig's config, emailSMS.team.tExistingUserInvitationUrl is required to be set to a value that points to the correct teams/account page.
  If emailSMS.team is not defined at all in the current environment, the value of externalUrls.teamSettings (or, if not present, externalUrls.nginz) will be used to construct the correct url, and no configuration change is necessary. (#4229)

• charts/wire-server: There is a new config value called background-worker.config.enableFederation which defaults to false. This must be kept in sync with tags.federation. (#4243)

• If you are mapping an email address to the externalId field in the
  scim schema, please check the following list for items that apply to
  you and recommended steps before/during/after upgrade.

  • Situation: the emails field of in your scim user records is
    empty.

    What you need to do: change your schema mapping to contain the
    same address in externalId and (as a record with one element) in
    emails.

  • Situation: the emails field of your scim user records is
    non-empty.

    What you need to do: make sure emails contains exactly one
    entry, which is the email from externalId. If there is a
    discrepancy, the address from emails will become the new
    (unvalidated) address of the user, and the user will receive an
    email to validate it. If the email cannot be sent or is ignored
    by the recipient, the valid address will not be changed. (#4221)

• A schema migration drops column 'phone' from Brig's 'team_invitation' table. Previous releases were still reading this column. As there is no Team Settings UI action to enter a phone number, this reading will not miss to read actual phone numbers. Therefore, during deployment this will lead to benign 5xx errors. (#4149)

• Password hashing can now be done using argon2id instead of scrypt. The argon2id parameters can be configured using these options:

  brig:
    optSettings:
      setPasswordHashingOptions:
        algorithm: argon2id
        iterations: ...
        memory: ... # memory needed in KiB
        parallelism: ...
  galley:
    settings:
      passwordHashingOptions:
        algorithm: argon2id
        iterations: ...
        memory: ... # memory needed in KiB
        parallelism: ...

  The default option is still to use scrypt as moving to argon2id might require
  allocating more resources according to configured parameters.

  When configured to use argon2id, the DB will be migrated slowly over time as the
  users enter their passwords (either to login or to do other operations which
  require explicit password entry). This migration is NOT done in reverse,
  i.e., if a deployment started with argon2id as the algorithm then chose to move
  to scrypt, the passwords will not get rehashed automatically, instead the users
  will have to reset their passwords if that is desired.

  NOTE It is highly recommended to move to argon2id as it will be made the
  only available choice for the algorithm config option in future.

  (#4291, #4291)

• Config value gundeck.config.bulkPush has been removed. This is purely an
  internal change, in case the value was overriden to false, operators might see
  more spiky usage of CPU and memory from gundeck due to bulk processing. (#4290)

API changes

• A new endpoint POST /teams/invitations/accept allows a non-team user to accept an invitation to join a team (#4229)

• Services allowlist are blocked by 409 (mls-services-not-allowed) for teams with default protocol MLS. (#4266)

• The POST /clients and PUT /clients/:cid endpoints support a new capability "consume-notifications" (#4259)

• New variant in API version 7 of endpoints for creating and listing SCIM tokens that support a name field. New endpoint in version 7 for updating a SCIM token name. (#4307)

• All the phone number-based functionality is removed from the client API v6 (#4149)

• The team CSV export endpoint has gained two extra columns: last_active and status. The streaming behaviour has also been improved. (#4293)

• The changes to the capabilities field of the Client structure, introduced in v6, have now been postponed to v7 (#4179)

• Finalise version 6 and introduce new development version 7 (#4179, #4179)

• From API version 7 the GET /mls/public-key and GET /conversations/one2one/:domain/:uid endpoints now take a format query parameter which can be either raw (default, for raw base64-encoded keys) or jwk (for JWK keys) (#4216, #4224)

• GET /conversations/one2one/:domain/:uid now returns public_keys along with the conversation containing all MLS public keys for the backend which will host this conversation (since v6). (#4224)

• Remove the ability to set the TTL of a feature flag. Existing TTLs are still retrieved and returned as before. Note that this only applies to the conferenceCalling feature, as none of the others supported TTL anyway. (#4164)

• Add useSFTForOneToOneCalls as a config option for the Conference Calling feature flag and make its lock status explicit. (#4164)

• Add endpoint to upgrade a personal user to a team owner (#4251)

Features

• DB migration for dropping phone column from user table (#4130)

• A text status field was added to user and user profile (#4155)

• Allow an existing non-team user to migrate to a team (#4229, #4268, #4315)

• Makes it impossible for a user to join an MLS conversation while already under legalhold (at least pending)

  This implies two things:

  1. If a user is under legalhold they cannot ever join an MLS conversation, not even an MLS self conversation.
  2. A user has to reject to be put under legalhold when they want to join an MLS conversation (ignoring the request to be put under legalhold is not enough). (#4242)

• Email template for inviting a personal user to a team added (#4310)

• Clients can declare to be supporting a capability for consuming notifications (#4259)

• New endpoint to revoke an OAuth session (#4213)

• Adds a field which contains a list of all active sessions to each OAuth application in the response of GET /oauth/applications (#4211)

• SCIM's emails field is now handled and the external ID is not restricted to being an email anymore (#4221)

• Added human readable names for SCIM tokens (#4307)

• allow subconversations for MLS 1-1 conversations (#4133)

• Allow choosing hashing algorithm and configuring argon2id parameters (#4291, #4291)

• Deny requests for a legalhold device for users who are part of any MLS conversations (#4245)

• Allow setting of Kubernetes annotations for the coturn Service. (#4189)

• Add initialConfig setting for the mls feature flag (#4262)

• Add federationProtocols setting to galley, which can be used to disable the creation of federated conversations with a given protocol (#4278)

• added open telemetry instrumentation for brig, galley, gundeck and cannon (#3901)

• Send confirmation email after adding a personal user to a new team (#4253)

• The SFT and turn usernames returned by /calls/config/v2 are now deterministically computed from the user ID (#4156)

• Use latest stable RabbitMQ version (3.13.7) and Helm chart (14.6.9). Please
  note that this minor RabbitMQ version upgrade (3.11.x to 3.13.x) may need
  special treatment regarding existing RabbitMQ instances. See
  https://www.rabbitmq.com/docs/upgrade#rabbitmq-version-upgradability . The major
  Helm chart version upgrade may (depending on your setup/values) need attention
  as well: https://github.com/bitnami/charts/tree/main/bitnami/rabbitmq#upgrading (#4227)

Bug fixes and other updates

• Fixed API version check. It has now precedence over other checks like e.g. method check. (#4152)

• Fix handling of defaults of mlsE2EID feature config (#4233)

• Match cipher suite tag in query parameters against key packages on replacing key packages (#4158)

• Users with SAML-SSO are allowed to delete their email address on the rest api. If they do that, the search indices are not updated correctly, and finding the user by the removed email address is still possible. (#4260)

• Re-add accidentally removed add-bot@v6 route in nginz, fixes #4302 (#4318)

• Exclude exception message from error response (#4153)

• Return HTTP 400 instead of 500 when property key is not printable ASCII (#4148)

• move cipher suite updates into the commit lock (#4151)

• Fix feature flag default calculation for mlsMigration and enforceFileDownloadLocation (#4265)

• Allow setting existing properties even if we have max properties (#4148)

• removed spam from nginx (nginz) by using the new style http/2 directive (#3901)

• brig: Make GET /services/tags work again (#4250)

• Process bounce and complaint notifications from SES correctly. (#4301)

Documentation

• Call graph of federated endpoints was removed from the docs (#4299)

• Restored LegalHold internal API swagger as part of Brig. (#4191)

• Fix: show openapi docs for blocked versions (#4309)

• Move docs from docs.wire.com to generated helper page served by brig (#4311)

• Deleted proteus-specific test documentation tags and added some new ...

2024-07-09 (Chart Release 5.5.0)

Bug fixes and other updates

• Fix names of metrics so they do not contain any dots (#4134)

2024-07-08 (Chart Release 5.4.0)

• Phone registration and login is not supported anymore. All API endpoints dealing with phone numbers and phone activation codes now fail with a 400 error. Brig options related to phone number support have now been deleted, namely:
  • setTwilio
  • setNexmo
  • setAllowlistPhonePrefixes. (#4045)

API changes

• Internal API endpoints related to phone numbers have been removed.

  In brig:

  • iGetPhonePrefix
  • iDeletePhonePrefix
  • iPostPhonePrefix.

  In stern:

  • get-users-by-phone
  • put-phone. (#4045)

Features

• charts/coturn: support putting coturn into 'drain' mode when terminating pods, denying new incoming client connections. This speeds up graceful coturn restarts significantly. (#4098)

• Set SFT usernames's shared field according to team settings (#4117)

• Updated the mlsE2EId feature config with two additional fields crlProxy and useProxyOnMobile (#4051)

• reject MLS messages for future epochs (#4110)

• Introduce more configuration options to the coturn helm chart (#4083)

• Update email templates to v1.0.121. (#4064)

• Support connecting to RabbitMQ over TLS. See "Configure RabbitMQ" section in the documentation for details. (#4094)

• Support connecting to Redis over TLS

  It can be enabled by setting these options on the wire-server helm chart:

  gundeck:
    config:
      redis:
        enableTls: true
  
        # When custom CAs are required, one of these must be set:
        tlsCa: <PEM encoded CA certificates>
        tlsCaSecretRef:
          name: <Name of the secret>
          key: <Key in the secret containing pem encoded CA Cert>
  
        # When TLS needs to be used without verification:
        insecureSkipVerifyTls: true

  (#4016)

Bug fixes and other updates

• fixed stern endpoint /i/users/meta-info (#4101)

• Log password reset errors instead of propagating them (#4114)

• Log request ids in brig. (#4086)

• Do not set update origin "scim" in public brig api. (#4072)

• Disabling legalhold before user's approval doesn't result in an error (#4104)

• Make scim-delete-user idempotent. Hide information about existing users (make delete idempotent) (#4120)

• Expose /providers/assets via nginz (#4082)

• federator: Expect a client certificate to be the certificate chain

  Without this openssl doesn't forward to whole chain causing mTLS to not succeed. (#4089)

• Only resend proposals once after external commit (#4103)

• gundeck: Better tolerance for redis-cluster restarts (#4084)

• GHC does not support repeated --with-rtsopts options, and it simply applies the last one. This means many of the baked-in options were actually not being passed, including -N for some of the services and -T for cannon. (#4118)

• Ensure that a Request ID is logged whenever unexpected errors are caught in any service (#4059)

• charts/coturn: use allowed dir to write PID file (#4098)

• Make pending LH requests (with no LH devices listening yet) not throw LH policy errors. This helps eg. in cases where a LH request is issued to the wrong user by accident, and the user can clear up the mistake. (#4056)

Documentation

• Adjust documentation for migrated helm charts (#4058)

Internal changes

• Adapt EJPD data to current requirements. (#3945)

• Port team feature tests to the integration package (#4063)

• Ported flaky legalhold test to the new integration test suite (#4057)

• Added profile update operations to the user subsystem. (#4046)

• Introduce authentication subsystem with password reset. (#4086)

• update nixpkgs and hence GHC version as well as some other tooling. (#4071)

• nginz: Added allowlisted_fqdn_origins to nginx_conf value (#4087)

• Add weeder for dead code elimination. (#4088)

• Introduce email subsystem (#4111)

• replace cabal.project.local template and update cabal.project (#4119)

• Add HTTP proxy in the local setup for elasticsearch in federation-v0. This makes it possible to use a single elasticsearch instance for both the main backends and federation-v0. (#4062)

• federator: Add metrics for garbage collections and unexpected errors that were caught (#4085)

• federator: Simplify polysemy setup to make it similar to other services so the
  interpreter is only used for hoisting the servant application and not explicitly
  inside handler of an endpoint (#4059)

• Added prometheus enable and datacenter size variables for k8ssandra-test-cluster helm chart. (#4011)

• Make Handle type abstract to guarantee it always contains valid Handles. (#4076)

• metrics-core: Delete Data.Metrics in favour of defining metrics closer to where they are being emitted (#4085)

• add more metadata into the meta attribute of all nix derivations produced locally (#4069)

• Do not log anything when warp kills a worker thread. (#4112)

• Introduce VerificationCodSubsystem (#4121)

• add tests for bots that use self-signed certs and add documentation on why we cannot test the bots to work with PKI (#4027)

2024-05-21 (Chart Release 5.3.0)

API changes

• /mls/keys use JWK instead of bare keys as MLS removal keys (#3548)

• The cipher_suite field is not present anymore in objects corresponding to newly created conversations (#4009)

Features

• Upgrade rusty-jwt-tools to support ecdsa_secp256r1_sha256 (#4035)

• gundeck: Delete all APNS_VOIP and APNS_VOIP_SANDBOX push tokens (#4044)

Bug fixes and other updates

• gundeck: Fix parsing errors for SNS ARN for VOIP Tokens (#4040)

• Fix hardcoded ciphersuite when switching to mixed (#4048)

Internal changes

• Add tool to determine number of phone-only users (#4024)

• Log federator request ID on exceptions (#4037)

• Update mls-test-cli to version 0.12 (#4039)

• Remove inbucket helm chart. (#4032)

• Finish servantifying galley and remove wai-routing dependency (#4018)

• New subsystem for user management. (#3977)

• Clean up syntax of test cases that occur in BSI audit. (#4041)

2024-04-29 (Chart Release 5.2.0)

Important: Do not upgrade

If you're upgrading to this version, if there were users using the APNS_VOIP tokens this will cause issues with notifications to those users.

Bug fixes and other updates

• charts/brig: Fix template for settings ES CA certs (#4022)

2024-04-25 (Chart Release 5.1.0)

Release notes

• There is a new optional Boolean in Brig's Helm chart, config.multiSFT.enabled,
  signalling whether calls between federated SFT servers are allowed. (#3915)

  IMPORTANT: The value of this new option needs be set to the value of
  multiSFT.enabled in SFT's Helm chart. Otherwise federated SFT servers won't
  work.

  If provided, the field is_federating in the response of /calls/config/v2
  will reflect multiSFT.enabled's value.

  Example:

  # [brig/values.yaml]
  multiSFT:
    enabled: true
  

  Also, the optional object sftToken with its fields ttl and secret define
  whether an SFT credential would be rendered in the response of
  /calls/config/v2. The field ttl determines the seconds for the credential to
  be valid and secret is the path to the secret shared with SFT to create
  credentials.

  Example:

  # [brig.yaml]
  sft:
    sftBaseDomain: sft.wire.example.com
    sftSRVServiceName: sft
    sftDiscoveryIntervalSeconds: 10
    sftListLength: 20
    sftToken:
      ttl: 120
      secret: /path/to/secret
  

• The "addClient" internal endpoint of galley has been changed. This can cause temporary failures during upgrades if brig attempts to use this endpoint on a different version of galley. (#3904)

• Removed the deprecated and unused field geoDb from Brig's config. (#3975)

• Added support for 3 more MLS ciphersuites. To enable MLS, all supported signature schemes (ed25519 and the three ecdsa variants) now need to have private keys specified in galley's configuration file. (#3964)

API changes

• Create version 6 of client-related endpoints, fixing an oddity in the serialisation of capabilities. (#3904)

• Add gzip request support to spar and proxy (#4013)

Features

• Backend validates display name during DPoP challenge (#3890)

• Add Helm chart smallstep-accomp that provides a CRL endpoint proxy for federated E2EI (#3896)

• Support for Elasticsearch password authentication (#3989, #3959, #3994, #3984, #2093, #2079)

• Support unblocking a user in an MLS 1-to-1 conversation (#3940)

• Add E2EI configuration setup to smallstep-accomp chart (#3944)

• Remove Helm migrated charts webapp, team-settings, account-pages, sftd (#3927)

• charts/nginz: Rate limiting claiming MLS key-pacakges by requesting and target user (#3918)

• Support connecting to Elasticsearch over TLS

  It can be enabled by setting these options on the wire-server helm chart (#3989):

  brig:
    config:
      elasticsearch:
        scheme: https
  
        # When custom CAs are required, one of these must be set:
        tlsCa: <PEM encoded CA certificates>
        tlsCaSecretRef:
          name: <Name of the secret>
          key: <Key in the secret containing pem encoded CA Cert>
  
        # When TLS needs to be used without verification:
        insecureSkipVerifyTls: true
  
  elasticsearch-index:
    elasticsearch:
      scheme: https
  
      # When custom CAs are required, one of these must be set:
      tlsCa: <PEM encoded CA certificates>
      tlsCaSecretRef:
        name: <Name of the secret>
        key: <Key in the secret containing pem encoded CA Cert>
  
      # When TLS needs to be used without verification:
      insecureSkipVerifyTls: true

• Make gundeck's notificationTTL configurable. The value defines how long
  notifications are (at most) stored in the database. Decreasing this value e.g.
  helps to safe database space on test environments. (#3960)

• charts/nginz: Allow 3000 reqs/min on /conversations/one2one/:user_domain/:user (#3918)

• Support authenticating to redis (#3971)

Bug fixes and other updates

• Send connection cancelled event to local pending connection when user gets deleted (#3861)

• Optional apiProxy attribute added to deeplink.json in nginz chart (#3933)

• coturn cert-reloader sidecar config: process name should not contain the path (helm chart) (#3916)

• Prevent conflict on subsequent tries to provision a SCIM user (#3914)

• Avoid IO Exception when querying

  GET /converations/{cnv_domain}/{cnv}/groupinfo

  with public group state not set in galley.converation. (#3939)

• Return an actual list of other users in a remote MLS 1-to-1 conversation (#3998)

• charts/background-worker: Fix name of the service monitor (#3913)

• Fix crash when enqueing an empty list of notifications and federation is disabled (#PR_NOT_FOUND)

• Add the request ID to the request's execution environment in gundeck, such that it can be logged. (#3903)

• The AWS SNS ARN was parsed by accumulating the environment name up to the first
  dash ('-') such that parts of this name spilled over into the app name. Now, we
  accumulate up to the last dash. (#3894)

• Fix bug where welcome notifications were generated for each client instead of for each user (#3907)

• Do not deliver MLS one-to-one conversation messages to a user that blocked the sender (#3889, #3906)

• Optimize getting all feature configs (#4002)

Documentation

• adds new coding-conventions.md and talks about the decision we made for cs (#4006)

• Distinguish UTCTime and UTCTimeMillis in swagger (#3899)

• Patch hole in scim docs regarding wire team role manipulation. (#3897)

Internal changes

• Create a new script (Sbom.hs) to generate the wire-server sbom (bill of material) file. (#3942)

• port flaking LH tests to new integration and improve the ergonomics of our testing library (#3876)

• some small refactorings to make it more clear in code what is happening when registering a scim token and an IdP (#3966)

• In order for the CRL-proxy to function correctly, it needs to have CORS headers set.
  We are now setting the CORS headers on the ingress level. (#3956)

• drop cs in all production code and from Imports (#4001)

• Galley's internal DELETE /i/client/:clientID now early-exits before visiting all conversations if the client is already gone.
  Galley now reports debug logs for every call to Cassandra. (#3985)

• move formatting and linting of haskell files to treefmt, remove some of the now unneeded rules (#4000)

• Integration test cases for strangely behaving feature config settings. (#4007)

• Add ldap-scim-bridge chart to the wire-server release (#3999)

• Disable integration subchart of wire-server by default (#3682)

• Provide password as value in elasticsearch-ephemeral. This way we can use
  different passwords on our test systems. Ensuring that the password is really
  configurable (and not accidentally hardcoded somewhere.) (#3994)

• Upgraded fluent-bit chart to version 0.46.2
  Added example values for fluent-bit helm chart for output to syslog server (#4012)

• Ported 2FA tests to the new integration test suite (#3986)

• To ensure certificate revocations get active in a short time frame, disable
  caching of proxy results on client side by setting respective HTTP headers. (#3952)

• Ensure that targets of the smallstep nginx proxy are resolved at runtime via the
  configured DNS server. This has two benefits: The target gets adjusted when it's
  changed at the DNS server. And, nginx doesn't fail to start when the target
  doesn't exist yet. (#3947)

• Use schema-profunctor for user event serialisation and introduce golden tests (#3912)

• Setup federation-v0 environment for use in integration tests:

  • add federation-v0 domain to test environment
  • provision integration certificates with cert-manager (#3849, #3898)

• Add assets to output of ejpd-info end-point in stern; also:

  • [brig] now talks to carghold for profile picture extraction;
  • [integration] migrate ejpd tests;
  • [integration] enhanced shouldMatch shows a diff on failure now;
  • [integration] added shouldMatchLeniently for rule-based canonicalization of arguments (#3875)

• Bump hsaml2, saml2-web-sso dependencies. (#3995)

• Remove support for push token transport types APNSVoIP, APNSVoIPSandbox from gundeck. (#3967)

• Include remote domain in federator error logs (#3919)

• Remove remaining splinters of wai-routing, wai-predicate from brig. (#3996)

Federation changes

• The on-conversation-updated notification is now queued instead of being sent directly. A new version of the notification has been introduced with a different JSON format for the body, mostly for testing purposes of the versioning system.

  Since the notification is now sent asynchronously, some error conditions in case of unreachable backends cannot be triggered anymore. (#3831)

• Versioning of backend to backend notifications. Notifications are now stored in "bundles" containing a serialised payload for each supported version. The background worker then dynamically selects the best version to use and sends only the notification corresponding to that version.
  (#3831)

2024-02-12 (Chart Release 4.41.0)

Release notes

• Updates Gundeck to remove unused APNS_VOIP related code.
• This supersedes 2024-02-12 (Chart Release 4.40.0)

2024-02-12 (Chart Release 4.40.0)

Release notes

• The settings setDisabledAPIVersions (brig) and disabledAPIVersions (in cannon, cargohold, galley, gundeck, proxy, and spar) are now required.
  The default defined in charts/<service>/values.yaml is set to [ development ] and disables all development API versions.
  For more information see https://docs.wire.com/developer/reference/config-options.html#disabling-api-versions (#3772)

• The mls team feature now has a lock status. If the current installation has default settings defined in wire-server's values.yaml, the lockStatus has to be added, e.g.:

  mls:
    defaults:
      status: enabled
      config:
        protocolToggleUsers: []
        defaultProtocol: mls
        allowedCipherSuites: [1]
        defaultCipherSuite: 1
        supportedProtocols: [proteus, mls] # must contain defaultProtocol
      lockStatus: locked

(#3681)

• • Replace fake-sqs with ElasticMQ

  ElasticMQ is an actively maintained project, fake-sqs hasn't seen a commit since
  2018. This is not expected to have any noticeable effect on deployments that
  don't have any extra configurations for the SQS queues. If the fake-aws-sqs
  chart had configured custom queue names, they have couple of extra limitations:

  • The queue names must only contain alphanumeric characters and hyphens.
  • The FIFO queue names must end in .fifo. (#3750)

• Upgrade team-settings version to 4.15.1-v0.31.19-0-ee1dbce (#2180)

• Upgrade webapp to 2023-12-11-production.0-v0.31.17-0-1e91445

  Beside using up-to-date versions in Helm charts is generally beneficial,
  this version also provides multi-ingress support. (#3803)

• Upgrade webapp version to 2024-01-22-production.1-v0.31.17-0-7f83dbe (#2302)

API changes

• Un-verified users can no longer upload assets (#3604)

• • Remove IP request header from add-client endpoint.
  • Remove longitude and latitude response fields from add-client, list-clients, and get-client endpoints.

  These are not considered breaking changes, since clients are not using this information. (#3792)

• Create new API version V6 and finalize V5 (#3815)

• Block changes of userDisplayName, userHandle in mlsE2EI-enabled teams on the backend without SCIM; report "managed_by" == "scim" in GET /self, but only there (#3827)

• The federation API can now be versioned. Multiple versions of an RPC can be defined on the same path. After version negotiation, the federation client now sets the X-Wire-API-Version header, and federator propagates it to the destination service. (#3762)

• Improved formatting of federation errors. No extra copy of the response body, and nested errors are now part of the JSON structure, not quoted inside the message. (#3742)

• New endpoint for replacing MLS key packages in bulk: PUT /mls/key-packages/self/:client. It replaces all existing key packages that match the given ciphersuites with the new key packages provided in the body. (#3654)

Features

• The lifetime of conversation guest links is now configurable (#3796)

• Events for a member update, join and leave are not sent to everyone in the team any longer. Only team admins get them. (#3703)

• Allowlist for who on cloud can connect to on-prem:

  • Internal API to configure allowlist
  • Restrict federated user search according to team federation policy
    (#3697, #3732, #3758)

• The mls team feature now has a lock status (#3681)

• add a uniform timeout to the integration test-suite set by the environment variable TEST_TIMEOUT_SECONDS with a default of 10 seconds if the variable isn't set (#3692)

• Apply team-level federation policies when establishing and updating user connections (#3774)

• Introduce a feature flag that controls whether the limited event fanout should be used when a team member is deleted (#3797)

• Send a conversation.member-leave event to team admins for each conversation the deleted team member used to be part of (#3790)

• Allow the configuration of TLS-secured connections to Cassandra. TLS is used
  when a certificate is provided. This is either done with
  --tls-ca-certificate-file for cli commands or the configuration attribute
  cassandra.tlsCa for services. In Helm charts, the certificate is provided as
  literal PEM string; either as attribute cassandra.tlsCa (analog to service
  configuration) or by a reference to a secret (cassandra.tlsCaSecretRef.) (#3587)

• [SFT chart] Add option to enable serviceMonitor to scrape prometheus metrics (#3770)

Bug fixes and other updates

• galley's DB migrations fixed (#3680)

• The X509 client identity parser supports a new format: wireapp://{userid}!{deviceid}@{host} (#3808)

• Updated demo-smtp Helm chart from deprecated docker image namshi/smtp to ixdotai/smtp (#3791)

• External partners search restriction enforced by backend (#3708)

• File upload size is now limited to 100 MiB (#3752)

• Fix a bug where non-team conversation members that are remote would not get a conversation.member-leave event (#3745, #3764)

• Enforce external partner permissions on the backend (#3788)

• Various improvements around LH policy conflict detection:

  • Fix LH policy conflict detection logic when posting messages
  • Better policy conflict error messages (distinguish between old clients and missing consent)
  • Add first LH scaffolding and tests to /integration
  • Annotate some API functions in /integration with links to openapi3 docs (#3773)

• Do not match on the Accept header for service provider endpoints with no response body (#3766)

• Guests should not be added to conversations that are under legalhold (#3853)

• Intra-service calls from brig to galley's public API are now aware of disabled API versions (#3863)

• fix Helm pretty-printer for disabledAPIVersions (#3877)

• Adjust the requested memory and upper bound limit of nginz pods in the related Helm chart. (We experienced OOM errors with the old settings.) (#3821)

• don't use shell when communicating with mls-test-cli, move flaking brig tests over to new integration testsuite (#3701)

• set notificationTimeOut to 28 days, make it legible (#3714)

• Update coturn image with bugfix to its prestop-hook from wireapp/coturn#10 to allow coturn pods to terminate once their traffic has drained. (#3872)

• Extra remove proposals were being sent when a user was removed from a conversation (#3672)

• Remove client check for subconversations (#3677)

• Ensure that SCIM can find users even after the team admin has changed the SAML issuer for the user. (#3747)

• addClient used the internal brig API in the integration testsuite when it should use the public one (#3869)

• Ensure that HTTP 1.1 connections are grafully closed

  To fix this warp had to be patched to fix the bug upstream: yesodweb/wai#958 (#3775)

Documentation

• Fix missing code sections on docs.wire.com, notably on "configuring TLS" page. (#3839)

• Swagger generation no longer adds tags containing information about federation calls.

  Added the federation calling graph to the Federation API Conventions page. (#3674)

• Backend-to-backend OpenApi Docs added (#3666)

• Documentation for creating a new API version updated (#3817)

• Update documentation of MLS group ID (#3705)

• Turn long summaries in openapi documentation into descriptions (#3706)

• update the build instructions for wire-server (#3854)

Internal changes

• stern/backoffice PUT /teams/{teamId}/features/conferenceCalling fixed (#3723)

• Removed client ID conversion round trip (#3727)

• Migrate to Servant the Galley conversation internal endpoints (#3718)

• The development API version is now disabled by default (#3772)

• Attempt to fix flaky integration test provider.service.delete (#3689)

• The fedcalls tool no longer walks the Swagger/OpenAPI structure when generating call graphs. These graphs are now generated directly from the Servant API types. (#3674, #3691)

• Increased ingress payload size from 256k to 512k (#3756)

• Request tracing across federated requests (#3765)

• upgrade nixpkgs to upgrade haskell-language-server (#3650)

• upgrade the GHC version to GHC 9.4 (#3679)

• Removed APNS_VOIP code. (APNS_VOIP is a native push notification channel which we aren't using anymore.) (#3695)

• Improve error logs (#3782)

• Migrating tests for Cargohold to the new integration test suite. (#3741)

• Fix calendar integration setting in backoffice / stern (#3761)

• Reply-Nonce is added to Access-Control-Expose-Headers (#3729)

• Add custom feature flag; only supported for some on-prem installations; locked & disabled by default (#3779)

• Improved how tests are automatically extracted from the integration test suite.

  The test extractor parser has been improved to handle block comments, and to more strictly check for Haddock documentation for each test. (#3749)

• Additional logging on user/team suspension (#3795)

• cleanup the haskell-pins

  • remove many pins
  • remove many overrides
  • restructure the files such that it's easier to see what is going on (#3814)

• Version of rusty-jwt-tools bumped to v0.8.0 (#3805)

• Feature enforceFileDownloadLocation lockstatus can be set with basic auth on staging (#3802)

• Version of rusty-jwt-tools bumped to v0.8.5 (#3820)

• Translate integration tests: manually add / delete LH device (#3830)

• adds a new executable, hs-run, to quickly run haskell scripts (#3716)

• Represent client IDs as Word64 internally (#3713)

• Allow to install the coturn chart multiple times in multiple namespaces on the same cluster. (#3698)

• For some rust packages (cryptobox and libzauth-c), we now use crate2nix as a build tool, rather than the more coarse and FOD-based nixpkgs `rustPlatfo...

2023-10-23 (Chart Release 4.39.0)

Release notes

• New field for Supported protocols in Galley's MLS feature config

  Galley will refuse to start if the list supportedProtocols does not contain
  the value of the field defaultProtocol. Galley will also refuse to start if
  MLS migration is enabled and MLS is not part of supportedProtocols.

  The default value for supportedProtocols is:

  [proteus, mls]
  

(#3374)

API changes

• The JSON schema of NonConnectedBackends has changed to have its single field now called non_connected_backends. (#3518)

• Remove de-federation (to avoid a scalability issue). (#3582)

• Replace the placeholder self conversation id with the qualified conversation id for welcome events. (#3335)

• Add new endpoint DELETE /mls/key-packages/self/:client (#3295)

• Introduce an endpoint for deleting a subconversation (#2956, #3119, #3123)

• Remove MLS endpoints from API v4 and finalise it (#3545)

• Add new endpoint GET /conversations/one2one/:domain/:uid to fetch the MLS 1-1 conversation with another user (#3345)

• Introduce a subconversation GET endpoint (#2869, #2995)

• Add GET /conversations/:domain/:cid/subconversations/:id/groupinfo endpoint to fetch the group info object for a subconversation (#2932)

• Introduce v5 development version (#3527)

• It is now possible to use PUT /conversation/:domain/:id/protocol to transition from Mixed to MLS (#3334)

• Report a failure to add remote users to an MLS conversation (#3304)

• The key package API has gained a ciphersuite query parameter, which should be the hexadecimal value of an MLS ciphersuite, defaulting to 0x0001. The ciphersuite parameter is used by the claim and count endpoints. For uploads, the API is unchanged, and the ciphersuite is taken directly from the uploaded key package. (#3454)

• Add MLS migration feature config (#3299)

• Switch to MLS draft 20. The following endpoints are affected by the change:

  • All endpoints with message/mls content type now expect and return draft-20 MLS structures.
  • POST /conversations does not require creator_client anymore.
  • POST /mls/commit-bundles now expects a "stream" of MLS messages, i.e. a sequence of TLS-serialised messages, one after the other, in any order. Its protobuf interface has been removed.
  • POST /mls/welcome has been removed. Welcome messages can now only be sent as part of a commit bundle.
  • POST /mls/message does not accept commit messages anymore. All commit messages must be sent as part of a commit bundle. (#3172)

• Key packages and leaf nodes with x509 credentials are now supported (#3532)

Features

• Add reason field to conversation.member-leave (#3640)

• Support deleting a remote subconversation (#2964)

• Introduce support for resetting a subconversation (#2956)

• Introduce a "mixed" conversation protocol type. A conversation of "mixed" protocol functions as a Proteus converation as well as a MLS conversations. It's intended to be used for migrating conversations from Proteus to MLS. (#3258)

• Added support for post-quantum ciphersuite 0xf031. Correspondingly, MLS groups with a non-default ciphersuite are now supported. The first commit in a group determines the group ciphersuite. (#3454)

• Remove conversation size limit for MLS conversations (#3468)

• Added support for MSL 1-1 conversations (#3360)

• MLS application messages for older epochs are now rejected (#3438)

• The public key in an x509 credential is now checked against that of the client (#3542)

• Add federated endpoints to get subconversations (#2952)

• Add Helm chart (rabbitmq-external) to interface RabbitMQ instances outside of the Kubernetes cluster. (#3626)

• Removing or kicking a user from a conversation also removes the user's clients from any subconversation. (#2942)

• Add support for subconversations in POST /mls/commit-bundles (#2932)

• Implement endpoint for leaving a subconversation (#2969, #3080, #3085, #3107)

Bug fixes and other updates

• Fix nix derivations for rust packages (#3628)

• Ensure benchmarking dependencies are provided by nix development environment (#3628)

• Disable a guest user from creating a group conversation (#3622)

• Adding users to a conversation now enforces that all federation domains that will be in the conversation are federated with each other. (#3514)

• Fix ES migration script. (#3558)

• Fixed add user to conversation when one of the other participating backends is offline (#3585)

• Create a new http2 connection in every federator client request instead of using a shared connection. (#3602)

• list-clients returns with partial success even if one of the remote backends is unreachable (#3611)

• Defederation notifications, federation.delete and federation.connectionRemoved, now deduplicate the user list so that we don't send them more notifications than required. (#3515)

• Fix memory and TCP connection leak in brig, galley, caroghold and background-worker. (#3663)

• Fix bug where notifications for MLS messages were not showing up in all notification streams of clients (#3610)

• Map the MLS self-conversation creator's key package reference in Brig (#3055)

• This fixes a bug where a remote member is removed from a conversation while their backend is unreachable, and the backend does not receive the removal notification once it is reachable again. (#3537)

• Welcome messages are not sent anymore to the creator of an MLS group on the first commit (#3392)

Documentation

• Fix: support api versions other than v0 in swagger docs. (#3619)

• Updating the route documentation from Swagger 2 to OpenAPI 3. (#3570)

• Elaborate on internal user creation in prod (#3596)

• Adding a testing config entry to the PR guidelines. (#3624)

Internal changes

• remove leaving clients immediately from subconversations (#3096)

• Servantify internal end-points: brig/teams (#3634)

• add conversation type to group ID serialisation (#3344)

• Do not cache federation remote configs on non-brig services (#3612)

• JSON derived schemas have been changed to no longer pre-process record fields to drop prefixes that were required to disambiguate fields.
  Prefix processing still exists to drop leading underscores from field names, as we are using prefixed field names with makeLenses.
  Code has been updated to use OverloadedRecordDot with the changed field names. (#3518)

• Updating the route documentation library from swagger2 to openapi3.

  This also introduced a breaking change in how we track what federation calls each route makes.
  The openapi3 library doesn't support extension fields, and as such tags are being used instead in a similar way. (#3570)

• • Extending the information returned in errors for Federator. Paths and response bodies, if available, are included in error logs.
  • Prometheus metrics for outgoing and incoming federation requests added. They can be enabled by setting metrics.serviceMonitor.enabled, like in other charts. (#3556)

• CLI tool to consume messages from a RabbitMQ queue (#3589, #3655)

• Removed user and client threshold fields from mls migration feature. (#3364)

• Include timestamp in s3 upload path for test logs (#3621)

• Migrating the following routes to the Servant API form.

  POST /provider/services
  GET /provider/services
  GET /provider/services/:sid
  PUT /provider/services/:sid
  PUT /provider/services/:sid/connection
  DELETE /provider/services/:sid
  GET /providers/:pid/services
  GET /providers/:pid/services/:sid
  GET /services
  GET /services/tags
  GET /teams/:tid/services/whitelisted
  POST /teams/:tid/services/whitelist (#3554)

• Provider API has been migrated to servant (#3547)

• background-worker: Get list of domains from RabbitMQ instead of brig for pushing backend notifications (#3588)

• Avoid including MLS application messages in the sender client's event stream. (#3379)

• Avoid empty pushes when chunking pushes in galley (#3646)

• Introduce a Galley DB table for subconversations (#2869)

• Support mapping MLS group IDs to subconversations (#2869)

• change version and conversation type to 16 bit in group ID serialisation (#3353)

• Brig does not perform key package ref mapping anymore. Claimed key packages are simply removed from the mls_key_packages table. The mls_key_package_refs table is now unused, and will be removed in the future. (#3172)

• Add intermediate "mixed" protocol for migrating from Proteus to MLS (#3292)

• • Do not perform client checks for add and remove proposals in mixed conversations
  • Restrict protocol updates to team conversations
  • Disallow MLS application messages in mixed conversations
  • Send remove proposals when users leave mixed conversations (#3303)

• New cron job to save data usable to watch the progress of the Proteus to MLS migration in S3 bucket.

  IMPORTANT: This cron job is not meant for general use! It can leak data about one team to other teams. (#3579)

• Subconversations are now created on their first commit (#3355)

• Propagate messages in MLS subconversations (#2937)

• Move some MLS tests to new integration suite (#3286)

• Check validity of notification IDs in the notification API (#3550)

• stern: Optimize RAM usage of /i/users/meta-info (#3522)

• Additional integration test for federated connections (#3538)

• The bot API is now migrated to servant (#3540)

• rusty-jwt-tools is upgraded to version 0.5.0 (#3572)

• Refactored schema version tracking from manually managed to automatic. (#3643)

• Avoid unnecessary error logs on service shutdown (#3592)

• Introduce an effect for subconversations (#2869)

• Via the update path update the key package of the committer in epoch 0 of a subconversation (#2975)

• Add more tests for joining a sub...