Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 0 additions & 5 deletions static/env-vars/activitylog.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
# Autogenerated
# Filename: activitylog.yaml

tracing:
enabled: false
type: ""
endpoint: ""
collector: ""
log:
level: ""
pretty: false
Expand Down
8 changes: 2 additions & 6 deletions static/env-vars/activitylog_configvars.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,6 @@ Environment variables for the **activitylog** service

| Name | Introduction Version | Type | Description | Default Value |
|---|---|---|---|:---|
|`OC_TRACING_ENABLED`<br/>`ACTIVITYLOG_TRACING_ENABLED`| 1.0.0 |bool|`Activates tracing.`|`false`|
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dragonchaser from my pov, they should appear in the deletion section.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let us put that on the deleted list with a explanation why we did this without deprecation.

|`OC_TRACING_TYPE`<br/>`ACTIVITYLOG_TRACING_TYPE`| 1.0.0 |string|`The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now.`|``|
|`OC_TRACING_ENDPOINT`<br/>`ACTIVITYLOG_TRACING_ENDPOINT`| 1.0.0 |string|`The endpoint of the tracing agent.`|``|
|`OC_TRACING_COLLECTOR`<br/>`ACTIVITYLOG_TRACING_COLLECTOR`| 1.0.0 |string|`The HTTP endpoint for sending spans directly to a collector, i.e. \http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.`|``|
|`OC_LOG_LEVEL`<br/>`ACTIVITYLOG_LOG_LEVEL`| 1.0.0 |string|`The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'.`|``|
|`OC_LOG_PRETTY`<br/>`ACTIVITYLOG_LOG_PRETTY`| 1.0.0 |bool|`Activates pretty log output.`|`false`|
|`OC_LOG_COLOR`<br/>`ACTIVITYLOG_LOG_COLOR`| 1.0.0 |bool|`Activates colorized log output.`|`false`|
Expand Down Expand Up @@ -43,5 +39,5 @@ Environment variables for the **activitylog** service
|`OC_DEFAULT_LANGUAGE`| 1.0.0 |string|`The default language used by services and the WebUI. If not defined, English will be used as default. See the documentation for more details.`|`en`|
|`OC_SERVICE_ACCOUNT_ID`<br/>`ACTIVITYLOG_SERVICE_ACCOUNT_ID`| 1.0.0 |string|`The ID of the service account the service should use. See the 'auth-service' service description for more details.`|``|
|`OC_SERVICE_ACCOUNT_SECRET`<br/>`ACTIVITYLOG_SERVICE_ACCOUNT_SECRET`| 1.0.0 |string|`The service account secret.`|``|
|`ACTIVITYLOG_WRITE_BUFFER_DURATION`| next |Duration|`The duration to wait before flushing the write buffer. This is used to reduce the number of writes to the store.`|`10s`|
|`ACTIVITYLOG_MAX_ACTIVITIES`| next |int|`The maximum number of activities to keep in the store per resource. If the number of activities exceeds this value, the oldest activities will be removed.`|`6000`|
|`ACTIVITYLOG_WRITE_BUFFER_DURATION`| 4.0.0 |Duration|`The duration to wait before flushing the write buffer. This is used to reduce the number of writes to the store.`|`10s`|
|`ACTIVITYLOG_MAX_ACTIVITIES`| 4.0.0 |int|`The maximum number of activities to keep in the store per resource. If the number of activities exceeds this value, the oldest activities will be removed.`|`6000`|
2 changes: 1 addition & 1 deletion static/env-vars/activitylog_readme.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Activitylog
date: 2025-11-13T17:22:55.092232+01:00
date: 2025-11-27T22:56:02.331192+01:00
weight: 20
geekdocRepo: https://github.com/opencloud-eu/opencloud
geekdocEditPath: edit/master/services/activitylog
Expand Down
5 changes: 0 additions & 5 deletions static/env-vars/antivirus.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,6 @@ debug:
token: ""
pprof: false
zpages: false
tracing:
enabled: false
type: ""
endpoint: ""
collector: ""
infected-file-handling: delete
events:
endpoint: 127.0.0.1:9233
Expand Down
4 changes: 0 additions & 4 deletions static/env-vars/antivirus_configvars.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,6 @@ Environment variables for the **antivirus** service
|`ANTIVIRUS_DEBUG_TOKEN`| 1.0.0 |string|`Token to secure the metrics endpoint.`|``|
|`ANTIVIRUS_DEBUG_PPROF`| 1.0.0 |bool|`Enables pprof, which can be used for profiling.`|`false`|
|`ANTIVIRUS_DEBUG_ZPAGES`| 1.0.0 |bool|`Enables zpages, which can be used for collecting and viewing in-memory traces.`|`false`|
|`OC_TRACING_ENABLED`<br/>`ANTIVIRUS_TRACING_ENABLED`| 1.0.0 |bool|`Activates tracing.`|`false`|
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

how can i enable tracing now? @butonic

|`OC_TRACING_TYPE`<br/>`ANTIVIRUS_TRACING_TYPE`| 1.0.0 |string|`The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now.`|``|
|`OC_TRACING_ENDPOINT`<br/>`ANTIVIRUS_TRACING_ENDPOINT`| 1.0.0 |string|`The endpoint of the tracing agent.`|``|
|`OC_TRACING_COLLECTOR`<br/>`ANTIVIRUS_TRACING_COLLECTOR`| 1.0.0 |string|`The HTTP endpoint for sending spans directly to a collector, i.e. \http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.`|``|
|`ANTIVIRUS_INFECTED_FILE_HANDLING`| 1.0.0 |string|`Defines the behaviour when a virus has been found. Supported options are: 'delete', 'continue' and 'abort '. Delete will delete the file. Continue will mark the file as infected but continues further processing. Abort will keep the file in the uploads folder for further admin inspection and will not move it to its final destination.`|`delete`|
|`OC_EVENTS_ENDPOINT`<br/>`ANTIVIRUS_EVENTS_ENDPOINT`| 1.0.0 |string|`The address of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture.`|`127.0.0.1:9233`|
|`OC_EVENTS_CLUSTER`<br/>`ANTIVIRUS_EVENTS_CLUSTER`| 1.0.0 |string|`The clusterID of the event system. The event system is the message queuing service. It is used as message broker for the microservice architecture. Mandatory when using NATS as event system.`|`opencloud-cluster`|
Expand Down
2 changes: 1 addition & 1 deletion static/env-vars/antivirus_readme.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Antivirus
date: 2025-11-13T17:22:55.092547+01:00
date: 2025-11-27T22:56:02.331592+01:00
weight: 20
geekdocRepo: https://github.com/opencloud-eu/opencloud
geekdocEditPath: edit/master/services/antivirus
Expand Down
5 changes: 0 additions & 5 deletions static/env-vars/app-provider.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
# Autogenerated
# Filename: app-provider.yaml

tracing:
enabled: false
type: ""
endpoint: ""
collector: ""
log:
level: ""
pretty: false
Expand Down
4 changes: 0 additions & 4 deletions static/env-vars/app-provider_configvars.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,6 @@ Environment variables for the **app-provider** service
| Name | Introduction Version | Type | Description | Default Value |
|---|---|---|---|:---|
|`APP_PROVIDER_SERVICE_NAME`| 1.0.0 |string|`The name of the service. This needs to be changed when using more than one app provider. Each app provider configured needs to be identified by a unique service name. Possible examples are: 'app-provider-collabora', 'app-provider-onlyoffice', 'app-provider-office365'.`|`app-provider`|
|`OC_TRACING_ENABLED`<br/>`APP_PROVIDER_TRACING_ENABLED`| 1.0.0 |bool|`Activates tracing.`|`false`|
|`OC_TRACING_TYPE`<br/>`APP_PROVIDER_TRACING_TYPE`| 1.0.0 |string|`The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now.`|``|
|`OC_TRACING_ENDPOINT`<br/>`APP_PROVIDER_TRACING_ENDPOINT`| 1.0.0 |string|`The endpoint of the tracing agent.`|``|
|`OC_TRACING_COLLECTOR`<br/>`APP_PROVIDER_TRACING_COLLECTOR`| 1.0.0 |string|`The HTTP endpoint for sending spans directly to a collector, i.e. \http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.`|``|
|`OC_LOG_LEVEL`<br/>`APP_PROVIDER_LOG_LEVEL`| 1.0.0 |string|`The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'.`|``|
|`OC_LOG_PRETTY`<br/>`APP_PROVIDER_LOG_PRETTY`| 1.0.0 |bool|`Activates pretty log output.`|`false`|
|`OC_LOG_COLOR`<br/>`APP_PROVIDER_LOG_COLOR`| 1.0.0 |bool|`Activates colorized log output.`|`false`|
Expand Down
98 changes: 15 additions & 83 deletions static/env-vars/app-provider_readme.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Antivirus
date: 2025-11-12T11:19:19.198072+01:00
title: App Provider
date: 2025-11-27T22:56:02.331696+01:00
weight: 20
geekdocRepo: https://github.com/opencloud-eu/opencloud
geekdocEditPath: edit/master/services/app-provider
Expand All @@ -13,97 +13,29 @@ geekdocCollapseSection: true
## Abstract


The `antivirus` service is responsible for scanning files for viruses.
The `app-provider` service provides the CS3 App Provider API for OpenCloud. It is responsible for managing and serving applications that can open files based on their MIME types.

The service works in conjunction with the `app-registry` service, which maintains the registry of available applications and their supported MIME types. When a client requests to open a file with a specific application, the `app-provider` service handles the request and coordinates with the application to provide the appropriate interface.


## Table of Contents

* [Memory Considerations](#memory-considerations)
* [Integration](#integration)
* [Configuration](#configuration)
* [Antivirus Scanner Type](#antivirus-scanner-type)
* [Maximum Scan Size](#maximum-scan-size)
* [Antivirus Workers](#antivirus-workers)
* [Infected File Handling](#infected-file-handling)
* [Scanner Inaccessibility](#scanner-inaccessibility)
* [Operation Modes](#operation-modes)
* [Postprocessing](#postprocessing)
* [Scaling in Kubernetes](#scaling-in-kubernetes)

## Memory Considerations

The antivirus service can consume considerable amounts of memory.
This is relevant to provide or define sufficient memory for the deployment selected.
To avoid out of memory (OOM) situations, the following equation gives a rough overview based on experiences made.
The memory calculation comes without any guarantee, is intended as overview only and subject of change.
* [Scalability](#scalability)

`memory limit` = `max file size` x `workers` x `factor 8 - 14`
## Integration

With:
`ANTIVIRUS_WORKERS` == 1
```plaintext
50MB file --> factor 14 --> 700MB memory
844MB file --> factor 8,3 --> 7GB memory
```
The `app-provider` service integrates with:
- `app-registry` - For discovering which applications are available for specific MIME types
- `frontend` - The frontend service forwards app provider requests (default endpoint `/app`) to this service

## Configuration

### Antivirus Scanner Type

The antivirus service currently supports [ICAP](https://tools.ietf.org/html/rfc3507) and [ClamAV](http://www.clamav.net/index.html) as antivirus scanners.
The `ANTIVIRUS_SCANNER_TYPE` environment variable is used to select the scanner.
The detailed configuration for each scanner heavily depends on the scanner type selected.
See the environment variables for more details.

- For `icap`, only scanners using the `X-Infection-Found` header are currently supported.
- For `clamav` only local sockets can currently be configured.

### Maximum Scan Size

Several factors can make it necessary to limit the maximum filesize the antivirus service uses for scanning.
Use the `ANTIVIRUS_MAX_SCAN_SIZE` environment variable to scan only a given number of bytes,
or to skip the whole resource.

Even if it's recommended to scan the whole file, several factors like scanner type and version,
bandwidth, performance issues, etc. might make a limit necessary.

In such cases, the antivirus the max scan size mode can be handy, the following modes are available:

- `partial`: The file is scanned up to the given size. The rest of the file is not scanned. This is the default mode `ANTIVIRUS_MAX_SCAN_SIZE=partial`
- `skip`: The file is skipped and not scanned. `ANTIVIRUS_MAX_SCAN_SIZE=skip`

**IMPORTANT**
> Streaming of files to the virus scan service still [needs to be implemented](https://github.com/owncloud/ocis/issues/6803).
> To prevent OOM errors `ANTIVIRUS_MAX_SCAN_SIZE` needs to be set lower than available ram and or the maximum file size that can be scanned by the virus scanner.

### Antivirus Workers

The number of concurrent scans can be increased by setting `ANTIVIRUS_WORKERS`. Be aware that this will also increase memory usage.

### Infected File Handling

The antivirus service allows three different ways of handling infected files. Those can be set via the `ANTIVIRUS_INFECTED_FILE_HANDLING` environment variable:

- `delete`: (default): Infected files will be deleted immediately, further postprocessing is cancelled.
- `abort`: (advanced option): Infected files will be kept, further postprocessing is cancelled. Files can be manually retrieved and inspected by an admin. To identify the file for further investigation, the antivirus service logs the abort/infected state including the file ID. The file is located in the `storage/users/uploads` folder of the OpenCloud data directory and persists until it is manually deleted by the admin via the [Manage Unfinished Uploads](https://github.com/opencloud-eu/opencloud/tree/main/services/storage-users#manage-unfinished-uploads) command.
- `continue`: (not recommended): Infected files will be marked via metadata as infected, but postprocessing continues normally. Note: Infected Files are moved to their final destination and therefore not prevented from download, which includes the risk of spreading viruses.

In all cases, a log entry is added declaring the infection and handling method and a notification via the `userlog` service sent.

### Scanner Inaccessibility

In case a scanner is not accessible by the antivirus service like a network outage, service outage or hardware outage, the antivirus service uses the `abort` case for further processing, independent of the actual setting made. In any case, an error is logged noting the inaccessibility of the scanner used.

## Operation Modes

The antivirus service can scan files during `postprocessing`. `on demand` scanning is currently not available and might be added in a future release.

### Postprocessing

The antivirus service will scan files during postprocessing. It listens for a postprocessing step called `virusscan`. This step can be added in the environment variable `POSTPROCESSING_STEPS`. Read the documentation of the [postprocessing service](https://github.com/opencloud-eu/opencloud/tree/main/services/postprocessing) for more details.

The number of concurrent scans can be increased by setting `ANTIVIRUS_WORKERS`, but be aware that this will also increase the memory usage.
The service can be configured via environment variables. Key configuration options include:
- `APP_PROVIDER_EXTERNAL_ADDR` - External address where the gateway service can reach the app provider

### Scaling in Kubernetes
## Scalability

In kubernetes, `ANTIVIRUS_WORKERS` and `ANTIVIRUS_MAX_SCAN_SIZE` can be used to trigger the horizontal pod autoscaler by requesting a memory size that is below `ANTIVIRUS_MAX_SCAN_SIZE`. Keep in mind that `ANTIVIRUS_MAX_SCAN_SIZE` amount of memory might be held by `ANTIVIRUS_WORKERS` number of go routines.
The app-provider service can be scaled horizontally as it primarily acts as a coordinator between applications and the OpenCloud backend services.

5 changes: 0 additions & 5 deletions static/env-vars/app-registry.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
# Autogenerated
# Filename: app-registry.yaml

tracing:
enabled: false
type: ""
endpoint: ""
collector: ""
log:
level: ""
pretty: false
Expand Down
4 changes: 0 additions & 4 deletions static/env-vars/app-registry_configvars.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,6 @@ Environment variables for the **app-registry** service

| Name | Introduction Version | Type | Description | Default Value |
|---|---|---|---|:---|
|`OC_TRACING_ENABLED`<br/>`APP_REGISTRY_TRACING_ENABLED`| 1.0.0 |bool|`Activates tracing.`|`false`|
|`OC_TRACING_TYPE`<br/>`APP_REGISTRY_TRACING_TYPE`| 1.0.0 |string|`The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now.`|``|
|`OC_TRACING_ENDPOINT`<br/>`APP_REGISTRY_TRACING_ENDPOINT`| 1.0.0 |string|`The endpoint of the tracing agent.`|``|
|`OC_TRACING_COLLECTOR`<br/>`APP_REGISTRY_TRACING_COLLECTOR`| 1.0.0 |string|`The HTTP endpoint for sending spans directly to a collector, i.e. \http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.`|``|
|`OC_LOG_LEVEL`<br/>`APP_REGISTRY_LOG_LEVEL`| 1.0.0 |string|`The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'.`|``|
|`OC_LOG_PRETTY`<br/>`APP_REGISTRY_LOG_PRETTY`| 1.0.0 |bool|`Activates pretty log output.`|`false`|
|`OC_LOG_COLOR`<br/>`APP_REGISTRY_LOG_COLOR`| 1.0.0 |bool|`Activates colorized log output.`|`false`|
Expand Down
4 changes: 2 additions & 2 deletions static/env-vars/app-registry_readme.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: App Registry
date: 2025-11-13T17:22:55.092655+01:00
date: 2025-11-27T22:56:02.33178+01:00
weight: 20
geekdocRepo: https://github.com/opencloud-eu/opencloud
geekdocEditPath: edit/master/services/app-registry
Expand All @@ -20,7 +20,7 @@ Administrators can set default applications on a per MIME type basis and also al

## Table of Contents

* [MIME Type Configuration / Creation Allow List](#mime-type-configuration--creation-allow-list)
* [MIME Type Configuration / Creation Allow List](#mime-type-configuration-/-creation-allow-list)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, @ScharfViktor That is still wrong in the opencloud repo.

* [MIME Type Configuration](#mime-type-configuration)
* [Endpoint Access](#endpoint-access)
* [Listing available apps and mime types](#listing-available-apps-and-mime-types)
Expand Down
5 changes: 0 additions & 5 deletions static/env-vars/audit.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
# Autogenerated
# Filename: audit.yaml

tracing:
enabled: false
type: ""
endpoint: ""
collector: ""
log:
level: ""
pretty: false
Expand Down
4 changes: 0 additions & 4 deletions static/env-vars/audit_configvars.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,6 @@ Environment variables for the **audit** service

| Name | Introduction Version | Type | Description | Default Value |
|---|---|---|---|:---|
|`OC_TRACING_ENABLED`<br/>`AUDIT_TRACING_ENABLED`| 1.0.0 |bool|`Activates tracing.`|`false`|
|`OC_TRACING_TYPE`<br/>`AUDIT_TRACING_TYPE`| 1.0.0 |string|`The type of tracing. Defaults to '', which is the same as 'jaeger'. Allowed tracing types are 'jaeger' and '' as of now.`|``|
|`OC_TRACING_ENDPOINT`<br/>`AUDIT_TRACING_ENDPOINT`| 1.0.0 |string|`The endpoint of the tracing agent.`|``|
|`OC_TRACING_COLLECTOR`<br/>`AUDIT_TRACING_COLLECTOR`| 1.0.0 |string|`The HTTP endpoint for sending spans directly to a collector, i.e. \http://jaeger-collector:14268/api/traces. Only used if the tracing endpoint is unset.`|``|
|`OC_LOG_LEVEL`<br/>`AUDIT_LOG_LEVEL`| 1.0.0 |string|`The log level. Valid values are: 'panic', 'fatal', 'error', 'warn', 'info', 'debug', 'trace'.`|``|
|`OC_LOG_PRETTY`<br/>`AUDIT_LOG_PRETTY`| 1.0.0 |bool|`Activates pretty log output.`|`false`|
|`OC_LOG_COLOR`<br/>`AUDIT_LOG_COLOR`| 1.0.0 |bool|`Activates colorized log output.`|`false`|
Expand Down
2 changes: 1 addition & 1 deletion static/env-vars/audit_readme.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
title: Audit
date: 2025-11-13T17:22:55.092897+01:00
date: 2025-11-27T22:56:02.331876+01:00
weight: 20
geekdocRepo: https://github.com/opencloud-eu/opencloud
geekdocEditPath: edit/master/services/audit
Expand Down
5 changes: 0 additions & 5 deletions static/env-vars/auth-app.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,6 @@
# Autogenerated
# Filename: auth-app.yaml

tracing:
enabled: false
type: ""
endpoint: ""
collector: ""
log:
level: ""
pretty: false
Expand Down
Loading
Loading