Skip to content

Feat/genapi add shared responsibility model #5125

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
134 changes: 134 additions & 0 deletions pages/generative-apis/reference-content/security-and-reliability.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
---
meta:
title: Security and Reliability in Generative APIs
description: Learn more about shared responsibility in security and reliability practices for Generative APIs
content:
h1: Security and Reliability in Generative APIs
paragraph: Learn more about shared responsibility in security and reliability practices for Generative APIs
tags: generative-apis ai-data shared responsibility security reliability
dates:
validation: 2025-06-16
---

This page outlines key principles and best practices to help you ensure your applications' security and reliability when using Generative APIs.

## Resilience

Resilience ensures the continuity and availability of your applications and data, even in the face of disruptions or failures. In Generative APIs, you can promote resilience through the following pillars: availability, durability and performance.

### Availability and durability

Generative APIs SLAs target the following SLOs:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe explain what "SLA", and "SLO" stand for, for people who aren't familiar with the terms.


| Processing Type | Configuration Details | Availability |
| ------------ | ------------------- | ------ |
| Standard | Standard synchronous HTTP calls to Generative APIs providing the generated content directly in HTTP response. These calls include stream and non-stream requests. | 99.9% |
| Batch | Asynchronous processing of files sent to Generative APIs providing the generated content through files. | 99.9% |

The detailed SLAs measurements and guarantees can be found on the [Service Level Agreement for Generative APIs](https://www.scaleway.com/en/database/sla/) page.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The detailed SLAs measurements and guarantees can be found on the [Service Level Agreement for Generative APIs](https://www.scaleway.com/en/database/sla/) page.
The detailed SLA measurements and guarantees can be found on the [Service Level Agreement for Generative APIs](https://www.scaleway.com/en/database/sla/) page.


As we do not store any data with Standard processing durability requirements do not apply.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
As we do not store any data with Standard processing durability requirements do not apply.
As we do not store any data with standard processing, durability requirements do not apply.


When processing data using Batch processing, your input data is stored only during processing time (24 hours):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
When processing data using Batch processing, your input data is stored only during processing time (24 hours):
When processing data using batch processing, your input data is stored only during processing time (24 hours):

- As input data storage is only temporary, no specific durability guarantee apply.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- As input data storage is only temporary, no specific durability guarantee apply.
- As input data storage is only temporary, no specific durability guarantee applies.

- Output data (processing results) durability depends on the target storage system used (by default: Object Storage Standard Class)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Output data (processing results) durability depends on the target storage system used (by default: Object Storage Standard Class)
- Output data (processing results) durability depends on the target storage system used. The storage system used by default is the [Object Storage Standard Class](/object-storage/concepts/#storage-class)



## Performance

Standard processing (synchronous HTTP calls):
- Generative APIs runs models on mutualized infrastructure, and therefore ensures good performance in average utilization. We monitor and respond quickly to any drops in token generation throughput, but cannot guarantee strictly performance, especially during customer peak loads. As a consequence, [Rate Limits](https://www.scaleway.com/en/docs/generative-apis/reference-content/rate-limits/) apply, to ensure "fair use" of synchronous HTTP calls. Bigger requests volumes should be treated through Batch processing.
- Guaranteed performance can be provided using dedicated resources on Managed Inference product.
Comment on lines +40 to +41
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Generative APIs runs models on mutualized infrastructure, and therefore ensures good performance in average utilization. We monitor and respond quickly to any drops in token generation throughput, but cannot guarantee strictly performance, especially during customer peak loads. As a consequence, [Rate Limits](https://www.scaleway.com/en/docs/generative-apis/reference-content/rate-limits/) apply, to ensure "fair use" of synchronous HTTP calls. Bigger requests volumes should be treated through Batch processing.
- Guaranteed performance can be provided using dedicated resources on Managed Inference product.
- Generative APIs run models on mutualized infrastructure, and therefore ensure good performance in average usage. We monitor and respond quickly to any drops in token generation throughput, but cannot strictly guarantee performance, especially during customer peak loads. As a consequence, [rate limits](/generative-apis/reference-content/rate-limits/) apply, to ensure fair use of synchronous HTTP calls. Bigger volumes of requests should be treated through batch processing.
- Guaranteed performance can be provided using dedicated resources on the [Managed Inference](/managed-inference/) product.


Batch processing (asynchronous file processing):
- When using Batch processing, we handle scheduling of batch jobs to optimize both processing ressource allocation and processing time. Processing time is therefore only guaranteed to be lower than 24 hours and [Rate Limits](https://www.scaleway.com/en/docs/generative-apis/reference-content/rate-limits/) (larger than Standard processing) still apply.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- When using Batch processing, we handle scheduling of batch jobs to optimize both processing ressource allocation and processing time. Processing time is therefore only guaranteed to be lower than 24 hours and [Rate Limits](https://www.scaleway.com/en/docs/generative-apis/reference-content/rate-limits/) (larger than Standard processing) still apply.
- When using batch processing, we handle scheduling of batch jobs to optimize both processing resource allocation and processing time. Processing time is therefore only guaranteed to be lower than 24 hours and [rate limits](/generative-apis/reference-content/rate-limits/) (larger than Standard processing) still apply.


## Monitoring

Monitoring is an essential pillar to ensure the security and reliability of your services. The practice provides real-time insights into the performance, security, and resource consumption of your Generative API usage.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Monitoring is an essential pillar to ensure the security and reliability of your services. The practice provides real-time insights into the performance, security, and resource consumption of your Generative API usage.
Monitoring is an essential pillar to ensure the security and reliability of your services. The practice provides real-time insights into the performance, security, and resource consumption of your usage of Generative APIs.


### Metrics and logs

Generative APIs metrics and logs are stored inside [Cockpit](https://www.scaleway.com/en/docs/cockpit/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Generative APIs metrics and logs are stored inside [Cockpit](https://www.scaleway.com/en/docs/cockpit/).
Generative APIs metrics and logs are stored inside [Cockpit](/cockpit/).


This includes:
- **Metrics**: Input and Output tokens and API requests . Metrics are refreshed every minute (some dashboards may aggregate data by the hour for accuracy reasons, but metrics can be queried at a finer rate using Cockpit Custom Dashboards)
- **Logs**: No logs are currently stored inside Cockpit.
Comment on lines +55 to +56
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Metrics**: Input and Output tokens and API requests . Metrics are refreshed every minute (some dashboards may aggregate data by the hour for accuracy reasons, but metrics can be queried at a finer rate using Cockpit Custom Dashboards)
- **Logs**: No logs are currently stored inside Cockpit.
- **Metrics**: Input and output tokens and API requests. Metrics are refreshed every minute (some dashboards may aggregate data by the hour for accuracy reasons, but metrics can be queried at a finer rate using Cockpit custom dashboards)
- **Logs**: No logs are currently stored inside Cockpit.


## Configuration and version management

Configuration and version management are critical for maintaining reliability and performance across your services.

### Configuration

Currently, Generative APIs do not provide specific configuration properties stored within your account. All configuration parameters are the one you send through each API HTTP calls (such as `temperature`, `top_p` or `seed`) and you remain responsible for any change in outputs based on these parameters.

Since Generative AI models are by definition non-deterministic, we cannot guarantee the same input will provide the same output over time (for example when used two different HTTP calls). If you want deterministic processing, we encourage you to use Managed Inference with a specific model and set all randomness parameters to deterministic level (for example using for instance `temperature`:`0` and a specific `seed` value).
Comment on lines +64 to +66
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Currently, Generative APIs do not provide specific configuration properties stored within your account. All configuration parameters are the one you send through each API HTTP calls (such as `temperature`, `top_p` or `seed`) and you remain responsible for any change in outputs based on these parameters.
Since Generative AI models are by definition non-deterministic, we cannot guarantee the same input will provide the same output over time (for example when used two different HTTP calls). If you want deterministic processing, we encourage you to use Managed Inference with a specific model and set all randomness parameters to deterministic level (for example using for instance `temperature`:`0` and a specific `seed` value).
Currently, Generative APIs do not provide specific configuration properties stored within your account. All configuration parameters are the ones you send through each API HTTP call (such as `temperature`, `top_p` or `seed`) and you remain responsible for any change in outputs based on these parameters.
Since Generative AI models are by definition non-deterministic, we cannot guarantee the same input will provide the same output over time (for example when using two different HTTP calls). If you want deterministic processing, we encourage you to use [Managed Inference](/managed-inference) with a specific model and set all randomness parameters to deterministic levels (for example using for instance `temperature`:`0` and a specific `seed` value).


### Version management

#### Supported models

Any changes to supported models and associated guarantees are detailed in our [Model Lifecycle Policy](https://www.scaleway.com/en/docs/generative-apis/reference-content/model-lifecycle/).

Comment on lines +72 to +73
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Any changes to supported models and associated guarantees are detailed in our [Model Lifecycle Policy](https://www.scaleway.com/en/docs/generative-apis/reference-content/model-lifecycle/).
Any changes to supported models and associated guarantees are detailed in our [model lifecycle policy page](/generative-apis/reference-content/model-lifecycle/).

#### API versions

Two types of API version updates may be performed:

| Upgrade Type | Description|
| ------------ | ------------------- |
| Minor | These updates do not change the API current fields format and are backward compatible (no action is required on your side). New fields and feature can however be added. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| Minor | These updates do not change the API current fields format and are backward compatible (no action is required on your side). New fields and feature can however be added. |
| Minor | These updates do not change the API's current fields format and are backward compatible (no action is required on your side). New fields and features can however be added. |

| Major | These updates change the API current fields or path. They may require action from your side. In this case, we will notify you with at least 3 month notice before deprecating significant feature that might break your application. |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| Major | These updates change the API current fields or path. They may require action from your side. In this case, we will notify you with at least 3 month notice before deprecating significant feature that might break your application. |
| Major | These updates change the API's current fields or paths. They may require action from your side. In that case, we will notify you with at least 3 months' notice before deprecating significant features that might break your application. |


#### Third party tools compatibility

By following industry standards (such as targeting OpenAI API compatibility), we aim to provide compatibility with most AI ecosystem and tools by default. However, as ecosystem evolves quickly, we cannot guarantee strictly compatibility with third party tools, but provide extensive documentation:
- Current API supported features are available in our API Documentation
- Advanced errors and edge cases workaround in our [Troubleshooting section](https://www.scaleway.com/en/docs/generative-apis/troubleshooting/fixing-common-issues/).
- Integration with third party tools is available in [Integrating Scaleway Generative APIs with popular AI tools](https://www.scaleway.com/en/docs/generative-apis/reference-content/integrating-generative-apis-with-popular-tools/#openai-client-libraries)
Comment on lines +85 to +88
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
By following industry standards (such as targeting OpenAI API compatibility), we aim to provide compatibility with most AI ecosystem and tools by default. However, as ecosystem evolves quickly, we cannot guarantee strictly compatibility with third party tools, but provide extensive documentation:
- Current API supported features are available in our API Documentation
- Advanced errors and edge cases workaround in our [Troubleshooting section](https://www.scaleway.com/en/docs/generative-apis/troubleshooting/fixing-common-issues/).
- Integration with third party tools is available in [Integrating Scaleway Generative APIs with popular AI tools](https://www.scaleway.com/en/docs/generative-apis/reference-content/integrating-generative-apis-with-popular-tools/#openai-client-libraries)
By following industry standards (such as targeting OpenAI API compatibility), we aim to provide compatibility with most AI ecosystems and tools by default. However, as ecosystems evolve quickly, we cannot guarantee compatibility with third party tools, but provide extensive documentation:
- Current API supported features are available in our API Documentation
- Advanced errors and edge cases workarounds in our [Troubleshooting section](/generative-apis/troubleshooting/fixing-common-issues/).
- Integration with third party tools is available in our [dedicated documentation page](/generative-apis/reference-content/integrating-generative-apis-with-popular-tools/#openai-client-libraries)


## Data protection

Our Data protection measures are detailed in our [Privacy Policy](https://www.scaleway.com/en/docs/generative-apis/reference-content/data-privacy/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Our Data protection measures are detailed in our [Privacy Policy](https://www.scaleway.com/en/docs/generative-apis/reference-content/data-privacy/).
Our data protection measures are detailed in our [privacy policy page](/generative-apis/reference-content/data-privacy/).


Overall:
- We do not store sensitive data (such as the content of your prompt), unless we need to, to provide the service (such as temporarily when using Batch processing).
- When data is stored, it is protected using state of the art method (such as encryption at rest).
- During transit, your data is encrypted by relying on HTTPS protocol.
Comment on lines +94 to +97
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Overall:
- We do not store sensitive data (such as the content of your prompt), unless we need to, to provide the service (such as temporarily when using Batch processing).
- When data is stored, it is protected using state of the art method (such as encryption at rest).
- During transit, your data is encrypted by relying on HTTPS protocol.
- Scaleway does not store sensitive data (such as the content of your prompt), unless we need it to provide the service (such as when using batch processing)
- When data is stored, it is protected using a state of the art method (such as encryption at rest)
- During transit, your data is encrypted by relying on the HTTPS protocol


### Scaleway access

In order to perform maintenance operations and guarantee the reliability of Generative APIs, or comply with local regulations, we need to access servers hosting Generative APIs service.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
In order to perform maintenance operations and guarantee the reliability of Generative APIs, or comply with local regulations, we need to access servers hosting Generative APIs service.
In order to perform maintenance operations and guarantee the reliability of Generative APIs, or comply with local regulations, we need to access servers hosting the Generative APIs service.


Most of these accesses are automatic actions, in cases where Scaleway needs update configuration or upgrade software versions.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Most of these accesses are automatic actions, in cases where Scaleway needs update configuration or upgrade software versions.
Most of the time, they are automatic actions, when Scaleway needs to update configuration or upgrade software versions, for example.


Manual interventions might be required occasionally, and for troubleshooting reasons (such as specific customer requests generating errors or performing malicious activity), we may access temporarily complete HTTP requests content to identify root cause issue or any security vulnerability. All Scaleway accesses are authenticated and traced following industry security standards.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Manual interventions might be required occasionally, and for troubleshooting reasons (such as specific customer requests generating errors or performing malicious activity), we may access temporarily complete HTTP requests content to identify root cause issue or any security vulnerability. All Scaleway accesses are authenticated and traced following industry security standards.
Manual interventions might be required occasionally, and for troubleshooting reasons (such as specific customer requests generating errors or performing malicious activity), we may temporarily access the content of HTTP requests to identify a root cause issue or any security vulnerability. All Scaleway accesses are authenticated and traced following industry security standards.


## Identity and access management

Identity and access management allows you to enable granular control over user permissions and to mitigate the risk of unauthorized access or data breaches.

All accesses to Generative APIs are authenticated and authorized relying on [Scaleway IAM permissions sets](https://www.scaleway.com/en/docs/iam/reference-content/permission-sets/).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
All accesses to Generative APIs are authenticated and authorized relying on [Scaleway IAM permissions sets](https://www.scaleway.com/en/docs/iam/reference-content/permission-sets/).
All accesses to Generative APIs are authenticated and authorized relying on [Scaleway IAM permissions sets](/iam/reference-content/permission-sets/).


You are responsible for attributing these permissions to the relevant users or applications and reviewing these accesses frequently.

## Compliance

Several regulations apply to us directly, whereas other apply to your usage. Even in this case, we help you ease your compliance process by providing you with the information you need from your Cloud Provider.

Comment on lines +117 to +118
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Several regulations apply to us directly, whereas other apply to your usage. Even in this case, we help you ease your compliance process by providing you with the information you need from your Cloud Provider.
Several regulations apply to us directly, whereas others apply to your usage. Even in this case, we help you ease your compliance process by providing you with the information you need from your cloud provider.

### AI Act

We (Scaleway) ensure our compliance with the AI Act within our scope of responsibilities. We also ensure that you have the information needed to comply with the requirements that apply to you. This means concretely:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provide link to the AI Act (this one maybe? https://artificialintelligenceact.eu/)

- Gathering AI Model Providers information about their model (such as whether its training capacity is above 10²⁵ FLOPs, and falls into a specific category) and provide you with a link to these documents when they are made available by AI Model Providers.
- Providing you with links towards licensing required by the AI Model Providers.
Comment on lines +122 to +123
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Gathering AI Model Providers information about their model (such as whether its training capacity is above 10²⁵ FLOPs, and falls into a specific category) and provide you with a link to these documents when they are made available by AI Model Providers.
- Providing you with links towards licensing required by the AI Model Providers.
- Gathering AI Model Providers information about their models (such as whether its training capacity is above 10²⁵ FLOPs, and falls into a specific category) and provide you with a link to these documents when they are made available by AI Model Providers.
- Providing you with links towards licensing required by the AI Model Providers.


Scaleway has no access to, nor knowledge of, any inputs and outputs generated by the Models. By using our AI products, you agree and acknowledge that you are (i) responsible for this use including any content integrated into the Models, and (ii) required to use the AI products in compliance with our General Terms of services.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Scaleway has no access to, nor knowledge of, any inputs and outputs generated by the Models. By using our AI products, you agree and acknowledge that you are (i) responsible for this use including any content integrated into the Models, and (ii) required to use the AI products in compliance with our General Terms of services.
Scaleway has no access to, nor knowledge of any inputs and outputs generated by the models. By using our AI products, you agree and acknowledge that you are (i) responsible for this use including any content integrated into the models, and (ii) required to use the AI products in compliance with our General Terms of services.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Provide link to the General terms of services


As a client of our AI products, you are likely to be considered an AI System Provider or Deployer under the AI Act. As such, it is your responsibility to ensure you comply with requirements that apply to you.

### Additional Local Regulation
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Additional Local Regulation
### Additional local regulation


If you require additional information to comply with specific regulation, you can create a [support ticket](https://console.scaleway.com/support/tickets/create) or contact your account manager.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you require additional information to comply with specific regulation, you can create a [support ticket](https://console.scaleway.com/support/tickets/create) or contact your account manager.
If you require additional information to comply with specific regulations, you can create a [support ticket](https://console.scaleway.com/support/tickets/create) or contact your account manager.