-
Notifications
You must be signed in to change notification settings - Fork 64
Add RSE configuration guide for operators #710
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
|
Overall all these changes are really good!! Only thing I didn't comment on directly in the body of the review is that we might want to mention how rse specific limits work vs account only limits |
|
@voetberg Added a few fixes in 317c0db. Also, I stumbled upon https://rucio.github.io/documentation/operator/configuration/#creating-new-rses; should I add a link to there pointing to this page? |
I would say this PR completely supersedes that page, and I would link to this page and drastically reduce what's on the config params page. (Maybe just summarizing into a TL;DR with "add rse", "add rse attribute", "add rse protocol", "add account limit") If you would like, I can take that over and you can just link your page there with something like "An in-depth guide to configuring RSEs can be found here" |
Cool, then I'll let you take care of the summary, while I'll just link this page in there. As soon as I have some info about |
|
@Soap2G , there seem to be existing section in doc title "Creating new RSEs" Quota stuff is also discussed in here: https://rucio.cern.ch/documentation/operator/configuration/#setting-quota-and-permissions We should consolidate the two and have a single place to have these information. So I would think: |
See this comment; Maggie will take care of that once the page is up. |
|
@voetberg @panta-123 We should be ready to go, with the reminder of merging the redundant RSE-related pages after this is done. |
|
@Soap2G Please rebase to grab the pre-commit ci! |
Documentation on setting up and configuring Rucio Storage Elements (RSEs) from an operator's perspective. Includes: - Overview of RSE types (POSIX, WebDAV, Disk, Tape) - Two setup methods: CLI and Python API with side-by-side examples - Configuration examples for each RSE type - WebDAV setup with Apache configuration and davs protocol - EOS disk RSE with https and root protocols - CTA tape RSE configuration with staging timeouts - RSE attributes, protocols, and account limits reference - Best practices and common pitfalls - Quick reference commands The examples use the latest `rucio` CLI commands Co-authored-by: Nikita Avdeev <naavdeev.astro@gmail.com> Co-authored-by: Luis Antonio Obis Aparicio <luis.antonio.obis@gmail.com>
…nstead of core Used RSEClient class for rse operations, and AccountLimitClient for account. Additionally added a paragraph about configuration concepts.
Removed istape from RSE config guide, as it's not needed by Rucio and it can be replaced by rse_type. Additionally, added a clearer description of istape in the attributes page.
d46af70 to
f80b901
Compare
voetberg
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm - nothing seems obviously wrong and if people want to add more examples we can do that in the future
|
@panta-123 Do you want to give this a read-over and review? |
This commit consolidates multiple documentation corrections and clarifications: - Corrected RSE settings vs attributes distinction - Fixed TypedDict field listings to match actual implementation - Removed non-existent protocol priority fields (priority_lan, priority_wan) - Clarified that lfn2pfn_algorithm is an RSE attribute, not a creation parameter - Restored geographic fields (city, country_name, latitude, longitude, region_code, time_zone) as valid RSE settings that can be set via gateway API - Clarified deterministic vs non-deterministic RSE behavior - lfn2pfn_algorithm: for deterministic RSEs (disk), computes paths from scope+name only - naming_convention: for non-deterministic RSEs (tape), uses metadata/timestamps - Added detailed comparison table explaining the differences - Updated all examples to reflect correct usage patterns - Fixed Python API examples** - Removed incorrect lfn2pfn_algorithm parameter from add_rse() calls - Shows correct attribute-based configuration via add_rse_attribute() - Updated workflow examples to match actual client implementation - CLI fixes - Minor fixes to commands structure
|
Thanks to @Geogouz for the AI-assisted review. It was very useful to spot some inconsistencies in the text. I've also slightly updated Reviews are welcome @voetberg @panta-123 |
At Rucio's service upon request :D. For reference, here is what the output was in case others would like to use it too in the future. May not be perfect, but I would say it clearly does more good than harm to have it as an additional review opinion: 1) “RSE Attributes” section mixes up settings vs attributesInaccurate / misleading in the guide
What’s correct
Why this matters: setting 2) The “Mandatory attributes” danger box is incorrectInaccurate
What’s correct
3) Protocol priority rules in the guide are wrong / misleadingInaccurate
Misleading / needs tightening
What’s correct directionally
4) CLI examples don’t match the current
|
| The RSE settings are set separately using `rucio.RSEClient.update_rse` or `rucio rse update`, and specifies RSE configuration used by the Rucio instance. | ||
| The RSE settings are set separately using `rucio.RSEClient.update_rse` or `rucio rse update`, and specify RSE configuration used by the Rucio instance. | ||
| Mutable settings are `deterministic`, `rse_type`, `staging_area`, `volatile`, `qos_class`, `availability_delete`, `availability_read`, `availability_write`, `city`, `country_name`, `latitude`, `longitude`, `region_code`, and `time_zone`. | ||
| Geographic fields (`city`, `country_name`, `latitude`, `longitude`, `region_code`, `time_zone`) can also be set at RSE creation time via API parameters, though they are stored and returned as part of the RSESettingsDict structure. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The wording here is really confusing (and sort of redundant). Do we mention the RSESettingsDict anywhere else? Is this construct useful for an operator to use or is just part of how they're displayed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed the last part of the sentence.
| - **fts**: String. Specify the REST API URL of the FTS3 transfer manager. No default. | ||
| - **greedyDeletion**: Boolean. Allow files without a rule locking them to be deleted by a Reaper Daemon. Default behavior only marks a file for deletion when there is no space on an RSE for a new required file. Default: `False`. | ||
| - **group_by_rse_attribute**: String. Control the RSE attribute (such as `country`) which transfer source RSEs will be grouped by when determining an appropriate transfer source. Default: `UNKNOWN`. | ||
| - **lfn2pfn_algorithm**: String. Name of the algorithm to be used for generating paths to files on **deterministic RSEs**. Must be defined in the configured policy package. If not set, defaults to `lfn2pfn_algorithm_default` from the `[policy]` section of the config file. Common values: `identity`, `hash`. Default: `default`. Note: This attribute is also included in the RSE settings dictionary when protocols are retrieved. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is the re-ordering just to put it in alphabetic order?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Part of the edits were to standardise the format (name:type. instead of name:type:). This one specifically was a leftover, I've removed it since lfn2pfn_algorithm is an attribute
|
|
||
| # Set backend type | ||
| rucio rse attribute add POSIX_RSE --key backend_type --value POSIX | ||
| rucio rse attribute add --key backend_type --value POSIX POSIX_RSE |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fwiw, placement of the RSE name here doesn't matter so either approach is fine (be it rucio rse attribute add POSIX_RSE --key backend_type --value POSIX or rucio rse attribute add --key backend_type --value POSIX POSIX_RSE)
I don't think we have a consistent style for how we write CLI commands bc it's sort of a free-for-all. I guess adding to the comment on 149 to say "Add the RSE, named POSIX_RSE" would avoid all possible confusion
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, I looked at the help from the CLI and since it looks like:
Usage: rucio rse add [OPTIONS] RSE_NAME
So I've preferred to stick to the order that it's in there. Good to know that order doesn't matter in any case.
docs/operator/rse_configuration.md
Outdated
| - `rse_type` is set to `TAPE` | ||
| - `archive_timeout` attribute specifies maximum time for file staging (86400 = 24 hours) | ||
| - `rse_type` is set to `TAPE` at creation time | ||
| - `archive_timeout` attribute controls maximum time for tape archival operations (86400 = 24 hours) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"file staging" is more precise. There are steps taken after the file is staged as part of the archiving process that Rucio wouldn't know about
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Rolled back to the previous phrasing
docs/operator/rse_configuration.md
Outdated
| - Tape systems with internal file organization requiring metadata-driven naming | ||
| - Storage systems that don't support directory structures | ||
| - Systems requiring custom naming schemes | ||
| - Systems where physical file names must be generated using business logic beyond scope/name |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"Business logic" here feels sticky for a reason I can't place. Either way, this says functionally the same thing as point 1
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Removed the bullet point
docs/operator/rse_configuration.md
Outdated
| For non-deterministic RSEs: | ||
| 1. Physical file paths must be registered explicitly during replication | ||
| 2. Files cannot be uploaded directly by clients (only via replication) | ||
| 3. The `naming_convention` attribute must reference a `non_deterministic_pfn` algorithm in your policy package |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would be a good time to link the policy package docs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
docs/operator/rse_configuration.md
Outdated
|
|
||
| - Set reasonable `archive_timeout` values (24 hours recommended) | ||
| - Use staging-aware clients | ||
| - Set reasonable `archive_timeout` values (24 hours recommended). This controls how long FTS waits for tape archival operations to complete |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Points again to this comment - a5a0b21#r2760241802
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Done.
docs/operator/rse_configuration.md
Outdated
| # 2. Add attributes | ||
| rse_client.add_rse_attribute('RSE_NAME', 'rse_type', 'DISK') | ||
| rse_client.add_rse_attribute('RSE_NAME', 'attribute_name', 'attribute_value') | ||
| rse_client.add_rse_attribute('RSE_NAME', 'lfn2pfn_algorithm', 'identity') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry this comment is so late in the review process. Would it be better to use full arg names so it's clear what is being set and how here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good point, done.
docs/operator/rse_configuration.md
Outdated
| ### RSE Settings vs Attributes | ||
|
|
||
| RSE attributes are key-value pairs that control the behavior and capabilities of an RSE. They define how Rucio interacts with the storage system. | ||
| Rucio distinguishes between **RSE settings** (properties of the RSE record itself) and **RSE attributes** (key-value metadata). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
RSE "record"?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, with record I wanted to say the RSE itself, but doesn't make much sense and it's redundant. Removed.
docs/operator/rse_configuration.md
Outdated
| ::: | ||
|
|
||
| An exhaustive list of RSE attributes can be found in the [RSE attributes page](configuration_parameters/#rse-attributes). | ||
| An exhaustive list of RSE attributes can be found in the [configuration parameters page](configuration_parameters/#rse-settings). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Linking to the settings section but talking about attributes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typo, I forgot to update it after copying the line from above. Fixed.
Closes #709
Documentation on setting up and configuring Rucio Storage Elements (RSEs) from an operator's perspective. Includes:
Overview of RSE types (POSIX, WebDAV, Disk, Tape)(removed in f80b901)WebDAV setup with Apache configuration and davs protocol(removed in 040ee73)The examples use the latest
rucioCLI commands