Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[filelogreceiver]: Add ability to sort by mtime #28691

Closed
wants to merge 4 commits into from
Closed

[filelogreceiver]: Add ability to sort by mtime #28691

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
46d20f8
add ordering by mtime
BinaryFissionGames Oct 26, 2023
7dc355a
add option to readme
BinaryFissionGames Oct 27, 2023
5b28b60
add changelog entry
BinaryFissionGames Oct 27, 2023
5ba5044
add featuregate for mtime configuration
BinaryFissionGames Oct 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
add option to readme
  • Loading branch information
@BinaryFissionGames
BinaryFissionGames committed Oct 30, 2023
commit 7dc355a1737a6b912c8446f005b13844b9529a1d
3 changes: 2 additions & 1 deletion receiver/filelogreceiver/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,8 @@ Tails and parses logs from files.
| `retry_on_failure.initial_interval` | `1s` | [Time](#time-parameters) to wait after the first failure before retrying. |
| `retry_on_failure.max_interval` | `30s` | Upper bound on retry backoff [interval](#time-parameters). Once this value is reached the delay between consecutive retries will remain constant at the specified value. |
| `retry_on_failure.max_elapsed_time` | `5m` | Maximum amount of [time](#time-parameters) (including retries) spent trying to send a logs batch to a downstream consumer. Once this value is reached, the data is discarded. Retrying never stops if set to `0`.
| `ordering_criteria.regex` | | Regular expression used for sorting, should contain a named capture groups that are to be used in `regex_key`. |
| `ordering_criteria.method` | `regex` | Specifies the mode to use for ordering. Can be `regex` (sort using regex and sort_by rules) or `mtime` (sort by file modified time). |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this serve basically the same purpose as ordering_criteria.sort_by.sort_type? Why not add mtime as an additional option there?

This would also allow users to compose multiple types of ordering and filtering.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I got a little lost looking at the current implementation where every sort type requires the regex to be set, I'll take a stab at reworking to have to be a separate sort type.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New implementation for sort_type here: #28850

| `ordering_criteria.regex` | | Regular expression used for sorting, should contain a named capture groups that are to be used in `regex_key`. Ignored if `mode` is not `regex`. |
| `ordering_criteria.top_n` | 1 | The number of files to track when using file ordering. The top N files are tracked after applying the ordering criteria. |
| `ordering_criteria.sort_by.sort_type` | | Type of sorting to be performed (e.g., `numeric`, `alphabetical`, `timestamp`) |
| `ordering_criteria.sort_by.location` | | Relevant if `sort_type` is set to `timestamp`. Defines the location of the timestamp of the file. |
Expand Down