Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QueryLog Interface #5099

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Conversation

elee1766
Copy link

@elee1766 elee1766 commented Nov 4, 2022

I have been looking for a while to reduce the footprint of the logs on my machine, while still keeping them.

this pr is the start of that work, and does the following:

  1. splits querylog package into two, "logs", and "jsonfile". jsonfile contains things specifically related to the existing on-disk format, while logs includes http handler, interfaces, and shared structs for a "logs.Api implementation" to use.
  2. changes the HTTP to use a properly tagged struct with omitempty tags which then marshals to json, instead of populating a map.
  3. Extends the "Store" interface to include methods "ApplyConfig", "ConfigInfo", and "Clear", interface "Searcher" which uses struct "SearchParams" to get result "LogsPayload", and interface "Api", a Search + Store.

my idea is that one could implement the "Store" and "Searcher" interfaces for different sorts of storage and indexing backends.

This would allow people to create their own log storage & search implementations. looking at #2290, it seems that people are eager to have custom log sinks, but the existing implementation makes it hard to add a new one.

I think more thought needs to be put into an actual interface, along with the configuration format, but I do not feel like the qualified person to do that. so my goal in this PR was mostly to keep compatibility with the existing logging implementation, keeping the changes simple. hopefully we can move from here to something better. that said, things have to start somewhere, so I thought I would make the PR anyways.

new to this project, so if i did anything wrong, please let me know. happy to make any changes / take any suggestions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant