Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Officially support aggregator Starlark scripts #8869

Closed
ssoroka opened this issue Feb 16, 2021 · 0 comments · Fixed by #9419
Closed

Officially support aggregator Starlark scripts #8869

ssoroka opened this issue Feb 16, 2021 · 0 comments · Fixed by #9419
Labels
area/starlark feature request Requests for new plugin and for new features to existing plugins rfc Request for comment - larger issues that require clarification

Comments

@ssoroka
Copy link
Contributor

ssoroka commented Feb 16, 2021

Feature Request

support calling push() in Starlark scripts so that Starlark processor scripts can fully implement all the features that aggregators would need.

Proposal:

allow running starlark as an aggregator. this would probably mean supporting something like this:

[[aggregators.starlark]]
  period = "30s"
  delay = "5m"
  grace = "5m" 
  drop_original = true
  source = '''
state = {}

def apply(metric): # or should this be named add() like the aggregators?
    last = state["last"]
    if last != None:
        metric.fields["value"] = last.fields["value'] + metric.fields["value"]
    state["last"] = deepcopy(metric) # do I need to copy it if I'm not returning it?
    return None

def push(): # called when we push this group of metrics down the line
    last = state["last"]
    state["last"] = None
    return last
'''

(see Aggregator Plugins for details on aggregator settings)

Current behavior:

There is no call to push, and as such it's not possible to guarantee the aggregate record will be written out in the right aggregation window. This can be problematic for infrequently collected metrics.

Desired behavior:

support writing aggregators in Starlark.

Use case:

People have asked for this, and will continue to build it in processors if we don't add it, though not ideal.

@ssoroka ssoroka added feature request Requests for new plugin and for new features to existing plugins rfc Request for comment - larger issues that require clarification labels Feb 16, 2021
@sspaink sspaink removed their assignment Mar 4, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/starlark feature request Requests for new plugin and for new features to existing plugins rfc Request for comment - larger issues that require clarification
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants