Skip to content

MySQL task resource usage storage #1928

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
ssalinas merged 21 commits into from
May 2, 2019
Merged

MySQL task resource usage storage #1928

ssalinas merged 21 commits into from
May 2, 2019

Conversation

@ssalinas
Copy link
Member

@ssalinas ssalinas commented Apr 29, 2019 •
edited
Loading

Last (for now) in the 'be nice to zookeeper' series. We store cluster, slave, request, and task level resource usage, currently all in zookeeper. This does a few things to make that more scalable:

  • For request/slave we only ever need the current. Stop saving history, and make sure recent slave usages are also leader cached
  • Task usages, if in zk, need to be namespaced by request id to avoid extra large list children calls of all task ids
  • Task usages will be stored in mysql if it is configured, which will require a migration to be run to create the table
  • A zk migration has been added to clean all current usage data. This data will be rewritten the next time the poller runs and is not required for scheduling
  • task usage timestamps are now stored in millis instead of seconds
  • usage/slaves/{slaveId}/tasks/current endpoint is removed

baconmania
baconmania reviewed Apr 29, 2019
View reviewed changes
SingularityBase/src/main/java/com/hubspot/singularity/SingularityTaskUsage.java Outdated

private final long memoryTotalBytes;
private final double timestamp; // seconds
private final long timestamp; // seconds
Copy link
Contributor

@baconmania baconmania Apr 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should update the comment here

Copy link
Member Author

@ssalinas ssalinas Apr 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, missed that

baconmania
baconmania reviewed Apr 29, 2019
View reviewed changes
SingularityService/src/main/java/com/hubspot/singularity/data/usage/TaskUsageJDBI.java Outdated

public abstract class TaskUsageJDBI {
static final String FILEDS = "requestId, taskId, memoryTotalBytes, timestamp, cpuSeconds, diskTotalBytes, cpusNrPeriods, cpusNrThrottled, cpusThrottledTimeSecs";
static final String FILED_VALUES = ":requestId, :taskId, :memoryTotalBytes, :timestamp, :cpuSeconds, :diskTotalBytes, :cpusNrPeriods, :cpusNrThrottled, :cpusThrottledTimeSecs";
Copy link
Contributor

@baconmania baconmania Apr 29, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: spelling

@ssalinas ssalinas added the hs_qa label Apr 30, 2019
@baconmania
Copy link
Contributor

baconmania commented May 1, 2019

🚢

@ssalinas ssalinas merged commit 2402934 into master May 2, 2019
@ssalinas ssalinas deleted the mysql_usage_storage branch May 2, 2019 14:46
@ssalinas ssalinas added this to the 0.23.0 milestone Jun 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@baconmania baconmania baconmania left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

0.23.0

Development

Successfully merging this pull request may close these issues.

3 participants

@ssalinas @baconmania