Development roadmap

[JMuff22]

Hi, first of all love the project! We are currently implementing this as a standalone scheduler and liking it so far!

I am wondering if the developers would like to share a possible roadmap? We are using hyperqueue standalone and we would be interested in potential upcoming features? For example:

• REST API implementation and use of JWT
• Tied into a central accounting DB

Hope to hear from you. Thanks!

[Kobzol]

Hi :) We usually implement new features on a use-case basis, based on what feature requests users (or various projects where HQ is used) bring. So the current roadmap is basically the union of all opened issues. For more specific and short term goals, we want to improve the behaviour of the autoallocator, extend resource support (add support for alternative and partial resources) and possibly add some basic server resilience.

Could you tell us a bit more about the use cases that you have mentioned?

[JMuff22]

Thanks for the info! Our use case is as a standalone scheduler for a Quantum Computer, so hearing about extending resource support is great! However, we recognise that Hyperqueue was more intended for usage along side SLURM/PBS therefore it is understandable that the development may not go towards this particular use case.

For us it would be great if Hyperqueue could accept jobs via HTTP requests from multiple source with JWT integration. And if the resource usage could be tied into some database.

[spirali]

For REST service you can write a simple flask app like this (pseudocode):

@app.route('/hq', methods = ['POST'])
def call_hq(args):
	return call_program_and_return_stdout("hq", "--output-mode=json", *args)

And you can basically have HQ via Rest API. Option "--output-mode=json" will cause that you will get JSON responses on stdout. In this way you can add your own authrization and authentication as you like. (to be honest, I do not know what is JWT:).

For the second question. HQ now produces event log where you can read task starts and finishes so it can be used for accounting. So far, we store log into a file, but we can support a unix socket. Then you can create a simple script that redirects events into some database.

[Kobzol]

To answer more broadly, we try to avoid feature creep and only add features that really have to be in the "core", either for performance reasons (most often) or when they form a core functionality of HQ that wouldn't be possible to implement externally.

We acknowledge that some people are using HQ "programmatically", and we are happy to improve support for better interfacing with such tools. Currently, we have machine-readable output (JSON) for CLI commands, we provide a Python API and soon we will add support for submitting tasks via a task data model (e.g. written in a YAML/TOML file). We can also add new forms of API interfacing if there would be a use-case for it (e.g. a C API). HQ can also export a JSON log of events like task submissions, PBS allocation start etc. to a file, which can then be read by external tools.

However, for features that can be reasonably implemented "outside" of HyperQueue, we prefer to keep them in an external tool. One of the design principles of HQ is to be simple to deploy, and adding things like HTTP servers into the binary goes slightly against this idea. While it's true that using Rust helps us avoid external dependencies, at the very least such features would increase the binary size (perhaps nontrivially).

If someone implements an external feature that is popular and used by more people, we could e.g. create some hyperqueue-ext repository with external extensions for HQ that would be "blessed" by us. And if it gets really popular, we might think about including it in HQ directly. But for now a REST API probably stays in the "best implemented externally" category.

Btw, we will have a quantum computer at our supercomputing server soon too and HQ might be a part of a solution to schedule tasks for it, so maybe this use-case will not be so niche :)

[JMuff22]

Btw, we will have a quantum computer at our supercomputing server soon too and HQ might be a part of a solution to schedule tasks for it, so maybe this use-case will not be so niche :)

This was precisely my reason for contacting. We would be very interested in such a solution for scheduling tasks and as we have a current system up and running I do not mind testing and developing this on a real machine :D

Thanks for all the information! This is very helpful indeed and what @spirali has written is precisely the current thinking on the implementation, it was just unclear if this would be further incorporated in the future.

Hope we can be in further contact!

[spirali]

For completeness. In general, we do not want to be dependent on 3rd party libraries that cannot be statically linked. There are purely rust performant http servers, so it may be possible that in a longer horizon we may implement a REST service. And the binary would not be that bigger as we may need http server for some other reasons (eg. serving some http dashboard).

On the other hand, for the second problem: We will not support a direct connect to a database as long as there is no pure Rust controller for that database. And as far as I know, there is no such thing for any major DB (they always link some dynamic C library)