Setup to publish benchmark results to nodejs.org

[mhdawson]

I've been doing some experimentation as described in this issue: nodejs/benchmarking#23

I'd like to get the required setup to publish benchmark graphs to nodejs.org, possibly using benchmarking.nodejs.org as the main entrypoint.

The machine generating the data will need to be able to push the graphs to nodejs.org on a nightly basis. Talking with @jbergstroem it sounds like we've done something similar and would set up a public key to be used to do the pushes.

[rvagg]

There's two approaches we could take here:

1. set up benchmarking account and point benchmarking.nodejs.org to at directory in the home for that and you could put what you need in there
2. replicate what we do for the website, i.e. watch a repo (or some other trigger) for changes and rebuild the site within a container

The nice thing about #2 is that we isolate the server more than if we make a new account that can be used. Personally I'd rather us keep non-build-core users off the server so the more we can do at a distance the better. Could the data be posted back to a GitHub repository that can be used as a basis for building the site?

For context, this is how we do it with the website, nobody from the website group needs any kind of direct access and any activity performed by their scripting is done within a container so it's mostly isolated: https://github.com/nodejs/build/blob/master/setup/www/resources/scripts/build-site.sh#L36-L59

[jbergstroem]

@rvagg you're right; the second approach is probably better.

[mhdawson]

@rvagg just to confirm I understand. For 2 we'd publish the contents for benchmarking.nodejs.org to a specific directory in the nodejs/benchmarking repository on github which we'd update daily and then this would be automatically replicated onto the website.

In this case we'd setup a public key so that change could be pushed from the benchmark machines to the nodejs/benchmarking repo on github, right ? If so, do we do anything special when setting that up or is it just a dummy github user (for example node-benchmarking) that we add and give access to the repo ?

[mhdawson]

Ok so based on this and discussion with @jbergstroem I think the plan is to do the following:

1. mirror from https://github.com/nodejs/benchmarking/tree/master/www to the www site
2. make nodejs.org/benchmarking point to the mirrored contents
3. at least daily have the www server pull all of the pngs from /home/benchmark/charts on the benchmark data machine (currently 50.97.245.4) as user benchmark and put then into the charts directory in the mirror from https://github.com/nodejs/benchmarking/tree/master/www on the www server.

This would allow the benchmark team to update the static www pages and also make the latest charts available on the www server for reference by those static pages.

For more info on the planned overall flow/benchmarking infrastructure see https://github.com/nodejs/benchmarking/blob/master/benchmarks/README.md

[rvagg]

sounds good, let us know when this is ready to happen and we can generate an ssh keypair for it and get it rolling.

[mhdawson]

@rvagg I'd like to get the mirroring going now as the graphs should have been generating for while and getting them onto the website even with the index will let us see them and provide the basis that we can start creating the index.

[mhdawson]

Anything I can do to help on this one ?

[mhdawson]

Have temporarily setup jobs to push to here http://nodejs.devrus.com:8080/

• cronjob on benchmark machines pushes updated charts nightly (as opposed to pull which is planed for node webiste)
• repo is pulled every 15 mins to update www content

[rvagg]

Done, I think. Server setup including 6-hourly sync (I don't know what "overnight" means tbh so this is will get it done!) plus github webhook for the benchmarking repo so when you update it there it'll get synced on the server and put in to place.

#344

https://benchmarking.nodejs.org/

You might want to ask @nodejs/website to have a go at your design since this is now a public presence.

[jbergstroem]

Design? This is perfect! 🎉