Add task to generate release body #233

csegarragonz · 2022-02-23T10:23:43Z

In this PR I introduce a task to generate the body for a new release.

We use the git-cliff project through their docker image to generate a changelog between the current tag and the latest release's tag.

I have customised the format of the release body a bit to fit our Github usage (i.e. squash commits). This is orthogonal to any automation efforts we want to do in the future.

I paste what the release body would look like now for discussion (note that we are missing this actual PR, hence why we see three dangling commits at the beginning).

=====

Faabric's Changelog

Here is what has changed since last release:

[0.3.1] - 2022-02-23

Ci: generate release body with changelog
Ci: add step to generate changelog
Bumping code version for release
Detect CPU information (Detect CPU information #231)
Executor context (Executor context #228)
Conan cache improvements (Conan cache improvements #230)
Remove SGX-related message fields (Remove SGX-related message fields #229)
Master to main (Master to main #227)

[0.3.0] - 2022-02-11

Version bump (Version bump #226)
Generate documentation (Generate documentation #223)
Executor decision caching and single-host optimisations (Executor decision caching and single-host optimisations #222)
XOR diffing (XOR diffing #224)
Use conan action (Use conan action in GHA #225)
Use dirty page flags instead of dirty regions (Use dirty page flags instead of dirty regions #219)
Periodically Check For Migration Opportunities (Periodically Check For Migration Opportunities #214)
Add gdbinit config (Add gdbinit config #217)
Remove timed mutex from ptp broker (Remove timed mutex from ptp broker #220)
Cmake minimum version and skylake (CMake minimum version and skylake #218)
Cmake: quote conan revisions (CMake: quote conan revisions #213)
Dirty tracking performance improvements (Dirty tracking performance improvements #210)
Introduce Fixed-Capacity Queues (Introduce Fixed-Capacity Queues #205)
Merge regions overhaul (Merge regions overhaul #201)
Add in-proc PAIR sockets (Add in-proc PAIR sockets #206)
Use matrix build for sanitisers (Use matrix build for sanitisers #208)
No need to acquire a lock to set a message id in mpi (Don't use util::generateGid() for MPI messages #203)
Snapshots overhaul (Snapshots overhaul #176)

[0.2.3] - 2021-12-20

Pin Conan dependencies to revision (Pin Conan dependencies to revision #200)
Locality aware gather (Locality aware gather #198)
Return executed time with successful status request (Return JSON of result with successful status request #197)
Fix reduce bug and add regression test (Fix bug in MPI_Reduce #195)
More race condition fixes (More race condition fixes #193)
Locality Aware Reduce (Locality Aware Reduce #191)
Minor fixes (Minor fixes #192)
Locality Aware Broadcast (Locality Aware Broadcast #185)
Enable/disable scheduling topology hints through an env. variable (Enable/disable scheduling topology hints through an env. variable #188)
Re-use conan cache for built dependencies (Re-use conan cache for built dependencies #187)
Adding support for sanitised builds (Adding support for sanitised builds #182)
Retry GET from Redis for messages in the Execution Graph (Retry GET from Redis for messages in the Execution Graph #181)
Use NEVER_ALONE for chained MPI calls (Use NEVER_ALONE for chained MPI calls #184)
Add Scheduling Topology Hints (Add Scheduling Topology Hints #180)
Fix race condition in scheduler reset (Fix race condition in scheduler reset #179)
Get ExecGraph hosts (Get ExecGraph hosts #178)
Make sure conan packages get compiled with C++20 enabled (Make sure conan packages get compiled with C++20 enabled #177)
Track Message Type in Execution Graph (Track Message Type in Execution Graph #175)
Snapshot diff merging fixes (Snapshot diff merging fixes #174)
Scheduler hints and fixes for distributed coordination bugs (Scheduler hints and fixes for distributed coordination bugs #169)
Fixes for MPI message graph (Fixes for MPI message graph #173)
Adding arbitrary information to execution graph (Adding arbitrary information to execution graph #166)
Add distributed coordination operations (Add distributed coordination operations #161)
Use faster lld as the linker (Use faster lld as the linker #168)

[0.2.1] - 2021-10-27

Cmake Cleanup Faabric Part 2 (Cmake Cleanup Faabric Part 2 #165)
Cmake cleanup Faabric Part 1 (Cmake cleanup Faabric Part 1 #164)
Initialise point-to-point mappings from scheduler output (groundwork) (Initialise point-to-point mappings from scheduler output (groundwork) #160)
README flair (README badges #159)
Allow merge regions to cross pages (Allow merge regions to cross pages #158)
Allow generating execution graphs from MPI runs (Allow generating execution graphs from MPI runs #154)
Generalised point-to-point messaging (Generalised point-to-point messaging #151)
Fix issue with hanging shutdown when not really started (Fix issue with hanging shutdown when not started #155)
Remove sgx check from faabric runner (Remove SGX check from faabric runner #153)
Add snapshot if exists function to snapshot registry (Add snapshot if exists function to snapshot registry #152)
Multi-threaded server message endpoints (Multi-threaded server message endpoints #147)
Change default mpi base port and allow value to be set through env variable (Change default MPI_BASE_PORT and turn into env. variable #150)
Add ignored snapshot regions (Add ignored snapshot regions #148)
Add crash handler to tests (Add crash handler to tests #149)
Merge docker compose files (Merge docker compose files #146)
Add a crash handler that prints backtraces to FaabricMain users (Add a crash handler that prints backtraces to FaabricMain users #145)
Compare pointers rather than ids in Scheduler::notifyExecutorShutdown (Compare pointers rather than ids in Scheduler::notifyExecutorShutdown #144)
Snapshot diff merge operations (Snapshot diff merge operations #142)
Not use emplace for editing vector elements (Not use emplace for editing vector elements #143)
Specify timeouts when timing out (Log timeouts when timing out #140)
Add snapshotExists function to registry (Add snapshotExists function to registry #136)
Set appid on messages (Set appid on messages #137)
Fix race condition in test (Fix race condition in executor test #138)
Latency fixes (Latency fixes #135)

csegarragonz · 2022-02-23T12:49:05Z

tasks/git.py

+    """
+    docker_cmd = [
+        "docker run -t -v",
+        "{}/..:/app/".format(PROJ_ROOT),


Note: this only works if faabric is used as a submodule in faasm. This is because, as a submodule, the .git folder is a softlink to ../.git/modules/faabric.

We shouldn't assume this, Faabric should be treated as a separate library and worked on in a separate checkout. Editing Faabric as a submodule of Faasm should be the exception not the rule.

Shillaker

Nice one, this looks good. I think we can go one step further and actually do the release from the command. See the equivalent command in Faasm: https://github.com/faasm/faasm/blob/main/faasmcli/faasmcli/tasks/release.py#L152

Splitting it into creating and then publishing the release is probably good for now, but can eventually be merged into one step.

Shillaker · 2022-02-23T13:06:04Z

tasks/git.py

+    """
+    docker_cmd = [
+        "docker run -t -v",
+        "{}/..:/app/".format(PROJ_ROOT),


We shouldn't assume this, Faabric should be treated as a separate library and worked on in a separate checkout. Editing Faabric as a submodule of Faasm should be the exception not the rule.

Shillaker · 2022-02-23T13:13:28Z

@csegarragonz let's also remove the commit bodies and just include the headline commits. There are links to the PRs that people can click on if they want more detail.

Shillaker · 2022-02-23T13:15:54Z

@csegarragonz finally, do we need to add git-cliff to requirements.txt?

… title

csegarragonz · 2022-02-23T15:20:00Z

@Shillaker we don't need to add git-cliff as a dependency, as we rely on the docker image to run it. further, git-cliff is written in rust, what means that we couldn't use pip to manage its installation anyway.

i add the new release body format in the updated pr description, i agree with you and think it is more concise and readable.

Shillaker · 2022-02-23T15:41:47Z

Ah whoops, sorry I'd assumed git-cliff was a python module. Interesting choice to go with Rust...

Shillaker

This looks great. Only possible issue is with the docs, I think those inv commands won't work but may be wrong.

Shillaker · 2022-02-23T17:07:49Z

docs/source/development.md

+First, generate a draft release:
+
+```bash
+inv git.release_create


I don't think this will work, I have a feeling Invoke will replace all underscores with hyphens.

It works for me locally (source ./bin/workon.sh && inv -l I see underscores). The same does not happen in faasm.

We use different invoke versions (1.6.0 and 1.5.0 respectively) but I can't see anything about such changes in their changelog.

csegarragonz added 2 commits February 23, 2022 10:10

bumping code version for release

315ea4c

ci: add step to generate changelog

29efee8

csegarragonz self-assigned this Feb 23, 2022

csegarragonz added 3 commits February 23, 2022 10:30

ci: generate release body with changelog

032ef9d

ci: add task to generate release body without dependencies

7e8c99c

docs: update docs with latest releasing info

41824ce

csegarragonz changed the title ~~Release~~ Add task to generate release body Feb 23, 2022

csegarragonz commented Feb 23, 2022

View reviewed changes

Shillaker requested changes Feb 23, 2022

View reviewed changes

csegarragonz added 2 commits February 23, 2022 15:16

cli: support running both as a submodule and as a standalone project

1103b51

ci: change format of release template so that it only includes the pr…

6888a8b

… title

csegarragonz added the ci Continuous integration label Feb 23, 2022

ci: automate release creation/publishing from tasks

52f1745

csegarragonz mentioned this pull request Feb 23, 2022

Release faasm/faasm#598

Merged

csegarragonz requested a review from Shillaker February 23, 2022 16:30

Shillaker approved these changes Feb 23, 2022

View reviewed changes

csegarragonz merged commit b11e8ab into main Feb 23, 2022

csegarragonz deleted the release branch February 23, 2022 17:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add task to generate release body #233

Add task to generate release body #233

csegarragonz commented Feb 23, 2022 •

edited

Loading

csegarragonz Feb 23, 2022

Shillaker Feb 23, 2022

Shillaker left a comment

Shillaker Feb 23, 2022

Shillaker commented Feb 23, 2022

Shillaker commented Feb 23, 2022

csegarragonz commented Feb 23, 2022

Shillaker commented Feb 23, 2022

Shillaker left a comment

Shillaker Feb 23, 2022

csegarragonz Feb 23, 2022 •

edited

Loading

Add task to generate release body #233

Add task to generate release body #233

Conversation

csegarragonz commented Feb 23, 2022 • edited Loading

Faabric's Changelog

[0.3.1] - 2022-02-23

[0.3.0] - 2022-02-11

[0.2.3] - 2021-12-20

[0.2.1] - 2021-10-27

csegarragonz Feb 23, 2022

Choose a reason for hiding this comment

Shillaker Feb 23, 2022

Choose a reason for hiding this comment

Shillaker left a comment

Choose a reason for hiding this comment

Shillaker Feb 23, 2022

Choose a reason for hiding this comment

Shillaker commented Feb 23, 2022

Shillaker commented Feb 23, 2022

csegarragonz commented Feb 23, 2022

Shillaker commented Feb 23, 2022

Shillaker left a comment

Choose a reason for hiding this comment

Shillaker Feb 23, 2022

Choose a reason for hiding this comment

csegarragonz Feb 23, 2022 • edited Loading

Choose a reason for hiding this comment

csegarragonz commented Feb 23, 2022 •

edited

Loading

csegarragonz Feb 23, 2022 •

edited

Loading