Skip to content

Commit

Permalink
Improve ci output (#850)
Browse files Browse the repository at this point in the history 
* Use wireit v0.11.0 in wireit.

* Add the `quiet-ci` logger option

* Address review comments.
  • Loading branch information
@rictic
rictic authored Sep 1, 2023
1 parent 7de30b6 commit 3ec0739
Show file tree
Hide file tree
Showing 8 changed files with 293 additions and 98 deletions.
6 changes: 6 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,12 @@ Versioning](https://semver.org/spec/v2.0.0.html).

<!-- ## [Unreleased] -->

## [Unreleased]

## Added

- Added a `quiet-ci` logger with output optimized for non-interactive environemnts, like a continuous integration builder (e.g. GitHub Actions). Writes less often, doesn't show a spinner, doesn't use \r to try to writeover previous output, and only prints a new status line if there's been a change.

## [0.11.0] - 2023-08-30

## Added
Expand Down
16 changes: 8 additions & 8 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -847,14 +847,14 @@ The following syntaxes can be used in the `wireit.<script>.dependencies` array:

The following environment variables affect the behavior of Wireit:

| Variable | Description |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `WIREIT_FAILURES` | [How to handle script failures](#failures-and-errors).<br><br>Options:<br><ul><li>[`no-new`](#failures-and-errors) (default): Allow running scripts to finish, but don't start new ones.</li><li>[`continue`](#continue): Allow running scripts to continue, and start new ones unless any of their dependencies failed.</li><li>[`kill`](#kill): Immediately kill running scripts, and don't start new ones.</li></ul> |
| `WIREIT_PARALLEL` | [Maximum number of scripts to run at one time](#parallelism).<br><br>Defaults to 2×logical CPU cores.<br><br>Must be a positive integer or `infinity`. |
| `WIREIT_CACHE` | [Caching mode](#caching).<br><br>Defaults to `local` unless `CI` is `true`, in which case defaults to `none`.<br><br>Automatically set to `github` by the [`google/wireit@setup-github-actions-caching/v1`](#github-actions-caching) action.<br><br>Options:<ul><li>[`local`](#local-caching): Cache to local disk.</li><li>[`github`](#github-actions-caching): Cache to GitHub Actions.</li><li>`none`: Disable caching.</li></ul> |
| `CI` | Affects the default value of `WIREIT_CACHE`.<br><br>Automatically set to `true` by [GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables) and most other CI (continuous integration) services.<br><br>Must be exactly `true`. If unset or any other value, interpreted as `false`. |
| `WIREIT_MAX_OPEN_FILES` | Limits the number of file descriptors Wireit will have open concurrently. Prevents resource exhaustion when checking large numbers of cached files. Set to a lower number if you hit file descriptor limits. |
| `WIREIT_LOGGER` | How to present progress and results on the command line.<br><br>Options:<br><ul><li>`quiet`: writes a single dynamically updating line summarizing progress. Only passes along stdout and stderr from commands if there's a failure, or if the command is a service. The planned new default, please try it out.</li><li>`simple` (default): A verbose logger that presents clear information about the work that Wireit is doing.</li><li>`metrics`: Like `simple`, but also presents a summary table of results once a command is finished.</li></ul> |
| Variable | Description |
| ----------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `WIREIT_FAILURES` | [How to handle script failures](#failures-and-errors).<br><br>Options:<br><ul><li>[`no-new`](#failures-and-errors) (default): Allow running scripts to finish, but don't start new ones.</li><li>[`continue`](#continue): Allow running scripts to continue, and start new ones unless any of their dependencies failed.</li><li>[`kill`](#kill): Immediately kill running scripts, and don't start new ones.</li></ul> |
| `WIREIT_PARALLEL` | [Maximum number of scripts to run at one time](#parallelism).<br><br>Defaults to 2×logical CPU cores.<br><br>Must be a positive integer or `infinity`. |
| `WIREIT_CACHE` | [Caching mode](#caching).<br><br>Defaults to `local` unless `CI` is `true`, in which case defaults to `none`.<br><br>Automatically set to `github` by the [`google/wireit@setup-github-actions-caching/v1`](#github-actions-caching) action.<br><br>Options:<ul><li>[`local`](#local-caching): Cache to local disk.</li><li>[`github`](#github-actions-caching): Cache to GitHub Actions.</li><li>`none`: Disable caching.</li></ul> |
| `CI` | Affects the default value of `WIREIT_CACHE`.<br><br>Automatically set to `true` by [GitHub Actions](https://docs.github.com/en/actions/learn-github-actions/environment-variables#default-environment-variables) and most other CI (continuous integration) services.<br><br>Must be exactly `true`. If unset or any other value, interpreted as `false`. |
| `WIREIT_MAX_OPEN_FILES` | Limits the number of file descriptors Wireit will have open concurrently. Prevents resource exhaustion when checking large numbers of cached files. Set to a lower number if you hit file descriptor limits. |
| `WIREIT_LOGGER` | How to present progress and results on the command line.<br><br>Options:<br><ul><li>`quiet`: writes a single dynamically updating line summarizing progress. Only passes along stdout and stderr from commands if there's a failure, or if the command is a service. The planned new default, please try it out.</li><li>`simple` (default): A verbose logger that presents clear information about the work that Wireit is doing.</li><li>`metrics`: Like `simple`, but also presents a summary table of results once a command is finished.</li><li>`quiet-ci`: like `quiet` but optimized for non-interactive environments, like GitHub Actions runners.</li></ul> |

### Glob patterns

Expand Down
13 changes: 13 additions & 0 deletions package.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@
"test:metrics": "wireit",
"test:optimize-mkdirs": "wireit",
"test:parallelism": "wireit",
"test:quiet": "wireit",
"test:service": "wireit",
"test:watch": "wireit"
},
Expand Down Expand Up @@ -105,6 +106,7 @@
"test:metrics",
"test:optimize-mkdirs",
"test:parallelism",
"test:quiet",
"test:service",
"test:watch"
]
Expand Down Expand Up @@ -357,6 +359,17 @@
"files": [],
"output": []
},
"test:quiet": {
"command": "uvu lib/test \"^quiet-logger\\.test\\.js$\"",
"env": {
"NODE_OPTIONS": "--enable-source-maps"
},
"dependencies": [
"build"
],
"files": [],
"output": []
},
"test:service": {
"command": "uvu lib/test \"^service\\.test\\.js$\"",
"env": {
Expand Down
5 changes: 4 additions & 1 deletion src/cli-options.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import {ScriptReference} from './config.js';
import {FailureMode} from './executor.js';
import {unreachable} from './util/unreachable.js';
import {Logger} from './logging/logger.js';
import {QuietLogger} from './logging/quiet-logger.js';
import {QuietCiLogger, QuietLogger} from './logging/quiet-logger.js';
import {DefaultLogger} from './logging/default-logger.js';

export const packageDir = await (async (): Promise<string | undefined> => {
Expand Down Expand Up @@ -194,6 +194,9 @@ export const getOptions = (): Result<Options> => {
if (str === 'quiet') {
return {ok: true, value: new QuietLogger(packageRoot)};
}
if (str === 'quiet-ci') {
return {ok: true, value: new QuietCiLogger(packageRoot)};
}
if (str === 'simple') {
return {ok: true, value: new DefaultLogger(packageRoot)};
}
Expand Down
32 changes: 25 additions & 7 deletions src/logging/quiet-logger.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@

import {Event} from '../event.js';
import {Logger} from './logger.js';
import {WriteoverLine} from './quiet/writeover-line.js';
import {
CiWriter,
StatusLineWriter,
WriteoverLine,
} from './quiet/writeover-line.js';
import {QuietRunLogger, noChange, nothing} from './quiet/run-tracker.js';

/**
Expand All @@ -21,18 +25,19 @@ import {QuietRunLogger, noChange, nothing} from './quiet/run-tracker.js';
export class QuietLogger implements Logger {
private runTracker;
private readonly _rootPackage: string;
private readonly _writeoverLine = new WriteoverLine();
private readonly _statusLineWriter: StatusLineWriter;

constructor(rootPackage: string) {
constructor(rootPackage: string, statusLineWriter?: StatusLineWriter) {
this._rootPackage = rootPackage;
this._statusLineWriter = statusLineWriter ?? new WriteoverLine();
this.runTracker = new QuietRunLogger(
this._rootPackage,
this._writeoverLine,
this._statusLineWriter,
);
}

printMetrics() {
this._writeoverLine.clearAndStopSpinner();
this._statusLineWriter.clearAndStopRendering();
this.runTracker.printSummary();
}

Expand All @@ -44,9 +49,9 @@ export class QuietLogger implements Logger {
if (line === noChange) {
// nothing to do
} else if (line === nothing) {
this._writeoverLine.clearAndStopSpinner();
this._statusLineWriter.clearAndStopRendering();
} else {
this._writeoverLine.writeLine(line);
this._statusLineWriter.updateStatusLine(line);
}
if (event.type === 'info' && event.detail === 'watch-run-end') {
this.printMetrics();
Expand All @@ -60,3 +65,16 @@ export class QuietLogger implements Logger {
return this;
}
}

/**
* A QuietLogger that is intended to be used in CI environments and other
* non-interactive environments.
*
* Doesn't use a spinner, updates less often, and doesn't use '/r' to writeover
* the previous line.
*/
export class QuietCiLogger extends QuietLogger {
constructor(rootPackage: string) {
super(rootPackage, new CiWriter());
}
}
24 changes: 11 additions & 13 deletions src/logging/quiet/run-tracker.ts
View file
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import {
import {Failure, Info, Success, Output, Event} from '../../event.js';
import {DefaultLogger, labelForScript} from '../default-logger.js';
import {DEBUG} from '../logger.js';
import {WriteoverLine} from './writeover-line.js';
import {StatusLineWriter} from './writeover-line.js';
import {StackMap} from './stack-map.js';

interface SimpleOutput {
Expand Down Expand Up @@ -156,7 +156,7 @@ export class QuietRunLogger {
private readonly _startTime = Date.now();
private readonly _rootPackage: string;
private readonly _defaultLogger: DefaultLogger;
private readonly _writeoverLine;
private readonly _statusLineWriter;
/**
* Sometimes a script will fail multiple times, but we only want to report
* about the first failure for it that we find. Sometimes a script will
Expand All @@ -169,11 +169,11 @@ export class QuietRunLogger {

constructor(
rootPackage: string,
writeoverLine: WriteoverLine,
statusLineWriter: StatusLineWriter,
defaultLogger?: DefaultLogger,
) {
this._rootPackage = rootPackage;
this._writeoverLine = writeoverLine;
this._statusLineWriter = statusLineWriter;
this._defaultLogger = defaultLogger ?? new DefaultLogger(rootPackage);
}

Expand All @@ -185,7 +185,7 @@ export class QuietRunLogger {
// Reuse the default logger, a minor savings.
const instance = new QuietRunLogger(
this._rootPackage,
this._writeoverLine,
this._statusLineWriter,
this._defaultLogger,
);
// Persistent services stay running between runs, so pass along what we
Expand Down Expand Up @@ -330,10 +330,8 @@ export class QuietRunLogger {
const peekResult = this._running.peek()?.[1];
let mostRecentScript = '';
if (peekResult !== undefined) {
mostRecentScript = labelForScript(
this._rootPackage,
peekResult.scriptReference,
);
mostRecentScript =
' ' + labelForScript(this._rootPackage, peekResult.scriptReference);
}
const done = this._finishedScriptsWithCommands.size;
const total = analysisInfo.scriptsWithCommands.size;
Expand All @@ -351,7 +349,7 @@ export class QuietRunLogger {
}
return `${percentDone} [${done.toLocaleString()} / ${total}] [${
this._running.size
} running]${servicesInfo}${failureInfo} ${mostRecentScript}`;
} running]${servicesInfo}${failureInfo}${mostRecentScript}`;
}

private _markScriptAsFinished(script: ScriptReference) {
Expand Down Expand Up @@ -511,7 +509,7 @@ export class QuietRunLogger {
this._encounteredFailures = true;
{
// eslint-disable-next-line @typescript-eslint/no-unused-vars
using _pause = this._writeoverLine.clearUntilDisposed();
using _pause = this._statusLineWriter.clearUntilDisposed();
this._reportFailure(event);
}
return noChange;
Expand Down Expand Up @@ -615,7 +613,7 @@ export class QuietRunLogger {
// Pause the status line while we print this real quick, but then resume
// it.
// eslint-disable-next-line @typescript-eslint/no-unused-vars
using _pause = this._writeoverLine.clearUntilDisposed();
using _pause = this._statusLineWriter.clearUntilDisposed();
if (event.stream === 'stdout') {
process.stdout.write(event.data);
} else {
Expand All @@ -628,7 +626,7 @@ export class QuietRunLogger {
// Unlike for a service, this is a terminal state, instead of pausing the
// status line, we stop it completely, because for the rest of the run
// we're just going to be printing the root script's output.
this._writeoverLine.clearAndStopSpinner();
this._statusLineWriter.clearAndStopRendering();
if (event.stream === 'stdout') {
process.stdout.write(event.data);
} else {
Expand Down
Loading

0 comments on commit 3ec0739

Please sign in to comment.