Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TT-1693] reusable Solidity Review Artifact workflow #197

Closed
wants to merge 50 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
426f5b4
reusable solidity artifact pipeline WIP#1
Tofel Sep 16, 2024
bd98536
move reusable workflow to .github/workflows
Tofel Sep 16, 2024
e352dae
skip should be a string
Tofel Sep 16, 2024
0eb0e94
reference actions with full path
Tofel Sep 16, 2024
dafe5a2
rename old variable
Tofel Sep 16, 2024
e427b83
pass correct config to setup node-js
Tofel Sep 16, 2024
d2394ab
checkout this repo to get the scripts
Tofel Sep 16, 2024
76ea30e
add repository name to checkout
Tofel Sep 16, 2024
952377f
checkout repo before JIRA
Tofel Sep 16, 2024
2c76552
fix jira path
Tofel Sep 16, 2024
ae1c2fa
run pnpm with prefix
Tofel Sep 16, 2024
1c18c23
read top level dir from env var if present
Tofel Sep 16, 2024
d4f881b
forget about checking out to a directory
Tofel Sep 16, 2024
53ccf64
add more debug
Tofel Sep 17, 2024
684b17a
inherit secrets
Tofel Sep 17, 2024
bc3068c
add jira secrets
Tofel Sep 17, 2024
698d5e3
add customisable slither config path
Tofel Sep 17, 2024
b6d81fb
create contracts diretory if it doesn't exist
Tofel Sep 17, 2024
500c941
install sol2uml with pnpm
Tofel Sep 17, 2024
63f3bdf
install sol2uml globally
Tofel Sep 17, 2024
f2ff22b
try different docs path
Tofel Sep 17, 2024
ca1e663
fix parenthesis
Tofel Sep 17, 2024
0413b3a
conditionally install semver
Tofel Sep 17, 2024
def526e
made no-changes notification more general
Tofel Sep 17, 2024
0090042
use customisable artifact directory
Tofel Sep 17, 2024
ecfe3bc
upload 2 docs dirs, just in case
Tofel Sep 17, 2024
8c6933a
try to fix artifact paths for various conditions
Tofel Sep 17, 2024
8e292de
add missing output
Tofel Sep 17, 2024
fe1e348
one more try with forge output
Tofel Sep 17, 2024
16fb286
hack for forge book
Tofel Sep 17, 2024
830d5ba
try to simplify
Tofel Sep 17, 2024
f88a1a6
fix directory creation
Tofel Sep 17, 2024
21dec75
add even more configuration
Tofel Sep 17, 2024
c929ae0
do install forge, always
Tofel Sep 17, 2024
024f049
run final step always()
Tofel Sep 18, 2024
13ed227
allow to disable jira traceablity
Tofel Sep 18, 2024
45a21d0
skip artifacts gathering if any step failed
Tofel Sep 18, 2024
4e75a50
add timeouts
Tofel Sep 18, 2024
ac4869a
try another approach to failed jobs
Tofel Sep 18, 2024
4c63711
try one more way
Tofel Sep 18, 2024
74bfa2f
remove test failure
Tofel Sep 18, 2024
620887a
use boolean instead of string, modify jira path
Tofel Sep 18, 2024
14cd930
use latest actions versions, remove non-reusable action
Tofel Sep 18, 2024
13e5ab6
do not require JIRA vars
Tofel Sep 18, 2024
d4ccf51
checkout caller repository
Tofel Sep 18, 2024
3dcd41c
fix order of actions
Tofel Sep 18, 2024
21b8f9d
update checkout ref
Tofel Sep 18, 2024
92e2e0c
allow to pass path to custom lcov prunning script
Tofel Sep 18, 2024
3c3b6de
remove slither diff scripts
Tofel Sep 18, 2024
2951e97
improve comments
Tofel Sep 18, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
466 changes: 466 additions & 0 deletions .github/workflows/review-artifacts.yml

Large diffs are not rendered by default.

97 changes: 97 additions & 0 deletions jira/scripts/axios.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
import {
AxiosRequestConfig,
AxiosResponse,
AxiosError,
InternalAxiosRequestConfig,
} from "axios";
import { Readable } from "stream";

interface AxiosErrorFormat<Data = any> {
config: Pick<AxiosRequestConfig, (typeof CONFIG_KEYS)[number]>;
code?: string;
response: Partial<Pick<AxiosResponse<Data>, (typeof RESPONSE_KEYS)[number]>>;
isAxiosError: boolean;
}

interface AxiosErrorFormatError<Data = any>
extends Error,
AxiosErrorFormat<Data> {}

export function formatAxiosError<Data = any>(
origErr: AxiosError<Data>
): AxiosErrorFormatError<Data> {
const { message, name, stack, code, config, response, isAxiosError } =
origErr;

const err: AxiosErrorFormatError = {
...new Error(message),
name,
stack,
code,
isAxiosError,
config: {},
response: {},
};

for (const k of CONFIG_KEYS) {
if (config?.[k] === undefined) {
continue;
}

err.config[k] = formatValue(config[k], k);
}

for (const k of RESPONSE_KEYS) {
if (response?.[k] === undefined) {
continue;
}

err.response[k] = formatValue(response[k], k);
}

return err as any;
}

const CONFIG_KEYS: (keyof InternalAxiosRequestConfig)[] = [
"url",
"method",
"baseURL",
"params",
"data",
"timeout",
"timeoutErrorMessage",
"withCredentials",
"auth",
"responseType",
"xsrfCookieName",
"xsrfHeaderName",
"maxContentLength",
"maxBodyLength",
"maxRedirects",
"socketPath",
"proxy",
"decompress",
] as const;

const RESPONSE_KEYS: (keyof AxiosResponse)[] = [
"data",
"status",
"statusText",
] as const;

function formatValue(
value: any,
key: (typeof CONFIG_KEYS)[number] | (typeof RESPONSE_KEYS)[number]
): any {
if (key !== "data") {
return value;
}

if (process.env.BROWSER !== "true") {
if (value instanceof Readable) {
return "[Readable]";
}
}

return value;
}
215 changes: 215 additions & 0 deletions jira/scripts/create-jira-traceability.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,215 @@
import * as jira from "jira.js";
import {
createJiraClient,
extractJiraIssueNumbersFrom,
generateIssueLabel,
generateJiraIssuesLink,
getJiraEnvVars,
handleError,
} from "./lib";
import * as core from "@actions/core";

/**
* Extracts the list of changeset files. Intended to be used with https://github.com/dorny/paths-filter with
* the 'csv' output format.
*
* @returns An array of strings representing the changeset files.
* @throws {Error} If the required environment variable CHANGESET_FILES is missing.
* @throws {Error} If no changeset file exists.
*/
function extractChangesetFiles(): string[] {
const changesetFiles = process.env.CHANGESET_FILES;
if (!changesetFiles) {
throw Error("Missing required environment variable CHANGESET_FILES");
}
const parsedChangesetFiles = changesetFiles.split(",");
if (parsedChangesetFiles.length === 0) {
throw Error("At least one changeset file must exist");
}

core.info(
`Changeset to extract issues from: ${parsedChangesetFiles.join(", ")}`
);
return parsedChangesetFiles;
}

/**
* Adds traceability to JIRA issues by commenting on each issue with a link to the artifact payload
* along with a label to connect all issues to the same chainlink product review.
*
* @param client The jira client
* @param issues The list of JIRA issue numbers to add traceability to
* @param label The label to add to each issue
* @param artifactUrl The url to the artifact payload that we'll comment on each issue with
*/
async function addTraceabillityToJiraIssues(
client: jira.Version3Client,
issues: string[],
label: string,
artifactUrl: string
) {
for (const issue of issues) {
await checkAndAddArtifactPayloadComment(client, issue, artifactUrl);

// CHECK: We don't need to see if the label exists, should no-op
core.info(`Adding label ${label} to issue ${issue}`);
await client.issues.editIssue({
issueIdOrKey: issue,
update: {
labels: [{ add: label }],
},
});
}
}

/**
* Checks if the artifact payload already exists as a comment on the issue, if not, adds it.
*/
async function checkAndAddArtifactPayloadComment(
client: jira.Version3.Version3Client,
issue: string,
artifactUrl: string
) {
const maxResults = 5000;
const getCommentsResponse = await client.issueComments.getComments({
issueIdOrKey: issue,
maxResults, // this is the default maxResults, see https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-issue-comments/#api-rest-api-3-issue-issueidorkey-comment-get
});
core.debug(JSON.stringify(getCommentsResponse.comments));
if ((getCommentsResponse.total ?? 0) > maxResults) {
throw Error(
`Too many (${getCommentsResponse.total}) comments on issue ${issue}, please increase maxResults (${maxResults})`
);
}

// Search path is getCommentsResponse.comments[].body.content[].content[].marks[].attrs.href
//
// Example:
// [ // getCommentsResponse.comments
// {
// body: {
// type: "doc",
// version: 1,
// content: [
// {
// type: "paragraph",
// content: [
// {
// type: "text",
// text: "Artifact URL",
// marks: [
// {
// type: "link",
// attrs: {
// href: "https://github.com/smartcontractkit/chainlink/actions/runs/10517121836/artifacts/1844867108",
// },
// },
// ],
// },
// ],
// },
// ],
// },
// },
// ];
const commentExists = getCommentsResponse.comments?.some((c) =>
c?.body?.content?.some((innerContent) =>
innerContent?.content?.some((c) =>
c.marks?.some((m) => m.attrs?.href === artifactUrl)
)
)
);

if (commentExists) {
core.info(`Artifact payload already exists as comment on issue, skipping`);
} else {
core.info(`Adding artifact payload as comment on issue ${issue}`);
await client.issueComments.addComment({
issueIdOrKey: issue,
comment: {
type: "doc",
version: 1,
content: [
{
type: "paragraph",
content: [
{
type: "text",
text: "Artifact Download URL",
marks: [
{
type: "link",
attrs: {
href: artifactUrl,
},
},
],
},
],
},
],
},
});
}
}

function fetchEnvironmentVariables() {
const product = process.env.CHAINLINK_PRODUCT;
if (!product) {
throw Error("CHAINLINK_PRODUCT environment variable is missing");
}
const baseRef = process.env.BASE_REF;
if (!baseRef) {
throw Error("BASE_REF environment variable is missing");
}
const headRef = process.env.HEAD_REF;
if (!headRef) {
throw Error("HEAD_REF environment variable is missing");
}

const artifactUrl = process.env.ARTIFACT_URL;
if (!artifactUrl) {
throw Error("ARTIFACT_URL environment variable is missing");
}
return { product, baseRef, headRef, artifactUrl };
}

/**
* For all affected jira issues listed within the changeset files supplied,
* we update each jira issue so that they are all labelled and have a comment linking them
* to the relevant artifact URL.
*/
async function main() {
const { product, baseRef, headRef, artifactUrl } =
fetchEnvironmentVariables();
const changesetFiles = extractChangesetFiles();
core.info(
`Extracting Jira issue numbers from changeset files: ${changesetFiles.join(
", "
)}`
);
const jiraIssueNumbers = await extractJiraIssueNumbersFrom(changesetFiles);

const client = createJiraClient();
const label = generateIssueLabel(product, baseRef, headRef);
try {
await addTraceabillityToJiraIssues(
client,
jiraIssueNumbers,
label,
artifactUrl
);
} catch (e) {
handleError(e);

process.exit(1);
}

const { jiraHost } = getJiraEnvVars();
core.summary.addLink(
"Jira Issues",
generateJiraIssuesLink(`${jiraHost}/issues/`, label)
);
core.summary.write();
}
main();
Loading