Skip to content

Improve Harper Application Management by optimizing installation using lockfile #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Parent: General Repo Updates
wants to merge 5 commits into
base: general-repo-updates
Choose a base branch
from
Draft

Improve Harper Application Management by optimizing installation using lockfile #11

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
7444417
add harper application lock file
Ethan-Arrowood Oct 27, 2025
fe19a1c
only skip if actually exists
Ethan-Arrowood Oct 27, 2025
ad4484c
start on application unit tests
Ethan-Arrowood Oct 27, 2025
465728f
format
Ethan-Arrowood Oct 27, 2025
72b3ab4
more tests - thx claude
Ethan-Arrowood Oct 27, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 88 additions & 20 deletions components/Application.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,19 @@
import logger from '../utility/logging/harper_logger.js';

import { dirname, extname, join } from 'node:path';
import { access, constants, cp, mkdir, mkdtemp, readdir, readFile, rm, stat, symlink } from 'node:fs/promises';
import {
access,
constants,
cp,
mkdir,
mkdtemp,
readdir,
readFile,
rm,
stat,
symlink,
writeFile,
} from 'node:fs/promises';
import { spawn } from 'node:child_process';
import { createReadStream, existsSync, readdirSync } from 'node:fs';
import { Readable } from 'node:stream';
Expand All @@ -13,43 +25,74 @@
import gunzip from 'gunzip-maybe';

interface ApplicationConfig {
// define known config properties
package: string;
install?: {
command?: string;
timeout?: number;
};
// an application config can have other arbitrary properties
[key: string]: unknown;
}

export class InvalidPackageIdentifierError extends TypeError {
constructor(applicationName: string, packageIdentifier: unknown) {
super(
`Invalid 'package' property for application ${applicationName}: expected string, got ${typeof packageIdentifier}`
);
}
}

export class InvalidInstallPropertyError extends TypeError {
constructor(applicationName: string, installProperty: unknown) {
super(
`Invalid 'install' property for application ${applicationName}: expected object, got ${typeof installProperty}`
);
}
}

export class InvalidInstallCommandError extends TypeError {
constructor(applicationName: string, command: unknown) {
super(
`Invalid 'install.command' property for application ${applicationName}: expected string, got ${typeof command}`
);
}
}

export class InvalidInstallTimeoutError extends TypeError {
constructor(applicationName: string, timeout: unknown) {
super(
`Invalid 'install.timeout' property for application ${applicationName}: expected non-negative number, got ${typeof timeout}`
);
}
}

export function assertApplicationConfig(
applicationName: string,
applicationConfig: object & Record<'package', unknown>
applicationConfig: Record<'package', unknown> & Record<string, unknown>
): asserts applicationConfig is ApplicationConfig {
if (typeof applicationConfig.package !== 'string') {
throw new TypeError(
`Invalid 'package' property for application ${applicationName}: expected string, got ${typeof applicationConfig.package}`
);
throw new InvalidPackageIdentifierError(applicationName, applicationConfig.package);
}

if ('install' in applicationConfig) {
if (typeof applicationConfig.install !== 'object' || applicationConfig.install === null) {
throw new TypeError(
`Invalid 'install' property for application ${applicationName}: expected object, got ${typeof applicationConfig.install}`
);
if (
typeof applicationConfig.install !== 'object' ||
applicationConfig.install === null ||
Array.isArray(applicationConfig.install)
) {
throw new InvalidInstallPropertyError(applicationName, applicationConfig.install);
}

if ('command' in applicationConfig.install && typeof applicationConfig.install.command !== 'string') {
throw new TypeError(
`Invalid 'install.command' property for application ${applicationName}: expected string, got ${typeof applicationConfig.install.command}`
);
throw new InvalidInstallCommandError(applicationName, applicationConfig.install.command);
}

if (
'timeout' in applicationConfig.install &&
(typeof applicationConfig.install.timeout !== 'number' || applicationConfig.install.timeout < 0)
) {
throw new TypeError(
`Invalid 'install.timeout' property for application ${applicationName}: expected non-negativenumber, got ${typeof applicationConfig.install.timeout}`
);
throw new InvalidInstallTimeoutError(applicationName, applicationConfig.install.timeout);
}
}
}
Expand Down Expand Up @@ -80,7 +123,7 @@
if (application.payload) {
// Given a payload, create a Readable from the Buffer or string
tarball = Readable.from(
application.payload instanceof Buffer ? application.payload : Buffer.from(application.payload, 'base64')

Check failure on line 126 in components/Application.ts

View workflow job for this annotation

GitHub Actions / Unit Test (Node.js v20) 

No overload matches this call.

Check failure on line 126 in components/Application.ts

View workflow job for this annotation

GitHub Actions / Unit Test (Node.js v22) 

No overload matches this call.

Check failure on line 126 in components/Application.ts

View workflow job for this annotation

GitHub Actions / Unit Test (Node.js v24) 

No overload matches this call.

Check failure on line 126 in components/Application.ts

View workflow job for this annotation

GitHub Actions / Integration Test (Node.js v24) 

No overload matches this call.

Check failure on line 126 in components/Application.ts

View workflow job for this annotation

GitHub Actions / Integration Test (Node.js v22) 

No overload matches this call.

Check failure on line 126 in components/Application.ts

View workflow job for this annotation

GitHub Actions / Integration Test (Node.js v20) 

No overload matches this call.
);
} else {
// Given a package, there are a a couple options
Expand Down Expand Up @@ -184,8 +227,9 @@
* This method should only be called from the main thread
*/
export async function installApplication(application: Application) {
let packageJSON: any;
try {
await access(join(application.dirPath, 'package.json'), constants.F_OK);
packageJSON = JSON.parse(await readFile(join(application.dirPath, 'package.json'), 'utf8'));
} catch (err) {
if (err.code !== 'ENOENT') throw err;
// If no package.json, nothing to install
Expand Down Expand Up @@ -225,8 +269,6 @@
}

// Next, try package.json devEngines field
const packageJSON = JSON.parse(await readFile(join(application.dirPath, 'package.json'), 'utf8'));

const { packageManager } = packageJSON.devEngines || {};

// Custom package manager specified
Expand Down Expand Up @@ -344,7 +386,7 @@
* during the installation process in order to actually resolve what the user specifies for a
* component matching some of npm's package resolution rules.
*/
function derivePackageIdentifier(packageIdentifier: string) {
export function derivePackageIdentifier(packageIdentifier: string) {
if (packageIdentifier.includes(':')) {
return packageIdentifier;
}
Expand Down Expand Up @@ -384,6 +426,18 @@
// Ensure component directory exists
await mkdir(componentsRootDirPath, { recursive: true });

const harperApplicationLockPath = join(getConfigValue(CONFIG_PARAMS.ROOTPATH), 'harper-application-lock.json');

let harperApplicationLock: any = { application: {} };
try {
harperApplicationLock = JSON.parse(await readFile(harperApplicationLockPath, 'utf8'));
} catch (error) {
// Ignore file not found error; will create new lock file after installations
if (error.code !== 'ENOENT') {
throw error;
}
}

const applicationInstallationPromises: Promise<void>[] = [];

for (const [name, applicationConfig] of Object.entries(config)) {
Expand All @@ -403,12 +457,27 @@
install: applicationConfig.install,
});

// Lock check: only install if not already installed with matching configuration
if (
existsSync(application.dirPath) &&
harperApplicationLock.applications[name] &&
JSON.stringify(harperApplicationLock.applications[name]) === JSON.stringify(applicationConfig)
) {
logger.info(`Application ${name} is already installed with matching configuration; skipping installation`);
continue;
}

applicationInstallationPromises.push(prepareApplication(application));

harperApplicationLock.applications[name] = applicationConfig;
}

const applicationInstallationStatuses = await Promise.allSettled(applicationInstallationPromises);
logger.debug(applicationInstallationStatuses);
logger.info('All root applications loaded');

// Finally, write the lock file
await writeFile(harperApplicationLockPath, JSON.stringify(harperApplicationLock, null, 2), 'utf8');
}

function getGitSSHCommand() {
Expand Down Expand Up @@ -455,7 +524,6 @@
env.GIT_SSH_COMMAND = gitSSHCommand;
}

// eslint-disable-next-line sonarjs/os-command
const childProcess = spawn(command, args, {
shell: true,
cwd,
Expand Down
Loading
Loading