Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3 dev server #894

Merged
merged 42 commits into from
Feb 12, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
d9bbd9a
WIP
ericallam Jan 15, 2024
910fa45
Move the v3 dev server over to the new CLI
ericallam Jan 15, 2024
2b6b962
Added rudimentary logout command
ericallam Jan 15, 2024
7da2afa
Adding project refs to uniquely identify projects through the API wit…
ericallam Jan 15, 2024
0366e17
Indexing and creating background workers and tasks in the server
ericallam Jan 16, 2024
3fefffa
Triggering tasks
ericallam Jan 16, 2024
f0c45f6
WIP web socket server experiments
ericallam Jan 17, 2024
2a6930b
Getting websockets to work inside the remix build
ericallam Jan 17, 2024
26af912
Update remix server to work better with v2 dev server model
ericallam Jan 17, 2024
f08d11f
More websocket connection stuff
ericallam Jan 17, 2024
8cbc350
Reserve and run tasks in a background worker
ericallam Jan 18, 2024
3783a27
Combine all tasks into one file when building
ericallam Jan 19, 2024
0460d02
Handling file updates and running multiple background workers at once
ericallam Jan 23, 2024
d564958
Improve dev logger and watch for config file changes
ericallam Jan 23, 2024
078e53e
Rebuild when files are added or removed from trigger dirs
ericallam Jan 23, 2024
a27caf0
Split task runs into task runs and attempts
ericallam Jan 24, 2024
fe89ea2
Much better error support
ericallam Jan 24, 2024
39dfcbd
Convert v3 cli to Node16 module resolution and separate out v3 export
ericallam Jan 25, 2024
355084e
Implement triggerAndWait
ericallam Jan 25, 2024
e3d994f
Upgrade cli/v3 tsup and improve tsconfig setup
ericallam Jan 25, 2024
e485420
Merge run ID with attempt number in logs
ericallam Jan 25, 2024
4c5eb82
Tasks run in their own process now, and task runs can be locked to a …
ericallam Jan 26, 2024
e451ada
WIP otel
ericallam Feb 1, 2024
260044a
Use detect resources sync
ericallam Feb 5, 2024
0e904a1
Honeycomb collector
ericallam Feb 5, 2024
b72573a
Use baselime to help test otel for now
ericallam Feb 5, 2024
f561022
WIP importing otlp traces and logs into TaskEvent table
ericallam Feb 8, 2024
a6443c7
Import events from otlp spans
ericallam Feb 9, 2024
499e789
Export logs to events
ericallam Feb 9, 2024
1a083ca
Remove tracing stuff from the v3-catalog
ericallam Feb 9, 2024
2e57659
Pass worder id and version through to the events
ericallam Feb 9, 2024
d4fcc54
Extract out the CLI tracing stuff into core/v3
ericallam Feb 9, 2024
784b52c
Setup instrumentations
ericallam Feb 9, 2024
8a7cf38
Get typechecking working and building otlp-importer
ericallam Feb 9, 2024
841f50e
Fixed pnpm lock file after rebasing with main
ericallam Feb 12, 2024
b30394e
Fixed failing typecheck
ericallam Feb 12, 2024
5682749
Try to fix e2e tests
ericallam Feb 12, 2024
5726f82
Build the server before running e2e tests
ericallam Feb 12, 2024
6425ccb
Another attempt to fix e2e
ericallam Feb 12, 2024
9a16fe2
Fixed depds
ericallam Feb 12, 2024
80c2a91
Another attempt to fix e2e
ericallam Feb 12, 2024
1868494
Add external ref to e2e setup
ericallam Feb 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
WIP web socket server experiments
  • Loading branch information
ericallam committed Feb 12, 2024
commit f0c45f6d16e1a7944695e1863999d1a1bd877fd1
3 changes: 3 additions & 0 deletions apps/webapp/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
"build:db:seed": "esbuild --platform=node --bundle --minify --format=cjs ./prisma/seed.ts --outdir=prisma",
"build:remix": "remix build",
"build:server": "esbuild --platform=node --format=cjs ./server.ts --outdir=build",
"build:socket-server": "esbuild --platform=node --format=cjs ./socket-server.ts --outdir=build",
"dev": "cross-env PORT=3030 remix dev",
"dev:manual": "cross-env PORT=3030 remix dev -c \"node ./build/server.js\"",
"format": "prettier --write .",
Expand Down Expand Up @@ -132,6 +133,7 @@
"tiny-invariant": "^1.2.0",
"ulid": "^2.3.0",
"ulidx": "^2.2.1",
"ws": "^8.11.0",
"zod": "3.22.3",
"zod-error": "1.5.0",
"zod-validation-error": "^1.5.0"
Expand Down Expand Up @@ -176,6 +178,7 @@
"@types/simple-oauth2": "^5.0.4",
"@types/slug": "^5.0.3",
"@types/tar": "^6.1.4",
"@types/ws": "^8.5.3",
"@typescript-eslint/eslint-plugin": "^5.59.6",
"@typescript-eslint/parser": "^5.59.6",
"autoprefixer": "^10.4.13",
Expand Down
51 changes: 51 additions & 0 deletions apps/webapp/server.ts
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,31 @@ app.all(

const port = process.env.REMIX_APP_PORT || process.env.PORT || 3000;

import { SocketServer } from "./socket-server";
import { WebSocketServer } from "ws";

if (process.env.HTTP_SERVER_DISABLED !== "true") {
const wss = new WebSocketServer({ noServer: true });

wss.on("connection", (ws, req) => {
const apiKey = req.headers.authorization;

if (!apiKey || typeof apiKey !== "string") {
console.log("Invalid API Key, closing the connection");

ws.close(1008, "Invalid API Key");
return;
}

const keyPart = apiKey.split(" ")[1];

console.log("Initialization the TriggerServer now...");

const triggerServer = new SocketServer(ws);

console.log("TriggerServer initialized, sending the API Key...");
});

const server = app.listen(port, () => {
// require the built app so we're ready when the first request comes in
require(BUILD_DIR);
Expand All @@ -73,6 +97,33 @@ if (process.env.HTTP_SERVER_DISABLED !== "true") {
}
});
});

server.on("upgrade", async (req, socket, head) => {
console.log(
`Attemping to upgrade connection at url ${req.url} with headers: ${JSON.stringify(
req.headers
)}`
);

const url = new URL(req.url ?? "", "http://localhost");

// Only upgrade the connecting if the path is `/ws`
if (url.pathname !== "/ws") {
socket.destroy(
new Error(
"Cannot connect because of invalid path: Please include `/ws` in the path of your upgrade request."
)
);
return;
}

console.log(`Client connected, upgrading their connection...`);

// Handle the WebSocket connection
wss.handleUpgrade(req, socket, head, (ws) => {
wss.emit("connection", ws, req);
});
});
} else {
require(BUILD_DIR);
console.log(`✅ app ready (skipping http server)`);
Expand Down
5 changes: 5 additions & 0 deletions apps/webapp/socket-server.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { WebSocket } from "ws";

export class SocketServer {
constructor(private socket: WebSocket) {}
}
4 changes: 4 additions & 0 deletions pnpm-lock.yaml

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.