Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

v3 dev server #894

Merged
merged 42 commits into from
Feb 12, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
d9bbd9a
WIP
ericallam Jan 15, 2024
910fa45
Move the v3 dev server over to the new CLI
ericallam Jan 15, 2024
2b6b962
Added rudimentary logout command
ericallam Jan 15, 2024
7da2afa
Adding project refs to uniquely identify projects through the API wit…
ericallam Jan 15, 2024
0366e17
Indexing and creating background workers and tasks in the server
ericallam Jan 16, 2024
3fefffa
Triggering tasks
ericallam Jan 16, 2024
f0c45f6
WIP web socket server experiments
ericallam Jan 17, 2024
2a6930b
Getting websockets to work inside the remix build
ericallam Jan 17, 2024
26af912
Update remix server to work better with v2 dev server model
ericallam Jan 17, 2024
f08d11f
More websocket connection stuff
ericallam Jan 17, 2024
8cbc350
Reserve and run tasks in a background worker
ericallam Jan 18, 2024
3783a27
Combine all tasks into one file when building
ericallam Jan 19, 2024
0460d02
Handling file updates and running multiple background workers at once
ericallam Jan 23, 2024
d564958
Improve dev logger and watch for config file changes
ericallam Jan 23, 2024
078e53e
Rebuild when files are added or removed from trigger dirs
ericallam Jan 23, 2024
a27caf0
Split task runs into task runs and attempts
ericallam Jan 24, 2024
fe89ea2
Much better error support
ericallam Jan 24, 2024
39dfcbd
Convert v3 cli to Node16 module resolution and separate out v3 export
ericallam Jan 25, 2024
355084e
Implement triggerAndWait
ericallam Jan 25, 2024
e3d994f
Upgrade cli/v3 tsup and improve tsconfig setup
ericallam Jan 25, 2024
e485420
Merge run ID with attempt number in logs
ericallam Jan 25, 2024
4c5eb82
Tasks run in their own process now, and task runs can be locked to a …
ericallam Jan 26, 2024
e451ada
WIP otel
ericallam Feb 1, 2024
260044a
Use detect resources sync
ericallam Feb 5, 2024
0e904a1
Honeycomb collector
ericallam Feb 5, 2024
b72573a
Use baselime to help test otel for now
ericallam Feb 5, 2024
f561022
WIP importing otlp traces and logs into TaskEvent table
ericallam Feb 8, 2024
a6443c7
Import events from otlp spans
ericallam Feb 9, 2024
499e789
Export logs to events
ericallam Feb 9, 2024
1a083ca
Remove tracing stuff from the v3-catalog
ericallam Feb 9, 2024
2e57659
Pass worder id and version through to the events
ericallam Feb 9, 2024
d4fcc54
Extract out the CLI tracing stuff into core/v3
ericallam Feb 9, 2024
784b52c
Setup instrumentations
ericallam Feb 9, 2024
8a7cf38
Get typechecking working and building otlp-importer
ericallam Feb 9, 2024
841f50e
Fixed pnpm lock file after rebasing with main
ericallam Feb 12, 2024
b30394e
Fixed failing typecheck
ericallam Feb 12, 2024
5682749
Try to fix e2e tests
ericallam Feb 12, 2024
5726f82
Build the server before running e2e tests
ericallam Feb 12, 2024
6425ccb
Another attempt to fix e2e
ericallam Feb 12, 2024
9a16fe2
Fixed depds
ericallam Feb 12, 2024
80c2a91
Another attempt to fix e2e
ericallam Feb 12, 2024
1868494
Add external ref to e2e setup
ericallam Feb 12, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
WIP importing otlp traces and logs into TaskEvent table
  • Loading branch information
ericallam committed Feb 12, 2024
commit f561022050c7ca6b8348fa77af3dd9589f2a5923
3 changes: 3 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[submodule "packages/otlp-importer/protos"]
path = packages/otlp-importer/protos
url = https://github.com/open-telemetry/opentelemetry-proto.git
12 changes: 10 additions & 2 deletions apps/webapp/app/routes/otel.v1.logs.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
import { ActionFunctionArgs } from "@remix-run/server-runtime";
import { ExportLogsServiceRequest, ExportLogsServiceResponse } from "@trigger.dev/otlp-importer";
import { otlpExporter } from "~/v3/otlpExporter.server";

export function action({ request }: ActionFunctionArgs) {
return new Response(null, { status: 200 });
export async function action({ request }: ActionFunctionArgs) {
const buffer = await request.arrayBuffer();

const exportRequest = ExportLogsServiceRequest.decode(new Uint8Array(buffer));

const exportResponse = await otlpExporter.exportLogs(exportRequest);

return new Response(ExportLogsServiceResponse.encode(exportResponse).finish(), { status: 200 });
}
12 changes: 10 additions & 2 deletions apps/webapp/app/routes/otel.v1.traces.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
import { ActionFunctionArgs } from "@remix-run/server-runtime";
import { ExportTraceServiceRequest, ExportTraceServiceResponse } from "@trigger.dev/otlp-importer";
import { otlpExporter } from "~/v3/otlpExporter.server";

export function action({ request }: ActionFunctionArgs) {
return new Response(null, { status: 200 });
export async function action({ request }: ActionFunctionArgs) {
const buffer = await request.arrayBuffer();

const exportRequest = ExportTraceServiceRequest.decode(new Uint8Array(buffer));

const exportResponse = await otlpExporter.exportTraces(exportRequest);

return new Response(ExportTraceServiceResponse.encode(exportResponse).finish(), { status: 200 });
}
18 changes: 18 additions & 0 deletions apps/webapp/app/v3/eventRepository.server.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import { Prisma, TaskEventStatus, type TaskEventKind } from "@trigger.dev/database";
import { PrismaClient, prisma } from "~/db.server";

export type CreatableEvent = Omit<Prisma.TaskEventCreateInput, "id" | "createdAt">;
export type CreatableEventKind = TaskEventKind;
export type CreatableEventStatus = TaskEventStatus;

export class EventRepository {
constructor(private db: PrismaClient = prisma) {}

async insert(event: CreatableEvent) {
return this.db.taskEvent.create({ data: event });
}

async insertMany(events: CreatableEvent[]) {
return this.db.taskEvent.createMany({ data: events });
}
}
Loading