refactor: backend and removal of temporal worker #111

vikaxsh · 2025-07-01T07:03:44Z

Description

Fixes # (issue)

Type of change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
This change requires a documentation update

How Has This Been Tested?

Scenario A
Scenario B

Screenshots or Recordings

Related PR's (If Any):

… into refactor/bff

server/internal/database/job.go

server/cmd/temporal-worker/main.go

server/internal/handlers/destination.go

server/internal/handlers/user.go

… into refactor/bff

…er operations

hash-data

Once change the go project name to olake-ui

server/cmd/temporal-worker/main.go

server/internal/database/destination.go

server/internal/database/job.go

server/internal/handlers/auth.go

server/internal/handlers/handlers_utils.go

server/internal/handlers/user.go

vishalm0509 · 2025-07-29T07:05:23Z

There are few lint errors and warnings please look into them as well

server/internal/database/job.go

vishalm0509

Please check the comments and make changes if relevant.

vikaxsh · 2025-08-04T06:49:59Z

jobs should be in sorted order of updated at time

hash-data · 2025-08-09T10:26:35Z

Temporal and Main server should use same db passed in env

tanishaAtDatazip · 2025-08-11T10:36:13Z

move routes folder inside internal and each router should have its seperate file like routes/source.go:

server/
  routes/
    auth.go
    users.go
    sources.go
    destinations.go
    routes.go   // central Init() that calls sub-inits

…ection

server/internal/handlers/auth.go

server/internal/handlers/destination.go

server/internal/handlers/job.go

.github/workflows/build-and-release.yml

server/internal/constants/constants.go

server/internal/handlers/job.go

server/internal/services/etl/destination.go

server/internal/services/etl/source.go

server/internal/services/temporal/client.go

…t connection

* fix: fix job history retry * fix: fix destination spec api

* fix: send id in job edit * fix: add modal for spec error, fix double error messages * fix: integration test fix * fix: minor fix

* chore: remove temporalAddr constant and make retry simple * chore: remove logs

fix: add cleanup on cancelling fetchSpec API

hash-data

LGTM, nice work @vikaxsh, and on worker side @vishalm0509

vishalm0509

Approved

vikaxsh added 6 commits June 12, 2025 11:26

fix(api): refactor raw queries and add constants

0ca9bab

chore(api): Adding service layer

b6c00d6

chore(api): adding service layer

190b89f

Merge branch 'staging' of https://github.com/datazip-inc/olake-frontend…

922c2da

… into refactor/bff

fix: resolve conflict errors

9a00631

feat: improve Worker design by passing existing Client

b4bb565

hash-data reviewed Jul 7, 2025

View reviewed changes

vikaxsh added 7 commits July 10, 2025 15:32

Merge branch 'staging' of https://github.com/datazip-inc/olake-frontend…

679e4ce

… into refactor/bff

feat: improving error handling a

934cb1e

feat: add info logging

d152357

fix: changes after testing

a1142de

Merge branch 'staging' of https://github.com/datazip-inc/olake-frontend…

3bdd367

… into refactor/bff

feat(docker): use Docker Go SDK instead of shell commands for contain…

c4e93e8

…er operations

feat: updated GoSec security ci

489a2ea

vikaxsh changed the title ~~Refactor/bff~~ feat(refactor): introduce service layer and migrate from shell to Docker SDK Jul 18, 2025

vikaxsh marked this pull request as ready for review July 18, 2025 05:18

feat: delete config files

5942237

hash-data reviewed Jul 25, 2025

View reviewed changes

server/cmd/temporal-worker/main.go Outdated Show resolved Hide resolved

vishalm0509 reviewed Jul 25, 2025

View reviewed changes

server/internal/database/destination.go Show resolved Hide resolved

vishalm0509 reviewed Jul 25, 2025

View reviewed changes

server/internal/database/job.go Outdated Show resolved Hide resolved

vishalm0509 reviewed Jul 25, 2025

View reviewed changes

server/internal/database/job.go Outdated Show resolved Hide resolved

vishalm0509 requested changes Jul 29, 2025

View reviewed changes

server/internal/handlers/auth.go Show resolved Hide resolved

server/internal/handlers/auth.go Show resolved Hide resolved

server/internal/handlers/handlers_utils.go Outdated Show resolved Hide resolved

server/internal/handlers/user.go Show resolved Hide resolved

vishalm0509 reviewed Jul 29, 2025

View reviewed changes

server/internal/database/job.go Outdated Show resolved Hide resolved

vishalm0509 requested changes Jul 29, 2025

View reviewed changes

vikaxsh added 2 commits July 30, 2025 15:07

chore: update project name to olake-ui

376a12b

chore: addressed review comments

8213098

vikaxsh added 6 commits November 2, 2025 18:04

fix: security ci

7e527f9

fix: panic issue and add unique name check in job creation

b12fafc

feat: added job cancel in job updatea and return error from test conn…

a6ad22a

…ection

chore: move temporal and etl into services folder

0f566d8

chore: use same name in sevice and handlers

68b2397

fix: source test connection

b9bbc49

hash-data reviewed Nov 3, 2025

View reviewed changes

vikaxsh and others added 4 commits November 3, 2025 19:07

fix: move result nil check before error in source and destiantion tes…

ce8b05c

…t connection

fix: fix job history retry and destination spec api (#244)

6ffd9f6

* fix: fix job history retry * fix: fix destination spec api

fix: remove hardcoding branch name in build-ui workflow

d0053fb

fix: remove k8s and docker task queue

268e7b2

vishalm0509 temporarily deployed to Build UI November 3, 2025 17:07 — with GitHub Actions Inactive

vishalm0509 and others added 4 commits November 3, 2025 22:41

fix: lint issue

7b62dd2

fix: error in login and test destiantion

e5dfb84

fix: add retry logic for temporal client init

a849ec9

fix: send id in job edit (#245)

4091964

* fix: send id in job edit * fix: add modal for spec error, fix double error messages * fix: integration test fix * fix: minor fix

vishalm0509 temporarily deployed to Build UI November 6, 2025 09:38 — with GitHub Actions Inactive

vishalm0509 and others added 7 commits November 7, 2025 12:00

chore: update worker tag in docker-compose for int. test

a825e8c

chore: remove temporal address from constant (#246)

029ebfb

* chore: remove temporalAddr constant and make retry simple * chore: remove logs

fix: add cleanup on cancelling fetchSpec API

dad4009

chore: resolve review comments

f54819f

chore: remove unused messages

2c09805

Merge pull request #248 from datazip-inc/fix/add-cleanup-fetchSpec-api

df41e5d

fix: add cleanup on cancelling fetchSpec API

fix: integration test - attempt 1

58f5edd

vishalm0509 had a problem deploying to Build UI November 7, 2025 12:18 — with GitHub Actions Failure

chore: added a todo

292b8a3

hash-data approved these changes Nov 7, 2025

View reviewed changes

vishalm0509 approved these changes Nov 7, 2025

View reviewed changes

hash-data merged commit ace4121 into staging Nov 7, 2025
10 checks passed

hash-data deleted the refactor/bff branch November 7, 2025 12:39

refactor: backend and removal of temporal worker #111

refactor: backend and removal of temporal worker #111

Uh oh!

Conversation

vikaxsh commented Jul 1, 2025

Description

Type of change

How Has This Been Tested?

Screenshots or Recordings

Related PR's (If Any):

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hash-data left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

vishalm0509 commented Jul 29, 2025

Uh oh!

Uh oh!

vishalm0509 left a comment

Choose a reason for hiding this comment

Uh oh!

vikaxsh commented Aug 4, 2025

Uh oh!

hash-data commented Aug 9, 2025

Uh oh!

tanishaAtDatazip commented Aug 11, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hash-data left a comment

Choose a reason for hiding this comment

Uh oh!

vishalm0509 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants