Repository structure to accommodate frontend code

[pavish]

This discussion is regarding how we are going to structure our repository for frontend code.

Requirements:

1. Requires it’s own pipeline for lint, test and build steps.
2. The built frontend assets need to be placed on the static served files directory of the server.
3. Contributors should be able to contribute to both frontend and backend code without much hassle.

Approaches:

1. Mono-repo

We could have a mono repo with frontend code within a subfolder of the Mathesar repo.

This folder could contain different tooling, and config files of it’s own and would function independent, as needed for frontend development. The build scripts of the frontend would ensure that the built assets are moved to the static folder of the backend web server.

Pros:

1. Single repo - easier for people to work across frontend and backend.
2. Initial setup for development is simple, docker-compose can take care of setting up the frontend and building it.

Cons:

1. The tooling for frontend and backend are independent and each require separate setup/structure.
2. There will be no repo-wide consistency in structure. Repo wide configuration such as linting will have to be specified separately for backend and frontend.
3. Single Pipeline would need steps to handle both frontend and backend.

---

2. Individual repositories

We could have a completely different repo for the frontend. docker-compose can have a step to download the latest/specific supported frontend build and place it within the webserver’s static folder.

Pros:

1. Clean separation of concerns: code, tooling, reviews and pipelines.
2. Development and releases can happen independently.

Cons:

1. Requires working across two repos for each new feature.
2. Initial setup for development across both would take more steps. In-order to work on the frontend, in their local environment, the developer would need scripts to move the built assets to the Mathesar server.

---

3. Using Git Submodules

We could use git Submodules to have a mixed structure. Mathesar can have a submodule that references to the mathesar client repo. This allows tracking, and being able to access/work on the submodule repo from the parent repo.

Pros:

1. Works similar to a monorepo, has both advantages of it.
2. Since frontend is also a separate repo, pipelines and tooling can be independent.

Cons:

1. Adds additional complexity to PRs when they have different branches in parent tracking their own feature branches in frontend. First the frontend will have to be released/merged and that commit has to be mentioned in the parent branch before merging it.
2. Committing changes from parent would require committing changes to submodule and then updating parent to track latest commit.

---

Currently, I’m inclined with approach 2, which will offer clean separation of concerns, and it allows us to shift easily to approach 3 in the future, if such a need is warranted.

I may have overlooked some requirements/approaches here. I’d like to hear more of your thoughts around this.

[kgodey]

I think option 1 is best for our use case. The frontend and backend are a single product, it's not like the frontend is just a client of the backend. We do not want to release them separately, we just want a single version of Mathesar, and coordinating across repositories will make that harder.

In addition, we should use the backend to pre-render data on the frontend and avoid network calls wherever possible, and that's going to be easiest if they are in a single repo.

[pavish]

Rethinking over the approaches, I agree with this. I also think a mono repo will be the best approach for us.

[pavish]

We would need:

• A separate service in docker-compose to build the client files for development - logistics need to be figured out.
• For production builds, we will have to build client and place it in static web server folder before creating a bundle/container.
• Separate workflow for lint, test and build steps for client.

[kgodey]

@pavish sounds good, could you make a GitHub issue to set these up?

[pavish]

@kgodey Created #59

We would need to decide upon the frontend framework before proceeding with this. #55

[mathemancer]

My default choice would be (2), but only because if we're wrong, it's easier (in my experience) to make the change (2) -> (1) or (2) -> (3) than it is to make a change like (1) -> (2).

I'm not overly worried either way, though.

[kgodey]

I'm not worried about portability, the frontend code will be in its own folder.