Skip to content

feat: Refactor backend to a rest api #346

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 24 commits into from
Jul 2, 2025
Merged

Conversation

ix-56h
Copy link
Contributor

@ix-56h ix-56h commented Jul 1, 2025

Main topic

The current webapp doesn't provide API endpoints to communicate with the backend, instead, we got endpoints that returns fully generated HTML pages, then, the frontend replace the whole DOM with the new HTML.

This PR provide a REST api and fix the auto generated openapi.json and Swagger UI.

Changes

  • /api old endpoints doesn't redirect to an deprecated docs
  • /api/ingest new endpoint provides an easy way to ingest any repository with a simple POST request.
  • new pydantic models for the ingest endpoint, both request body and responses.
  • The new endpoint returns a JSON response and provides well defined JSON response object for each possible scenarios (OK, ERROR, INTERNAL ERROR, MALFORMED REQUEST)
  • The new Swagger ui (and the generated openapi.json file used to render it) list only useful endpoints with comprehensive description (refer to the screen bellow)
    image

Testing

The old test_integration_flow.py have been fixed to make it works with the new endpoint.

To test it manually, run the uvicorn server and see the /docs endpoint to perform requests.

Please test the usual workflow (the url trick and ingest form submission), it works for me but i don't have enough experience with the product.

@ix-56h ix-56h self-assigned this Jul 1, 2025
@cyclotruc cyclotruc marked this pull request as draft July 1, 2025 00:45
filipchristiansen and others added 13 commits July 1, 2025 11:05
…eanup (coderamp-labs#349)

* refactor: centralize PAT validation, streamline repo checks & housekeeping

* `.venv*` to `.gitignore`
* `# type: ignore[attr-defined]` hints in `compat_typing.py` for IDE-agnostic imports
* Helpful PAT string in `InvalidGitHubTokenError` for easier debugging

* Bump **ruff-pre-commit** hook → `v0.12.1`
* CONTRIBUTING:
  * Require **Python 3.9+**
  * Recommend signed (`-S`) commits
* PAT validation now happens **only** in entry points
  (`utils.auth.resolve_token` for CLI/lib, `server.process_query` for Web UI)
* Unified `_check_github_repo_exists` into `check_repo_exists`, replacing
  `curl -I` with `curl --silent --location --write-out %{http_code} -o /dev/null`
* Broaden `_GITHUB_PAT_PATTERN`
* `create_git_auth_header` raises `ValueError` when hostname is missing
* Tests updated to expect raw HTTP-code output

* Superfluous “token can be set via `GITHUB_TOKEN`” notes in docstrings
* `.gitingestignore` & `.terraform` from `DEFAULT_IGNORE_PATTERNS`
* Token validation inside `create_git_command`
* Obsolete `test_create_git_command_invalid_token`

* Adjust `test_clone.py` and `test_git_utils.py` for new status-code handling
* Consolidate mocks after token-validation relocation

BREAKING CHANGE:
`create_git_command` no longer validates GitHub tokens; callers must ensure
tokens are valid (via `validate_github_token`) before invoking lower-level
git helpers.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
… front to make it works

fix weird behavior after rebase attempt

clean after rebase

refactor: centralize PAT validation, streamline repo checks & misc cleanup (coderamp-labs#349)

* refactor: centralize PAT validation, streamline repo checks & housekeeping

* `.venv*` to `.gitignore`
* `# type: ignore[attr-defined]` hints in `compat_typing.py` for IDE-agnostic imports
* Helpful PAT string in `InvalidGitHubTokenError` for easier debugging

* Bump **ruff-pre-commit** hook → `v0.12.1`
* CONTRIBUTING:
  * Require **Python 3.9+**
  * Recommend signed (`-S`) commits
* PAT validation now happens **only** in entry points
  (`utils.auth.resolve_token` for CLI/lib, `server.process_query` for Web UI)
* Unified `_check_github_repo_exists` into `check_repo_exists`, replacing
  `curl -I` with `curl --silent --location --write-out %{http_code} -o /dev/null`
* Broaden `_GITHUB_PAT_PATTERN`
* `create_git_auth_header` raises `ValueError` when hostname is missing
* Tests updated to expect raw HTTP-code output

* Superfluous “token can be set via `GITHUB_TOKEN`” notes in docstrings
* `.gitingestignore` & `.terraform` from `DEFAULT_IGNORE_PATTERNS`
* Token validation inside `create_git_command`
* Obsolete `test_create_git_command_invalid_token`

* Adjust `test_clone.py` and `test_git_utils.py` for new status-code handling
* Consolidate mocks after token-validation relocation

BREAKING CHANGE:
`create_git_command` no longer validates GitHub tokens; callers must ensure
tokens are valid (via `validate_github_token`) before invoking lower-level
git helpers.

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

remove api ingest endpoint test (duplicate)

add uv.lock to gitignore
@ix-56h ix-56h changed the title WIP: Refactor backend to a rest api feat: Refactor backend to a rest api Jul 1, 2025
Co-authored-by: Nicolas Iragne <nicoragne@hotmail.fr>
…, remove sensitive data from the Success Response object, make the integration unit test compatible
@ix-56h ix-56h marked this pull request as ready for review July 1, 2025 23:32
@filipchristiansen filipchristiansen self-requested a review July 2, 2025 00:45
@filipchristiansen
Copy link
Contributor

I think you can make these changes

git_form.jinja

-              onsubmit="handleSubmit(event{% if is_index %}, true{% endif %})">
+              onsubmit="handleSubmit(event, true)">

git.jinja
-    {% with is_index=false, show_examples=false %}
+    {% with show_examples=false %}


index.jinja
-    {% with is_index=true, show_examples=true %}
+    {% with show_examples=true %}

@ix-56h ix-56h merged commit 2b1f228 into coderamp-labs:main Jul 2, 2025
18 checks passed
BareninVitalya pushed a commit to BareninVitalya/gitingest that referenced this pull request Jul 6, 2025
* refactor: Refactor backend to a rest api and make weak changes to the front to make it works

* fix: remove result bool in Success response object since it's useless, remove sensitive data from the Success Response object, make the integration unit test compatible

* fix: clean query processor return types, remove deprecated fields, better handling for few errors

* fix: unit tests, remove deprecated is_index from jinja templates

---------

Co-authored-by: ix <n.guintini@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants