Open
Description
Hi! First off, thank you for your fantastic work—gitingest is incredibly helpful to many of us.
Feature Proposal
I’d like to propose an optional toggle that outputs the directory structure in a more token-efficient format, optimized for LLM consumption.
Motivation
The current ASCII tree output is beautifully readable for humans, but it adds unnecessary overhead in token count, given the intended consumers are LLMs... An idea to add an option enabling a more compact format comes to my mind.
Comparison of token counts for various formats
Summary
| Format | Token Count |
|---|---|
| Fully indented ASCII tree as currently implemented | 1059 |
| Flattened ASCII tree | 646 |
| Markdown List-like | 584 |
| Double-space indentation | 484 |
Details
Current output
Fully indented ASCII tree 1059 tokens
Directory structure:
└── cyclotruc-gitingest/
├── README.md
├── CODE_OF_CONDUCT.md
├── CONTRIBUTING.md
├── Dockerfile
├── LICENSE
├── package.json
├── pyproject.toml
├── requirements-dev.txt
├── requirements.txt
├── SECURITY.md
├── tailwind.config.js
├── .dockerignore
├── .pre-commit-config.yaml
├── src/
│ ├── gitingest/
│ │ ├── __init__.py
│ │ ├── cli.py
│ │ ├── clone.py
│ │ ├── config.py
│ │ ├── entrypoint.py
│ │ ├── ingestion.py
│ │ ├── output_formatter.py
│ │ ├── query_parser.py
│ │ ├── schemas/
│ │ │ ├── __init__.py
│ │ │ ├── filesystem.py
│ │ │ └── ingestion.py
│ │ └── utils/
│ │ ├── __init__.py
│ │ ├── auth.py
│ │ ├── compat_func.py
│ │ ├── compat_typing.py
│ │ ├── exceptions.py
│ │ ├── file_utils.py
│ │ ├── git_utils.py
│ │ ├── ignore_patterns.py
│ │ ├── ingestion_utils.py
│ │ ├── notebook.py
│ │ ├── os_utils.py
│ │ ├── path_utils.py
│ │ ├── query_parser_utils.py
│ │ └── timeout_wrapper.py
│ ├── server/
│ │ ├── __init__.py
│ │ ├── form_types.py
│ │ ├── main.py
│ │ ├── models.py
│ │ ├── query_processor.py
│ │ ├── server_config.py
│ │ ├── server_utils.py
│ │ ├── routers/
│ │ │ ├── __init__.py
│ │ │ ├── download.py
│ │ │ ├── dynamic.py
│ │ │ ├── index.py
│ │ │ └── ingest.py
│ │ └── templates/
│ │ ├── base.jinja
│ │ ├── git.jinja
│ │ ├── index.jinja
│ │ └── components/
│ │ ├── _macros.jinja
│ │ ├── footer.jinja
│ │ ├── git_form.jinja
│ │ ├── navbar.jinja
│ │ └── result.jinja
│ └── static/
│ ├── llm.txt
│ ├── robots.txt
│ ├── css/
│ │ └── tailwind.css
│ └── js/
│ ├── git.js
│ ├── git_form.js
│ ├── index.js
│ ├── navbar.js
│ ├── posthog.js
│ └── utils.js
├── tests/
│ ├── __init__.py
│ ├── conftest.py
│ ├── test_cli.py
│ ├── test_clone.py
│ ├── test_flow_integration.py
│ ├── test_git_utils.py
│ ├── test_gitignore_feature.py
│ ├── test_ingestion.py
│ ├── test_notebook_utils.py
│ ├── .pylintrc
│ └── query_parser/
│ ├── __init__.py
│ ├── test_git_host_agnostic.py
│ └── test_query_parser.py
└── .github/
├── dependabot.yml
└── workflows/
├── ci.yml
├── publish.yml
└── scorecard.yml
Some variants I tried at first
including these just for the record...
flattened ASCII tree 646 tokens
Directory structure:
cyclotruc-gitingest/
├README.md
├CODE_OF_CONDUCT.md
├CONTRIBUTING.md
├Dockerfile
├LICENSE
├package.json
├pyproject.toml
├requirements-dev.txt
├requirements.txt
├SECURITY.md
├tailwind.config.js
├.dockerignore
├.pre-commit-config.yaml
├src/
│├gitingest/
││├__init__.py
││├cli.py
││├clone.py
││├config.py
││├entrypoint.py
││├ingestion.py
││├output_formatter.py
││├query_parser.py
││├schemas/
│││├__init__.py
│││├filesystem.py
│││└ingestion.py
││└utils/
││├__init__.py
││├auth.py
││├compat_func.py
││├compat_typing.py
││├exceptions.py
││├file_utils.py
││├git_utils.py
││├ignore_patterns.py
││├ingestion_utils.py
││├notebook.py
││├os_utils.py
││├path_utils.py
││├query_parser_utils.py
││└timeout_wrapper.py
│├server/
││├__init__.py
││├form_types.py
││├main.py
││├models.py
││├query_processor.py
││├server_config.py
││├server_utils.py
││├routers/
│││├__init__.py
│││├download.py
│││├dynamic.py
│││├index.py
│││└ingest.py
││└templates/
││├base.jinja
││├git.jinja
││├index.jinja
││└components/
││├_macros.jinja
││├footer.jinja
││├git_form.jinja
││├navbar.jinja
││└result.jinja
│└static/
│├llm.txt
│├robots.txt
│├css/
││└tailwind.css
│└js/
│├git.js
│├git_form.js
│├index.js
│├navbar.js
│├posthog.js
│└utils.js
├tests/
│├__init__.py
│├conftest.py
│├test_cli.py
│├test_clone.py
│├test_flow_integration.py
│├test_git_utils.py
│├test_gitignore_feature.py
│├test_ingestion.py
│├test_notebook_utils.py
│├.pylintrc
│└query_parser/
│├__init__.py
│├test_git_host_agnostic.py
│└test_query_parser.py
└.github/
├dependabot.yml
└workflows/
├ci.yml
├publish.yml
└scorecard.yml
Markdown List-like 584 tokens
It allows collapsing if opened some editor as Markdown, but then the `__` characters would need to be escaped.
Directory structure:
cyclotruc-gitingest/
- README.md
- CODE_OF_CONDUCT.md
- CONTRIBUTING.md
- Dockerfile
- LICENSE
- package.json
- pyproject.toml
- requirements-dev.txt
- requirements.txt
- SECURITY.md
- tailwind.config.js
- .dockerignore
- .pre-commit-config.yaml
- src/
- gitingest/
- __init__.py
- cli.py
- clone.py
- config.py
- entrypoint.py
- ingestion.py
- output_formatter.py
- query_parser.py
- schemas/
- __init__.py
- filesystem.py
- ingestion.py
- utils/
- __init__.py
- auth.py
- compat_func.py
- compat_typing.py
- exceptions.py
- file_utils.py
- git_utils.py
- ignore_patterns.py
- ingestion_utils.py
- notebook.py
- os_utils.py
- path_utils.py
- query_parser_utils.py
- timeout_wrapper.py
- server/
- __init__.py
- form_types.py
- main.py
- models.py
- query_processor.py
- server_config.py
- server_utils.py
- routers/
- __init__.py
- download.py
- dynamic.py
- index.py
- ingest.py
- templates/
- base.jinja
- git.jinja
- index.jinja
- components/
- _macros.jinja
- footer.jinja
- git_form.jinja
- navbar.jinja
- result.jinja
- static/
- llm.txt
- robots.txt
- css/
- tailwind.css
- js/
- git.js
- git_form.js
- index.js
- navbar.js
- posthog.js
- utils.js
- tests/
- __init__.py
- conftest.py
- test_cli.py
- test_clone.py
- test_flow_integration.py
- test_git_utils.py
- test_gitignore_feature.py
- test_ingestion.py
- test_notebook_utils.py
- .pylintrc
- query_parser/
- __init__.py
- test_git_host_agnostic.py
- test_query_parser.py
- .github/
- dependabot.yml
- workflows/
- ci.yml
- publish.yml
- scorecard.yml
Most economical variant (preferable)
Structure using double space indentation only 484 tokens
Directory structure:
cyclotruc-gitingest/
README.md
CODE_OF_CONDUCT.md
CONTRIBUTING.md
Dockerfile
LICENSE
package.json
pyproject.toml
requirements-dev.txt
requirements.txt
SECURITY.md
tailwind.config.js
.dockerignore
.pre-commit-config.yaml
src/
gitingest/
__init__.py
cli.py
clone.py
config.py
entrypoint.py
ingestion.py
output_formatter.py
query_parser.py
schemas/
__init__.py
filesystem.py
ingestion.py
utils/
__init__.py
auth.py
compat_func.py
compat_typing.py
exceptions.py
file_utils.py
git_utils.py
ignore_patterns.py
ingestion_utils.py
notebook.py
os_utils.py
path_utils.py
query_parser_utils.py
timeout_wrapper.py
server/
__init__.py
form_types.py
main.py
models.py
query_processor.py
server_config.py
server_utils.py
routers/
__init__.py
download.py
dynamic.py
index.py
ingest.py
templates/
base.jinja
git.jinja
index.jinja
components/
_macros.jinja
footer.jinja
git_form.jinja
navbar.jinja
result.jinja
static/
llm.txt
robots.txt
css/
tailwind.css
js/
git.js
git_form.js
index.js
navbar.js
posthog.js
utils.js
tests/
__init__.py
conftest.py
test_cli.py
test_clone.py
test_flow_integration.py
test_git_utils.py
test_gitignore_feature.py
test_ingestion.py
test_notebook_utils.py
.pylintrc
query_parser/
__init__.py
test_git_host_agnostic.py
test_query_parser.py
.github/
dependabot.yml
workflows/
ci.yml
publish.yml
scorecard.yml