feat(ui): make exclude/include pattern field easier to read & edit

[filipchristiansen]

Once a few patterns are added, the single-line input can’t display them all, so editing becomes awkward (scrolling sideways, guessing hidden text, etc.). A multiline, auto-expand, or chip-style control would make managing patterns much smoother.

If you have an idea how this can be handled better and implemented, please let us know!

[NicolasIRAGNE]

Here's a quick mockup of something I was thinking of
Pretty much the same thing as current but with a dynamically updated tree and bigger textbox

chrome_qOKKyROx4A.mp4

[filipchristiansen]

Looks great—thanks for the mock-up, @NicolasIRAGNE! cc @cyclotruc for thoughts.

One additional nice-to-have we could explore: letting a click on a directory collapse it (toggling the caret from ▼ to ►).

[cyclotruc]

Love this mockup, we should definitly aim for that, I think next important step for the project is to split the front-end logic with a /api route that returns a json, this will make this kind of front-end processing much easier once we have a proper json object to parse in frontend

We should indeed include @filipchristiansen's idea for the collapse as well

[Cheelax]

I am working on my own saas related to AI and I think your tool might be interesting to use in my project.
This project imo is very interesting to optimize the context and make it lighter to use in llm

[cyclotruc]

@Cheelax happy to hear that,
If you have any questions/needs related to your project, don't hesitate to reach me at romain@coderamp.io or on discord (easier) at @yasbaltrine

[zamora-carlos]

Hi, I’d like to help with this issue. I really like the mockup and the idea of making the tree structure collapsible.

I do have a couple of questions, though. When the user interacts with the exclude patterns text area, it updates the tree structure, and vice versa. But what happens to the previous exclude/include pattern field? Will it be replaced by the new pattern text area, or will it remain on the page in the same place, simply reflecting updates from the tree and the text area, so that when the user submits the form this is the data that gets send? I was thinking the latter might make more sense, but I’d love to hear your thoughts.

Regarding the .gitignore-style patterns, what do you think would be the best approach for updating the tree structure as the user types? Should we use a frontend library, or perhaps an endpoint in the API?

I also created a Figma showing a possible layout for the updated results section. I’d appreciate any feedback on the layout and the order of the sections. For instance, should the summary appear before or after the file system tree? I was initially concerned that the summary might be too long, but I saw there's an open issue to include more information in that section, so maybe that is okay. I left out the Use .gitignore from repo checkbox because I don't think it affects the output. I created two versions: one that includes the quick examples section and one that omits it.

I put together a CodePen with a working example of the new section. For the .gitignore-style pattern matching, I’m using a bundled version of the picomatch package. Since that package doesn’t support negated patterns, I’m handling that part manually in the createGitIgnoreMatcher function. The rest of the code consists of a class to organize the tree and more functions to render and update the UI.

[cyclotruc]

@zamora-carlos Wow incredible work you did here, thanks a lot!
We've reviewed your Figma and it gave us a lot of food for thought on the future UX improvements

Here's my full feedback:
First, let's state once again that simplicity is what people love about the tool so far

currently, we can say the web UI is divided in 2 parts:

The issue with introducing more capabilities via the tree selection is that it would make the output section act as an input one as well, which I'm not a fan of (even if it's already the case with the current selection method)

So one thing that appears quite clear to me is that this new selection menu should be in the first section and not the result one

The easiest way I can think of would be to introduce an "advanced settings" dropdown that reveals all the power-user features such as this one

It could look like this:

and would reveal all the settings that we don't want to expose straight up on the landing page

Maybe it can be a checkbox that reads:
"simple mode" -> "advanced mode"

Other than that, I think we're on the right track with the UI you made

• About the selection menu:

My personal pick would be this one, the idea is "what you see is what you get" and I want people to get the same clipboard content if they click the copy button or if they select + ctrl C themselves in the text area

That being said, I like the idea of improving the subfolder scructure like this (and like you did in the codepen)

So in the end, I think the best of both world would be to have this in the webUI:

But making sure that either when we click the "copy" button or when we select manually, we always get an output that looks like:

Directory structure:
└── cyclotruc-gitingest/
    ├── README.md
    ├── CODE_OF_CONDUCT.md
    ├── CONTRIBUTING.md
    ├── Dockerfile
    ├── LICENSE
    ├── pyproject.toml
    ├── requirements-dev.txt

[cyclotruc]

Just a heads up before implementing anything: We should make an important refactor of the webUI beforehand

Currently, the webUI returns the whole HTML page containing the result as a response to the POST request that is triggered on clicking ingest (I didn't know better at the time sorry)

A much better approach now seems obvious to me:
gitingest.com or gitingest.com/user/repo only return the empty HTML pages
and the actual data for the digest is fetched through a classic REST endpoint

I'm thinking api.gitingest.com, in order to keep the url trick as easy as possible

(the other option would be to serve on gitingest.com/api/... but I feel like it's more complicated to replace github.com by gitingest.com/api than api.gitingest.com )

I need to make an Issue for this one but I planned to do it myself eventually

[zamora-carlos]

Hello @cyclotruc, thank you for the feedback. I agree it's a good idea to separate the inputs from the results section. I've added some updated screenshots to the same Figma file, showing how I think the Advanced Settings section could look. Here’s a CodePen version in case you'd like to tweak the layout or colors.

Considerations

• What other elements should be included in the Advanced Settings?
  Besides the text area with the .gitignore style patterns and the directory structure, should we also include the Private Access Token field?

• What should we show before the user submits the form?

  • Directory structure: At this point we don't have the tree structure of the repository. Should we display an empty div, or just remove it?

  • Textarea: Since we don't have the directory structure, Its only purpose would be to update the include/exclude pattern field.
    In the CodePen I sent last time, I used the textarea + directory structure to determine which files to include/exclude, then updated the input field with only the full paths that matched. Another approach would be to send the textarea content directly to the server and let it handle the matching. If we go that route, I think we can remove the include/exclude pattern input field when showing the Advanced Settings and just keep the selector for include/exclude mode. What I don't like about this approach is sending a request every time the user types just to update the UI. Adding a debounce could help, but it might make the website feel slow.

One idea I’ve been thinking is including the full directory structure of the repository in the Advanced Settings section. This would give users better control when selecting which files to include or exclude. Then, in the results section, we could show the plain-text non-interactive structure of only the ingested files.

Might be a little confusing but it's worth considering. The other only thing we would have to take into account is preserving the text area content used in the last ingest, so users don’t lose their previous patterns.

Selection menu

I also created the version you said you liked the most. You can check it out here

This version is much simpler in terms of markup and styling. Now the user can actually see full text selection. The only drawback is that the expand/collapse button leaves visual gap when selecting text:

In the first version, I'd added hidden elements with box-drawing characters so the format would be preserved when copy-pasting. But the user can't see the characters when copying text. I think both versions could work.

[cyclotruc]

Once again wonderful work @zamora-carlos, thank you!!

What other elements should be included in the Advanced Settings?

For now, no other elements are needed but we'll introduce new options with time in this section

One idea I’ve been thinking is including the full directory structure of the repository in the Advanced Settings section. This would give users better control when selecting which files to include or exclude. Then, in the results section, we could show the plain-text non-interactive structure of only the ingested files.

Yes, that's what I was thinking as well, it would be a bit redundant but I don't mind, since people will mostly:
a - not open advanced settings
b - get scrolled down to the result section after ingesting

Maybe we should even make the directory tree in the result section interactive identical to the one in advanced settings, idk yet

What I don't like about this approach is sending a request every time the user types just to update the UI. Adding a debounce could help, but it might make the website feel slow.

Maybe I'm not understanding right, but with the rest API refactor that is currently being worked on, we should be able to handle a "virtual" tree in browser (a bit like virtual DOM) and update the frontend/digest without requiring any calls to the backend

What should we show before the user submits the form?
Good question, since it won't be possible to show the advanced settings tree without first doing an ingest, I think not showing the div is fine (or maybe a spinner?)

The selection menu is wonderful! so great that you made that interactive codepen, love to play with the collapsible folders, makes me think we might want to collapse by default some deeply nested folders (maybe always keep depth 1 and 2 expanded, and collapse the rest)

One idea I'm curious but not convinced about:
should we have the include/exclude list displayed as a list of pills/tags ? by pills I mean like the labels here:

with a cross icon to remove them maybe

Not sure tho because keeping it very text-looking would stick to the habit of .gitignore

Thanks again for this work, this really helps a lot to get an idea of what this future UI could feel like

[zamora-carlos]

Hi @cyclotruc,

No other elements are needed, but we'll introduce new options in this section over time.

Ohh okay, that makes it clearer for me. Just one question, would the section to include the Private Access Token be considered an advanced setting or a regular one?

Maybe I'm not understanding this correctly, but with the REST API refactor currently in progress, we should be able to handle a "virtual" tree in the browser (a bit like a virtual DOM) and update the frontend/digest without requiring backend calls.

Yes, you're right, but I was referring to a potential issue. Sorry I didn’t explain it very well.

When I was working on the sample CodePen with the interactive tree and textarea, I wanted to see if in the website I could send negated patterns in the form, but I couldn’t. That left me with the impression that the backend didn’t support them, I’m not sure though.

As a workaround, I took the patterns the user typed in the textarea, matched them against the full file paths of files/folders, and generated a list of matches. Obviously that list of matches was necessary to update the UI, but I also pasted it into the input element that's sent to the backend. As you can see, those inputs have different values:

What I meant in the previous comment is that this approach won’t work without having the directory structure in the frontend. In that case, we’d need to send all the patterns the user typed in the textarea directly to the backend.

I mentioned the part about doing requests because there might be differences between how the library matches patterns and how the backend does it, but maybe that won’t be an issue. Either way, I think the backend should at least be able to handle negated patterns for ingestion.

We might want to collapse deeply nested folders by default (maybe always keep depth 1 and 2 expanded, and collapse the rest).

That sounds like a good idea. Maybe we could even add a couple of buttons to collapse or expand everything.

Should we display the include/exclude list as a list of pills/tags?

Are you referring to the list inside the textarea of the advanced settings, or the other input? Do you mean something like this?

It might look a bit strange in a textarea, but it’s definitely worth a try. I like the version on this page, though it’s a little different, it initially looks like an input and then it expands vertically.