Automation / Testing / Visual documentation framework

[ocornut]

(EDIT 2019: Even though this is an active topic (in 2019) note that this thread started a looong time ago and the early messages are not representive of where we are going now. When the new testing framework is a mature enough to be shared I'll close this topic and open a new one.)

(EDIT 2022: Released Dear ImGui Test Engine + Dear ImGui Test Suite
https://github.com/ocornut/imgui_test_engine)

Looking for someone who would be interested in working on that, probably under my guidance.

Mentioned it in #259

Add a framework for generating screenshots/animations for given pieces of code. This is probably going to be done along with the "testing" framework. I would like the ImGui documentation to be a very visual, well organized listing with C++ code on one side, screenshot (or animated GIF) on the other side and tags + commentary. I think this will easily convey a lot of ideas and tips about ways to use ImGui.
Update: I've created some helper to help me take manual screenshots and its been very helpful in the past few days. This will probably evolve into such framework.

The idea would be have a separate test application, could be or not on the same repo, that would A/ generate screenshots/gif + code markup output for the wiki and B/ defacto run tests.

So we could describe those documentation elements in the code, the framework would execute the test, simulate inputs to interact, take screenshots or gif, crop them as desired, parse its own .cpp file to retrieve the code (which itself could have marking in comments to hide some lines) and generate the help section that would demonstrate ways to use ImGui by showing the code and the screenshot. We would have hundreds of short examples, grouped, chaptered, tagged, indexed.

I was originally primarily interested in the value of documentation but I think the overlap with testing will be enormous and natural. Just the act of generating the documentation from runtime will be an act of testing. The testing wouldn't be unit tests at ultra fine granularity but rather functional tests (at least this is what I think they are called in the real world?): say, move mouse to corner, click and drag. We can compare screenshot, regression easily caught. Tests would be merely the unpublished part of the documentation system. Any tricky area of the code or fixed bug can be subject to a test, etc.

It's not really a trivial task to set all that up and I think it'd be highly valuable to the project. Realistically I don't think I'd be able to tackle by myself this year, so looking around if someone is interested to contribute :)

The person would be expected to be proficient in writing terse, optimal C++ and willing to adopt the general programming style of ImGui.

Thanks!

[ocornut]

PS: to clarify even if we could aim to have lots of it in a separate file, that would still involve touching a lot of the core library, and some of the changes will probably lead to imgui's own public api development.

[emoon]

This sound like an awesome idea. I have thought about this before as I want to document the UI API in my debugger. The API is close the exactly the same but with a different naming scheme but that is very minor. I would likely try to use this as well (given permission of course) in a slightly altered form.

[emoon]

I'm bit interested in helping out with this.

I wonder if you have some ideas on how to do it? That being said I don't have lots of time doing this but I want to help out at least.

[ocornut]

Thanks Daniel and sorry for not answering to this earlier. :)
Some out-of-order notes on top of my head.

That picture is essentially showing the intent:

Except it's all wrong because:

• the code is embedded in a picture
• it's just showing random unrelated stuff
• the example isn't complete and lack context
• and mostly, it was painfully and manually authored

I think a first step would be to manually draft more example documentation. Which would naturally lead to creating the framework for this. It would probably embed something like the screenshot helper here https://github.com/ocornut/imgui/wiki/screenshot_tool

How should the documentation be presented? When and how can we be skip context? Then we figure step by step out how we can make this work easier and more automated. Believe it or not I feel pretty bad with my English when it comes to presenting things in a consistent manner so I could do with some help :)

I am not too concerned about tackling the testing part too explicitly at first. ImGui doesn't really crash so tackling this possibility in an automated manner isn't an important priority ihmo. I think for a start the documentation itself can be defacto testing.

Overview style vs Single-function/symbol style

I think there would be more value in providing chapters/article about series of functions rather than individual functions. There's nothing more frustrating that documentation which have a line about each functions but you never get to understand the whole picture. So I would rather have a chapters called "Menus & Popups" and say, BeginMenu() EndMenu() are documented together, than separate entries. It's still possibly to have separate entries for every functions as the second half of those chapters if we feel it makes sense. As the output will be automated we will be able to build a full index anyway.

Also, the documentation shall be seen as a general "usage guide" which include suggestions, tips, making uses of several functions to benefit from a certain pattern, etc. so I feel it is nice to not overly focus on single function/symbols.

Note about repo size: I would prefer to keep the main repo small because any commit ends up in user clone. So if we end up uploading pictures they could be on the Wiki repo (which would make sense for the documentation anyway) or we create a imgui-docs repo just to host the data

[ocornut]

Here's a quick example I just hacked
EDIT and here's the code: https://gist.github.com/ocornut/4d9001336d07a6818f96
Code is pretty bad/hacky/shortsighted but just giving the general idea of where things might go. It only generate the pictures for now. It should eventually generate a full .md file with text and links to screenshots?

---

Buttons

bool Button(const char* label, const ImVec2& size = ImVec2(0,0))

Button() returns true when it has been pressed

    if (ImGui::Button("Save"))
        MySaveCode();

Button() can takes an optional size, set individually for each axis.
A value of zero sets the size automatically based on the label.

    ImGui::Button("Click");
    ImGui::Button("Click", ImVec2(0, 0));

A positive value sets the size

    ImGui::Button("Click", ImVec2(100, 0));   // Specify width
    ImGui::Button("Click", ImVec2(0, 100));   // Specify height
    ImGui::Button("Click", ImVec2(100, 100)); // Specify width and height

A negative value align to the edge of the window or current contents region

    ImGui::Button("Click", ImVec2(-1, 0));    // Align to right edge
    ImGui::Button("Click", ImVec2(-20, 0));   // Align to right edge 
    ImGui::Button("Click", ImVec2(0, -1));    // Align to bottom edge 

---

[emoon]

This sounds good.

I also would like to separate the code examples into separate files that may or may not be included when generating the doc. That way it's possible to have allow others to use the documentation but having slightly different syntax (for example in Rust or C wrapper case) but still use the guts of the documentation.

Still the generated doc can include samples with the code in question but selectable in some way.

Also using markdown for the documentation would be great.

[ocornut]

Yes ideally it would become a set of helpers that can be used in different contexts, locally or for languages ports. It could potentially even be useful for people to document their own imgui-based helpers or even post repros here :) but we'd have to see how it evolve. Shorter term I would just suggest tackling the basic problems: parsing source code, outputting to markdown, adapting the system so that it works in more contexts. Some context may include or exclude the creation of a window, some context may need to hide some code, etc. I could create a imgui-doc repo right now and grant you access if you want to fiddle with it together and we can always scrap the repo if it ends up back inside the main repo later

[emoon]

I was thinking of doing it the other way around.

Parsing markdown (with inline or ref code) and generating code to be executed and then screenshoted. That way one doesn't need to parse source code as markdown is likely way easier to parse.

Thoughts?

[ocornut]

I would guess it is easy to parse source code as long as we have known markers and don't try to support edge cases, and it's nice to write and iterate/run actual code directly in place, rather than indirectly via a MD layer. But either way would work.

I semi expect the code to include various markers/oddities. (e.g. a marker to request "hiding a line" or hiding unnecessary scaffolding from from the user-visible output) so even if MD is used as source, we would have to regenerate a different MD either way. For example for a particular series of example demo we may need to setup certain conditions but maybe the setup is only visible once (or never if it's not really important in context).

[emoon]

Yeah if we use markers then it should be fine.

[ocornut]

Created an empty repo for code https://github.com/ocornut/imgui-doc
I don't mind keeping the discussions here as the other repo may or not last

[emoon]

Thanks! I will give this some more though and come up with a suggestion.

[emoon]

Sorry for the lack of updates on this one. I will try to get it done this week.