Update examples to require installable dependencies

[hyuri]

What is the problem or limitation you are having?

I downloaded one of the Toga examples — positron-static — from this repo, and only that folder, set up the venv, and when I ran briefcase create, it gave me an error saying it couldn't install requirements:

Then I noticed that, in its pyproject.toml, some of its requirements point to folders/packages relative to itself (e.g.: ../../cocoa) — probably expecting the entire repo to be there:

Source: https://github.com/beeware/toga/blob/main/examples/positron-static/pyproject.toml

So I downloaded the entire repo and tried again, but got another error:

Finally, I changed the requirements in the pyproject.toml file so it asks to install all the required packages:

And it worked!

Describe the solution you'd like

Since I don't know the technicalities behind this, I'll pose this as a question:

Isn't it better to only list installable required packages, like toga-cocoa~=0.4.5, instead of ../../cocoa, in all of the examples, so that anyone can download them — entire repo or pick and choose individual examples — and try without running into these issues?

This kind of issue is even noted in Step 7 of the official Tutorial, when talking about pyproject.toml's requires=[]:

Describe alternatives you've considered

None

Additional context

I noticed other examples also suffer from the same issue.

[freakboy3742]

Since I don't know the technicalities behind this, I'll pose this as a question:

Isn't it better to only list installable required packages, like toga-cocoa~=0.4.5, instead of ../../cocoa, in all of the examples, so that anyone can download them — entire repo or pick and choose individual examples — and try without running into these issues?

No, for three reasons:

Firstly, there's nothing documented anywhere (that I'm aware of) that says that these examples are standalone, or can be downloaded on their own. I'm not even sure how you were able to download just the positron-static example - if that's a feature of Github, it's not one I was previously aware of. However, if you have a checkout of the Toga repository, or a complete source tarball of the repository (something that Github does provide), the relative links will work fine.

Secondly, while the examples are examples, they're also used as manual test cases for widgets. If I (or any other developer) is adding a new feature to a widget, it's extremely helpful to have a working example that will use the current state of the Toga repository in deployment.

Lastly - and perhaps the most significant - saying "toga-cocoa~=0.4.5" wouldn't be accurate. When a new feature is added, it will frequently be accompanied by an addition to one or more example apps. For example, a couple of months back, the current Toga version was 0.4.5, and I landed #2244, which added a number of features to the App class. That PR includes a number of changes to example apps to demonstrate those new features - including some completely new example apps. If you try to run those apps with Toga 0.4.5 - which was the most recent version available on PyPI, and thus the version that pip would install - the apps would crash, because they rely on features that aren't available in Toga 0.4.5.

Interestingly, if you go back into the archive: once upon a time, the apps did reference a specific version, and we had tooling that would bump that version every time we did a release. We moved to using relative path installs because it was more reliable, and caused less problems than having a specific version named. See #1374 for details.

---

As an aside: When providing examples of text (like error messages or configuration files), it's really unhelpful to use screenshots. Screenshots are great for visual details ("these two widgets aren't vertically aligned..."); but when quoting text, it's a lot easier for readers to work with:

[tool.briefcase.app.positron.macOS]
requires = [
    "../../cocoa",
    "std-nslog>=1.0.0",
]

than a screenshot of the same. Completely aside from the accessibility issue (images don't work with screenreaders); images don't come through inline on email updates, and you can't copy-and-paste details from an image.

[hyuri]

Firstly, there's nothing documented anywhere (that I'm aware of) that says that these examples are standalone, or can be downloaded on their own. I'm not even sure how you were able to download just the positron-static example - if that's a feature of Github, it's not one I was previously aware of.

That is all true. I had to use a browser extension called "GitZip" to be able to download only the example I was looking for.

Still tremendously helpful to have standalone examples for Toga users, so maybe I should add a feature request to figure out a way to include that in a maintainable way on the website. I think this would really go a long way to easing adoption. Toga is more capable than it seems from just the official tutorial, but we never know that until we see some examples. And quick standalone download-and-play examples are excellent for reference or as starting points.

However, if you have a checkout of the Toga repository, or a complete source tarball of the repository (something that Github does provide), the relative links will work fine.

Downloading the whole repo as a .zip and trying to run positron-static still gave me errors, as I said in the feature request. I could debug and ask around but it's better if the user can avoid these issues in the first place when they simply want to try an example.

Secondly, while the examples are examples, they're also used as manual test cases for widgets. If I (or any other developer) is adding a new feature to a widget, it's extremely helpful to have a working example that will use the current state of the Toga repository in deployment.

So the real problem is that two concepts — "examples for users" and "test cases for Toga" — are mixed. I acknowledge the technical challenge, but I still think it's useful to have some standalone examples. Like most websites of game engines have a section with standalone examples you can quickly download, try, learn from and see what the engine is capable of; and you don't have to download all the examples to try one. That helps convince users that the framework can work for them and their use case.

Lastly - and perhaps the most significant - saying "toga-cocoa~=0.4.5" wouldn't be accurate. When a new feature is added, it will frequently be accompanied by an addition to one or more example apps.

Perhaps I should've been more clear: I meant to propose changing it to "requires=[toga-cocoa, core, etc.]", not any specific version. That way, the latest version will always be downloaded.

As an aside: When providing examples of text (like error messages or configuration files), it's really unhelpful to use screenshots.

Understood.

Thanks.

[freakboy3742]

Downloading the whole repo as a .zip and trying to run positron-static still gave me errors, as I said in the feature request. I could debug and ask around but it's better if the user can avoid these issues in the first place when they simply want to try an example.

I can't think of any reason why this would be the case.

The error suggests that it's trying to include a binary requirement for which a wheel isn't available, but because you've only provided a portion of a screenshot of a log (as opposed to the content of the log file that is generated specifically to diagnose problems like this), it's impossible to diagnose what has gone on here.

Perhaps I should've been more clear: I meant to propose changing it to "requires=[toga-cocoa, core, etc.]", not any specific version. That way, the latest version will always be downloaded.

That still won't help. If an example is dependent on a change in main, there is no version of Toga that can be downloaded from PyPI that will include that change. The only version that will work is a reference to the checked out code - the version that is referenced by a relative path.

[hyuri]

[...] but because you've only provided a portion of a screenshot of a log (as opposed to the content of the log file that is generated specifically to diagnose problems like this), it's impossible to diagnose what has gone on here.

But that's exactly the point I made: We could debug this, but as an end user I shouldn't have to because it's an error that doesn't even need to happen. Maybe you are thinking from a framework developer's perspective, of someone who is happy to download the whole repo and run a bunch of tests? I'm thinking from an end user's perspective, who is looking for just one or a few examples and doesn't need the whole repo: I just want to download an example and run. So adding as few layers of complexity as possible to that process, and reducing friction is ideal.

That still won't help. If an example is dependent on a change in main, there is no version of Toga that can be downloaded from PyPI that will include that change. The only version that will work is a reference to the checked out code - the version that is referenced by a relative path.

I see. But that would work for standalone examples for end users, who will be using a stable release, right?

So I think the only solution is to have separation of concerns: 1) A set of test cases for Toga and its developers, purely for internal testing purposes; 2) Standalone examples for regular users, for reference and for trying out Toga features.

This is especially important because let's say the user downloaded the whole repo and tried an example and it worked for them. They might copy the whole example folder to use as a starting point for their project, and be surprised when it suddenly doesn't run anymore because there is this relative local path ../../core etc. in the .toml that no longer exists and bars briefcase from installing the packages. See the problem?

If it's too much work to maintain a separate repo for end-user examples, perhaps only offering a smaller set of examples for users at first, until a more maintainable solution is found?

I think it's critical to offer a good set of plug-and-play examples for end users. I was playing around with the positron-static example, adding buttons and PyScript logic, and amazed that I could do all of that in Toga. But I could only do that after erros, then going through the hassle of downloading the whole repo, ignoring a bunch of folders I didn't care about and took extra space in my disk, all the erros I had to figure out and then figure out I had to replace the relative paths in the .toml file. And that example is buried in that error-prone examples folder. Isn't it better to make it easier for more people to be amazed at Toga's capabilities?

[freakboy3742]

[...] but because you've only provided a portion of a screenshot of a log (as opposed to the content of the log file that is generated specifically to diagnose problems like this), it's impossible to diagnose what has gone on here.

But that's exactly the point I made: We could debug this, but as an end user I shouldn't have to because it's an error that doesn't even need to happen.

I agree. An end user shouldn't need to debug this problem.

But right now, I - the expert in this particular domain - can't debug it either, because I don't have any information to work off, it doesn't break like this for me, and I cannot even begin to imagine why it would be happening. As I've indicated - a full source checkout should work. If it doesn't, that's very weird, and I'd like to understand why.

Before we start radically redesigning how examples are being shipped on the basis that they're not usable as standalone examples, I don't think it's unreasonable to understand the problem that is actually occuring.

That still won't help. If an example is dependent on a change in main, there is no version of Toga that can be downloaded from PyPI that will include that change. The only version that will work is a reference to the checked out code - the version that is referenced by a relative path.

I see. But that would work for standalone examples for end users, who will be using a stable release, right?

A source tarball of a stable release will reference... the stable version of the code.

A stable version of the examples could reference a PyPI release - but then we have to manage the publication of a bunch of source code samples as part of our release process. And I disagree that this would result in a better end result.

This is especially important because let's say the user downloaded the whole repo and tried an example and it worked for them. They might copy the whole example folder to use as a starting point for their project, and be surprised when it suddenly doesn't run anymore because there is this relative local path ../../core etc. in the .toml that no longer exists and bars briefcase from installing the packages. See the problem?

Sure, but that seems like a pretty esoteric way for the problem to manifest. The example apps aren't really great starting points for your own app - they're all fairly eclectic demonstrators of explicit problems. They are testbeds that demonstrate API usage. I challenge your assertion that most users - or even a non-trivial subset of users - start developing a new app by copying one of Toga's example apps. I would have thought that was especially true of users who are new to Briefcase - who have been introduced to using briefcase new to start a new project via the tutorial, and would be more likely to copy the contents of app.py from the examples folder into their own app.

The Positron examples are the only exception to that - and the fix for that isn't to make the code in the Toga repository "reusable" in the way you describe. The fix is to turn the code in the positron example into a Briefcase bootstrap plugin, so that an end user can use Briefcase to generate a Positron app - which was one of the reasons that we added the bootstrap API in the first place (see beeware/briefcase#1288 and beeware/briefcase#1524)

If it's too much work to maintain a separate repo for end-user examples, perhaps only offering a smaller set of examples for users at first, until a more maintainable solution is found?

I think it's critical to offer a good set of plug-and-play examples for end users.

I disagree. There's a need to provide a good set of documentation.

Code is not documentation. The examples are, at best, a stop-gap measure to work around the fact that we don't have extensive topic guides for performing common tasks. The examples that are there aren't especially good documentation either, because they are, in most cases, too complex to be pedagogically helpful.

The long term goal should be (and is - see #1873) to remove the example apps, in favor of a single app to replace "toga-demo" that demonstrates "one of everything", plus more extensive documentation on specific widgets.

I was playing around with the positron-static example, adding buttons and PyScript logic, and amazed that I could do all of that in Toga. But I could only do that after erros, then going through the hassle of downloading the whole repo, ignoring a bunch of folders I didn't care about and took extra space in my disk, all the erros I had to figure out and then figure out I had to replace the relative paths in the .toml file. And that example is buried in that error-prone examples folder. Isn't it better to make it easier for more people to be amazed at Toga's capabilities?

Yes. But the fix for that is documentation - plus, in the Positron case specifically, a Briefcase bootstrap.

[hyuri]

I agree. An end user shouldn't need to debug this problem.

But right now, I - the expert in this particular domain - can't debug it either [...] As I've indicated - a full source checkout should work. If it doesn't, that's very weird, and I'd like to understand why.

Before we start radically redesigning how examples are being shipped on the basis that they're not usable as standalone examples, I don't think it's unreasonable to understand the problem that is actually occuring.

Fair enough. I'll run through the process again and provide logs.

Sure, but that seems like a pretty esoteric way for the problem to manifest. The example apps aren't really great starting points for your own app - they're all fairly eclectic demonstrators of explicit problems. [...]

The Positron examples are the only exception to that - and the fix for that isn't to make the code in the Toga repository "reusable" in the way you describe. The fix is to turn the code in the positron example into a Briefcase bootstrap plugin, so that an end user can use Briefcase to generate a Positron app - which was one of the reasons that we added the bootstrap API in the first place (see beeware/briefcase#1288 and beeware/briefcase#1524)

Ok, if Positron is the only example of that kind, and considering that it does demonstrate a complete and common use case, it does make more sense to have it as a bootstrap plugin, and leave the examples for code snippets indeed.

I think it's critical to offer a good set of plug-and-play examples for end users.

I disagree. There's a need to provide a good set of documentation.

Code is not documentation. The examples are, at best, a stop-gap measure to work around the fact that we don't have extensive topic guides for performing common tasks. The examples that are there aren't especially good documentation either, because they are, in most cases, too complex to be pedagogically helpful.

I agree that great, comprehensive and well-designed documentation is critical for success, but I would challenge the notion that code samples are not important, as we can see in Defold(game engine)'s recent survey, where they found overwhelming demand for more code samples (and they already have plenty):
(The red slice is "not enough code samples")

Source: https://defold.com/2024/09/03/Defold-user-survey-results/

I know it's one survey of one cohort, but I've seen similar survey results in other projects before, it's not a one-off. Some people learn faster via examples and looking at code, and don't do as well with pure documentation, simply because they get exhausted of the often endless reading and give up. And it's always exciting to simply download a few samples, see them running, tweak and push them further and start imagining the possibilities for your use case.

[freakboy3742]

I agree that great, comprehensive and well-designed documentation is critical for success, but I would challenge the notion that code samples are not important, as we can see in Defold(game engine)'s recent survey, where they found overwhelming demand for more code samples
...
I know it's one survey of one cohort, but I've seen similar survey results in other projects before, it's not a one-off.

Without knowing the context of that (and other) surveys, I'm hesitant to read anything into it. A request for "more code samples" could be interpreted as "More standalone projects I can just copy and paste" (which appears to be how you're interpreting it); or "More documentation of specific solutions to practical problems" (which is how I would read it).

Diataxis highlights that there are four roles for documentation. Unfortunately, a very large number of projects never move beyond an API guide autogenerated from source, and maybe a bare bones tutorial. In such a project, "more code samples" are absolutely required - because over half of the Diataxis documentation map that are unserved. But that doesn't mean a project needs "more copy-and-paste standalone projects". It means the project needs HowTo guides.