[Docs]: Tutorial is missing step to create User SDL

[webstackdev]

Summary and description

I'm coming back to Redwood after being away some time, and just worked through the Tutorial for v4. There is no step to add an SDL for the User model (created in Chapter 4, Authentication). That creates a fatal error in Chapter 7, Accessing currentUser in the API side, when the user field is added to the Posts type in api/src/graphql/posts.sdl.js.

Was there a page in the Tutorial for setting up the User model and SDLs that got removed? I can't find where it was generated using Git blame. Maybe related to using yarn rw g dbAuth that autogenerates the sign up page?

I'd be happy to push a PR for the doc, but I'm not sure where it should be (Chapter 4, right after the User model is created in schema.prisma, or Chapter 7, when User is added to the Posts SDL file). The SDL file in the tutorial example repo shows sensitive User model fields being commented out after running yarn rw g sdl User --no-crud (hashedPassword, salt, resetToken, and resetTokenExpiresAt). There should probably be some explanatory text as to why.

Are you interested in working on this?

• I'm interested in working on this

[thedavidprice]

Welcome back @webstackdev Help here would be great — PR is definitely appreciated.

Suggestions:

• Add in Chapter 7; don't add until necessary; explain as "this is needed for the user in Posts... etc"
• Yes to the explanatory text regarded commented out fields

If you take this on, could you also confirm the Tutorial Example Repo matches the changes in the Tutorial Doc?

FYI @cannikin is out this week. I'll loop him in but just know his response will be delayed.

[webstackdev]

@thedavidprice Will do. I also have a TypeScript version of redwood-tutorial that I generated walking through the tutorial. There's an open issue on that repo suggesting turning the tutorial repo to TypeScript, and using yarn rw ts-to-js to create a JS version.

I'll push two PRs, one with the TypeScript version of the tutorial, and one with an update to the docs. I'll step through the tutorial again first to make sure it matches what I push for the TS version of the repo first.

I followed the tutorial to the end, so it has the adminPosts SDL and service (instead of just posts). There is some code and information about that service to add to docs to get it to work (context.currentUser can return null, so needs checked before using in where clauses).

I'll have some time on Saturday. If it's not ready then, I'll try to finish and push the two PRs Monday-Tuesday of next week (April 3/4).

[thedavidprice]

Woah, that's amazing 🤯

I'm looping in @Tobbe who can possibly help get the TS PR in. Include Rob on the Docs PR.

Do know that I'll be out next week. Give me (and others) nudges as needed so this doesn't get lost.

[cannikin]

In the Intermission in the tutorial we highly recommend that you continue on from the redwood-tutorial repo, which does have the User SDL defined, but we probably haven't had anyone go through the tutorial without doing that, so we haven't had anyone complain about this before! My first thought is just to add a section under the Using Your Current Codebase section that gives instructions for creating that User SDL so it's ready to go for Chapter 7.

I think the reason we've been hesitant about moving the repo to TS is that if you convert to JS, the code examples we show in the tutorial need to match what's generated by ts-to-js or it's going to be confusing or just look plain sloppy on our part. Doing this is a lot of work—checking and potentially updating each and every code block throughout the entire second half of the tutorial. Someone did most of that work, but then we made some major changes to the auth system, the TS PR got out of sync, and no one ever picked it back up. You're welcome to try it again, but be aware that we have very high standards for the tutorial! :)

[webstackdev]

@cannikin I agree the tutorial is very high quality, and after recently working through it (for TS) I have a few thoughts I'd like to share (and an offer to help if I can).

Redwood is using the repo accompanying the tutorial in a way that was non-obvious to me as a consumer of the tutorial. My expectation was that it would be a source of truth for the steps I was following, in case I made a mistake or something was out of sync with the tutorial text. That's been my experience with other repos accompanying tutorials.

Redwood is using the tutorial repo to add some additional code to the tutorial consumer's WIP repo when they reach the intermission, by suggesting they abandon what they've already done and start from a pull of the tutorial repo. The docs explaining the reason for this are a little hit and miss:

• In Intermission's Using the Example Repo (Recommended) and the README for the tutorial repo, it says the reason is to add some additional styling ("This repo contains much more styling than the one we built together in the tutorial, but is functionally identical").
• In Intermission's Using Your Current Codebase (before the above text), it gives a little more detail for the reason: "You'll also be missing out on a good starting test suite that we've added" (which also means the statement on the README isn't true, and the reasons in the section on using the example repo are incomplete).

Rob had this comment in the PR to convert the tutorial repo to TS:

A TS person will have no problem understanding a JS repo, but the inverse is not true.

I disagree with that idea. I originally explored the tutorial repo to try and understand the correct typings to use for the tutorial. As it stands, following the tutorial using TS code won't result in a working end result repo - it will have type errors. I'd have to walk back through it to identify exactly where I ran into type errors (there were several of them, also I'm willing to do that to help), and I'm a little uncertain if my solutions are canonical for Redwood.

I did some research to see if other projects had good solutions to keeping tutorial code blocks and an associated repo in sync. Meteor used a system some years ago that involved sync'ing code blocks to particular commits, and rebasing updates to the code blocks (which seems like a nightmare and which Meteor seems to have abandoned). I came across a few other approaches, but they all also involved rebasing to commits. There just don't seem to be any good solutions to avoid a lot of manual work.

There's a few ideas I'd like to throw out.

• First is to rename the default branch from main to intermission so it's more obvious what's going on.
• Second is to add a branch that's the end-state of the repo so users can check their work against a completed repo (that passes tests) when they finish the tutorial. That will help catch people who run into problems, and make it easier for them to see what happened if the generators are changed. The end-state branch can be forwarded to the head of the intermission default branch if changes are made in the first half of the tutorial.
• Third is a GH Action to throw a flag up if any code block in the tutorial is changed, saying the tutorial repo needs to be adjusted also if necessary. I haven't really thought that through but without any CI automation it's likely to happen and slip through.
• Fourth is to also have a TS repo, either separate from the JS repo (I know it expands the workload substantially) or as a branch from an empty initial commit following the above scheme (except named intermission-ts and not the default, and maybe end-state-ts). The CI should run type-check and tests on the TS branch (or repo) also.
• Fifth is to consider skipping the idea of an Intermission repo, and have a codegen command instead that would apply the requisite changes (like rw g tutorial-intermission) or instructions to set a remote origin and do a pull from the intermission branch suggested above. I think that would be more intuitive.

I agree ts-to-js doesn't seem like a workable approach. Unfortunately it seems like there's no avoiding a lot of work in managing a high-quality tutorial and associated repo. I do think the way it stands now, it breaks expectations (it's unusual that the purpose of the tutorial repo is to do a fast-forward at the Intermission so the tutorial can avoid getting into the weeds unnecessarily and be kept concise). There's also no reference repo for users who get into trouble to help them get back on track (for JS or TS).

As a more involved idea, I'm willing to work on a POC to extract the CLI commands and Git diffs from the tutorial, and build the repo automatically (for both JS and TS) from the Tutorial. My motivation is that I expect to have this same problem for a project that I'm working on long-term (at a very low intensity) that I intend to base on Redwood. I think it's doable. My project will require a number of end-user tutorials, and I'll be severely hampered on time to maintain them.

I'm willing to put in work to help if there's direction.

[cannikin]

Thanks for the in depth explanation! I can tell you're really invested in this, which is awesome. Not may people find docs so enthralling!

So are you thinking that people are finding and checking out the redwood-tutorial from day 1, before even getting to the intermission, and getting confused? We don't mention it exists until the Intermission, and if you follow along with the recommendations (check this repo out and keep going) it should be working, no? (I can't speak to type errors: I hate TS with the firey passion of 1,000 suns.) But I can see if you checked out redwood-tutorial and THEN started on Chapter 1 of the tutorial you'd be in a world of pain. But I can't imagine that's many people's path.

The docs explaining the reason for this are a little hit and miss:

Yeah I don't think the README has been updated since it was first written, it could probably be updated at this point, with a better explanation of why to go with this repo (and to make sure you're not using it to start from scratch).

Second is to add a branch that's the end-state of the repo so users can check their work against a completed repo (that passes tests) when they finish the tutorial.

There is a final-state branch but we don't promote it. I don't know that changes to main have been pulled into it recently, and it was more of a sanity check for myself as I went through the second half of the tutorial.

A TS person will have no problem understanding a JS repo, but the inverse is not true.

I disagree with that idea. I originally explored the tutorial repo to try and understand the correct typings to use for the tutorial.

The fact that you used the phrase "correct typings" here tells me that you're already a TS person, and so have probably forgot what it's like for a purely JS person (or someone new to programming altogether) to gaze upon a full-typed source file. I'm not a TS person and want to quit programming every time I look at a file riddled with Typescript. ;)

I saw a tweet recently that said something along the lines of "if your tutorial isn't teaching CSS, don't include a bunch of CSS" and that put into words to something I've felt for years while writing docs. I feel the same way about Typescript, personally. I'm trying to teach someone the 25 things about Redwood that are brand new concepts to them (many people will have no experience with GraphQL, Prisma, Apollo, Jest, some have never worked on the frontend, others have never worked on the back...) I'm trying to keep the amount of new things to learn to the bare essentials, and for me that list doesn't include Typescript (or CSS). I understand that for TS adherents that having types feels safer and more correct, but you don't need TS in order to get the app working. It's like when you're teaching someone to drive, they don't need to know how the engine works or the physics of weight offsets in a turn when braking. Knowing that may make them better drivers in the future, but first they need to know how to put the car in drive and not hit anyone else!

This is the reason the first half of the tutorial builds an app that's so "ugly"—I don't want to bog people down in styling (CSS being possibly another concept that's new to them) and I don't like saying "just copy this code, trust me." But at some point a bunch of text boxes in a line and everything in Times New Roman starts to obscure what you're actually building in each step, so in the Intermission you get a chance to "start over" with something that looks a little nicer and now you can focus on the new concepts (testing and storybook). (Not to mention you'd have a bunch of failing tests and no stories based on what we built in the first half, and I'm not ready to introduce tests or storybook yet.) You're actually the first person I've heard complain about this approach—although maybe you're just the first person that's taken the time to tell us, everyone else just quit in frustration! I can see another approach to keep working with your existing repo, which is to give you just the tests and stories you need to bring your own repo up to the same point as redwood-tutorial (I think just having people copy and paste at this point is okay—you don't understand what you're pasting, but it's going to be explained in the following chapters.)

You're more than welcome to try and come up with a solution that keeps a TS and JS version of the tutorial in sync, or builds it automatically based on CLI tools, but it needs to be something that can be maintained with minimal effort from us going forward. There are several folks that have attempted something similar and then we never hear from them again, and now it's one more thing us on the core team have to try and keep running forever. :( Part of our end-to-end test suite actually tests that the steps taken in the tutorial continue to work, although we only got up to the end of Chapter 3.

[webstackdev]

@cannikin Thank you for sharing the insight on tutorial scope (keeping it very narrow). It does seem like the risk/reward of building tooling is unfavorable. Sometimes a shovel is the best tool for a job, even if an excavator makes the work easier.

I came across the tutorial repo in the Intermission text. I had several TS errors while following the tutorial, but none of them broke the dev server. I was confused about what the correct type should be at several steps. The first TS error that broke my dev server was after the Intermission. I then read through the TypeScript section of the Redwood docs carefully, and studied the generated TS types in my Tutorial repo. That's when I started looking for a an end-state TS tutorial repo and came across the issue and link to one from a PR in the JS tutorial repo.

My goal at the end of working through the Tutorial was to have a repo that passed the test suite, that passed the other tests mentioned in the documentation (yarn rw check and yarn rw type-check), and that didn't have linting errors. There was a lot of friction reaching that point with a TS project. I still have no certainty that I did it correctly, e.g. making the best use of the types that Redwood provides vs. creating my own.

I think the help that I can offer is to (1) walk through the Tutorial for a TS project from a clean repo, (2) make notes on the points where I had type errors or other problems, (3) post in an issue what the problem was, the solution I came up with, and ask if my solution is the best practice, and (4) submit a PR revising the TS portion of the Tutorial docs with the result. Should I open a new issue for that?

I think the Tutorial would also benefit from a feedback form at the end, and a "Stuck? Let us know!" kind of box on each page. I'd be happy to contribute the code for that if it seems like a worthwhile effort and someone gives me some direction. I don't know what endpoint would make sense for it. Maybe moderated comments?

[cannikin]

Ahh, I haven't gone through the tutorial with the TS version of the code blocks, those were added at a later date by another person.

I think your suggestions (1-4) would be amazing! And we've also talked about adding a comment form like that, we just haven't taken the time to implement. If you could add one we'd be eternally grateful! Any suggestions on where it should submit to? A database somewhere? Email? A thread on the forums? A discord channel?

[webstackdev]

@cannikin I'll do the suggestions about generating a TS repo from the tutorial, probably finishing by the end of this coming weekend. I'll make a post in the Community forum asking for feedback on any types I have to write myself to avoid type errors, and ask for feedback (and why). I'll push a PR with any changes to the TS code sections that generates.

I'd be happy to work on a comment form for the tutorial. I'd suggest two parts:

Get Support

Each tutorial page could have a small sticky modal pinned to the right-hand side and middle of the page for users to get help if they get stuck. It could be as simple as a slide-out modal with a link to the Discord tutorial-help channel. Or it could work like a popup chat window, letting the user post a message with optional image from the chat window into the Discord tutorial-help channel and giving a "live help" kind of user experience.

Discord can call back over a websocket (a "Gateway" in Discord dev docs) if someone responds to a message in a channel. Netlify Background Functions can run up to 15 minutes, so that's the window between a user posting a help request and getting a response (without going to a persistent process). If that time expires, the help modal could inform them that no one is currently in the channel to help, give them the link to the Discord message, and ask them if they would like to open an Issue on the tutorial repo (giving them a link to the Issue if they do). The latter would require them authenticating to GitHub to post the Issue as their user, and could also update the Discord tutorial-help channel message with what happened and a link to the GH Issue.

The reason for using a Lambda is I don't think Discord would allow the user's browser to maintain the websocket connection directly to Discord because of permissions. They'd have to be a registered user on Discord and invited to the "Guild". The client ID and secret provided to the bot can be narrowed in scope to just reading and writing in the tutorial-help channel, but I think it would violate Discord's ToS to share that (though not sure).

I know there's a monthly hourly limit on Netlify lambdas. My guess is that the monthly run-time is pretty low, and it could maybe be put in a separate Free-tier account (100 hours per month).

Comment Form on Afterword Page of Tutorial

It might make sense to have a single well-known Issue on the tutorial repo for comments to be posted to, and use a GitHub App to post comments to that Issue. A link to the well-known Issue could be added to the tutorial repo README as well as an explanation of what it is. That Issue should have decent visibility in Open Issues as new comments come in.

The comment form could have a star-rating field, labeled something like "not useful - useful", and anything under a threshold (three stars or under) could be handled specially (since they're probably serious issues that need looked at), like posting a message into the Discord tutorial-help channel with a BOT tag. Or all comments could also have notifications sent to the Discord tutorial-help channel, with a colored flag (red for three stars or less, green for four or five stars maybe). I think keeping them posted as GH Issues would be helpful since they're be consolidated in one place and visible.

The comment form could either have a single field for text, or maybe two fields ("What was good about your tutorial experience?" and "What could be improved in the tutorial?"), and any other data you'd like to see collected.

What do you think?

[cannikin]

I like the idea of pointing people to the #tutorial-help channel on Discord. I don't know that it's watched 24/7 for posts though, so I don't think a live chat-like experience would really work out. But even just a "Need help? Get help on Discord" would be great, linking directly to the channel.

For the end-of-tutorial questionnaire, having them on a GitHub issue would be great for accountability, but we'd want people to be honest, and I wonder if having the responses be public would discourage some from doing that. Also, I'm worried it would give a negative impression of the tutorial since you hear from dissatisfied customers orders of magnitude more times than you hear from happy ones! Someone going through that issue might think the tutorial is riddled with problems. :(

The site is deployed on Netlify, so the exit interview could be something as simple as a form that posts to the Netlify Forms feature.

From looking at our analytics it appears about 8.5% of users who start the tutorial end up making it to the Afterword. And if you assume, generously, that 10% of those will actually fill out the form, we're talking just a couple of responses a month, so I wouldn't spend too much effort on a super complex integration!

[webstackdev]

@cannikin Sounds like a plan ≧◉◡◉≦ I'll set aside some time to work on it this weekend.