Newsletter: new format and more delegation

[ozkriff]

So, I'm struggling with editing and releasing the newsletter in its current format and workflow - it takes something like 40 to 50 pure hours every month to collect all the news, talk with all the people, prepare the plan and coordination issue, review/edit/merge PRs, write what's left, prepare the final draft, and publish it.

Thus, it'd be cool to reduce the bus factor and delegate the editing/merging responsibilities. But it doesn't seem like delegating the editing responsibilities as-is will work well - most of the contributors are struggling to follow the guidelines from CONTRIBUTING.md and there's only one person who regularly helps woth incoming PRs reviews.

As I see it, there're two ways to solve this:

1. To sacrifice the consistency of the newsletter (merge incoming PRs without much scrutiny)
2. or to simplify the newsletter's format and thus simplify the rules.

I still believe that for a collective and periodic project of this scale consistency is extremely important, so I concentrated my thoughts on the second option and this is what I come up with:

• Make rules much shorter and stricter, and move them to the coordination issues header (so that every contributor would actually read it).
• Introduce a pool of editors and a monthly rotated role of a lead("main"? "publishing"?) editor.

The lead editor is rotated from the pool every month and is responsible for:

• Collecting news from reddit/twitter/discord/etc and creating the initial newsletter plan from it.
• Creating and maintaning the coordination issue.
• Pinging or inviting the possible contributors in Discord/Twitter/Reddit's DMs.
• Reviewing, editing if needed, and mergeing the PRs.
• Preparing the final draft and releasing the newsletter.

Non-lead editors help with PRs reviews and can edit and merge correct/fixed PRs themselves - if something is off in the merged PRs the lead editor can still fix it during the preparation of the final draft.

The requirements for merging a PR are reduced to something like:

• Only one image (<300kb) or GIF (<2.5mb) before the text. With an optional caption and a mandatory alt text.
• All the (rendered) text should be under 1000 characters (including spaces and punctuation) and under 6 paragraphs (without any subsections, etc).
• No bold/italic/etc formatting - only links and one plain list without nesting per section.
• Third-person perspective.
• 80 chars per MD line and no other markdownlint warnings on CI.
• Only the following simple templates are allowed:
  • For games/apps/libs:

    # [Gamename]
    
    ![alt text](img)
    _optional image label_
    
    [Gamename] ([GitHub], [Discord], [Twitter]) by [@nickname]
    is ... {short project description in one sentence}.
    
    {An overview of the recent updates with links to more details}.
    
    _Discussions: [/r/rust_gamedev](link), [Twiter](link), [etc](link)_
    
    {md links block}

  • For articles/videos/etc:

    # [Articlename]
    
    ![alt text](img)
    _optional image label_
    
    [@nickname] published an [article] about ...
    {overview what the resource is about}.
    
    _Discussions: [/r/rust_gamedev](link), [Twiter](link), [etc](link)_
    
    {md links block}

Atm, @17cupsofcoffee and @AngelOnFira agreed to join the editors team.

If there're no objections, I'd like to try the new rules and workflow starting with the current newsletter: the coordination issue for the 18th newsletter will be created this evening.

[Eliah-Lakhin]

In my opinion maybe it would be worth to consider turning the Digest publications into paid service. Since the regular maintenance taking too many actual working hours, it probably cannot be performed on a pure volunteering basis in long term even with a pool of volunteers. And the editors should be paid for their job even if the payment would be relatively small in the beginning. Also since the Digest has about ~2000 regular readers(it would be worth to also make a more precise research on this), there is an implicit promotion interest for the publishers too which is usually not a free service in common. And as a paid service it would make publishers more organized and be more responsive for their content and format too, and could even raise the content quality that would benefit readers as well. So there are a lot of benefits for all sides of such initiative.

Basically, The Newsletter Digest is naturally evolving into a form of scientific/engineers theme Journal. And such Journals are usually not free either for readers or for publishers.

Just as an idea. @ozkriff, @17cupsofcoffee, @AngelOnFira your thoughts?

[Uriopass]

I wonder how hard it would be to write a Python script that automatically enforces thoses rules, for example less than 80 characters per line.
This would run as part of CI so that reviewing is purely on content and not on form.
I'm also thinking that all the different game/lib updates could be written in separate files so that the final edit can be more automated and reduces merge conflicts to practically zero.

Basically, automate as much as possible to lessen the work required

[17cupsofcoffee]

@Uriopass We already have MarkdownLint running in CI to enforce those rules, but a big chunk of time gets spent chasing people to fix them - as a first step I did a PR to automatically comment when MarkdownLint fails: #453

Hopefully that'll improve things a bit - I definitely agree the more we can automate the better :)

[ozkriff]

I'm also thinking that all the different game/lib updates could be written in separate files so that the final edit can be more automated and reduces merge conflicts to practically zero.

@Uriopass Something very similar was propesed by @AngelOnFira just yesterday. It's definitely worth discussing, though my immidiate concerns are:

1. It'll forever tie the newsletter to Zola. Zola seems strong atm, but there's no guarantee that it'll last or that we wouldn't decide to migrate to something in a few years.
2. It'll wreck GitHub previews that we've been trying to keep fully functional.

[Uriopass]

@Uriopass We already have MarkdownLint running in CI to enforce those rules, but a big chunk of time gets spent chasing people to fix them - as a first step I did a PR to automatically comment when MarkdownLint fails: #453

Hopefully that'll improve things a bit - I definitely agree the more we can automate the better :)

To avoid chasing people, I think a fixed schedule and deadlines could help saying "I'm sorry you couldn't be put into this newsletter this month but you didn't respect the rules and as volunteers we don't have time for this". (There's probably a better way to say it)

I would never have thought that it would take 40 to 50 hours to make the newsletter. At this point I feel you have the right to not chase people around if that can save time.

[iolivia]

Crazy idea, could we have a google form with the required fields you mentioned in the template (author link, paragraph about the project, type of project, link to an image and paragraph with most recent update), post that every month (or keep an evergreen link and filter responses by date) and whoever wants to be featured needs to fill in the google form. Then we write a script which takes the google form responses and converts them to nicely formatted markdown to produce the final newsletter.

I am thinking a lot of the overhead here is actually to do with people creating the PRs, not following the formatting rules and also overhead on the maintainers when it comes to reviewing the PRs and fixing merge conflicts, etc, which automating using the google form would help. wdyt?

[AngelOnFira]

@Eliah-Lakhin I do appreciate the <3 for the contributors, but I'd personally prefer to stay away from a paid subscription model. My motivation behind contributing a few sections each month is that it promotes Rust's gamedev ecosystem. I wouldn't be particularly interested if it was hidden behind a paywall.

I do like a lot of ideas here, but I think we need to move towards them slowly rather than making a radical shift. I think some items to try for January's edition might be:

• Have multiple primary editors that merge sections early on. This should allow us to see what issues arise from this method, but also take the load off @ozkriff.
• Leave shuffling sections until the end, that way everything can just be merged as is
• Don't worry too much about every PR, letting them build up over time is a real killer. Also, after a PR is made, allow the editors to have veto power over it, i.e. make any required changes themselves, and merge right after. It's better to move fast on this, we're not going to bring down production :P

I think some of the items I like for the long term:

• @iolivia's idea of Google forms is quite nice, it allows the editors to grab content quite easily and distribute the work
• A Python script could be helpful, would be nice to experiment with that next month to see what is helpful. Maybe it could automatically read the Google form and generate a "good enough" section that we can then edit.

[17cupsofcoffee]

👍 to that!

When the PRs pile up it means that all of them end up with merge conflicts that have to be resolved, and while it's not a ton of work to do that, it adds up when there's like 50 PRs coming in. I think if we can merge stuff quicker between us, people's PRs will be more likely to be starting from a recent version of the file, and that'll avoid a lot of the conflicts.

[Eliah-Lakhin]

@AngelOnFira Thank you for your response!

My proposal was about paid content promotions(paid contributions), not paid subscriptions. I agree that paid subscription would be bad idea. Moreover, maybe a paid promotion should be an option, so the community is still be able to PR into the digest free of charge. But I would consider the payment services at least in the future. It is not necessary bad move for the community openness and the ecosystem grow, in my opinion it could even help in long term.

Anyway, just bringing an idea. :)

[ozkriff]

@Eliah-Lakhin I think content promotions and payouts to editors is a thing that is worth keeping in mind, but it's extremely hard to implement correctly and could cause loads of ethical questions. I'd prefer to stay away from this (at least for now) - atm editors can open patreon/ghsponsors/etc for donations to them personally.

[ozkriff]

To avoid chasing people, I think a fixed schedule and deadlines could help saying "I'm sorry you couldn't be put into this newsletter this month but you didn't respect the rules and as volunteers we don't have time for this". (There's probably a better way to say it)

@Uriopass I believe we already did this a few times (also, in situations when folks take sections but don't sends corresponding PRs in time) - that's why I define a "soft deadline" in the coordination issue. So it seems like we're already doing this to some extent.

I would never have thought that it would take 40 to 50 hours to make the newsletter.

Note, that maybe I'm just not a very productive person, hah. It used to take 20 to 30 hrs before, but we have more rust gamedev news now and everything just feels subjectively harder and more depressive since 2020 started.

Also, I'm still not happy with my English. Newsletter coordination is a great practice, but I get tired of writing/reading lots of text in English - not even mentioning all the required communications and discussions. I guess a well-rested native speaker will be able to do all this work much quicker.

[ozkriff]

@iolivia idea about google form seems interesting, though it seems to work nice as an extension to the thread's original proposal - an issue should be created to discuss how exactly it should work. (One possible downside that comes to my mind: how should an interaction between an editor and a contributor work if there're some issues with the proposed section?)

Don't worry too much about every PR, letting them build up over time is a real killer. Also, after a PR is made, allow the editors to have veto power over it, i.e. make any required changes themselves, and merge right after. It's better to move fast on this, we're not going to bring down production :P

@AngelOnFira an important note about merged the PRs quickly and fixing them later: the images and especially GIFs should be finalized and have a correct size before they get into the main branch, otherwise we may quickly blow up the repo with useless binary data.

[ozkriff]

If there're no objections, I'd like to try the new rules and workflow starting with the current newsletter: the coordination issue for the 18th newsletter will be created this evening.

Soo, it seems like no one is against the changes, right?

[kvark]

I read the OP but not all the comments. The 40-50hours is totally nuts. We should bring that to below 10. Editor rotation sounds a bit unrealistic to me (as in, at least I may not be able to rotate, maybe others will?).

The suggested requirements and rules are great. We should just turn them into github PR templates, which means 95% of submissions will follow them naturally. Cutting the date of more sharply, and not actively soliciting submissions is something we need to do: given the growing popularity of this newsletter, it should be in the authors incentives to submit and do it in time, not yours @ozkriff .

I think you did amazing job at leading the effort, and you were efficient at this. Regardless of how it goes from here, your efforts are highly appreciated, thank you!