[Discussion]: Revamped Getting Started guide

[Saransh-cpp]

Hi everyone! I have successfully added doctests and missing docstrings and have fixed miscellaneous documentation-related issues in Flux and Flux's close neighbor repositories! Some of the PRs have been merged, and some are still under the review process.

According to the timeline submitted by me, the next step would be to revamp the Getting Started section. I had a basic idea about the same, but I want to discuss it here with the community for better implementation! The text submitted by me in the proposal -

The “Getting Started” section, the “Overview” section, and the “Basics” section have valuable information for the beginners, but the information is scattered among these 3 sections. Additionally, one of these 3 sections is on Flux’s website, and 2 of them are available on the documentation website, making it difficult for newcomers to navigate between these hackable yet basic examples. Instead, these three tutorials can be moved out of their current places and combined under a single section named "Getting Started", which can then be added to the documentation and linked on the website.

The new “Getting Started” section will contain the contents that currently reside in the current “Getting Started” section, “Overview” section, and “Basic” section. The eight different pages under the “Getting Started” section -

1. Linear regression with and without Flux
   Currently, there is no tutorial on building the linear regression algorithm with and without using Flux. Linear regression is usually the first algorithm a person learns while trying to grasp Machine Learning. This page would aim to create the algorithm from scratch and use Flux’s functionalities parallelly. The algorithm would be used on a custom-generated dataset as well as a proper regression dataset.
2. Logistic regression with and without Flux
   Similar to the linear regression algorithm, there is no tutorial in Flux's documentation on building a logistic regression algorithm. This page will describe how to implement the algorithm from scratch, i.e. define the loss function, define the activation functions, etc. The page would also show the users the corresponding functionality provided by Flux for the same. In the end, it would convert the complete tutorial to a Flux-based tutorial showing how one can define custom functions and use them with Flux or can directly use the functions provided by Flux. Again, the tutorial will show the algorithm working on a sample-generated dataset and a proper classification dataset.
3. Approximating a function using Neural Networks
   This page would take on from the current “Overview” section and would expand on it. It would run the model on a real dataset and would also go ahead to take up a more complicated function. This would aim to show how adding the right amount of layers would help the model converge. This page might also introduce regularisation methods and drop-out layers to address the problem of high variance. I will use a mixture of self-defined functions and Flux functions to show how composable Flux is.
4. Approximating parameters using Neural Networks
   This page would build upon the current “Getting Started” section. I would be re-arranging, auditing, and updating the section.
5. Building Neural Networks from scratch
   This page would build upon the current “Basics” section. I would be re-arranging (for example, the gradient description should be shifted below to the backpropagation step) auditing, and updating the section.
6. Neural Networks in real-life
   Running algorithms on dummy data does not let the users know some particular functionalities of Flux. This page would aim to run a neural network on a real dataset to show Flux’s wide range of abilities. This page would ideally include loading data with DataLoader, pre-processing the data with utility functions, using neural networks with different configurations, solving the problem of high bias using more layers or more features, and solving the problem of high variance using regularisation and drop-out layers.
7. Computer vision with Flux
   Though Flux has some computer vision tutorials on the website, it lacks basic computer vision tutorials in the documentation. This tutorial would aim to teach users how to handle large datasets, pre-process images, normalize data, use convolution and
   max pool layers, etc.
8. Natural Language Processing with Flux
   Flux completely lacks NLP tutorials in the documentation. This section would add a basic example for the same, describing how one can use recurrent neural networks in Flux.

The only thing that is bothering me is that I don't want to make the "Getting Started" section into a "Tutorials" section since that has a separate place in Flux's website. Other minor considerations include adding a code block at the end of each page to make copy-pasting easier and using doctests to ensure that the code isn't outdated. Should I start by adding a "Logistic Regression" guide and then move ahead in the chronological order?

[theabhirath]

I'd also like to add one thing about the current docs structure that strikes me as a little unintuitive - the API reference is not grouped together but scattered through the docs, which means that users often hit upon a page full of functions that looks quite intimidating right after a gentle tutorial. Perhaps this could be remedied somehow?

There's also the fact that NNlib, MLUtils and Functors are referred to by name in the sidebar, which counts on users knowing what those libraries provide (or alternatively going through another ton of API references to figure this out). While the uses of these libraries are documented under the Ecosystem tab, that page is positioned near the right end of the docs and also has an enormous list of libraries to go through. Maybe the sidebar could be renamed to something more intuitive and a small note attached to the top of these pages detailing what exactly these libraries provide?

[Saransh-cpp]

This is what I will be working on next haha! More precisely, re-structuring the documentation. I agree, the API docs are scattered weirdly. For instance, all the API docs for layers are present on a single page whereas other such API docs are present in between different pages. Most of the open-source software has a dedicated API docs section for this, which I think should be the case with Flux's docs too.

Yes, the documentation of the re-exported packages is not very user-friendly at the moment. This was also pointed out by @mcabbott in one of my previous PRs. They will need some changes in the text as well as their position in the sidebar.

Quoting some examples from my proposal -

The current documentation structure buries plenty of helpful information and details under different headings. For instance, some textual information about loss functions is available on the "Training/Loss Functions" page here, when it should instead be on the "Building Model/Loss Functions" page here. In fact, the “Loss Functions” page can be moved out of the “Building Model” section, as these functions can be used independently of the model.

Another such instance of “buried” documentation is the documentation of re-exported Julia packages. For example, the API reference of activation functions is buried under the page heading “NNLib” making it hard to discover.

Thank you for the suggestions! :)

[mcabbott]

#2069 is a possible vision for "getting started". Its main addition addresses a different need to the points above: If you already know something about neural networks, just not Flux/Julia, then you should be able to see the major parts in action very quickly.

At least for points 6-8, I think having one "tutorials" section somewhere would be nice. These sound too detailed / advanced to live under "getting started".

One question is whether they are going to stay current, or should instead have an honest publication / last updated date. Having outdated things which pretend to be current is dreadful. The website has dates, and does not (I believe) run them on CI.

[Saransh-cpp]

#2069 is a possible vision for "getting started". Its main addition addresses a different need to the points above: If you already know something about neural networks, just not Flux/Julia, then you should be able to see the major parts in action very quickly.

Yes, this looks very nice to me! It paves a very clear way for the new "getting started" section!

At least for points 6-8, I think having one "tutorials" section somewhere would be nice. These sound too detailed / advanced to live under "getting started".

Agreed. They can maybe go in Flux's website (the tutorials section)?

One question is whether they are going to stay current, or should instead have an honest publication / last updated date. Having outdated things which pretend to be current is dreadful. The website has dates, and does not (I believe) run them on CI.

There is an open issue (or discussion) about this here - FluxML/fluxml.github.io#141. The main blockers in the way of an automated testing pipeline are the CI resources. I think having dates for now should be fine, but in the long run they should be tested somehow.

[Saransh-cpp]

#2105 - a unified discussion point.