SDKs Documentation in API References - MVP

[curquiza]

Currently, there is no SDK documentation except the READMEs and the Getting Started in the docs.

But, in this part of the docs, the code is hard-written: the code snippets are not automatically generated. If the SDK methods change, the documentation needs to be changed too.

We should find a way to link the docs repository and each of SDKs.
In one PR, the SDK maintainers should be able to change the code of the SDK and the docs at the same time.

With this kind of solution, it would be able to add more code snippets in API References.

Related issue

How?

A "simple" solution would be to add a template file in each SDK that the documentation would be able to read and then generate code snippets according to this file.

Example of a template file:

###
# guides/introduction/quick_start_guide.html#create-your-index
$ gem install meilisearch
---
require 'meilisearch'

client = MeiliSearch::Client.new('http://127.0.0.1:7700')
index = client.create_index('movies')
###

###
# guides/introduction/quick_start_guide.html#add-documents
require 'json'

movies_json = File.read('movies.json')
movies = JSON.parse(movies_json)
index.add_documents(movies)
###

###
# guides/introduction/quick_start_guide.html#search
index.search('botman')
###

Steps

• Find a great syntax for the template files: thinking about the delimiters, the "title" of the code snippet...
• In the docs part: create a parser able to read the template and then, generate the code snippets => most of the work is in the docs part.
• Find a way to regenerate the docs if a template changes (there is scheduler with GHA to check them on regular bases)
• The docs should provide a script that generates an empty template.

Bonus:

• If the template changes, a bot should create a PR (or at least an issue) to change the current template for each SDK repository.

Bonus ++:

• Find a way to test the lines of code in the template files, to be sure that we provide good examples in the docs.

[bidoubiwa]

SDK code samples in documentation

State

As of today, only the getting started guide has code samples of most of the SDK's.

Objective

Our objective is to get every example in the documentation to have different tabs with the different SDK's like this:

Now to achieve this results those are the steps:

• Creating the templates
• Filling the samples
• Collecting the samples
• Showcasing the samples in tabs

Creating the templates

Because it is more easy to write code in a file that has a good extension (.js for javascript) I made a little CLI that generated the files based on a list of samples:

( for this example I only took the list of examples for the API references of the indexes route)

For this section to be completed:

• review CLI
• Finish the list of all samples list that helps generate the files

Filling the samples

In each generated file I have put the link to the place in the documentation where the sample will be. This is to help find a context to each example.

(link must be improved, they are bad for example purpose)

• Better links to documentation
• Complete the samples in every language

Collecting the samples

Once every sample has been completed, it would be a hassle for the documentation to require each sample one by one.

I made this little tool that compiles every sample into a json file. This is the file that will be gathered by the documentation:
(i have put the same code in each sample file just for the example)

As you can see this example generated a php-sample.json. This sample contains the following:

[
  {
    "id": "create_index_examples_1",
    "content": "<?php\n    $index = $client->createIndex('books');\n?>"
  },
  {
    "id": "delete_index_examples_1",
    "content": "<?php\n    $index->deleteIndex();\n?>"
  },
 ...
]

This file will be fetched on build in the documentation. Because it is in JSON file, no parsing is required at all by the documentation to read them as JSON objects.

• Have the compiled file language-sample.json at the root of every SDK repository. Or in an another repo.
• Script to fetch all files in the different SDK's on build

Showcasing the samples in tabs

In the documentation, we will use the following component to generate the tabs with all the fetched content:

<Codesample id="create_index_examples_1" />

This will add at that position a tab element with all the fetched SDK's.

• Vue Component that will make the tabs with the different language samples

Bonus

To make it more easy for a user of a specific SDK to see all the examples in a given language this is what stripe suggest in the documentation :

• Link based: https://stripe.com/docs/api?lang=go
  Every code sample is now by default on go
• Clickable choice:

By clicking on one of this logo it will change all the default samples to the one chosen.

[tpayet]

• We have approximately 40 method to cover.

• We have methods in the wrapper that are not in the original API, how do we cover them?

[bidoubiwa]

We have methods in the wrapper that are not in the original API, how do we cover them?

@tpayet Could we discuss this here : #15

[eskombro]

In my opinion, the proposition made by @bidoubiwa makes sense, thanks! First of all, this will only work if it is really easy to identify which is the file example that you have to modify when you change a method in a SDK.

I am not sure about some details:

1. CLI

• PROBLEM:
  The need to install a CLI for creating the examples. It seems simple, but is one additional step and tool, and adds complexity for contributions, we don't want to discourage contributions.
• POSSIBLE SOLUTIONS:
  Have a shell(sh/bash?) script that generates the files, hosted in the docs repo, and that can be called with (for example) curl https://github.com/meilisearch/documentation/examples/generate-examples.sh | sh. We can also have a script in the root of each repo, it would make a simple call to that main script from the docs. In that case, we don't need to maintain any script in the repos, because it is calling an external script, but when you clone the repo you can just run sh create-exaples.sh in the root and you will have your files.

2. Compiling examples

• PROBLEM:
  Adding additional steps for contibuting (same as CLI)
• POSSIBLE SOLUTIONS:
  As we discussed, it might be better to do the compilation of the examples in the DOCS and not in the SDK. In this way, the developer who wants to contribute just need to clone the repo, run the script if he needs to generate examples and modify the files he needs to modify (which shouldn't be very often.

3. Two different files, one for the example,one for the response

• PROBLEM:
  Adding unnecesary complexity
• POSSIBLE SOLUTIONS:
  Integrate response and code example in the same template

[tpayet]

What we are trying to achieve:

• SDK's examples in the API Reference part of the documentation
• There is no tool to automate this, so we will try to build a workflow that ask for the minimum of work from the SDKs contributors and the minimum of work for us to automate the job.

Real-time update: Since VuePress is built as a static website, we may not be able to use Vue as a cliend-site rendering which is sad because we could have imagined the Vue app to get on client-side the examples in each SDK GitHub repository.
So we will have update on SDKs examples, every time a build is done. Am I right?

We should have a VuePress component that will get the SDKs' examples on each GitHub repository at build time. If no example is available, the component won't build the tab associated with that language.

We can have one or multiple file in each repository that should be hidden

./meilisearch-php
  | scripts/
  | src/
  | tests/
  | .documentation-samples #hidden file or directory
  | ...

This way the VuePress component can get each sdks sample based on this url: https://github.com/meilisearch/meilisearch-<language>/blob/master/.documentation-samples

Using the described workflow, we only have one component to write and no github actions do setup or CLI to install

Documentation sample

How to format this document or this repository?

I would prefer a simple document to ease the work on the component side and because we only have around 40 examples to write down for each sdk. This is something that could evolve later ofc, but we can start with a simple file.
We should have a slug or "human readable id" to link code samples in each SDKs to the right documentation snippet tab.

E.g. using YAML for examples in Ruby:

./documentation-samples
---
create-your-index: |-
  require 'meilisearch'

  client = MeiliSearch::Client.new('http://127.0.0.1:7700')
  index = client.create_index('movies')

add-documents: |-
  require 'json'

  movies_json = File.read('movies.json')
  movies = JSON.parse(movies_json)
  index.add_documents(movies)

What format?

• JSON: difficult to write multiple line with indentation
• YAML: difficult to manage indentation but we can have easily multiple lines
• CUSTOM: Easy to parse, but we have to make our own parser

In Rust documentation examples are compiled to ensure that examples are working. In Ruby, you can use your test to generate documentation (so your documentation is tested). We can imagine a future where we introduce special comments in our tests suite to get examples from our test suite in order to ensure that every example is always working.

Conclusion

We may not have something very modular here but I think this solution would provide us with a working solution ASAP since we only have to agree on a format and write the VuePress component that will get each sample based on convention we can agree on. We are avoiding here CLIs tools or github-action or even transpilation scripts ^^
I get that this solution is not the best for the sdk developer since there is a YAML/chooseyourformat but I think it is a compromise against the simplicity of deployment.

[curquiza]

Why not using markdown as a format?

• Everybody knows that markdown is for docs, not for code execution
• The is no highlight or delimiters issue thanks to ``` delimiters.
• The delimiters in marknow are explicit: the SDK maintainer knows if he/she has to write a bash example or or a Python example.

[tpayet]

Markdown is a markup language, not a data-serialization format so we won't have slug or ids natively supported but we can work around something for sure

[eskombro]

I really like your idea @tpayet !

The format/language discussion is indeed an important one for this implementation. Doing our own format will for sure create a lot of work not only now, but also in the future, as it evolves. YAML and MarkDown (with a slug/id workarround) seem like the most promising solutions for that! In fact, the md solution is great because it would be easy for the developer to rely on other tools for highligthing etc, and be sure about the result he will have in the published documentation.