Python: Split Python docs #5727
Conversation
Fokko
force-pushed
the
fd-add-docs
branch
2 times, most recently
from
1be0b8f to
f0f9680
Compare
|
The deploy CI is failing, but I'm confident that we can make it work. Arrow has something similar. I'd love to know what other people think 👍🏻 |
I prefer this clean split, especially given the different release cycle for pyiceberg. Just a minor nit-pick: the two levels of |
|
Hey @dhruv-pratap thanks, I didn't think of that, but it makes a lot of sense. I've changed |
This PR will split the Python docs in a separate site. The main reason for this is that the docs are part of the Java release, which is not in sync with the Python release cylce. Meaning that there is a high probability that the docs does not match with current version of the code. This will publish the docs to Github pages, by pushing this to the `gh-pages` branch. We can set up an alias from Apache, and point pyiceberg.apache.org to the github pages endpoint. I also tried readthedocs, but I found that not straightforward. Mostly because they have a build process on their end that will pull the code, and build the docs. This involves another pipeline that we have to monitor, and we have to set up webhooks. I am a simple man, and I like simple things, therefore I went for mkdocs. This can push the docs to github pages in a single command: https://www.mkdocs.org/user-guide/deploying-your-docs/#project-pages Considerations: - Decided to keep it to a single page for now, we can break it out into different pages later on. Let me know what you think of this. - We build the docs now when we push to master, probably we'll change this later to trigger on tags. - I've removed the Python docs from the other docs to avoid confusion and make sure that we have a single source of truth. An example is shown here: https://fokko.github.io/incubator-iceberg/ (Once this is merged, I'll remove that one) Closes #363 Closes apache#3283
| - name: Copy | ||
| working-directory: ./python/mkdocs | ||
| run: mv ./site /tmp/site | ||
| - name: Push changes to gh-pages branch |
There was a problem hiding this comment.
Why not use mkdocs gh-deploy? That should automatically push to gh-pages.
There was a problem hiding this comment.
I wasn't sure what mkdocs gh-deploy is doing under the hood, therefore I made the steps explicit.
.github/workflows/python-ci-docs.yml
Outdated
| push: | ||
| branches: | ||
| - 'master' | ||
| pull_request: |
There was a problem hiding this comment.
Do we want to publish for pull requests? Why not just master?
There was a problem hiding this comment.
That was mostly for testing, removed that one 👍🏻
| on: | ||
| push: | ||
| branches: | ||
| - 'master' |
There was a problem hiding this comment.
Can we add paths here with python/docs/**?
| additional_dependencies: | ||
| - mdformat-black | ||
| - mdformat-config | ||
| - mdformat-beautysh |
There was a problem hiding this comment.
This is my personal favorite. This will actually detect shell, python, and config blocks in the code, and format it automatically. This way the code, script, and config will be nicely formatted 👍🏻 If it isn't formatted properly, it will fail the CI.
There was a problem hiding this comment.
Sounds reasonable. In the past, I've just configured those things in mkdocs, but if it's working that's great.
This PR will split the Python docs into a separate site. The main reason for this is that the docs are part of the Java release, which is not in sync with the Python release cycle. Meaning that there is a high probability that the docs do not match with the current version of the code.
This will publish the docs to Github pages, by pushing this to the
gh-pagesbranch. We can set up an alias from Apache, and point pyiceberg.apache.org to the GitHub pages endpoint.I also tried readthedocs, but I found that not straightforward. Mostly because they have a build process on their end that will pull the code, and build the docs. This involves another pipeline that we have to monitor, and we have to set up webhooks. I am a simple man, and I like simple things, therefore I went for mkdocs. This can push the docs to GitHub pages in a single command: https://www.mkdocs.org/user-guide/deploying-your-docs/#project-pages
Considerations:
An example is shown here: https://fokko.github.io/incubator-iceberg/ (Once this is merged, I'll remove that one)
python/README.mdwill be shown on PyPi, and refers to the documentation.