Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Convert docs to use EEP-59 and Markdown #75

Merged
merged 912 commits into from
Jan 18, 2024
Merged

Convert docs to use EEP-59 and Markdown #75

merged 912 commits into from
Jan 18, 2024

Conversation

garazdawi
Copy link
Owner

@garazdawi garazdawi commented Oct 20, 2023 •
edited
Loading

Things left to do:

  • Use EDoc syntax to document EDoc (for now). This makes sure we test EDoc to application/erlang+html support.
  • Figure out how to do xmlsrc style inclusions in markdown...
    • I think this will need to be solved using Makefiles for now.
  • general_info should be in index application
  • Move EEP48 docs to Erlang Reference Manual Moved erlang+html docs to edoc, the EEP48 docs were already in kernel
  • Fix shell_docs support
    • We should create a io_ansi module and then steal/borrow as much as we can from the Elixir markdown renderer.
  • Fix release of docs
  • Fix markdown linter
  • Fix redirects on erlang.org
    • Can we create a script that tries to re-create the original layout? No we cannot, we need to do everything with redirects. If we want to create the original layout, we need to heavily modify ExDoc.
  • Fix search on erlang.org
  • Fix version navigation on erlang.org
    • This is done through a .js file that erlang-org should generate from otp_versions.table. We want one file per major release that lists all patches in that release and also the latest patch of all other major releases. The files have to be per application.
  • Add umbrella navigation to ExDoc?
  • Fix since checking
  • Fix release note generation
  • Fix spell checking
    • Can we use https://github.com/tbroadley/spellchecker-cli? We could start with running find . -name "*.md" | xargs npx spellchecker-cli -p indefinite-article repeated-words syntax-mentions syntax-urls -q -f which does not do a spell check, but only check for repeated words and inproper use of a vs an. Running with spell checking on all our .md files takes a very long time...
  • Fix erts docs
    • When all bootstraps have been compiled we compile new .beam files for erts that include docs. These are then released as part of make release.
  • Add a generated markdown file to top with all modules in all applications, just like man_index.
    • Maybe we want to add such a page for User's Guides also? Or maybe it is just Kernel/Stdlib/Erts User's Guides that need a more prominent index?

bugs

  • ssl:state_name type should not be in docs
  • link to wx app does not work from top
  • table in ssl -> standard_compliance.md is broken
  • with multiple duplicate docs -> add equiv to datatypes.
    • equiv added to chunks
    • equiv for datatypes added to ex_doc
  • math:atan.... needs manyal rewrite

Many things I probably forget...

@garazdawi garazdawi self-assigned this Oct 20, 2023
@garazdawi garazdawi force-pushed the lukas/convert-docs-to-markdown branch from fee192f to a0653d4 Compare October 20, 2023 10:38
@garazdawi garazdawi force-pushed the lukas/convert-docs-to-markdown branch 2 times, most recently from 0c7b28b to d91b5f5 Compare November 2, 2023 16:03
@garazdawi garazdawi marked this pull request as draft November 3, 2023 11:03
@garazdawi garazdawi force-pushed the lukas/convert-docs-to-markdown branch from 01c0813 to ebea1b3 Compare November 6, 2023 14:55
@garazdawi garazdawi force-pushed the lukas/convert-docs-to-markdown branch from ebea1b3 to fc3da45 Compare November 17, 2023 09:40
@garazdawi garazdawi force-pushed the lukas/convert-docs-to-markdown branch 4 times, most recently from 85338d2 to 307c285 Compare December 8, 2023 09:46
@garazdawi garazdawi force-pushed the lukas/convert-docs-to-markdown branch 2 times, most recently from aee10c1 to e41cb06 Compare December 14, 2023 14:33
@garazdawi garazdawi force-pushed the lukas/convert-docs-to-markdown branch from c30e4c1 to 6c61693 Compare December 20, 2023 14:48
garazdawi and others added 16 commits January 2, 2024 12:55
@garazdawi
Merge pull request erlang#7958 from garazdawi/lukas/prepare-for-eep59…
ee17c2e 
…/OTP-18913

Prepare for EEP59
@garazdawi
Merge pull request erlang#7988 from stevensonmt/fix-diagraph-edge-docs
f6d6092 
Update digraph.xml
@garazdawi
Merge branch 'maint'
7fa980c
@jhogberg
Merge pull request erlang#7973 from Maria-12648430/optimize_timer
96da8b9 
Optimize the `timer`module by using a private table

OTP-18914
@garazdawi
stdlib: Fix io_lib:get_until to return binaries at eof
038a7bf 
Before this fix if a get_until function would return
data when eof is seen, the result would not be correctly
converted to a binary when returned.

closes erlang#4992
@garazdawi
diameter: Fix appup for 2.4
0d9a0bc
@kikofernandez
compiler: implementation of EEP-59
f2f48e3 
Implementation of EEP-59 - Documentation Attributes

- Documentation attributes are added to the binary beam file, following
format of [EEP-48](https://www.erlang.org/eeps/eep-0048), via
`+beam_docs` compiler flag

- Warnings related to documentation attributes are dealt with in the
`beam_docs.erl` instead of adding them to `erl_lint.erl`

*Example 1*

```erlang
-module(warn_missing_doc).

-export([test/0, test/1, test/2]).
-export_type([test/0, test/1]).

-type test() :: ok.
-type test(N) :: N.

-callback test() -> ok.

-include("warn_missing_doc.hrl").

test() -> ok.
test(N) -> N.
```

Using the compiler flag `warn_missing_doc` will raise a warning when
doc. attributes are missing in exported functions, types, and callbacks.

*Example 2*

```erlang
-module(doc_with_file).

-export([main/1]).

-moduledoc {file, "README"}.

-doc {file, "FUN"}.
-spec main(Var) -> foo(Var).
main(X) ->
    X.
```

`moduledoc`s and `doc`s may refer to external files to be embedded.

*Example 3 - Warnings and Types*

```erlang
-export([uses_public/0]).
-export_type([public/0]).

-doc false.
-type hidden_type() :: integer().

-type intermediate() :: hidden_type().
-type public() :: intermediate().

-spec uses_public() -> public().
uses_public() ->
    ok.
```

Compiler warns about exported functions whose specs refer to hidden
types. In the example above, the `hidden_type()` is set as `hidden`
either via `-doc false` or `-doc hidden` and `public() :> intermediate()
:> hidden_type()`. When documentation attributes mark a type as hidden,
they won't be part of the documentation. Thus, the warning that the
`hidden_type()` is not part of the documentation, yet used in an
exported function.
@kikofernandez
Merge pull request erlang#7936 from kikofernandez/kiko/EEP-59-doc-att…
922ef22 
…ributes/PR-7936/OTP-18916

Implementation of EEP-59 - Documentation Attributes

- Documentation attributes are added to the binary beam file, following
format of [EEP-48](https://www.erlang.org/eeps/eep-0048), via
`+beam_docs` compiler flag

- Warnings related to documentation attributes are dealt with in the
`beam_docs.erl` instead of adding them to `erl_lint.erl`

*Example 1*

```erlang
-module(warn_missing_doc).

-export([test/0, test/1, test/2]).
-export_type([test/0, test/1]).

-type test() :: ok.
-type test(N) :: N.

-callback test() -> ok.

-include("warn_missing_doc.hrl").

test() -> ok.
test(N) -> N.
```

Using the compiler flag `warn_missing_doc` will raise a warning when
doc. attributes are missing in exported functions, types, and callbacks.

*Example 2*

```erlang
-module(doc_with_file).

-export([main/1]).

-moduledoc {file, "README"}.

-doc {file, "FUN"}.
-spec main(Var) -> foo(Var).
main(X) ->
    X.
```

`moduledoc`s and `doc`s may refer to external files to be embedded.

*Example 3 - Warnings and Types*

```erlang
-export([uses_public/0]).
-export_type([public/0]).

-doc false.
-type hidden_type() :: integer().

-type intermediate() :: hidden_type().
-type public() :: intermediate().

-spec uses_public() -> public().
uses_public() ->
    ok.
```

Compiler warns about exported functions whose specs refer to hidden
types. In the example above, the `hidden_type()` is set as `hidden`
either via `-doc false` or `-doc hidden` and `public() :> intermediate()
:> hidden_type()`. When documentation attributes mark a type as hidden,
they won't be part of the documentation. Thus, the warning that the
`hidden_type()` is not part of the documentation, yet used in an
exported function.
@u3s
ssh: fix cross compiled test builds
b0a38be
@paulswartz
fix: rename openssl/md5.h to avoid include conflicts with OpenSSL
eace72d 
Ref: erlang#7987
@paulswartz
fix: ifdef for preprocessor variables
287a807
@paulswartz
ci: don't use cache if an .h file is deleted
d27de65
@garazdawi
stdlib: Fix shell_SUITE type test
5c98665
@garazdawi
gh: Change always to !cancelled
278ff1e 
This way we can cancel the workflow faster, but it will
still run when previous jobs/steps fail.
@garazdawi
Merge branch 'lukas/gh/maint-26' into maint
57ce5c7 
* lukas/gh/maint-26:
  gh: Change always to !cancelled
@garazdawi
Merge branch 'maint'
94c2730
@garazdawi
snmp: Update cmd links
41da609
@garazdawi
Use guesslang to guess which code we have
e97210e
@garazdawi
New documentation
3bc03d4
@garazdawi
Update preloaded modules
d332a5e
@garazdawi
docs: Add tools to generate markdown from xml
a744fc3 
The tools take application/html+erlang EEP-48 style documentation
and convert it to EEP-59 style markdown. All erl_docgen types of
documentation files are supported.

This commit also adds some rudimentory support for generating HTML
docs using ex_doc. The support is basically there to test that
the XML to Markdown convertion works as intended and is not meant
for production use.
@garazdawi
edoc: Propagate type links from {@type macro}
dac69ef
@garazdawi
Fix edoc @hidden chunk handling
c7ca53b
@garazdawi
stdlib: Update shell_docs:normalize to always have blocks at top level
40794aa 
This change makes ExDoc rendering look nicer for edoc content.
@garazdawi
Fix all code and pre tags to be correct
7a129fa
@garazdawi
Add equiv handling for datatypes
a95fa23
@garazdawi
megaco: Generate chunks for all modules
1484adc
@garazdawi
Fix snmp to create chunks for all modules
01eb59e
@garazdawi
otp: Generate markdown documentation
c335dab
@garazdawi
Remove all EEP-59 gen things
579fa70
@garazdawi
Remove all traces of xml docs
9d69cdc
@garazdawi
Compile docs by default
7fd2be1
@garazdawi
otp: Add html link checker
7a07daa
@garazdawi
asn1, parsetools: Add -moduledoc false to generated files
1bd1c9c
@garazdawi
edoc: Revert to edoc style docs for edoc
0e9b0ac 
We do this so that we also test that edoc still works
as it should and that ExDoc can render edoc docs nicely.
@garazdawi
erts: Hide t:bool/0 in favor for t:boolean/0
90fd72f
@garazdawi
otp: Add support for generating docs with ExDoc
073e025
@garazdawi
erts: Build erts beams during compilation
ecdc4d8 
Instead of using the preloaded beam files for the erts
applications beam files, we build new beam files when
we have a working compiler. The beam files in the erts
application are only used for tools such as dialyzer
and EEP-59 documentation.
@garazdawi
peer: Make compilable from PRE 27 releases
b1108ee
@garazdawi
Fix megaco things
766dc58
@garazdawi
Fix snmp
5c4e17b
@garazdawi
Temp fixes for dev
451bfe1
@garazdawi garazdawi force-pushed the lukas/convert-docs-to-markdown branch from 9cb6ebd to 451bfe1 Compare January 16, 2024 18:37
@garazdawi garazdawi merged commit 84cdc58 into master Jan 18, 2024
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@garazdawi garazdawi

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.