Add occurrence count in the json output for search engine #1076

panglesd · 2024-01-31T09:54:30Z

Since #972 there is a compile-index command which outputs a json index containing all searchable entries. The intended purpose of this json index is to pass it to a search engine.

Since #976 there is also a count-occurrences command which creates a table containing the number of occurrences for each identifier.

This PR makes the compile-index be able to use the information from count-occurrence. It adds a --occurrences argument to get the path to the table.

The first commit isolate the occurrence code in its own library.
The second commit implements the --occurrences argument of ``compile-index`.

panglesd · 2024-01-31T09:55:35Z

Note that there will be a conflict with #1067 which we will need to resolve (in whichever PR gets merged last).

Signed-off-by: Paul-Elliot <peada@free.fr>

gpetiot

Minor comment about the displaying of the result, otherwise looks good to me!

gpetiot · 2024-02-12T14:25:16Z

src/search/json_index/json_search.ml

+            ("direct", `Float (float_of_int occ.Odoc_occurrences.Table.direct));
+            ("indirect", `Float (float_of_int occ.indirect));
+          ]
+    | None -> `Null


Nit-picking but it would look more consistent to me if we had 0 for both counters here (see the output in the tests).

What about "no 'occurrences' field at all"?
I'm not very keen on choosing a wrong value when it was not possible to compute one because the table was not given...

Well, at least if we don't put this field at all it reduces the diff of this PR!

With 6205f97, if the occurrence table is not given, the field is not included.
However, if the occurrence table is given, but does not contain the occurrence, a use of 0 is counted.

Indeed, that removed that the diff for search without occurrences.

Thanks for the comment, I think that's better now!

Signed-off-by: Paul-Elliot <peada@free.fr>

doc/driver.mld

EmileTrotignon · 2024-07-09T14:16:20Z

What's the status of this ?

panglesd · 2024-07-09T14:41:26Z

It was ready for review. Now there are many conflicting files.

Signed-off-by: Paul-Elliot <peada@free.fr>

panglesd · 2024-07-12T07:47:45Z

It is now rebased and ready for review.

Julow

Looks good! Finally a way to consume the occurrences data.

Julow · 2024-07-15T15:31:36Z

src/search/json_index/json_search.ml

+        (* We don't want to include the [sub] field of occurrence tables. We use
+           a "polymorphic record" to avoid defining a type, but still get named
+           fields! *)


This is original but I find it confusing and heavy in syntax. What about defining a record with just that instead ?

This record could even replace the item type in Table as the sub field is never used outside of this module.
Here's an other idea: Julow@9cafc01

I was thinking that the sub field could be useful to some application of the occurrence table.

However, there are no such application. I removed sub from the exposed type in the last commit, and use that type instead of this weird tuple. If an application requires exposing the sub field, we'll do it at this moment.

Signed-off-by: Paul-Elliot <peada@free.fr>

This adds the occurrence count to each element of the json index Signed-off-by: Paul-Elliot <peada@free.fr>

Signed-off-by: Paul-Elliot <peada@free.fr>

Co-authored-by: Guillaume "Liam" Petiot <guillaume@tarides.com>

Signed-off-by: Paul-Elliot <peada@free.fr>

Julow

Perfect, let's merge!

Signed-off-by: Paul-Elliot <peada@free.fr>

@jonludlam

CHANGES: ### Highlight - Hierarchical documentation (@jonludlam, @panglesd, @Julow) Pages can now be organized in a directory tree structure. Relative and absolute references are added: `{!./other_page.label}`, `{!//other_page}`. - Improved sidebar and breadcrumbs navigation (@panglesd, @gpetiot) The documentation pages and the libraries of the entire package are shown on the left sidebar. - Added support for images, videos, audio and other assets The syntax is `{image!/reference/to/asset}` or `{image:URL}` for images. The syntax for `{video...}` and `{audio...}` is the same. (@panglesd, @EmileTrotignon, ocaml/odoc#1170, ocaml/odoc#1171, ocaml/odoc#1184, ocaml/odoc#1185) - Search using Sherlodoc (@panglesd, @EmileTrotignon, @Julow) A new search bar that supports full-text and type-based search. ### Added - Experimental driver (@jonludlam, @panglesd) The driver builds the documentation for a collection of Opam packages using the newer Odoc features. It supports linking external packages to ocaml.org and markdown files. This is experimental and will break in the future. - Cross-package references (@panglesd, @Julow) Pages and modules from other packages can be referenced: `{!/otherpackage/page}`, `{!/otherpackage/Module.t}`. - Option to remap links to other packages to ocaml.org or other site. See the `--remap` option of the driver or the `--remap-file` option of `odoc html-generate`. (@jonludlam, ocaml/odoc#1189, ocaml/odoc#1248) - Option to compute occurrences of use of each identifiers The commands `aggregate-occurrences` and `count-occurrences` are added. (@panglesd, ocaml/odoc#976, ocaml/odoc#1076, ocaml/odoc#1206) - Added the `odoc classify` command (@jonludlam, ocaml/odoc#1121) Helps driver detecting which modules belong to which libraries. - Added `--suppress-warnings` to the CLI to remove warnings from a unit, even if they end up being raised in another unit through expansion (@jonludlam, ocaml/odoc#1260) - Add clock emoji before `@since` tag (@yawaramin, ocaml/odoc#1089) - Navigation for the search bar : use '/' to enter search, up and down arrows to select a result, and enter to follow the selected link. (@EmileTrotignon, ocaml/odoc#1088) - Fix a big gap between the preamble and the content of a page (@EmileTrotignon, ocaml/odoc#1147) - Add a marshalled search index consumable by sherlodoc (@EmileTrotignon, @panglesd, ocaml/odoc#1084) - Allow referencing of polymorphic constructors in polymorphic variant type aliases (@panglesd, ocaml/odoc#1115) - Added a home icon in the breacrumbs (@panglesd, ocaml/odoc#1251) It can be disabled with a CLI option. - Add a frontmatter syntax for mld pages (@panglesd, ocaml/odoc#1187, ocaml/odoc#1193, ocaml/odoc#1243, ocaml/odoc#1246, ocaml/odoc#1251) Allows to specify the title of a page, the order of sub-pages and other behaviors in the sidebar. - Added `odoc-md` to process standalone Markdown pages (@jonludlam, ocaml/odoc#1234) ### Changed - The command line interface changed to support the new features. + Packages and libraries: `odoc link` must now be aware of packages and libraries with the `-L libname:path` and `-P pkgname:path` options. The module search path should still be passed with the `-I` option. The current package should be specified with `--current-package=pkgname`. + Hierarchy: `odoc compile` now outputs `.odoc` in the directory tree specified with `--output-dir=DIR` and the parent identifier must be specified with `--parent-id=PARENT`. The option `--source-parent-file` is removed. + Source code: Implementations are compiled with `compile-impl` instead of with `compile`. The options `--cmt=..` and `--source-name=..` are removed. Source code pages are generated with `html-generate-source`. + Assets: The commands `compile-asset`, `html-generate-asset` are added. The option `html-generate --asset` is removed. + Sidebar: The index is built using `compile-index`. The sidebar data is extracted from the index with `sidebar-generate` and passed to `html-generate --sidebar=..`. - The syntax for `@tag` is now delimited (@panglesd, ocaml/odoc#1239) A `@tag` can now be followed by a paragraph or other elements. - Updated colors for code fragments (@EmileTrotignon, ocaml/odoc#1023) - Fixed complexity of looking up `.odoc` files (@panglesd, ocaml/odoc#1075) - Normalize whitespaces in codespans (@gpetiot, ocaml/odoc#1085) A newline followed by any whitespaces is normalized as one space character. - Reduce size of `Odoc_html_frontend` when compiled to javascript (@EmileTrotignon, ocaml/odoc#1072) - Overhaul of module-type-of expansions and shadowing code (@jonludlam, ocaml/odoc#1081) - Output file paths and labels in the man and latex backends changed to avoid name clashes (@Julow, ocaml/odoc#1191) ### Fixed - Fix variant constructors being hidden if they contain hidden types (@jonludlam, ocaml/odoc#1105) - Fix rare assertion failure due to optional parameters (@jonludlam, ocaml/odoc#1272, issue ocaml/odoc#1001) - Fix resolution of module synopses in {!modules} lists that require --open (@jonludlam, ocaml/odoc#1104} - Fix top comment not being taken from includes often enough (@panglesd, ocaml/odoc#1117) - Fixed 404 links from search results (@panglesd, ocaml/odoc#1108) - Fixed title content not being picked up across pages when rendering references (ocaml/odoc#1116, @panglesd) - Fix wrong links to standalone comments in search results (ocaml/odoc#1118, @panglesd) - Remove duplicated or unwanted comments with inline includes (@Julow, ocaml/odoc#1133) - Fix bug where source rendering would cause odoc to fail completely if it encounters invalid syntax (@jonludlam ocaml/odoc#1208) - Add missing parentheses in 'val (let*) : ...' (@Julow, ocaml/odoc#1268) - Fix syntax highlighting not working for very large files (@jonludlam, @Julow, ocaml/odoc#1277)

@jonludlam

CHANGES: - Hierarchical documentation (@jonludlam, @panglesd, @Julow) Pages can now be organized in a directory tree structure. Relative and absolute references are added: `{!./other_page.label}`, `{!//other_page}`. - Improved sidebar and breadcrumbs navigation (@panglesd, @gpetiot) The documentation pages and the libraries of the entire package are shown on the left sidebar. - Added support for images, videos, audio and other assets The syntax is `{image!/reference/to/asset}` or `{image:URL}` for images. The syntax for `{video...}` and `{audio...}` is the same. (@panglesd, @EmileTrotignon, ocaml/odoc#1170, ocaml/odoc#1171, ocaml/odoc#1184, ocaml/odoc#1185) - Search using Sherlodoc (@panglesd, @EmileTrotignon, @Julow) A new search bar that supports full-text and type-based search. - Experimental driver (@jonludlam, @panglesd) The driver builds the documentation for a collection of Opam packages using the newer Odoc features. It supports linking external packages to ocaml.org and markdown files. This is experimental and will break in the future. - Cross-package references (@panglesd, @Julow) Pages and modules from other packages can be referenced: `{!/otherpackage/page}`, `{!/otherpackage/Module.t}`. - Option to remap links to other packages to ocaml.org or other site. See the `--remap` option of the driver or the `--remap-file` option of `odoc html-generate`. (@jonludlam, ocaml/odoc#1189, ocaml/odoc#1248) - Option to compute occurrences of use of each identifiers The commands `aggregate-occurrences` and `count-occurrences` are added. (@panglesd, ocaml/odoc#976, ocaml/odoc#1076, ocaml/odoc#1206) - Added the `odoc classify` command (@jonludlam, ocaml/odoc#1121) Helps driver detecting which modules belong to which libraries. - Added `--suppress-warnings` to the CLI to remove warnings from a unit, even if they end up being raised in another unit through expansion (@jonludlam, ocaml/odoc#1260) - Add clock emoji before `@since` tag (@yawaramin, ocaml/odoc#1089) - Navigation for the search bar : use '/' to enter search, up and down arrows to select a result, and enter to follow the selected link. (@EmileTrotignon, ocaml/odoc#1088) - Fix a big gap between the preamble and the content of a page (@EmileTrotignon, ocaml/odoc#1147) - Add a marshalled search index consumable by sherlodoc (@EmileTrotignon, @panglesd, ocaml/odoc#1084) - Allow referencing of polymorphic constructors in polymorphic variant type aliases (@panglesd, ocaml/odoc#1115) - Added a home icon in the breacrumbs (@panglesd, ocaml/odoc#1251) It can be disabled with a CLI option. - Add a frontmatter syntax for mld pages (@panglesd, ocaml/odoc#1187, ocaml/odoc#1193, ocaml/odoc#1243, ocaml/odoc#1246, ocaml/odoc#1251) Allows to specify the title of a page, the order of sub-pages and other behaviors in the sidebar. - Added `odoc-md` to process standalone Markdown pages (@jonludlam, ocaml/odoc#1234) - The command line interface changed to support the new features. + Packages and libraries: `odoc link` must now be aware of packages and libraries with the `-L libname:path` and `-P pkgname:path` options. The module search path should still be passed with the `-I` option. The current package should be specified with `--current-package=pkgname`. + Hierarchy: `odoc compile` now outputs `.odoc` in the directory tree specified with `--output-dir=DIR` and the parent identifier must be specified with `--parent-id=PARENT`. The option `--source-parent-file` is removed. + Source code: Implementations are compiled with `compile-impl` instead of with `compile`. The options `--cmt=..` and `--source-name=..` are removed. Source code pages are generated with `html-generate-source`. + Assets: The commands `compile-asset`, `html-generate-asset` are added. The option `html-generate --asset` is removed. + Sidebar: The index is built using `compile-index`. The sidebar data is extracted from the index with `sidebar-generate` and passed to `html-generate --sidebar=..`. - The syntax for `@tag` is now delimited (@panglesd, ocaml/odoc#1239) A `@tag` can now be followed by a paragraph or other elements. - Updated colors for code fragments (@EmileTrotignon, ocaml/odoc#1023) - Fixed complexity of looking up `.odoc` files (@panglesd, ocaml/odoc#1075) - Normalize whitespaces in codespans (@gpetiot, ocaml/odoc#1085) A newline followed by any whitespaces is normalized as one space character. - Reduce size of `Odoc_html_frontend` when compiled to javascript (@EmileTrotignon, ocaml/odoc#1072) - Overhaul of module-type-of expansions and shadowing code (@jonludlam, ocaml/odoc#1081) - Output file paths and labels in the man and latex backends changed to avoid name clashes (@Julow, ocaml/odoc#1191) - Fix variant constructors being hidden if they contain hidden types (@jonludlam, ocaml/odoc#1105) - Fix rare assertion failure due to optional parameters (@jonludlam, ocaml/odoc#1272, issue ocaml/odoc#1001) - Fix resolution of module synopses in {!modules} lists that require --open (@jonludlam, ocaml/odoc#1104} - Fix top comment not being taken from includes often enough (@panglesd, ocaml/odoc#1117) - Fixed 404 links from search results (@panglesd, ocaml/odoc#1108) - Fixed title content not being picked up across pages when rendering references (ocaml/odoc#1116, @panglesd) - Fix wrong links to standalone comments in search results (ocaml/odoc#1118, @panglesd) - Remove duplicated or unwanted comments with inline includes (@Julow, ocaml/odoc#1133) - Fix bug where source rendering would cause odoc to fail completely if it encounters invalid syntax (@jonludlam ocaml/odoc#1208) - Add missing parentheses in 'val (let*) : ...' (@Julow, ocaml/odoc#1268) - Fix syntax highlighting not working for very large files (@jonludlam, @Julow, ocaml/odoc#1277)

@jonludlam

CHANGES: ### Highlight - Hierarchical documentation (@jonludlam, @panglesd, @Julow). Pages can now be organized in a directory tree structure. Relative and absolute references are added: `{!./other_page.label}`, `{!//other_page}`. - Improved sidebar and breadcrumbs navigation (@panglesd, @gpetiot). The documentation pages and the libraries of the entire package are shown on the left sidebar. - Added support for images, videos, audio and other assets. The syntax is `{image!/reference/to/asset}` or `{image:URL}` for images. The syntax for `{video...}` and `{audio...}` is the same. (@panglesd, @EmileTrotignon, ocaml/odoc#1170, ocaml/odoc#1171, ocaml/odoc#1184, ocaml/odoc#1185) - Search using Sherlodoc (@panglesd, @EmileTrotignon, @Julow). A new search bar that supports full-text and type-based search. ### Added - Experimental driver (@jonludlam, @panglesd) The driver builds the documentation for a collection of Opam packages using the newer Odoc features. It supports linking external packages to ocaml.org and markdown files. This is experimental and will break in the future. - Cross-package references (@panglesd, @Julow) Pages and modules from other packages can be referenced: `{!/otherpackage/page}`, `{!/otherpackage/Module.t}`. - Option to remap links to other packages to ocaml.org or other site. See the `--remap` option of the driver or the `--remap-file` option of `odoc html-generate`. (@jonludlam, ocaml/odoc#1189, ocaml/odoc#1248) - Option to compute occurrences of use of each identifiers The commands `aggregate-occurrences` and `count-occurrences` are added. (@panglesd, ocaml/odoc#976, ocaml/odoc#1076, ocaml/odoc#1206) - Added an `extract-code` subcommand to extract code blocks from mld/mli files (@panglesd, ocaml/odoc#1326) - Added the `odoc classify` command (@jonludlam, ocaml/odoc#1121) Helps driver detecting which modules belong to which libraries. - Added `--warnings-tag` options to the CLI to silence warnings from a unit, even if they end up being raised in another unit through expansion (@jonludlam, ocaml/odoc#1260) - Add clock emoji before `@since` tag (@yawaramin, ocaml/odoc#1089) - Navigation for the search bar : use '/' to enter search, up and down arrows to select a result, and enter to follow the selected link. (@EmileTrotignon, ocaml/odoc#1088) - Fix a big gap between the preamble and the content of a page (@EmileTrotignon, ocaml/odoc#1147) - Add a marshalled search index consumable by sherlodoc (@EmileTrotignon, @panglesd, ocaml/odoc#1084) - Allow referencing of polymorphic constructors in polymorphic variant type aliases (@panglesd, ocaml/odoc#1115) - Added a home icon in the breacrumbs (@panglesd, ocaml/odoc#1251) It can be disabled with a CLI option. - Add a frontmatter syntax for mld pages (@panglesd, ocaml/odoc#1187, ocaml/odoc#1193, ocaml/odoc#1243, ocaml/odoc#1246, ocaml/odoc#1251) Allows to specify the title of a page, the order of sub-pages and other behaviors in the sidebar. - Added `odoc-md` to process standalone Markdown pages (@jonludlam, ocaml/odoc#1234) - Added CSS selectors to style version and and nav links when they appear within page titles, as produced by odig (@katrinafyi, ocaml/odoc#1290) - Added support for (local) images in the latex backend (@Octachron, ocaml/odoc#1297) ### Changed - The command line interface changed to support the new features. + Packages and libraries: `odoc link` must now be aware of packages and libraries with the `-L libname:path` and `-P pkgname:path` options. The module search path should still be passed with the `-I` option. The current package should be specified with `--current-package=pkgname`. + Hierarchy: `odoc compile` now outputs `.odoc` in the directory tree specified with `--output-dir=DIR` and the parent identifier must be specified with `--parent-id=PARENT`. The option `--source-parent-file` is removed. + Source code: Implementations are compiled with `compile-impl` instead of with `compile`. The options `--cmt=..` and `--source-name=..` are removed. Source code pages are generated with `html-generate-source`. + Assets: The commands `compile-asset`, `html-generate-asset` are added. The option `html-generate --asset` is removed. + Sidebar: The index is built using `compile-index`. The sidebar data is extracted from the index with `sidebar-generate` and passed to `html-generate --sidebar=..`. - The syntax for `@tag` is now delimited (@panglesd, ocaml/odoc#1239) A `@tag` can now be followed by a paragraph or other elements. - Updated colors for code fragments (@EmileTrotignon, ocaml/odoc#1023) - Fixed complexity of looking up `.odoc` files (@panglesd, ocaml/odoc#1075) - Normalize whitespaces in codespans (@gpetiot, ocaml/odoc#1085) A newline followed by any whitespaces is normalized as one space character. - Reduce size of `Odoc_html_frontend` when compiled to javascript (@EmileTrotignon, ocaml/odoc#1072) - Overhaul of module-type-of expansions and shadowing code (@jonludlam, ocaml/odoc#1081) - Output file paths and labels in the man and latex backends changed to avoid name clashes (@Julow, ocaml/odoc#1191) - Added a `header` field to the json output (@panglesd, ocaml/odoc#1314) - Changed indentation rules for code block and verbatim content (@panglesd, ocaml/odoc#1317) - odoc-parser: Store raw content in verbatim and code block, and expose a function to process it (@panglesd, ocaml/odoc#1325) ### Fixed - Fix variant constructors being hidden if they contain hidden types (@jonludlam, ocaml/odoc#1105) - Fix rare assertion failure due to optional parameters (@jonludlam, ocaml/odoc#1272, issue ocaml/odoc#1001) - Fix resolution of module synopses in {!modules} lists that require --open (@jonludlam, ocaml/odoc#1104} - Fix top comment not being taken from includes often enough (@panglesd, ocaml/odoc#1117) - Fixed 404 links from search results (@panglesd, ocaml/odoc#1108) - Fixed title content not being picked up across pages when rendering references (ocaml/odoc#1116, @panglesd) - Fix wrong links to standalone comments in search results (ocaml/odoc#1118, @panglesd) - Remove duplicated or unwanted comments with inline includes (@Julow, ocaml/odoc#1133) - Fix bug where source rendering would cause odoc to fail completely if it encounters invalid syntax (@jonludlam ocaml/odoc#1208) - Add missing parentheses in 'val (let*) : ...' (@Julow, ocaml/odoc#1268) - Fix syntax highlighting not working for very large files (@jonludlam, @Julow, ocaml/odoc#1277) - Fix backtrace on invalid input in compile-deps (@jonludlam, ocaml/odoc#1313) - Fix bug in our CSS hitting verbatim blocks in tags (@jonludlam, ocaml/odoc#1312) - Fix issue ocaml/odoc#610 where `odoc html-fragment` wasn't rendering headings correctly (@jonludlam, ocaml/odoc#1306)

@jonludlam

CHANGES: ### Highlight - Hierarchical documentation (@jonludlam, @panglesd, @Julow). Pages can now be organized in a directory tree structure. Relative and absolute references are added: `{!./other_page.label}`, `{!//other_page}`. - Improved sidebar and breadcrumbs navigation (@panglesd, @gpetiot). The documentation pages and the libraries of the entire package are shown on the left sidebar. - Added support for images, videos, audio and other assets. The syntax is `{image!/reference/to/asset}` or `{image:URL}` for images. The syntax for `{video...}` and `{audio...}` is the same. (@panglesd, @EmileTrotignon, ocaml/odoc#1170, ocaml/odoc#1171, ocaml/odoc#1184, ocaml/odoc#1185) - Search using Sherlodoc (@panglesd, @EmileTrotignon, @Julow). A new search bar that supports full-text and type-based search. ### Added - Experimental driver (@jonludlam, @panglesd) The driver builds the documentation for a collection of Opam packages using the newer Odoc features. It supports linking external packages to ocaml.org and markdown files. This is experimental and will break in the future. - Cross-package references (@panglesd, @Julow) Pages and modules from other packages can be referenced: `{!/otherpackage/page}`, `{!/otherpackage/Module.t}`. - Option to remap links to other packages to ocaml.org or other site. See the `--remap` option of the driver or the `--remap-file` option of `odoc html-generate`. (@jonludlam, ocaml/odoc#1189, ocaml/odoc#1248) - Option to compute occurrences of use of each identifiers The commands `aggregate-occurrences` and `count-occurrences` are added. (@panglesd, ocaml/odoc#976, ocaml/odoc#1076, ocaml/odoc#1206) - Added an `extract-code` subcommand to extract code blocks from mld/mli files (@panglesd, ocaml/odoc#1326) - Added the `odoc classify` command (@jonludlam, ocaml/odoc#1121) Helps driver detecting which modules belong to which libraries. - Added `--warnings-tag` options to the CLI to silence warnings from a unit, even if they end up being raised in another unit through expansion (@jonludlam, ocaml/odoc#1260) - Add clock emoji before `@since` tag (@yawaramin, ocaml/odoc#1089) - Navigation for the search bar : use '/' to enter search, up and down arrows to select a result, and enter to follow the selected link. (@EmileTrotignon, ocaml/odoc#1088) - Fix a big gap between the preamble and the content of a page (@EmileTrotignon, ocaml/odoc#1147) - Add a marshalled search index consumable by sherlodoc (@EmileTrotignon, @panglesd, ocaml/odoc#1084) - Allow referencing of polymorphic constructors in polymorphic variant type aliases (@panglesd, ocaml/odoc#1115) - Added a home icon in the breacrumbs (@panglesd, ocaml/odoc#1251) It can be disabled with a CLI option. - Add a frontmatter syntax for mld pages (@panglesd, ocaml/odoc#1187, ocaml/odoc#1193, ocaml/odoc#1243, ocaml/odoc#1246, ocaml/odoc#1251) Allows to specify the title of a page, the order of sub-pages and other behaviors in the sidebar. - Added `odoc-md` to process standalone Markdown pages (@jonludlam, ocaml/odoc#1234) - Added CSS selectors to style version and and nav links when they appear within page titles, as produced by odig (@katrinafyi, ocaml/odoc#1290) - Added support for (local) images in the latex backend (@Octachron, ocaml/odoc#1297) ### Changed - The command line interface changed to support the new features. + Packages and libraries: `odoc link` must now be aware of packages and libraries with the `-L libname:path` and `-P pkgname:path` options. The module search path should still be passed with the `-I` option. The current package should be specified with `--current-package=pkgname`. + Hierarchy: `odoc compile` now outputs `.odoc` in the directory tree specified with `--output-dir=DIR` and the parent identifier must be specified with `--parent-id=PARENT`. The option `--source-parent-file` is removed. + Source code: Implementations are compiled with `compile-impl` instead of with `compile`. The options `--cmt=..` and `--source-name=..` are removed. Source code pages are generated with `html-generate-source`. + Assets: The commands `compile-asset`, `html-generate-asset` are added. The option `html-generate --asset` is removed. + Sidebar: The index is built using `compile-index`. The sidebar data is extracted from the index with `sidebar-generate` and passed to `html-generate --sidebar=..`. - The syntax for `@tag` is now delimited (@panglesd, ocaml/odoc#1239) A `@tag` can now be followed by a paragraph or other elements. - Updated colors for code fragments (@EmileTrotignon, ocaml/odoc#1023) - Fixed complexity of looking up `.odoc` files (@panglesd, ocaml/odoc#1075) - Normalize whitespaces in codespans (@gpetiot, ocaml/odoc#1085) A newline followed by any whitespaces is normalized as one space character. - Reduce size of `Odoc_html_frontend` when compiled to javascript (@EmileTrotignon, ocaml/odoc#1072) - Overhaul of module-type-of expansions and shadowing code (@jonludlam, ocaml/odoc#1081) - Output file paths and labels in the man and latex backends changed to avoid name clashes (@Julow, ocaml/odoc#1191) - Added a `header` field to the json output (@panglesd, ocaml/odoc#1314) - Changed indentation rules for code block and verbatim content (@panglesd, ocaml/odoc#1317) - odoc-parser: Store raw content in verbatim and code block, and expose a function to process it (@panglesd, ocaml/odoc#1325) ### Fixed - Fix variant constructors being hidden if they contain hidden types (@jonludlam, ocaml/odoc#1105) - Fix rare assertion failure due to optional parameters (@jonludlam, ocaml/odoc#1272, issue ocaml/odoc#1001) - Fix resolution of module synopses in {!modules} lists that require --open (@jonludlam, ocaml/odoc#1104} - Fix top comment not being taken from includes often enough (@panglesd, ocaml/odoc#1117) - Fixed 404 links from search results (@panglesd, ocaml/odoc#1108) - Fixed title content not being picked up across pages when rendering references (ocaml/odoc#1116, @panglesd) - Fix wrong links to standalone comments in search results (ocaml/odoc#1118, @panglesd) - Remove duplicated or unwanted comments with inline includes (@Julow, ocaml/odoc#1133) - Fix bug where source rendering would cause odoc to fail completely if it encounters invalid syntax (@jonludlam ocaml/odoc#1208) - Add missing parentheses in 'val (let*) : ...' (@Julow, ocaml/odoc#1268) - Fix syntax highlighting not working for very large files (@jonludlam, @Julow, ocaml/odoc#1277) - Fix backtrace on invalid input in compile-deps (@jonludlam, ocaml/odoc#1313) - Fix bug in our CSS hitting verbatim blocks in tags (@jonludlam, ocaml/odoc#1312) - Fix issue ocaml/odoc#610 where `odoc html-fragment` wasn't rendering headings correctly (@jonludlam, ocaml/odoc#1306)

@jonludlam

CHANGES: - Hierarchical documentation (@jonludlam, @panglesd, @Julow). Pages can now be organized in a directory tree structure. Relative and absolute references are added: `{!./other_page.label}`, `{!//other_page}`. - Improved sidebar and breadcrumbs navigation (@panglesd, @gpetiot). The documentation pages and the libraries of the entire package are shown on the left sidebar. - Added support for images, videos, audio and other assets. The syntax is `{image!/reference/to/asset}` or `{image:URL}` for images. The syntax for `{video...}` and `{audio...}` is the same. (@panglesd, @EmileTrotignon, ocaml/odoc#1170, ocaml/odoc#1171, ocaml/odoc#1184, ocaml/odoc#1185) - Search using Sherlodoc (@panglesd, @EmileTrotignon, @Julow). A new search bar that supports full-text and type-based search. - Experimental driver (@jonludlam, @panglesd) The driver builds the documentation for a collection of Opam packages using the newer Odoc features. It supports linking external packages to ocaml.org and markdown files. This is experimental and will break in the future. - Cross-package references (@panglesd, @Julow) Pages and modules from other packages can be referenced: `{!/otherpackage/page}`, `{!/otherpackage/Module.t}`. - Option to remap links to other packages to ocaml.org or other site. See the `--remap` option of the driver or the `--remap-file` option of `odoc html-generate`. (@jonludlam, ocaml/odoc#1189, ocaml/odoc#1248) - Option to compute occurrences of use of each identifiers The commands `aggregate-occurrences` and `count-occurrences` are added. (@panglesd, ocaml/odoc#976, ocaml/odoc#1076, ocaml/odoc#1206) - Added an `extract-code` subcommand to extract code blocks from mld/mli files (@panglesd, ocaml/odoc#1326) - Added the `odoc classify` command (@jonludlam, ocaml/odoc#1121) Helps driver detecting which modules belong to which libraries. - Added `--warnings-tag` options to the CLI to silence warnings from a unit, even if they end up being raised in another unit through expansion (@jonludlam, ocaml/odoc#1260) - Add clock emoji before `@since` tag (@yawaramin, ocaml/odoc#1089) - Navigation for the search bar : use '/' to enter search, up and down arrows to select a result, and enter to follow the selected link. (@EmileTrotignon, ocaml/odoc#1088) - Fix a big gap between the preamble and the content of a page (@EmileTrotignon, ocaml/odoc#1147) - Add a marshalled search index consumable by sherlodoc (@EmileTrotignon, @panglesd, ocaml/odoc#1084) - Allow referencing of polymorphic constructors in polymorphic variant type aliases (@panglesd, ocaml/odoc#1115) - Added a home icon in the breacrumbs (@panglesd, ocaml/odoc#1251) It can be disabled with a CLI option. - Add a frontmatter syntax for mld pages (@panglesd, ocaml/odoc#1187, ocaml/odoc#1193, ocaml/odoc#1243, ocaml/odoc#1246, ocaml/odoc#1251) Allows to specify the title of a page, the order of sub-pages and other behaviors in the sidebar. - Added `odoc-md` to process standalone Markdown pages (@jonludlam, ocaml/odoc#1234) - Added CSS selectors to style version and and nav links when they appear within page titles, as produced by odig (@katrinafyi, ocaml/odoc#1290) - Added support for (local) images in the latex backend (@Octachron, ocaml/odoc#1297) - The command line interface changed to support the new features. + Packages and libraries: `odoc link` must now be aware of packages and libraries with the `-L libname:path` and `-P pkgname:path` options. The module search path should still be passed with the `-I` option. The current package should be specified with `--current-package=pkgname`. + Hierarchy: `odoc compile` now outputs `.odoc` in the directory tree specified with `--output-dir=DIR` and the parent identifier must be specified with `--parent-id=PARENT`. The option `--source-parent-file` is removed. + Source code: Implementations are compiled with `compile-impl` instead of with `compile`. The options `--cmt=..` and `--source-name=..` are removed. Source code pages are generated with `html-generate-source`. + Assets: The commands `compile-asset`, `html-generate-asset` are added. The option `html-generate --asset` is removed. + Sidebar: The index is built using `compile-index`. The sidebar data is extracted from the index with `sidebar-generate` and passed to `html-generate --sidebar=..`. - The syntax for `@tag` is now delimited (@panglesd, ocaml/odoc#1239) A `@tag` can now be followed by a paragraph or other elements. - Updated colors for code fragments (@EmileTrotignon, ocaml/odoc#1023) - Fixed complexity of looking up `.odoc` files (@panglesd, ocaml/odoc#1075) - Normalize whitespaces in codespans (@gpetiot, ocaml/odoc#1085) A newline followed by any whitespaces is normalized as one space character. - Reduce size of `Odoc_html_frontend` when compiled to javascript (@EmileTrotignon, ocaml/odoc#1072) - Overhaul of module-type-of expansions and shadowing code (@jonludlam, ocaml/odoc#1081) - Output file paths and labels in the man and latex backends changed to avoid name clashes (@Julow, ocaml/odoc#1191) - Added a `header` field to the json output (@panglesd, ocaml/odoc#1314) - Changed indentation rules for code block and verbatim content (@panglesd, ocaml/odoc#1317) - odoc-parser: Store raw content in verbatim and code block, and expose a function to process it (@panglesd, ocaml/odoc#1325) - Fix variant constructors being hidden if they contain hidden types (@jonludlam, ocaml/odoc#1105) - Fix rare assertion failure due to optional parameters (@jonludlam, ocaml/odoc#1272, issue ocaml/odoc#1001) - Fix resolution of module synopses in {!modules} lists that require --open (@jonludlam, ocaml/odoc#1104} - Fix top comment not being taken from includes often enough (@panglesd, ocaml/odoc#1117) - Fixed 404 links from search results (@panglesd, ocaml/odoc#1108) - Fixed title content not being picked up across pages when rendering references (ocaml/odoc#1116, @panglesd) - Fix wrong links to standalone comments in search results (ocaml/odoc#1118, @panglesd) - Remove duplicated or unwanted comments with inline includes (@Julow, ocaml/odoc#1133) - Fix bug where source rendering would cause odoc to fail completely if it encounters invalid syntax (@jonludlam ocaml/odoc#1208) - Add missing parentheses in 'val (let*) : ...' (@Julow, ocaml/odoc#1268) - Fix syntax highlighting not working for very large files (@jonludlam, @Julow, ocaml/odoc#1277) - Fix backtrace on invalid input in compile-deps (@jonludlam, ocaml/odoc#1313) - Fix bug in our CSS hitting verbatim blocks in tags (@jonludlam, ocaml/odoc#1312) - Fix issue ocaml/odoc#610 where `odoc html-fragment` wasn't rendering headings correctly (@jonludlam, ocaml/odoc#1306)

@jonludlam

CHANGES: - Hierarchical documentation (@jonludlam, @panglesd, @Julow). Pages can now be organized in a directory tree structure. Relative and absolute references are added: `{!./other_page.label}`, `{!//other_page}`. - Improved sidebar and breadcrumbs navigation (@panglesd, @gpetiot). The documentation pages and the libraries of the entire package are shown on the left sidebar. - Added support for images, videos, audio and other assets. The syntax is `{image!/reference/to/asset}` or `{image:URL}` for images. The syntax for `{video...}` and `{audio...}` is the same. (@panglesd, @EmileTrotignon, ocaml/odoc#1170, ocaml/odoc#1171, ocaml/odoc#1184, ocaml/odoc#1185) - Search using Sherlodoc (@panglesd, @EmileTrotignon, @Julow). A new search bar that supports full-text and type-based search. - Experimental driver (@jonludlam, @panglesd) The driver builds the documentation for a collection of Opam packages using the newer Odoc features. It supports linking external packages to ocaml.org and markdown files. This is experimental and will break in the future. - Cross-package references (@panglesd, @Julow) Pages and modules from other packages can be referenced: `{!/otherpackage/page}`, `{!/otherpackage/Module.t}`. - Option to remap links to other packages to ocaml.org or other site. See the `--remap` option of the driver or the `--remap-file` option of `odoc html-generate`. (@jonludlam, ocaml/odoc#1189, ocaml/odoc#1248) - Option to compute occurrences of use of each identifiers The commands `aggregate-occurrences` and `count-occurrences` are added. (@panglesd, ocaml/odoc#976, ocaml/odoc#1076, ocaml/odoc#1206) - Added an `extract-code` subcommand to extract code blocks from mld/mli files (@panglesd, ocaml/odoc#1326) - Added the `odoc classify` command (@jonludlam, ocaml/odoc#1121) Helps driver detecting which modules belong to which libraries. - Added `--warnings-tag` options to the CLI to silence warnings from a unit, even if they end up being raised in another unit through expansion (@jonludlam, ocaml/odoc#1260) - Add clock emoji before `@since` tag (@yawaramin, ocaml/odoc#1089) - Navigation for the search bar : use '/' to enter search, up and down arrows to select a result, and enter to follow the selected link. (@EmileTrotignon, ocaml/odoc#1088) - Fix a big gap between the preamble and the content of a page (@EmileTrotignon, ocaml/odoc#1147) - Add a marshalled search index consumable by sherlodoc (@EmileTrotignon, @panglesd, ocaml/odoc#1084) - Allow referencing of polymorphic constructors in polymorphic variant type aliases (@panglesd, ocaml/odoc#1115) - Added a home icon in the breacrumbs (@panglesd, ocaml/odoc#1251) It can be disabled with a CLI option. - Add a frontmatter syntax for mld pages (@panglesd, ocaml/odoc#1187, ocaml/odoc#1193, ocaml/odoc#1243, ocaml/odoc#1246, ocaml/odoc#1251) Allows to specify the title of a page, the order of sub-pages and other behaviors in the sidebar. - Added `odoc-md` to process standalone Markdown pages (@jonludlam, ocaml/odoc#1234) - Added CSS selectors to style version and and nav links when they appear within page titles, as produced by odig (@katrinafyi, ocaml/odoc#1290) - Added support for (local) images in the latex backend (@Octachron, ocaml/odoc#1297) - The command line interface changed to support the new features. + Packages and libraries: `odoc link` must now be aware of packages and libraries with the `-L libname:path` and `-P pkgname:path` options. The module search path should still be passed with the `-I` option. The current package should be specified with `--current-package=pkgname`. + Hierarchy: `odoc compile` now outputs `.odoc` in the directory tree specified with `--output-dir=DIR` and the parent identifier must be specified with `--parent-id=PARENT`. The option `--source-parent-file` is removed. + Source code: Implementations are compiled with `compile-impl` instead of with `compile`. The options `--cmt=..` and `--source-name=..` are removed. Source code pages are generated with `html-generate-source`. + Assets: The commands `compile-asset`, `html-generate-asset` are added. The option `html-generate --asset` is removed. + Sidebar: The index is built using `compile-index`. The sidebar data is extracted from the index with `sidebar-generate` and passed to `html-generate --sidebar=..`. - The syntax for `@tag` is now delimited (@panglesd, ocaml/odoc#1239) A `@tag` can now be followed by a paragraph or other elements. - Updated colors for code fragments (@EmileTrotignon, ocaml/odoc#1023) - Fixed complexity of looking up `.odoc` files (@panglesd, ocaml/odoc#1075) - Normalize whitespaces in codespans (@gpetiot, ocaml/odoc#1085) A newline followed by any whitespaces is normalized as one space character. - Reduce size of `Odoc_html_frontend` when compiled to javascript (@EmileTrotignon, ocaml/odoc#1072) - Overhaul of module-type-of expansions and shadowing code (@jonludlam, ocaml/odoc#1081) - Output file paths and labels in the man and latex backends changed to avoid name clashes (@Julow, ocaml/odoc#1191) - Added a `header` field to the json output (@panglesd, ocaml/odoc#1314) - Changed indentation rules for code block and verbatim content (@panglesd, ocaml/odoc#1317) - odoc-parser: Store raw content in verbatim and code block, and expose a function to process it (@panglesd, ocaml/odoc#1325) - Fix variant constructors being hidden if they contain hidden types (@jonludlam, ocaml/odoc#1105) - Fix rare assertion failure due to optional parameters (@jonludlam, ocaml/odoc#1272, issue ocaml/odoc#1001) - Fix resolution of module synopses in {!modules} lists that require --open (@jonludlam, ocaml/odoc#1104} - Fix top comment not being taken from includes often enough (@panglesd, ocaml/odoc#1117) - Fixed 404 links from search results (@panglesd, ocaml/odoc#1108) - Fixed title content not being picked up across pages when rendering references (ocaml/odoc#1116, @panglesd) - Fix wrong links to standalone comments in search results (ocaml/odoc#1118, @panglesd) - Remove duplicated or unwanted comments with inline includes (@Julow, ocaml/odoc#1133) - Fix bug where source rendering would cause odoc to fail completely if it encounters invalid syntax (@jonludlam ocaml/odoc#1208) - Add missing parentheses in 'val (let*) : ...' (@Julow, ocaml/odoc#1268) - Fix syntax highlighting not working for very large files (@jonludlam, @Julow, ocaml/odoc#1277) - Fix backtrace on invalid input in compile-deps (@jonludlam, ocaml/odoc#1313) - Fix bug in our CSS hitting verbatim blocks in tags (@jonludlam, ocaml/odoc#1312) - Fix issue ocaml/odoc#610 where `odoc html-fragment` wasn't rendering headings correctly (@jonludlam, ocaml/odoc#1306)

panglesd added a commit to panglesd/odoc that referenced this pull request Jan 31, 2024

Add changelog entry for ocaml#1076

0aa165f

Signed-off-by: Paul-Elliot <peada@free.fr>

gpetiot approved these changes Feb 12, 2024

View reviewed changes

panglesd added a commit to panglesd/odoc that referenced this pull request Feb 19, 2024

Add changelog entry for ocaml#1076

46b4461

Signed-off-by: Paul-Elliot <peada@free.fr>

panglesd force-pushed the occurrences-in-index branch from 6205f97 to 69306f6 Compare February 19, 2024 08:33

gpetiot approved these changes Feb 19, 2024

View reviewed changes

doc/driver.mld Outdated Show resolved Hide resolved

panglesd added a commit to panglesd/odoc that referenced this pull request Jul 12, 2024

Add changelog entry for ocaml#1076

ada479d

Signed-off-by: Paul-Elliot <peada@free.fr>

panglesd force-pushed the occurrences-in-index branch from 822c54a to 1b71952 Compare July 12, 2024 07:45

panglesd added a commit to panglesd/odoc that referenced this pull request Jul 12, 2024

Add changelog entry for ocaml#1076

1ee07db

Signed-off-by: Paul-Elliot <peada@free.fr>

panglesd force-pushed the occurrences-in-index branch from 1b71952 to fd15b16 Compare July 12, 2024 07:47

Julow approved these changes Jul 15, 2024

View reviewed changes

panglesd force-pushed the occurrences-in-index branch from 6d8cc9a to 447e6ac Compare July 15, 2024 16:16

panglesd and others added 8 commits July 15, 2024 18:21

Move occurrences to its own folder

4b2203d

Signed-off-by: Paul-Elliot <peada@free.fr>

Add --occurrences arg to compile-index

9f026db

This adds the occurrence count to each element of the json index Signed-off-by: Paul-Elliot <peada@free.fr>

Add changelog entry for ocaml#1076

fbbc0a0

Signed-off-by: Paul-Elliot <peada@free.fr>

Occurrence count in index: only include non-empty fields

3da4c87

Signed-off-by: Paul-Elliot <peada@free.fr>

update documentation for json index

a269f79

Signed-off-by: Paul-Elliot <peada@free.fr>

Driver: include occurrences count in the search index

195496c

Signed-off-by: Paul-Elliot <peada@free.fr>

Fix typo in doc/driver.mld

3812d93

Co-authored-by: Guillaume "Liam" Petiot <guillaume@tarides.com>

Occurrences: don't expose sub in occurrence table

d4c8046

Signed-off-by: Paul-Elliot <peada@free.fr>

panglesd force-pushed the occurrences-in-index branch from 447e6ac to d4c8046 Compare July 15, 2024 16:21

Julow approved these changes Jul 16, 2024

View reviewed changes

panglesd merged commit 58a4cb3 into ocaml:master Jul 16, 2024
13 checks passed

panglesd added a commit that referenced this pull request Jul 16, 2024

Add changelog entry for #1076

c2a6d35

Signed-off-by: Paul-Elliot <peada@free.fr>

jonludlam mentioned this pull request Jan 23, 2025

[new release] odoc (5 packages) (3.0.0~beta1) ocaml/opam-repository#27309

Merged

jonludlam mentioned this pull request Mar 18, 2025

[new release] odoc (5 packages) (3.0.0) ocaml/opam-repository#27641

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add occurrence count in the json output for search engine #1076

Add occurrence count in the json output for search engine #1076

Uh oh!

panglesd commented Jan 31, 2024

Uh oh!

panglesd commented Jan 31, 2024

Uh oh!

gpetiot left a comment

Uh oh!

gpetiot Feb 12, 2024

Uh oh!

panglesd Feb 13, 2024

Uh oh!

gpetiot Feb 13, 2024

Uh oh!

panglesd Feb 13, 2024 •

edited

Loading

Uh oh!

Uh oh!

EmileTrotignon commented Jul 9, 2024

Uh oh!

panglesd commented Jul 9, 2024

Uh oh!

panglesd commented Jul 12, 2024 •

edited

Loading

Uh oh!

Julow left a comment

Uh oh!

Julow Jul 15, 2024

Uh oh!

panglesd Jul 15, 2024

Uh oh!

Julow left a comment

Uh oh!

Uh oh!

Uh oh!

Add occurrence count in the json output for search engine #1076

Add occurrence count in the json output for search engine #1076

Uh oh!

Conversation

panglesd commented Jan 31, 2024

Uh oh!

panglesd commented Jan 31, 2024

Uh oh!

gpetiot left a comment

Choose a reason for hiding this comment

Uh oh!

gpetiot Feb 12, 2024

Choose a reason for hiding this comment

Uh oh!

panglesd Feb 13, 2024

Choose a reason for hiding this comment

Uh oh!

gpetiot Feb 13, 2024

Choose a reason for hiding this comment

Uh oh!

panglesd Feb 13, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

EmileTrotignon commented Jul 9, 2024

Uh oh!

panglesd commented Jul 9, 2024

Uh oh!

panglesd commented Jul 12, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Julow left a comment

Choose a reason for hiding this comment

Uh oh!

Julow Jul 15, 2024

Choose a reason for hiding this comment

Uh oh!

panglesd Jul 15, 2024

Choose a reason for hiding this comment

Uh oh!

Julow left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

panglesd Feb 13, 2024 •

edited

Loading

panglesd commented Jul 12, 2024 •

edited

Loading