Add all versions in version dropdown and filter out non-existing URLs #1367

reakaleek · 2025-06-13T16:27:00Z

This pull request introduces significant updates to the Elastic documentation ecosystem, including the addition of a Bloom filter-based legacy page checker, new tooling commands, and project restructuring. The changes aim to improve the efficiency of legacy page validation, enhance project modularity, and refine the user experience in documentation navigation.

Documentation Enhancements

Bloom Filter Implementation: Added a BloomFilter class in src/Elastic.Documentation.LegacyDocs/BloomFilter.cs to enable efficient legacy page existence checks. The filter supports persistence and optimal parameter calculation for false positive probability.
Legacy Page Checker: Introduced LegacyPageChecker in src/Elastic.Documentation.LegacyDocs/LegacyPageChecker.cs for checking legacy page existence using the Bloom filter. Includes methods for loading and generating the Bloom filter binary file.
Pages Provider: Added IPagesProvider and LocalPagesProvider in src/Elastic.Documentation.LegacyDocs/PagesProvider.cs to fetch pages from a local repository.
README Update: Documented usage of the legacy page checker and instructions for creating/updating the Bloom filter file in src/Elastic.Documentation.LegacyDocs/README.md.

Project Restructuring

New Projects: Added Elastic.Documentation.LegacyDocs and Elastic.Documentation.LegacyDocs.Tests projects to the solution, along with their configurations in docs-builder.sln. [1] [2]
Embedded Resource: Included legacy-pages.bloom.bin as an embedded resource in Elastic.Documentation.LegacyDocs.csproj.
Cross-Project References: Added project references to integrate Elastic.Documentation.LegacyDocs with other components, such as Elastic.Markdown.

Tooling Updates

Legacy Docs Commands: Added commands in src/tooling/docs-assembler/Cli/LegacyDocsCommands.cs for generating and validating Bloom filter binaries via CLI.
Repository Commands: Updated BuildAll in src/tooling/docs-assembler/Cli/RepositoryCommands.cs to use the new LegacyPageChecker for page validation.

UI Improvements

Table of Contents: Updated _TableOfContents.cshtml to display legacy page links conditionally based on their existence. Improved styling for better visual hierarchy. [1] [2]
Version Dropdown: Modified HtmlWriter.cs to dynamically populate version dropdown items based on legacy page mappings.

Miscellaneous Fixes

Package Updates: Corrected the casing of a package reference in Elastic.Documentation.Tooling.csproj.
Mapping Key Adjustment: Removed trailing slash from mapping keys in docs-assembler/AssembleSources.cs.

Copilot

Pull Request Overview

This PR adds comprehensive legacy version support by wiring in a bloom-filter–backed page existence checker and updating the UI to list all historical versions while disabling links for pages that no longer exist.

Introduces a new Elastic.Documentation.LegacyDocs project with LegacyPageChecker and CLI commands to generate/check against a bloom filter.
Enhances PageLegacyUrlMapper to mark legacy mappings as existing or not, and updates the Razor UI (_TableOfContents.cshtml) and HTML writer to render/disable links accordingly.
Updates version dropdown logic to include all versions (Skip(1).ToArray()), adjusts YAML mappings to use trailing slashes, and fixes project references.

Reviewed Changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
src/tooling/docs-assembler/Legacy/PageLegacyUrlMapper.cs	Added existence check via `LegacyPageChecker`, updated mapping API
src/tooling/docs-assembler/Cli/LegacyDocsCommands.cs	New CLI for bloom-filter creation and page-existence queries
src/tooling/docs-assembler/Cli/RepositoryCommands.cs	Wired up `PageLegacyUrlMapper` with `LegacyPageChecker`
src/tooling/docs-assembler/AssembleSources.cs	Adjusted YAML key handling (removed forced trailing slash)
src/Elastic.Markdown/Slices/Layout/_TableOfContents.cshtml	Conditional rendering/disabling of legacy version links
src/Elastic.Markdown/Slices/HtmlWriter.cs	Changed `LegacyPages` to include all prior versions
src/tooling/docs-assembler/legacy-url-mappings.yml	Added trailing slashes to keys for mapping consistency

Comments suppressed due to low confidence (3)

src/tooling/docs-assembler/Legacy/PageLegacyUrlMapper.cs:21

There are no existing unit tests covering the new filtering logic in MapLegacyUrl. Adding tests for scenarios where pages exist and do not exist will help prevent regressions.

public IReadOnlyCollection<LegacyPageMapping> MapLegacyUrl(IReadOnlyCollection<string>? mappedPages)

src/tooling/docs-assembler/Cli/LegacyDocsCommands.cs:17

[nitpick] The logger is typed as Program, but this class is LegacyDocsCommands. For clarity and consistency, consider using ILogger<LegacyDocsCommands>.

private readonly ILogger<Program> _log = logger.CreateLogger<Program>();

src/tooling/docs-assembler/AssembleSources.cs:141

Interpolating mappingValues (a list) directly will call its ToString() instead of listing items. Consider using string.Join(", ", mappingValues) for a clearer warning message.

reader.EmitWarning($"'{mappingKey}' is already mapped to '{mappingValues}'");

src/tooling/docs-assembler/Legacy/PageLegacyUrlMapper.cs

src/tooling/docs-assembler/Cli/RepositoryCommands.cs

…ropdown

src/tooling/docs-assembler/legacy-url-mappings.yml

Mpdreamz

LGTM, just a few nitpicks.

Mpdreamz · 2025-06-16T09:11:47Z

Directory.Packages.props

@@ -19,7 +19,9 @@
    <PackageVersion Include="FakeItEasy" Version="8.3.0" />
    <PackageVersion Include="Elastic.Ingest.Elasticsearch" Version="0.11.3" />
    <PackageVersion Include="Microsoft.OpenApi" Version="2.0.0-preview9" />
+    <PackageVersion Include="Octokit" Version="14.0.0" />


Looks unreferenced, can remove this dep?

Mpdreamz · 2025-06-16T09:13:30Z

Directory.Packages.props

    <PackageVersion Include="System.Text.Json" Version="9.0.5" />
+    <PackageVersion Include="TestableIO.System.IO.Abstractions.Wrappers" Version="22.0.14" />


Also not used (we use System.IO.Abstractions which is the same thing).

Mpdreamz · 2025-06-16T09:21:45Z

src/Elastic.Documentation.LegacyDocs/BloomFilter.cs

+		var itemList = new List<string>(items);
+		var filter = new BloomFilter(itemList.Count, falsePositiveProbability);
+		foreach (var item in itemList)
+		{


Inconsistent braces style

Mpdreamz · 2025-06-16T09:22:00Z

src/Elastic.Documentation.LegacyDocs/BloomFilter.cs

+		for (var i = 0; i < size; i++)
+		{
+			if ((bitArrayBytes[i / 8] & (1 << (i % 8))) != 0)
+			{


Inconsistent braces style

Mpdreamz · 2025-06-16T09:23:22Z

src/Elastic.Documentation.LegacyDocs/BloomFilter.cs

+	/// <returns>False if the item is definitely not in the set, True if it might be.</returns>
+	public bool Check(string item)
+	{
+		var itemBytes = Encoding.UTF8.GetBytes(item.Replace("/guide/en/", ""));


Lets do the item.Replace("/guide/en/", "") at the call site of Check so the bloom filter implementation stays agnostic.

782b42c

I did it a little different.

I'm now building the bloom filter binary with /guide/en instead.

Hence, the BloomFilter class itself only knows it's storing a list of strings.

In my first attempt, I thought the string size might influence the size of the bloom.bin file, which is why I removed the /guide/en prefix in the first place.

But obviously, it doesn't, since it's only storing the hashes.

Add all versions in version dropdown and filter out non-existing URLs

3c91aad

reakaleek requested a review from a team as a code owner June 13, 2025 16:27

Fix test setup

b8041ea

reakaleek added the enhancement label Jun 13, 2025

reakaleek added 2 commits June 13, 2025 19:12

Fix format

cc0db74

Fix test setup

1b423ed

reakaleek marked this pull request as draft June 13, 2025 17:20

reakaleek added 2 commits June 13, 2025 19:25

test

721d826

Refactor GetHash method

e36a293

reakaleek mentioned this pull request Jun 13, 2025

[Enhancement] Version dropdown #1327

Open

reakaleek added 2 commits June 13, 2025 23:22

Refactor naming

abef44d

Naming refactor, cleanup and docs

eaa39b2

reakaleek requested a review from Copilot June 13, 2025 22:10

Copilot AI reviewed Jun 13, 2025

View reviewed changes

src/tooling/docs-assembler/Legacy/PageLegacyUrlMapper.cs Show resolved Hide resolved

src/tooling/docs-assembler/Cli/RepositoryCommands.cs Outdated Show resolved Hide resolved

reakaleek added 2 commits June 14, 2025 00:16

Use existing filesystem

d011275

Merge branch 'main' into feature/add-only-existing-pages-to-version-d…

b9bfc74

…ropdown

reakaleek commented Jun 13, 2025

View reviewed changes

src/tooling/docs-assembler/legacy-url-mappings.yml Outdated Show resolved Hide resolved

Update src/tooling/docs-assembler/legacy-url-mappings.yml

b6b7aa0

reakaleek marked this pull request as ready for review June 13, 2025 22:25

bmorelli25 mentioned this pull request Jun 15, 2025

Implement dynamic applies_to tags #1369

Open

Mpdreamz approved these changes Jun 16, 2025

View reviewed changes

reakaleek added 3 commits June 16, 2025 11:36

cleanup: Remove unused dependencies

70b3862

format: brace styles

f38002c

Refactor: Make BloomFilter more generic

782b42c

reakaleek requested a review from Mpdreamz June 16, 2025 09:46

reakaleek merged commit 9ecb990 into main Jun 16, 2025
15 checks passed

reakaleek deleted the feature/add-only-existing-pages-to-version-dropdown branch June 16, 2025 09:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add all versions in version dropdown and filter out non-existing URLs #1367

Add all versions in version dropdown and filter out non-existing URLs #1367

reakaleek commented Jun 13, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Mpdreamz left a comment

Uh oh!

Mpdreamz Jun 16, 2025

Uh oh!

reakaleek Jun 16, 2025

Uh oh!

Mpdreamz Jun 16, 2025

Uh oh!

reakaleek Jun 16, 2025

Uh oh!

Mpdreamz Jun 16, 2025

Uh oh!

reakaleek Jun 16, 2025

Uh oh!

Mpdreamz Jun 16, 2025

Uh oh!

reakaleek Jun 16, 2025

Uh oh!

Mpdreamz Jun 16, 2025

Uh oh!

reakaleek Jun 16, 2025

Uh oh!

reakaleek Jun 16, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

		<PackageVersion Include="System.Text.Json" Version="9.0.5" />
		<PackageVersion Include="TestableIO.System.IO.Abstractions.Wrappers" Version="22.0.14" />

Add all versions in version dropdown and filter out non-existing URLs #1367

Add all versions in version dropdown and filter out non-existing URLs #1367

Conversation

reakaleek commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation Enhancements

Project Restructuring

Tooling Updates

UI Improvements

Miscellaneous Fixes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Mpdreamz left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

reakaleek Jun 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

reakaleek commented Jun 13, 2025 •

edited

Loading

reakaleek Jun 16, 2025 •

edited

Loading