Task: Optimize and Document End-to-End Demo and Benchmark Experience

[NightOwl888]

Deliverables for this task

• Update demo documentation and code snippets to show the ".NET way" of running demos

  • src\Lucene.Net.Demo\overview.md

• Determine the best way to show "live" code on the documentation (link to it in the tag at the repo, do some sort of code to doc conversion, manual), and decide what to do with these that are "frozen" at a point in time while the code evolves

  • websites\apidocs\apiSpec\demo\Lucene_Net_Demo_Facet_AssociationsFacetsExample.md
  • websites\apidocs\apiSpec\demo\Lucene_Net_Demo_Facet_DistanceFacetsExample.md
  • websites\apidocs\apiSpec\demo\Lucene_Net_Demo_Facet_ExpressionAggregationFacetsExample.md
  • websites\apidocs\apiSpec\demo\Lucene_Net_Demo_Facet_MultiCategoryListsFacetsExample.md
  • websites\apidocs\apiSpec\demo\Lucene_Net_Demo_Facet_RangeFacetsExample.md
  • websites\apidocs\apiSpec\demo\Lucene_Net_Demo_Facet_SimpleFacetsExample.md
  • websites\apidocs\apiSpec\demo\Lucene_Net_Demo_Facet_SimpleSortedSetFacetsExample.md
  • src\Lucene.Net.Facet\overview.md (links to the "live" demo, not a tag that corresponds to the documentation version, so the link will eventually become invalid)

  NOTE: There is an issue where the LuceneDocsPlugins (tracked in Docs cleanup/improvements #300) do not replace the tokens in the docs that represent a git release tag that if fixed would go a long way toward solving this.

• Update the benchmark documentation and code snippets to show the ".NET way" of running benchmarks (depends on Investigate how we can load external assemblies when running Lucene.Net.Benchmarks on the command line #305)

  • src\Lucene.Net.Benchmark\ByTask\package.md
  • src\Lucene.Net.Benchmark\package.md
  • src\Lucene.Net.Benchmark\Quality\package.md

• Update the website at https://lucenenet.apache.org/ to make it clear how to run the source code (prerequisites, dependencies, etc)

• Update the website at https://lucenenet.apache.org/ to make the demos more visible

Goal

The goal is to make this information easy to find, easy to understand, and test it to make sure the examples work as documented.

---

Based on the Lucene demo document here:

https://github.com/apache/lucenenet/blob/Lucene.Net_4_8_0_beta00014/src/Lucene.Net.Demo/overview.md

it is clear that there are gaps in our knowledge of how these demos were meant to be delivered to Java developers and how we can provide an optimal experience in .NET.

What Lucene does

• The demo "package" is shipped as compiled code and hosted on Maven
• The documentation seems to assume that you know how to get from there to "adjust your classpath" to being able to run from the command line, without giving much detail about how
• It is not clear whether the command line is run against a compiled .jar file or if they are building from source

How we "ported" it

• We don't ship the Lucene.Net.Demo package on NuGet
• Since it is not possible to "run" a .NET Standard assembly, the demos were compiled into a .NET Core 3.1 console application named lucene-cli.
• We compile the code from the Lucene.Net.Demo project into lucene-cli and ship that on NuGet. The demos can be run directly from the console, provided the right input options are provided (which have all been documented in the lucene-cli docs).
• The code is also embedded as resources inside of lucene-cli so it can be exported on disk by running the --output-source-code option
• There are references in the documentation that link to the code files in the Lucene.Net.Demo project in the repository so the user has an alternate way to see the demo code than installing lucene-cli
• Most of the demo code has also been added to Markdown files (Docs - convert java snippets to c# #284 (comment)), but AFAIK these are not being referenced in other documentation

Problems

• The demos are not mentioned anywhere on the home page, you have to drill down pretty deep in the documentation to find them
• The source code alone is not enough for the user to take advantage of it by using dotnet run or putting it in Visual Studio (a .csproj file is also required, I believe, but there may also be other dependencies to investigate)
• The links to the demo source code are not functional yet because docs.ps1 build doesn't replace the tags in URLs
• Keeping the demos in Markdown means that they could go out of date when we upgrade to the next version of Lucene. Note that Microsoft has info about how to include code from a .cs file in the docs dynamically without copy and paste.
• The example code on the home page is missing details about what NuGet packages to reference, what namespaces to import, and a basic way to make the code run (Code snippets on https://lucenenet.apache.org/ doesn't works. #453)
• There is also an external demo project, LuceneNetDemo that is being kept up to date, but is not being referenced in documentation. Instructions on how to get it running are also pretty thin.

The bottom line is that it is not easy to locate the demos and if you do find them there is little information about how to run them.

Research

We need to research to find the best way to get these demos running on developer machines and update the documentation and website to make this experience go smoother. I suspect we can use the dotnet new command in combination with lucene-cli and dotnet run to do most of the heavy lifting. But it also might make sense to use the templating feature of dotnet new. We should investigate how other open source projects are providing runnable demos to end users to see if there are any ideas we can borrow.

It would also certainly be useful to run the Java demos to determine exactly how they are intended to be used.

For Lucene.Net.Benchmarks, we have similar issues because it is unclear how the app is intended to be used by end users. So, what is learned about the demos will also help us work out how to create the optimal user experience for Lucene.Net.Benchmarks.

• Like Lucene.Net.Demo, we compile Lucene.Net.Benchmarks into lucene-cli.
• Lucene.Net.Benchmarks is not exportable as source code.
• Unlike Lucene.Net.Demo, Lucene.Net.Benchmarks is compiled and distributed on NuGet.

It is clear that Lucene.Net.Benchmark must be referenced to extend types, but the extensions need to be loaded via Reflection on the command line to run (#305). It is not clear if there was a workflow in Java where you are expected to compile from source to do the benchmarks, although it may also have been possible for end users to make their own runner implementation.