SenseNet Index Rebuilder Console Application #2221
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Implements standalone SenseNet index rebuilder console application - Uses IsOuterSearchEngineEnabled special working mode to prevent processing old indexing activities - Clears legacy indexing activities from database before rebuild - Successfully rebuilds index from scratch using ClearAndPopulateAllAsync - Provides progress monitoring and error handling - Tested successfully: indexed 62,685 nodes in ~28 minutes - No core SenseNet modifications required - uses existing infrastructure only Features: - Service registration using AddSenseNet pattern from integration tests - Proper repository startup with indexing disabled during initialization - Automatic cleanup of old IndexingActivities table entries - Clean index rebuild from current database state - Comprehensive logging and progress tracking This approach solves the issue of old indexing activities being processed during normal repository startup, enabling truly clean index rebuilds.
…ive progress tracking Features: - Add dual ETA display showing both average and worst-case time estimates - Implement IndexingProgressTracker class with advanced progress monitoring - Add comprehensive CLI argument parsing (--clear-activities, --help) - Add Serilog integration for dual console+file logging - Implement two rebuild approaches: 1. Clean rebuild without clearing activities (default) 2. Complete clean rebuild with activities table clearing (--clear-activities) - Add phantom activities issue resolution with TRUNCATE + DBCC CHECKIDENT - Add index directory clearing to remove cached LastActivityId - Enhanced error handling and user feedback - Professional progress display with total node counts and completion times Technical improvements: - Real-time progress updates every 100 nodes or 5 seconds - Worst-case scenario tracking using maximum time per node - Convergent ETA estimation as process stabilizes - Structured logging for troubleshooting and monitoring - Complete SQL identity seed management - Comprehensive help documentation
VargaJoe
changed the title
VargaJoe
changed the title
- Add System.Collections.Generic for Queue<double> - Add System.Linq for Average() extension method - Complete conservative ETA estimation implementation
…build paths - Extract common functionality into helper methods: - ClearIndexingActivitiesAsync() for database cleanup - ClearIndexDirectoryAsync() for file system cleanup - PerformIndexRebuildAsync() for shared rebuild logic - Consolidate duplicate progress tracking, error handling, and populator setup - Reduce code duplication from ~150 lines to ~20 lines of shared logic - Maintain identical functionality for both --clear-activities and default modes - Fix Task ambiguity by using fully qualified System.Threading.Tasks.Task
- Refactored Program.cs to eliminate 143 lines of duplicate code - Extracted helper methods: ClearIndexingActivitiesAsync, ClearIndexDirectoryAsync, PerformIndexRebuildAsync - Fixed --clear-activities mode by letting ClearAndPopulateAllAsync handle indexing engine startup internally - Resolved Lucene29 compatibility issue with explicit indexing engine startup - Updated comprehensive documentation with latest behavioral findings - Confirmed identity reset behavior: --clear-activities resets to ID 1, default mode preserves counter - Both modes now work correctly with streamlined codebase
marosvolgyiz
approved these changes
Sep 22, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
SenseNet Index Rebuilder Console Application
Overview
Adds a new standalone console application for rebuilding search indexes with real-time progress tracking and professional logging.
What's Added
New Console Tool:
src/Tools/SnIndexRebuilder/Command Line Interface:
dotnet run- Default clean rebuild (recommended)dotnet run --clear-activities- Complete reset with IndexingActivities pre-clearingdotnet run --help- Show help documentationFiles Added:
src/Tools/SnIndexRebuilder/Program.cs(413 lines) - Main applicationsrc/Tools/SnIndexRebuilder/SnIndexRebuilder.csproj- Project filesrc/Tools/SnIndexRebuilder/appsettings.json- Configurationdocs/index-rebuilder-console.md(308 lines) - DocumentationBenefits
Provides a professional alternative to existing index rebuilding methods with enhanced operational control.