Merge 4.14.0 into 4.14.1 #7840
Open
Merge 4.14.0 into 4.14.1 #7840
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add process.user.name field * Add changelog * Fix Process user name suggested filter
* Relocate Browser extensions and rename to Web browsers * Add changelog * Rename Extension to Package
chore: change tag references
* Rename Web browsers tab to Browser extensions * Add changelog * Change suggested filter Name to Browser name
) * Remove Top 5 agents visualization and update Groups label in IT Hygiene - Remove Top 5 agents visualization from IT Hygiene > Identity > Groups - Change 'Top 5 groups' label to 'Groups' for better clarity Fixes #7791 * Fix fieldCustomLabel: change 'Group name' to 'Groups' - Update fieldCustomLabel from 'Group name' to 'Groups' to match the title - Completes the label standardization requested in issue #7791 * Update label for groups visualization in IT Hygiene dashboard - Change label from 'Groups' to 'Top 5 groups' for improved clarity in the overview users groups tab. - This update aligns with the ongoing label standardization efforts. * Update CHANGELOG.md with PR link for IT Hygiene Groups changes * Fix CHANGELOG.md: Move PR #7792 to correct section * Refactor: Use getVisStateMetricUniqueCountByField for unique groups metric - Replace manual getVisStateUniqueGroupsMetric implementation with existing getVisStateMetricUniqueCountByField utility * Cleanup: Remove unnecessary blank line in dashboard.ts
* Removes columns process.user.name and process.executable * Change vis * Update changelog * Remove process.user.name filter
* Remove top 5 user auth failures count visualization * Add changelog * Add top 5 users visualization --------- Co-authored-by: Gonzalo Arancibia <72573241+gonzaarancibia@users.noreply.github.com>
* fix(endpoints): deprecate /agents/outdated and use api version to verify agents outdated Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * refactor(endpoints): delete getOutdatedAgentsService services Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * docs(changelog): add entry Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * docs(changelog): modify entry Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * fix(endpoints): remove the 'v' prefix if it exists Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * refactor(endpoints): use semver library to compare number versions Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * refactor: add getWazuhAPIVersion service and refactor code Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * refactor(endpoints-summary): delete useless code about outdated agents Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * refactor: run prettier Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> --------- Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com>
* Add Wazuh states index types and known fields for states patterns - Introduced new constants for various Wazuh states index types in `constants.ts`. - Updated `saved-objects.js` to include handling for the new states index types. - Added detailed known fields for Wazuh states, including vulnerabilities, inventory systems, and FIM, in `known-fields.js`. * Remove Elasticsearch references from known fields comments - Replace 'Basic Elasticsearch fields' with 'Basic index fields' - Addresses review comment about removing Elasticsearch references Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Fix states patterns to avoid fallback to alerts KnownFields - Remove fallback to KnownFields (alerts fields) for states patterns - Throw specific error when no known fields are found for states pattern - Prevents mixing alerts fields with states index patterns - Addresses review comment about inappropriate fallback logic Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Refactor getIndexTypeFromPattern to use clean object mapping - Replace multiple if/else statements with PATTERN_TO_INDEX_TYPE_MAP object - Use Object.keys().find() for cleaner pattern matching - Improve code readability and maintainability - Addresses review feedback about code elegance Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Restore switch statement and apply code formatting improvements - Restore original switch statement logic for states index types - Apply code formatting improvements to pattern mapping object - Ensure proper line breaks and indentation for better readability - Complete solution for passing correct indexType to known fields selection Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Add automated known fields generation system Features: - Create generate-known-fields.js script to automatically fetch and convert Wazuh index templates to known fields format - Support vulnerabilities and alerts templates from official Wazuh repos - Generate JSON files with properly mapped field types and properties - Add known-fields-loader.js for dynamic loading of generated fields - Update saved-objects.js to use generated fields instead of hardcoded ones - Add comprehensive documentation in scripts/README.md Benefits: - Ensures known fields stay synchronized with official templates - Eliminates manual maintenance of field definitions - Reduces risk of field mismatches between server and dashboard - Provides foundation for supporting all states index patterns Generated: - plugins/main/public/utils/known-fields/alerts.json (614 fields) - plugins/main/public/utils/known-fields/states-vulnerabilities.json (52 fields) This addresses the automation requirement mentioned in review feedback to keep known fields synchronized with upstream template definitions. Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Migrate to use generated JSON fields as primary source - Update known-fields-loader.js to export generated alerts fields as KnownFields - Remove dependency on manually maintained known-fields.js - Update all imports in saved-objects.js and test files to use loader - Apply code formatting improvements to generate-known-fields.js - Now all known fields come from automatically generated JSON files This completes the migration from hardcoded fields to generated fields, ensuring all field definitions stay synchronized with official templates. Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Complete implementation of automated known fields for all states patterns - Add all inventory and FIM templates to generation script (14 new templates) - Generate JSON files for all states patterns: vulnerabilities, FIM, and inventory types - Update known-fields-loader with proper imports for all generated fields - Remove fallback dependencies and use specific fields for each pattern type Generated files include: - states-fim-files.json (24 fields) - states-fim-registries.json (30 fields) - states-inventory-system.json (29 fields) - states-inventory-hardware.json (21 fields) - states-inventory-networks.json (20 fields) - states-inventory-packages.json (26 fields) - states-inventory-ports.json (24 fields) - states-inventory-processes.json (23 fields) - states-inventory-protocols.json (18 fields) - states-inventory-users.json (44 fields) - states-inventory-groups.json (20 fields) - states-inventory-services.json (45 fields) - states-inventory-interfaces.json (27 fields) - states-inventory-hotfixes.json (14 fields) - states-inventory-browser-extensions.json (35 fields) This resolves the issue by ensuring all Wazuh states index patterns have proper known fields definitions, preventing visualization errors when indices don't exist yet. Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Complete TypeScript parsing support and generate all remaining templates - Fix TypeScript template parsing with improved regex and safer evaluation - Generate monitoring.json (12 fields) from monitoring-template.ts - Generate statistics.json (75 fields) from statistics-template.ts - Update known-fields-loader to import monitoring and statistics fields - Now all Wazuh index patterns have automatically generated known fields Total coverage achieved: ✅ Alerts: 614 fields ✅ Monitoring: 12 fields ✅ Statistics: 75 fields ✅ States Vulnerabilities: 52 fields ✅ States FIM: 2 types (24-30 fields each) ✅ States Inventory: 13 types (14-45 fields each) This completes the full automation of known fields generation, ensuring 100% synchronization with official Wazuh templates and resolving all visualization errors when indices don't exist yet. Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Add comprehensive tests for known-fields-loader functionality - Create known-fields-loader.test.tsx with full test coverage - Test KnownFields exports and GeneratedKnownFields structure - Validate pattern extraction and field mapping functions - Test states-specific field validation for all types - Ensure proper field structure and metadata for all templates - Verify vulnerabilities package/vulnerability fields - Validate inventory system host/OS fields - Test monitoring timestamp/status fields - Verify statistics analysisd/remoted fields Tests cover all new functionality introduced by the automated known fields generation system. Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Update CHANGELOG.md with automated known fields feature - Add entry for automated known fields generation system - Reference PR #7777 in changelog - Document feature in Wazuh v4.14.0 Added section This completes the documentation requirements for the comprehensive known fields automation implementation. Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Fix script path resolution and add npm script for known fields generation - Fix generate-known-fields.js to use absolute paths relative to script location - Prevent creation of duplicate directory structures when run from different locations - Add 'generate:known-fields' script to plugins/main/package.json - Now script works correctly whether executed from project root or plugins/main directory This ensures the known fields generation script can be run reliably from any location within the project structure. Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Use neutral terminology instead of vendor-specific references - Replace OpenSearch/Elasticsearch references with neutral 'index' terminology - Update mapOpenSearchType → mapIndexFieldType in generate-known-fields.js - Update comments to use 'index fields' instead of vendor-specific terms - Regenerate JSON files with updated function references - Maintain only necessary URL references to official repository paths This ensures vendor-neutral terminology while maintaining functionality. Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * Apply Prettier formatting to generated JSON files - Format all generated known fields JSON files according to project standards - Ensures consistent formatting for generated content - Only applies to files we created, not the entire project Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> * fix: prettier formatting for known-fields-loader test file - Apply prettier formatting to known-fields-loader.test.tsx - Ensure code style consistency across the project - Fix CI/CD formatting warnings * Remove known-fields.js file containing predefined field definitions. This change eliminates the reliance on hardcoded fields, streamlining the known fields management process. All known fields are now generated dynamically, ensuring synchronization with official templates. * Integrate known fields with sample data generation for IT Hygiene Groups - Add known-fields-reader utility to load and process known fields JSON files - Update states-inventory-groups generator to use known fields as source of truth - Ensure perfect synchronization between field definitions and generated sample data - Maintain backward compatibility with fallback to legacy generation - Eliminate code duplication by using shared data arrays - Reduce code size by 45% while maintaining full functionality This resolves field misalignment issues and ensures sample data buttons generate consistent data based on the official field definitions. All 7 group fields are perfectly aligned with known fields JSON structure. * Make generate-known-fields script use dynamic version from package.json - Load version dynamically from plugins/main/package.json instead of hardcoding 4.14.0 - Add helper functions wazuhUrl() and dashboardPluginsUrl() for clean URL generation - Update all template URLs to use dynamic versioning - Ensure script remains sustainable across different Wazuh versions - Maintain backward compatibility and same functionality This resolves the issue where the script was hardcoded to version 4.14.0, making it unsustainable as branches update to newer versions. * Apply code formatting to generate-known-fields script - Improve readability with proper line breaks for long URLs - Format function calls with consistent indentation - Apply prettier-style formatting for better maintainability * Complete dynamic versioning for all template URLs in generate-known-fields script * Update scripts/README.md Co-authored-by: Antonio <34042064+Desvelao@users.noreply.github.com> * refactor: pass indexType as parameter to existsIndices function instead of inferring the index type internally * Apply code formatting to existsIndices calls - Format multi-line function calls for better readability - Apply consistent indentation style * Remove fallback to ALERTS in getIndexTypeFromPattern and specify index types explicitly * Use index type to get known fields and remove GeneratedKnownFields duplication * Move known fields JSON to common directory to fix server-client import issue - Move all known fields JSON files from public/utils/known-fields to common/known-fields - Update known-fields-reader.js to read from common instead of public - Update known-fields-loader.js and tests to import from common directory - Update generate-known-fields.js script to output to common directory - Fix production mode issue where server cannot import from public folder * Fix nested field handling in generate-known-fields script - Add proper handling for nested type fields with properties - Create field entry for the nested type itself before recursing into properties - Add 'nested' type to mapIndexFieldType function - Update isAggregatable and shouldReadFromDocValues to handle nested type correctly - Regenerate alerts.json with 7 additional nested fields (614 -> 621 fields) This fixes the issue where nested fields like configurationManagerClientEnabledFeatures were not being properly processed - now both the nested field and its sub-properties are correctly included in the known fields output. * Apply code formatting to generate-known-fields script * Simplify template extraction by converting TypeScript templates to JSON - Create monitoring-template.json from monitoring-template.ts - Create statistics-template.json from statistics-template.ts - Remove TypeScript parsing logic from generate-known-fields.js - Simplify fetchTemplate() to only handle JSON - Update script configuration to use .json files instead of .ts - Update README.md to reflect changes and improvements This addresses code review feedback to simplify template extraction by using JSON files instead of parsing TypeScript, making the script simpler and more maintainable. * Apply code formatting to statistics-template.json and README.md * Update README to reflect completed inventory states support - Remove outdated TODO about adding inventory states support (already implemented) - Update template sources section with complete list of supported templates - Clarify that all inventory states are now supported (13 types) - Remove references to TypeScript parsing issues (resolved) * Apply code formatting to monitoring-template.ts (add trailing commas) * Move known fields back to public and fix server path resolution - Move known-fields JSON files back from common/ to public/utils/known-fields/ - Update known-fields-loader.js to import from ./known-fields/ (public) - Update known-fields-reader.js (server) to dynamically resolve path - Add fallback path resolution for production vs development environments - Remove common/known-fields/ directory to avoid duplication - Update generate-known-fields.js to output to public/utils/known-fields/ The server now uses fs.existsSync() to check development path first, then falls back to production path, ensuring it works in both environments without duplicating files or hardcoding paths. This resolves webpack build errors where imports from common/ were not resolved correctly, while still addressing the code review concern about server-client imports by using dynamic path resolution. * Apply code formatting to generate-known-fields.js output paths * Exclude monitoring from auto-generation and use manual fields - Remove monitoring from generate-known-fields.js script configuration - Add TODO explaining why monitoring is not auto-generated - Delete auto-generated monitoring.json file - Update known-fields-loader.js to import from monitoring-fields.ts - Update test to use FieldsMonitoring from monitoring-fields.ts - Update README to document monitoring exclusion with explanation The monitoring template only contains basic index-level fields, but the actual monitoring index includes additional agent fields (dateAdd, lastKeepAlive, etc.) that are inserted dynamically by Wazuh server. These are maintained manually in monitoring-fields.ts to ensure all required fields are available. * Convert statistics-template from TypeScript to JSON - Add index_patterns property to statistics-template.json for dynamic assignment - Update import in scheduler-handler.ts to use JSON format - Remove statistics-template.ts as it's now replaced by JSON This aligns with the script-generated templates approach and fixes the 'Cannot set properties of undefined' error in the cron scheduler by ensuring the index_patterns property exists before assignment. * Fix extractPatternType regex to correctly handle patterns with multiple hyphens The previous regex pattern.match(/wazuh-states-(.+?)-\*/) used lazy matching which stopped at the first hyphen, incorrectly extracting: - 'wazuh-states-fim-files-*' → 'fim' (wrong, should be 'fim-files') - 'wazuh-states-inventory-system-*' → 'inventory' (wrong, should be 'inventory-system') This caused getKnownFieldsForStatesPattern() to return null for most states patterns with hyphens in their names. Solution: Replace regex with explicit string extraction: 1. Check if pattern starts with 'wazuh-states-' and ends with '-*' 2. Extract the substring between these markers 3. Verify extracted type exists in KnownFieldsStatesGenerated This approach is more robust and won't break if pattern prefixes change in the future, as it validates against the actual known pattern types. Added test case to verify unknown pattern types are correctly rejected. * Update changelog entry to be more user-oriented Changed from technical 'Automated known fields generation system' to user-friendly 'Add known fields for Wazuh states index patterns creation' which better describes the feature benefit for end users. * Apply code formatting (prettier) to modified files - Format check-index.ts - Format all known-fields JSON files - Format monitoring-template.json - Format known-fields-reader.js * Fix tests to use correct index type keys for getKnownFieldsByIndexType The function expects keys with 'wazuh-index-type-' prefix, not the raw constant values. Updated tests to use the correct string literals instead of the WAZUH_INDEX_TYPE_* constants which have different values. * Remove deprecated getKnownFieldsForStatesPattern function and its related tests. This change streamlines the known fields loader by eliminating unused code, improving maintainability. * Unify known fields mapping and fix getKnownFieldsByIndexType Created KnownFieldsByIndexType as a single unified mapping that uses the actual WAZUH_INDEX_TYPE_* constant values ('alerts', 'monitoring', etc.) as keys instead of prefixed strings. Benefits: - Eliminates inconsistency between constant values and map keys - getKnownFieldsByIndexType() now works correctly with constants - Single source of truth for all index types (main + states) - Kept KnownFieldsStatesGenerated for backwards compatibility - Tests now use constants directly instead of hardcoded strings This fixes the issue where getKnownFieldsByIndexType() was always returning null because it was looking for 'wazuh-index-type-*' keys that didn't exist in the map. * Use constants as keys in KnownFieldsByIndexType object Instead of hardcoded strings, now using computed property names with WAZUH_INDEX_TYPE_* constants for better maintainability and type safety. Example: Before: 'alerts': alertsFields After: [WAZUH_INDEX_TYPE_ALERTS]: alertsFields This ensures the keys are always in sync with the constant definitions and makes refactoring safer. * Refactor known fields loader by removing deprecated states pattern functions This commit eliminates the `KnownFieldsStatesGenerated` mapping and related functions, streamlining the known fields loader for better maintainability. The `getKnownFieldsForPattern` and `extractPatternType` functions have been removed as they are no longer necessary. The `KnownFieldsByIndexType` now serves as the single source of truth for all index types, enhancing clarity and reducing redundancy in the codebase. * Remove deprecated monitoring-template.json file to streamline integration files. This change eliminates unused configuration, enhancing maintainability and reducing clutter in the project. * Refactor sample data generation for states inventory groups This commit unifies the group data structure by consolidating descriptions, names, and user groups into a single format. The deprecated known fields reader has been removed, streamlining the sample data generation process. This change enhances maintainability and ensures consistency in the generated data. * Enhance index pattern validation by introducing indexType parameter This commit updates the ensureIndexPatternIsCreated function to accept an indexType parameter, allowing for more precise validation of index patterns. Additionally, various data source components have been modified to pass the appropriate indexType constants, improving maintainability and clarity in the codebase. Unused imports and deprecated functions have been removed to streamline the implementation. * Update CHANGELOG.md to reflect enhanced visualizations field validations for Wazuh states index patterns and support * Refactor pattern data source to ensure fields are updated correctly. This change moves the field retrieval and update logic into a try block, improving error handling and maintainability of the code. * Update pattern data source tests to improve error handling and assertions. The test for pattern not found now uses async/await syntax for clarity, and the test for index pattern field retrieval has been modified to ensure it does not throw errors when fields retrieval fails, reflecting the updated behavior of the select() method. * Add states inventory index type and associated fields This commit introduces the WAZUH_INDEX_TYPE_STATES_INVENTORY constant and integrates it into various components, including the validation logic and known fields loader. A new JSON file for states inventory fields is created, consolidating relevant data for improved management and accessibility. Additionally, a script is added to generate combined inventory fields, enhancing the overall structure and maintainability of the codebase. * Refactor combined inventory fields generation for improved readability This commit enhances the clarity of the combined inventory fields generation by formatting the function call to `generateCombinedInventoryFields` across multiple lines. This change improves code readability and maintainability without altering the functionality. --------- Signed-off-by: Felipe Gonzalez <felipegonzalezmv@hotmail.com> Co-authored-by: Antonio <34042064+Desvelao@users.noreply.github.com> Co-authored-by: Ian Yenien Serrano <63758389+yenienserrano@users.noreply.github.com>
* fix(scan-settings): add missing providers Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * fix(imposter-wmodules): add missing providers to syscollector Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * fix(scan-settings): restore old provider Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> * docs(changelog): add entry Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com> --------- Signed-off-by: gonzaarancibia <gonzaloarancibia48@gmail.com>
chore: change tag references
5 tasks
|
|
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Merge 4.14.0 into 4.14.1
Caution
Merge PR strategy: Create a merge commit