Skip to content

Conversation

@UseTheFork
Copy link
Collaborator

Overview

This PR introduces multiple features, bug fixes, refactoring, and testing improvements as described in the changelog.

Notable Changes

  • 🚀 Features:
    • Add hybrid Elasticsearch and SQLite models.
    • Implement dynamic schema operations.
    • Enhance query builder with pagination and nested query support.
  • 🐛 Bug Fixes:
    • Correct Elasticsearch primary key usage.
    • Improve error handling in QueryException.
  • 🚜 Refactors:
    • Modularize schema classes and streamline aggregation handling.
    • Remove legacy components and unused traits.
  • 🧪 Testing:
    • Update query and model tests for new functionality.

Breaking Changes

  • Changes to primary key handling in Elasticsearch.
  • Updated schema management for enhanced functionality.

UseTheFork and others added 30 commits October 26, 2024 11:40
- Added dependency on `Illuminate\Support\Str` for UUID generation.
- Auto-generate UUIDs when entity ID is not provided.
- Simplified ID assignment process for better consistency with `saveWithOutRefresh`.

UUIDs: because who can keep track of all those entities? 🧑‍💻
- Replaced occurrences of 'product->_id' with 'product->id' in tests.
- Added new methods to handle MorphToMany relationships in QueriesRelationships.php.
- Refactored Model.php to streamline Elasticsearch model identification and handling.
- Revised HybridRelations.php to properly handle relationships with Elasticsearch models.

Tidying the namespace and cleaning up as if it's spring already! 🌷
- Replaced waitForPendingTasks method with refreshIndex in Query Builder.
- Updated related test cases to use refreshIndex.
- Added handling for ClientResponseException and ServerResponseException.

Fresh index, who dis? 🧹🎉
- Added HasFactory trait and newFactory method to Soft model.
- Introduced SoftFactory with comprehensive field definitions.
- Refreshed product index in chunk and delete tests for better consistency.
- Fixed DSL query structuring in ElasticsearchSpecificTest.

Indexes, refreshing more often than your morning coffee! ☕️
…tests

- Added `getDelete` method in `WaitFor` enum for deletion flag.
- Utilized `getDelete` in `DSL/Bridge` for dynamic refresh parameter.
- Removed redundant `Product::refreshIndex` calls in deletion tests.
- Eliminated unnecessary `sleep(3)` in truncate test.

Who knew deletions could get this smart? 🚀
…d relationships

- Introduced new Elasticsearch models: Group, Role, Item, Skill, Experience, Label, and Book.
- Added SQLite models: SqlUser, SqlRole, and SqlBook with schema execution methods.
- Updated User model to implement hybrid relationships and additional attributes.
- Created EloquentBuilder helper class for custom query relationships.
- Modified test setup for SQLite in-memory database configuration.

Hybrid models: bridging gaps, building apps (and dreams)! 🤖🧩
- Updated belongsToMany method to properly handle `$table` argument.
- Enhanced `BelongsToMany` class with pivot column qualification and optimized data retrieval.
- Added `executeSchema` method to handle schema setup for `Skill`, `Label`, and `Experience` models.
- Introduced comprehensive tests for hybrid relations in `HybridRelationsTest.php`.

Hybrid relations just got a glow-up! 🌟
- Added setup schema execution in model tests.
- Simplified and enhanced assertions for model properties and state.
- Introduced modifications in schema execution within the User model.
- Implemented upsert, manual ID handling, date validation, and data scope tests.
- Ensured comprehensive CRUD operation tests on User and Item models.
- Incorporated soft delete and restore validation for Soft model.

Testing 🧪: Making sure the models behave better than my unwatered plant! 🌵
… operations

- Added `executeSchema` method to `HiddenAnimal` model for managing Elasticsearch index.
- Updated unit tests to utilize the new `executeSchema` method.
- Refined property hiding test assertions for clarity.

Executed schemas are like fine wine, best enjoyed dynamically. 🍷
- Added `executeSchema` method to several models (`HiddenAnimal`, `Address`, `Client`, `Photo`, `Role`, `Item`) for dynamic schema operations.
- Replaced `BelongsTo` relationships with `BelongsToMany` in `Group` and `Client`.
- Improved methods in `BelongsToMany` and `HybridRelations` for better handling of relationship constraints and syncing.
- Introduced `Address` model with schema definition.

Schemas now work harder so you can work smarter! 💡
… query handling

- Replaced WaitFor `getDelete` method with `getOperation` method for flexibility.
- Introduced `ArrayStore` contract and repository for flexible option handling.
- Added `HasOptions` trait to manage options in relevant classes.
- Created comprehensive `QueryTest` for testing user-related operations.
- Refactored queries to use `ArrayStore` for options management.
- Added `Birthday` model schema management to the project.

Operation Overload: Now with more options than a buffet! 🍽️
…andling

- Added `BulkInsertQueryException` for detailed bulk insert error reporting.
- Renamed and relocated classes for better organization (e.g., `Results` to `Result`, `QueryBuilder` to `Traits/Bridge/QueryBuilder`).
- Introduced new traits such as `Aggregates` and `SanitizeResponses` to modularize functionality.
- Refactored methods for building and parsing queries to maintain consistency.
- Enhanced meta-data handling in query responses.
- Added comprehensive error handling to better manage exceptions.

Refactor tennis: because even code needs some exercise 🏋️‍♂️
- Added try-catch block for client get requests in Bridgeable trait.
- Changed return type in Model to Result for consistency.
- Implemented delete method for deleting records.
- Enhanced `getOption` method to accept a default value.
- Improved delete method in Builder and added pre-query callbacks.
- Refined truncate method in Builder to invoke delete.
- Introduced compileOptions method in Grammar to manage options.
- Added new Helpers class for common utility functions.

Crushed more bugs than a colony of ants at a picnic. 🐜💥
- Renamed `IndexBlueprint` to `Blueprint` for clarity.
- Moved and enhanced methods in `Grammar` for better structure and readability.
- Added `Aggregates` and `AggregatesGrammar` traits to support query aggregates.
- Updated various models to use the new `Blueprint` naming.

The schema just got a makeover, runway ready! 💃🕺
…nctionality

- Removed Aggregates and AggregatesGrammar traits.
- Integrated aggregation methods directly into Builder class.
- Added cursor and scroll methods in Connection class for efficient data fetching.
- Enhanced tests to cover new aggregation logic and cursor features.
- Adjusted grammar to compile various aggregation types.
- Refactor `BulkInsertQueryException` to remove commented-out code for streamlined maintenance.
- Update `QueryException` constructor to utilize a new `formatMessage` method.
- Add `formatMessage` and helper methods to construct detailed error messages for specific exception types.
- Leverage `Illuminate\Support\Collection` for assembling error messages efficiently.

Now your errors come with a touch of elegance! 🎩
- Changed BulkInsertQueryException constructor to accept Elasticsearch instance.
- Updated parent::__construct invocation to process queryResult as an array.
- Added detailed error formatting for different Elasticsearch exceptions.
- Introduced individual methods to handle 'MissingParameterException', 'ClientResponseException', and other specific errors.
- Improved error messaging for 'search_phase_execution_exception', 'parse_exception', and 'script_exception'.

Making errors cry less and developers smile more! 😎
- Added push method to append values to an array field with optional uniqueness.
- Added pull method to remove specific values from an array field.
- Improved increment and decrement methods to handle null values.
- Added new date query operators and updated tests.

Because why not add some push and pull drama to arrays? 🎭
…ed methods

- Created a reusable `buildCrementEach` method for increment and decrement operations.
- Updated `dynamicFilter` method to correct the case transformation.
- Removed `AnalyzerBlueprint.php` as unused.
- Eliminated `aggregation` and `getAggregationResults` methods from `QueryBuilder`.
- Ensured raw response conversion in `Processor`.
- Added additional test cases for coverage.

Just flexin' those refactor muscles! 💪
…ition enhancements

- Added `PropertyDefinition` class to better define Elasticsearch properties.
- Updated `Grammar` to handle new `NullValue` modifier.
- Overhauled `Blueprint` class to support multiple new column types and methods.
- Enhanced error handling in `Connection` for `createIndex` method.
- Introduced new comprehensive tests for schema functionality.

Coding wizardry at its finest! 🧙✨
- Added index settings and metadata tests for Elasticsearch.
- Introduced support for additional data types in schema tests.
- Fixed alias creation logic in `Grammar.php`.
- Updated `Blueprint.php` to allow dynamic index settings.
…trait

- Deleted `Bridge` class from `src/DSL/Bridge.php`.
- Removed `HasCollection` trait usage from `ElasticsearchModel`.
- Added `Searchable` trait usage to `ElasticsearchModel`.
- Removed `setTable` method from `ElasticsearchModel`.
…oss models, improve keyword field handling

- Updated all model schemas to use `dropIfExists` instead of `deleteIfExists`.
- Refactored `getKeywordField` method to use collections for cleaner logic.
- Introduced `BuilderException`.
- Fixed typing in `RelationsTest`.
…atial tests

- Deleted ElasticSearchRelationshipsTest.php, MySQLRelationshipsTest.php, PolymorphicElasticSearchRelationshipsTest.php, and RelationshipTest.php.
- Introduced GeospatialTest.php for testing geospatial queries.
- Added Location model for the new geospatial tests.
- Minor updates in QueryException and Builder for relevant functionalities.
…just tests

- Renamed all model files from `workbench/app/Models` to `tests/Models`.
- Updated namespaces in models from `Workbench\App\Models` to `PDPhilip\Elasticsearch\Tests\Models`.
- Remove use of `HasFactory` trait from models.
- Replaced `deleteIfExists` with `dropIfExists` in schema methods.
- Adjusted `HybridRelationsTest` to run model schema setup and tweaks.
- Modified `Processor.php` to handle raw Elasticsearch response correctly.
- Changed `insert` method return type in `Connection.php` to `Elasticsearch`.
- Simplified model properties and removed superfluous docblocks in models.

There, shuffling schemas and namespaces like a boss! 🃏
…apabilities

- Added `whereRegex` method to support regex queries with parameters.
- Included `withParameters` and `withFilterParameters` traits to manage where/filter conditions.
- Expanded support for operators including 'not like' and '<>'.
- Adjusted the script handling for date-based queries.
- Deprecated unnecessary traits and cleaned up use statements.
- Updated test cases to cover new `whereRegex` method and parameter management.

Parameterize everything like it's nobody's business! 😜
…operations

- Added multiple test cases for grouped queries, subquery conditions, pagination, sorting, updates, and deletions.
- Introduced `whereRaw` for raw condition filtering in queries.
- Implemented `exists` and `update` methods in query builder.
- Improved handling of distinct columns and extraction of indexable fields in grammar.
- Adjusted date and keyword field handling for better accuracy in grammar.

One ring to rule all clauses and operations! 🧙‍♂️✨
- Refactored and improved `whereNot` and `whereIn` tests for better clarity.
- Consolidated aggregation tests for succinctness.
- Removed redundant group and pagination tests.
- Updated `SchemaTest` to use the correct model namespace.
- Improved query grammar handling for aggregations and source extraction.
- Fixed grammar issues related to query presence checks.

Refactoring and testing, because precise querying deserves it! 🛠🔍
Also:
- groupBy param parses an array
- Added some backwards compatible methods
Group by with aggregation returns models and meta (ex get doc_count)
Since they are model instances, they can also get any relationships
- Tests passing
@UseTheFork
Copy link
Collaborator Author

@pdphilip This feels so close! Really loving it; it looks like you're getting the hang of the updates.

- Timebase UUID upgrade for MAC address lookup and caching
- connection configurable for id preference
- ElasticClient class for single source of truth when using the Elasticsearch client
- Simplifies logic when working with the connection class or the Elasticsearch client
- Option to intercept/manage anything at client level
GeneratesUuids:  Base64 ordered UUID
GeneratesElasticIds:  Elasticsearchy type ID
+ Tests with GeneratesUuids
Chunking: PIT default when sort on ID connection option is not enabled
PIT under the hood
- Highlight model Attrs
- Query Exception clean/clear details
- WhereTimestamp()
- withSort()
- wildcards wrapping if excluded
- Search style queries to manage options/fields as input better
Previously queryNested()
@pdphilip pdphilip merged commit 86b8a6d into pdphilip:main Mar 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants