-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
feat: API for search entities (#29203)
### Description As per updated homepage experience search functionality will be handled by server and will be applicable for all the entities present on homepage. Request format: ``` curl --location 'https://dev.appsmith.com/api/v1/search-entities?keyword=test&page=0&size=10&entities={comma separated entity names e.g.Application,Workspace}' \ --header 'Cookie: SESSION={logged_in_user's_session_cookie}' ``` Note: We will be running a couple of experiment to optimise the search, with this PR we have implemented basic search with contains functionality and index is applied on the searchable fields. Mongo does offer [text-search functionality](https://www.mongodb.com/docs/manual/text-search/) based on tokenisation which may tackle the incorrect spellings scenario. But as the searches are for names we have avoided that route for now as language tokenisation was not providing the expected results which basic search was able to. Design handoff: https://app.zeplin.io/project/653f7de4c1d563203f817bce/screen/653f7eea5d02e7233ede382c <img width="1095" alt="Screenshot 2023-12-07 at 3 09 02 AM" src="https://github.com/appsmithorg/appsmith/assets/41686026/c26225ef-9d78-4969-b445-a78bc58c18a0"> #### PR fixes following issue(s) Fixes #28793 #### Type of change - New feature (non-breaking change which adds functionality) ## Testing > #### How Has This Been Tested? - [x] Manual - [x] JUnit - [ ] Jest - [ ] Cypress > > #### Test Plan > Add Testsmith test cases links that relate to this PR > > #### Issues raised during DP testing > Link issues raised during DP testing for better visiblity and tracking (copy link from comments dropped on this PR) > > > ## Checklist: #### Dev activity - [ ] My code follows the style guidelines of this project - [ ] I have performed a self-review of my own code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [ ] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes - [ ] PR is being merged under a feature flag #### QA activity: - [ ] [Speedbreak features](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#speedbreakers-) have been covered - [ ] Test plan covers all impacted features and [areas of interest](https://github.com/appsmithorg/TestSmith/wiki/Guidelines-for-test-plans#areas-of-interest-) - [ ] Test plan has been peer reviewed by project stakeholders and other QA members - [ ] Manually tested functionality on DP - [ ] We had an implementation alignment call with stakeholders post QA Round 2 - [ ] Cypress test cases have been added and approved by SDET/manual QA - [ ] Added `Test Plan Approved` label after Cypress tests were reviewed - [ ] Added `Test Plan Approved` label after JUnit tests were reviewed <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced a new search functionality that allows users to find and sort applications and workspaces. - Added the ability to view recently used applications in a sorted order. - **Improvements** - Enhanced the application sorting mechanism to prioritize recently used items. - Streamlined the process of checking if an application is connected to Git. - **Bug Fixes** - Fixed an issue where the list of applications was not updating correctly for recently used items. - **Deprecated Features** - Marked the `getAllApplicationsForHome` method as deprecated. - **Tests** - Added new test cases to ensure the reliability of the search and sorting features. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
- Loading branch information
Showing
21 changed files
with
838 additions
and
18 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
16 changes: 16 additions & 0 deletions
16
...appsmith-server/src/main/java/com/appsmith/server/controllers/SearchEntityController.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| package com.appsmith.server.controllers; | ||
|
|
||
| import com.appsmith.server.constants.Url; | ||
| import com.appsmith.server.controllers.ce.SearchEntityControllerCE; | ||
| import com.appsmith.server.searchentities.SearchEntitySolution; | ||
| import org.springframework.web.bind.annotation.RequestMapping; | ||
| import org.springframework.web.bind.annotation.RestController; | ||
|
|
||
| @RequestMapping(Url.SEARCH_ENTITY_URL) | ||
| @RestController | ||
| public class SearchEntityController extends SearchEntityControllerCE { | ||
|
|
||
| public SearchEntityController(SearchEntitySolution searchEntitySolution) { | ||
| super(searchEntitySolution); | ||
| } | ||
| } |
38 changes: 38 additions & 0 deletions
38
...ith-server/src/main/java/com/appsmith/server/controllers/ce/SearchEntityControllerCE.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| package com.appsmith.server.controllers.ce; | ||
|
|
||
| import com.appsmith.external.views.Views; | ||
| import com.appsmith.server.constants.Url; | ||
| import com.appsmith.server.dtos.ResponseDTO; | ||
| import com.appsmith.server.dtos.SearchEntityDTO; | ||
| import com.appsmith.server.searchentities.SearchEntitySolution; | ||
| import com.fasterxml.jackson.annotation.JsonView; | ||
| import lombok.extern.slf4j.Slf4j; | ||
| import org.springframework.http.HttpStatus; | ||
| import org.springframework.web.bind.annotation.GetMapping; | ||
| import org.springframework.web.bind.annotation.RequestMapping; | ||
| import org.springframework.web.bind.annotation.RequestParam; | ||
| import reactor.core.publisher.Mono; | ||
|
|
||
| @RequestMapping(Url.SEARCH_ENTITY_URL) | ||
| @Slf4j | ||
| public class SearchEntityControllerCE { | ||
|
|
||
| private final SearchEntitySolution searchEntitySolution; | ||
|
|
||
| public SearchEntityControllerCE(SearchEntitySolution searchEntitySolution) { | ||
| this.searchEntitySolution = searchEntitySolution; | ||
| } | ||
|
|
||
| @JsonView(Views.Public.class) | ||
| @GetMapping("") | ||
| public Mono<ResponseDTO<SearchEntityDTO>> getAllUnpublishedActionCollections( | ||
| @RequestParam(required = false) String[] entities, | ||
| @RequestParam(required = false, defaultValue = "") String keyword, | ||
| @RequestParam(required = false, defaultValue = "0") int page, | ||
| @RequestParam(required = false, defaultValue = "20") int size) { | ||
| log.debug("Going to search for entities with search string: {}", keyword); | ||
| return searchEntitySolution | ||
| .searchEntity(entities, keyword, page, size, Boolean.TRUE) | ||
| .map(resources -> new ResponseDTO<>(HttpStatus.OK.value(), resources, null)); | ||
| } | ||
| } |
9 changes: 9 additions & 0 deletions
9
app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/SearchEntityDTO.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| package com.appsmith.server.dtos; | ||
|
|
||
| import com.appsmith.server.dtos.ce.SearchEntityCE_DTO; | ||
| import lombok.Getter; | ||
| import lombok.Setter; | ||
|
|
||
| @Getter | ||
| @Setter | ||
| public class SearchEntityDTO extends SearchEntityCE_DTO {} |
15 changes: 15 additions & 0 deletions
15
app/server/appsmith-server/src/main/java/com/appsmith/server/dtos/ce/SearchEntityCE_DTO.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| package com.appsmith.server.dtos.ce; | ||
|
|
||
| import com.appsmith.server.domains.Application; | ||
| import com.appsmith.server.domains.Workspace; | ||
| import lombok.Getter; | ||
| import lombok.Setter; | ||
|
|
||
| import java.util.List; | ||
|
|
||
| @Getter | ||
| @Setter | ||
| public class SearchEntityCE_DTO { | ||
| List<Application> applications; | ||
| List<Workspace> workspaces; | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
57 changes: 57 additions & 0 deletions
57
...com/appsmith/server/migrations/db/ce/Migration037AddCompoundIndexForNameAndDeletedAt.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| package com.appsmith.server.migrations.db.ce; | ||
|
|
||
| import com.appsmith.server.constants.FieldName; | ||
| import com.appsmith.server.domains.Application; | ||
| import com.appsmith.server.domains.Workspace; | ||
| import io.mongock.api.annotations.ChangeUnit; | ||
| import io.mongock.api.annotations.Execution; | ||
| import io.mongock.api.annotations.RollbackExecution; | ||
| import lombok.extern.slf4j.Slf4j; | ||
| import org.springframework.data.mongodb.UncategorizedMongoDbException; | ||
| import org.springframework.data.mongodb.core.MongoTemplate; | ||
| import org.springframework.data.mongodb.core.index.Index; | ||
|
|
||
| import static com.appsmith.server.migrations.DatabaseChangelog1.dropIndexIfExists; | ||
| import static com.appsmith.server.migrations.DatabaseChangelog1.ensureIndexes; | ||
| import static com.appsmith.server.migrations.DatabaseChangelog1.makeIndex; | ||
|
|
||
| @Slf4j | ||
| @ChangeUnit(order = "037", id = "add-compound-index-name-deleted", author = " ") | ||
| public class Migration037AddCompoundIndexForNameAndDeletedAt { | ||
|
|
||
| private final MongoTemplate mongoTemplate; | ||
|
|
||
| private static final String NAME_DELETED_COMPOUND_INDEX = "name_deleted_compound_index"; | ||
|
|
||
| public Migration037AddCompoundIndexForNameAndDeletedAt(MongoTemplate mongoTemplate) { | ||
| this.mongoTemplate = mongoTemplate; | ||
| } | ||
|
|
||
| /** | ||
| * mandatory to declare, but we don't have a use-case for this yet. | ||
| */ | ||
| @RollbackExecution | ||
| public void rollbackExecution() {} | ||
|
|
||
| @Execution | ||
| public void addIndexInWorkspaceAndApplicationsCollection() { | ||
|
|
||
| Index namedDeletedAtIndex = makeIndex(FieldName.NAME, FieldName.DELETED, FieldName.DELETED_AT) | ||
| .named(NAME_DELETED_COMPOUND_INDEX); | ||
|
|
||
| try { | ||
| dropIndexIfExists(mongoTemplate, Workspace.class, NAME_DELETED_COMPOUND_INDEX); | ||
| ensureIndexes(mongoTemplate, Workspace.class, namedDeletedAtIndex); | ||
|
|
||
| dropIndexIfExists(mongoTemplate, Application.class, NAME_DELETED_COMPOUND_INDEX); | ||
| ensureIndexes(mongoTemplate, Application.class, namedDeletedAtIndex); | ||
| } catch (UncategorizedMongoDbException mongockException) { | ||
| log.error( | ||
| "An error occurred while creating the index : {}, skipping the addition of index because of {}.", | ||
| NAME_DELETED_COMPOUND_INDEX, | ||
| mongockException.getMessage()); | ||
| } catch (Exception exception) { | ||
| log.error("An error occurred while creating the index : {}", NAME_DELETED_COMPOUND_INDEX, exception); | ||
| } | ||
| } | ||
| } |
3 changes: 3 additions & 0 deletions
3
...ppsmith-server/src/main/java/com/appsmith/server/searchentities/SearchEntitySolution.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,3 @@ | ||
| package com.appsmith.server.searchentities; | ||
|
|
||
| public interface SearchEntitySolution extends SearchEntitySolutionCE {} |
9 changes: 9 additions & 0 deletions
9
...smith-server/src/main/java/com/appsmith/server/searchentities/SearchEntitySolutionCE.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| package com.appsmith.server.searchentities; | ||
|
|
||
| import com.appsmith.server.dtos.SearchEntityDTO; | ||
| import reactor.core.publisher.Mono; | ||
|
|
||
| public interface SearchEntitySolutionCE { | ||
| Mono<SearchEntityDTO> searchEntity( | ||
| String[] entities, String searchString, int page, int size, Boolean isRequestedForHomepage); | ||
| } |
108 changes: 108 additions & 0 deletions
108
...h-server/src/main/java/com/appsmith/server/searchentities/SearchEntitySolutionCEImpl.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,108 @@ | ||
| package com.appsmith.server.searchentities; | ||
|
|
||
| import com.appsmith.server.applications.base.ApplicationService; | ||
| import com.appsmith.server.constants.FieldName; | ||
| import com.appsmith.server.domains.Application; | ||
| import com.appsmith.server.domains.Workspace; | ||
| import com.appsmith.server.dtos.SearchEntityDTO; | ||
| import com.appsmith.server.helpers.GitUtils; | ||
| import com.appsmith.server.helpers.ResponseUtils; | ||
| import com.appsmith.server.services.WorkspaceService; | ||
| import com.appsmith.server.solutions.ApplicationPermission; | ||
| import com.appsmith.server.solutions.WorkspacePermission; | ||
| import lombok.RequiredArgsConstructor; | ||
| import org.springframework.data.domain.Pageable; | ||
| import org.springframework.data.domain.Sort; | ||
| import org.springframework.util.StringUtils; | ||
| import reactor.core.publisher.Mono; | ||
|
|
||
| import java.util.ArrayList; | ||
| import java.util.List; | ||
|
|
||
| import static com.appsmith.server.searchentities.helpers.SearchEntityHelper.shouldSearchEntity; | ||
|
|
||
| @RequiredArgsConstructor | ||
| public class SearchEntitySolutionCEImpl implements SearchEntitySolutionCE { | ||
|
|
||
| private final WorkspaceService workspaceService; | ||
|
|
||
| private final ApplicationService applicationService; | ||
|
|
||
| private final WorkspacePermission workspacePermission; | ||
|
|
||
| private final ApplicationPermission applicationPermission; | ||
|
|
||
| private final ResponseUtils responseUtils; | ||
|
|
||
| /** | ||
| * This method searches for workspaces and applications based on the searchString provided. | ||
| * The search is performed with contains operator on the name field of the entities and is case-insensitive. | ||
| * The search results are sorted by the updated_at field in descending order. | ||
| * searchString = "test" will return all entities with name containing "test". | ||
| * e.g. "test_app", "test_workspace", "appTest", "wsTest_random" etc. | ||
| * | ||
| * @param entities The list of entities to search for. If null or empty, all entities are searched. | ||
| * @param searchString The string to search for in the name field of the entities. | ||
| * @param page The page number of the results to return. | ||
| * @param size Max number of results to return within each entity. | ||
| * @param isRequestedForHomepage Whether the search is requested for the homepage or not. | ||
| * | ||
| * @return A Mono of SearchEntityDTO containing the list of workspaces and applications. | ||
| */ | ||
| @Override | ||
| public Mono<SearchEntityDTO> searchEntity( | ||
| String[] entities, String searchString, int page, int size, Boolean isRequestedForHomepage) { | ||
| if (size == 0) { | ||
| return Mono.just(new SearchEntityDTO()); | ||
| } | ||
| Pageable pageable = Pageable.ofSize(size).withPage(page); | ||
| Sort sort = Sort.by(Sort.Direction.DESC, FieldName.UPDATED_AT); | ||
| searchString = StringUtils.hasLength(searchString) ? searchString.trim() : ""; | ||
| // If no entities are specified, search for all entities. | ||
| Mono<List<Workspace>> workspacesMono = Mono.just(new ArrayList<>()); | ||
| if (shouldSearchEntity(Workspace.class, entities)) { | ||
| workspacesMono = workspaceService | ||
| .filterByEntityFields( | ||
| List.of(FieldName.NAME), | ||
| searchString, | ||
| pageable, | ||
| sort, | ||
| workspacePermission.getReadPermission()) | ||
| .collectList(); | ||
| } | ||
|
|
||
| Mono<List<Application>> applicationsMono = Mono.just(new ArrayList<>()); | ||
| if (shouldSearchEntity(Application.class, entities)) { | ||
| applicationsMono = applicationService | ||
| .filterByEntityFields( | ||
| List.of(FieldName.NAME), | ||
| searchString, | ||
| pageable, | ||
| sort, | ||
| applicationPermission.getReadPermission()) | ||
| .filter(application -> { | ||
| if (Boolean.FALSE.equals(isRequestedForHomepage)) { | ||
| return true; | ||
| } | ||
| /* | ||
| * As the applications are requested on homepage filter applications based on the following | ||
| * criteria: | ||
| * - Applications that are not connected to Git. | ||
| * OR | ||
| * - Applications that, when connected, revert with default branch only. | ||
| */ | ||
| return !GitUtils.isApplicationConnectedToGit(application) | ||
| || GitUtils.isDefaultBranchedApplication(application); | ||
| }) | ||
| .map(responseUtils::updateApplicationWithDefaultResources) | ||
| .collectList(); | ||
| } | ||
|
|
||
| return Mono.zip(workspacesMono, applicationsMono).map(tuple2 -> { | ||
| SearchEntityDTO searchEntityDTO = new SearchEntityDTO(); | ||
| searchEntityDTO.setWorkspaces(tuple2.getT1()); | ||
| searchEntityDTO.setApplications(tuple2.getT2()); | ||
| return searchEntityDTO; | ||
| }); | ||
| } | ||
| } |
Oops, something went wrong.