Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: API for search entities #29203

Merged
merged 23 commits into from
Dec 8, 2023
Merged
Show file tree
Hide file tree
Changes from 19 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
4a27a58
feat: API to fetch workspaces for homepage
abhvsn Nov 16, 2023
169fc5c
test: Add testcases
abhvsn Nov 17, 2023
b64713c
Merge branch 'release' into feat/add-workspece-homepage-api
abhvsn Nov 20, 2023
98bfa7b
Add testcases for domain sorter class
abhvsn Nov 20, 2023
88eae0e
Create basic skeleton with API contract
abhvsn Nov 21, 2023
257bd61
feat(RecentlyUsedEntityDTO): Introduce RecentlyUsedEntityDTO and impl…
abhvsn Nov 24, 2023
40f175e
test: Add testcases
abhvsn Nov 27, 2023
2a267ef
Merge branch 'release' into feat/add-get-applications-for-homepage-api
abhvsn Nov 28, 2023
3beeae6
Add testcases for updating the entities
abhvsn Nov 28, 2023
4962ab9
Merge branch 'release' into feat/add-get-applications-for-homepage-api
abhvsn Nov 28, 2023
042df3d
feat: Add migration to track recently used entities for user data
abhvsn Nov 28, 2023
c62ee13
feat: Add search entity functionality for homepage
abhvsn Nov 29, 2023
4eb0876
feat: Add entity selector option to enhance search functionality
abhvsn Nov 29, 2023
fbedec8
chore: Add indexing comment for search field method
abhvsn Dec 4, 2023
8d8dd6e
Merge branch 'release' into feat/search-api-homepage
abhvsn Dec 4, 2023
0a80de2
feat: Add SearchEntityHelper and its test cases
abhvsn Dec 4, 2023
2f9ea02
test: Add testcases and fix get default application when connected to…
abhvsn Dec 5, 2023
34ff3cf
feat: Trim the search string before fetching results from DB
abhvsn Dec 5, 2023
86c6462
Merge branch 'release' into feat/search-api-homepage
abhvsn Dec 5, 2023
5a6c1bb
chore: Resolve merge conflicts with release
abhvsn Dec 5, 2023
f86074a
Merge branch 'release' into feat/search-api-homepage
abhvsn Dec 6, 2023
fb606c2
chore: Resolve merge conflicts with release
abhvsn Dec 6, 2023
aadb679
refactor method name
abhvsn Dec 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ public class UrlCE {
public static final String USAGE_PULSE_URL = BASE_URL + VERSION + "/usage-pulse";
public static final String TENANT_URL = BASE_URL + VERSION + "/tenants";
public static final String CUSTOM_JS_LIB_URL = BASE_URL + VERSION + "/libraries";

public static final String PRODUCT_ALERT = BASE_URL + VERSION + "/product-alert";
public static final String SEARCH_ENTITY_URL = BASE_URL + VERSION + "/search-entities";

// Sub-paths
public static final String MOCKS = "/mocks";
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
package com.appsmith.server.controllers;

import com.appsmith.server.constants.Url;
import com.appsmith.server.controllers.ce.SearchEntityControllerCE;
import com.appsmith.server.searchentities.SearchEntitySolution;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RequestMapping(Url.SEARCH_ENTITY_URL)
@RestController
public class SearchEntityController extends SearchEntityControllerCE {

public SearchEntityController(SearchEntitySolution searchEntitySolution) {
super(searchEntitySolution);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,7 @@ public Mono<ResponseDTO<List<Application>>> deleteMultipleApps(@Valid @RequestBo
.map(deletedResources -> new ResponseDTO<>(HttpStatus.OK.value(), deletedResources, null));
}

@Deprecated
@JsonView(Views.Public.class)
@GetMapping("/new")
public Mono<ResponseDTO<UserHomepageDTO>> getAllApplicationsForHome() {
Expand All @@ -169,6 +170,16 @@ public Mono<ResponseDTO<UserHomepageDTO>> getAllApplicationsForHome() {
.map(applications -> new ResponseDTO<>(HttpStatus.OK.value(), applications, null));
}

@JsonView(Views.Public.class)
@GetMapping("/home")
public Mono<ResponseDTO<List<Application>>> findByWorkspaceIdAndRecentlyUsedOrder(
@RequestParam(required = false) String workspaceId) {
log.debug("Going to get all applications by workspace id {}", workspaceId);
return service.findByWorkspaceIdAndDefaultApplicationsInRecentlyUsedOrder(workspaceId)
.collectList()
.map(applications -> new ResponseDTO<>(HttpStatus.OK.value(), applications, null));
}

@JsonView(Views.Public.class)
@GetMapping(Url.RELEASE_ITEMS)
public Mono<ResponseDTO<ReleaseItemsDTO>> getReleaseItemsInformation() {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
package com.appsmith.server.controllers.ce;

import com.appsmith.external.views.Views;
import com.appsmith.server.constants.Url;
import com.appsmith.server.dtos.ResponseDTO;
import com.appsmith.server.dtos.SearchEntityDTO;
import com.appsmith.server.searchentities.SearchEntitySolution;
import com.fasterxml.jackson.annotation.JsonView;
import lombok.extern.slf4j.Slf4j;
import org.springframework.http.HttpStatus;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestParam;
import reactor.core.publisher.Mono;

@RequestMapping(Url.SEARCH_ENTITY_URL)
@Slf4j
public class SearchEntityControllerCE {

private final SearchEntitySolution searchEntitySolution;

public SearchEntityControllerCE(SearchEntitySolution searchEntitySolution) {
this.searchEntitySolution = searchEntitySolution;
}

@JsonView(Views.Public.class)
@GetMapping("")
public Mono<ResponseDTO<SearchEntityDTO>> getAllUnpublishedActionCollections(
@RequestParam(required = false) String[] entities,
@RequestParam(required = false, defaultValue = "") String keyword,
@RequestParam(required = false, defaultValue = "0") int page,
@RequestParam(required = false, defaultValue = "10") int size) {
log.debug("Going to search for entities with search string: {}", keyword);
return searchEntitySolution
.searchEntity(entities, keyword, page, size, Boolean.TRUE)
abhvsn marked this conversation as resolved.
Show resolved Hide resolved
.map(resources -> new ResponseDTO<>(HttpStatus.OK.value(), resources, null));
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The method name getAllUnpublishedActionCollections does not accurately reflect its functionality, which is to search for various entities. Consider renaming it to better describe its purpose, such as searchEntities.

}
Original file line number Diff line number Diff line change
Expand Up @@ -82,4 +82,12 @@ public Mono<ResponseDTO<Workspace>> deleteLogo(@PathVariable String workspaceId)
return service.deleteLogo(workspaceId)
.map(workspace -> new ResponseDTO<>(HttpStatus.OK.value(), workspace, null));
}

@JsonView(Views.Public.class)
@GetMapping("/home")
public Mono<ResponseDTO<List<Workspace>>> getUserWorkspacesByRecentlyUsedOrder() {
return userWorkspaceService
.getUserWorkspacesByRecentlyUsedOrder()
.map(workspaces -> new ResponseDTO<>(HttpStatus.OK.value(), workspaces, null));
abhvsn marked this conversation as resolved.
Show resolved Hide resolved
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

import com.appsmith.external.models.BaseDomain;
import com.appsmith.external.views.Views;
import com.appsmith.server.dtos.RecentlyUsedEntityDTO;
import com.appsmith.server.helpers.CollectionUtils;
import com.fasterxml.jackson.annotation.JsonView;
import lombok.Getter;
Expand Down Expand Up @@ -56,13 +57,19 @@ public class UserData extends BaseDomain {
private List<String> recentlyUsedOrgIds;

// list of workspace ids that were recently accessed by the user
@Deprecated
@JsonView(Views.Public.class)
private List<String> recentlyUsedWorkspaceIds;

// list of application ids that were recently accessed by the user
@Deprecated
@JsonView(Views.Public.class)
private List<String> recentlyUsedAppIds;

// Map of workspaceId to list of recently used applicationIds. This field should be used to add entities
@JsonView(Views.Public.class)
private List<RecentlyUsedEntityDTO> recentlyUsedEntityIds;

// Map of defaultApplicationIds with the GitProfiles. For fallback/default git profile per user default will be the
// the key for the map
@JsonView(Views.Internal.class)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
package com.appsmith.server.dtos;

import com.appsmith.server.dtos.ce.RecentlyUsedEntityCE_DTO;
import lombok.Getter;
import lombok.NoArgsConstructor;
import lombok.Setter;

@Getter
@Setter
@NoArgsConstructor
public class RecentlyUsedEntityDTO extends RecentlyUsedEntityCE_DTO {}
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
package com.appsmith.server.dtos;

import com.appsmith.server.dtos.ce.SearchEntityCE_DTO;
import lombok.Getter;
import lombok.Setter;

@Getter
@Setter
public class SearchEntityDTO extends SearchEntityCE_DTO {}
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
package com.appsmith.server.dtos.ce;

import lombok.Getter;
import lombok.Setter;

import java.util.List;

@Getter
@Setter
public class RecentlyUsedEntityCE_DTO {
String workspaceId;
List<String> applicationIds;
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
package com.appsmith.server.dtos.ce;

import com.appsmith.server.domains.Application;
import com.appsmith.server.domains.Workspace;
import lombok.Getter;
import lombok.Setter;

import java.util.List;

@Getter
@Setter
public class SearchEntityCE_DTO {
List<Application> applications;
List<Workspace> workspaces;
}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package com.appsmith.server.helpers;

import com.appsmith.server.domains.Application;
import com.appsmith.server.domains.GitApplicationMetadata;
import com.appsmith.server.exceptions.AppsmithError;
import com.appsmith.server.exceptions.AppsmithException;
Expand Down Expand Up @@ -102,4 +103,29 @@ public static String getDefaultBranchName(GitApplicationMetadata gitApplicationM
? gitApplicationMetadata.getBranchName()
: gitApplicationMetadata.getDefaultBranchName();
}

/**
* This method checks if the application is connected to git and is the default branched.
*
* @param application application to be checked
* @return true if the application is default branched, false otherwise
*/
public static boolean isDefaultBranchedApplication(Application application) {
GitApplicationMetadata metadata = application.getGitApplicationMetadata();
return isApplicationConnectedToGit(application)
&& !StringUtils.isEmptyOrNull(metadata.getBranchName())
&& metadata.getBranchName().equals(metadata.getDefaultBranchName());
}

/**
* This method checks if the application is connected to Git or not.
* @param application application to be checked
* @return true if the application is connected to Git, false otherwise
*/
public static boolean isApplicationConnectedToGit(Application application) {
GitApplicationMetadata metadata = application.getGitApplicationMetadata();
return metadata != null
&& !StringUtils.isEmptyOrNull(metadata.getDefaultApplicationId())
&& !StringUtils.isEmptyOrNull(metadata.getRemoteUrl());
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -17,15 +17,13 @@
import com.appsmith.server.exceptions.AppsmithError;
import com.appsmith.server.exceptions.AppsmithException;
import com.appsmith.server.migrations.JsonSchemaVersions;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang.StringUtils;
import org.springframework.stereotype.Component;
import org.springframework.util.CollectionUtils;

import java.util.List;

@RequiredArgsConstructor
@Slf4j
@Component
public class ResponseUtils {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
package com.appsmith.server.helpers.ce;

import com.appsmith.external.models.BaseDomain;
import org.springframework.util.CollectionUtils;
import reactor.core.publisher.Flux;

import java.util.ArrayList;
import java.util.LinkedHashMap;
import java.util.List;
import java.util.function.Function;
import java.util.stream.Collectors;

public class DomainSorter {

/**
* Sorts a Flux of domains based on the provided list of ordered domain IDs.
*
* @param domainFlux The Flux of domains to be sorted.
* @param sortedDomainIds The list of domain IDs used for sorting.
* @param <Domain> The type of the domains, must extend BaseDomain.
* @return A Flux of sorted domains.
*/
public static <Domain extends BaseDomain> Flux<Domain> sortDomainsBasedOnOrderedDomainIds(
Flux<Domain> domainFlux, List<String> sortedDomainIds) {
if (CollectionUtils.isEmpty(sortedDomainIds)) {
return domainFlux;
}
return domainFlux
.collect(Collectors.toMap(Domain::getId, Function.identity(), (key1, key2) -> key1, LinkedHashMap::new))
.map(domainMap -> {
List<Domain> sortedDomains = new ArrayList<>();
for (String id : sortedDomainIds) {
if (domainMap.containsKey(id)) {
sortedDomains.add(domainMap.get(id));
domainMap.remove(id);
}
}
sortedDomains.addAll(domainMap.values());
return sortedDomains;
})
.flatMapMany(Flux::fromIterable);
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
package com.appsmith.server.migrations.db.ce;

import com.appsmith.server.domains.QUserData;
import com.appsmith.server.domains.UserData;
import com.appsmith.server.dtos.RecentlyUsedEntityDTO;
import com.appsmith.server.helpers.CollectionUtils;
import io.mongock.api.annotations.ChangeUnit;
import io.mongock.api.annotations.Execution;
import io.mongock.api.annotations.RollbackExecution;
import org.springframework.data.mongodb.core.MongoTemplate;
import org.springframework.data.mongodb.core.query.Query;
import org.springframework.data.mongodb.core.query.Update;

import java.util.ArrayList;
import java.util.List;

import static com.appsmith.server.repositories.ce.BaseAppsmithRepositoryCEImpl.fieldName;
import static org.springframework.data.mongodb.core.query.Criteria.where;
import static org.springframework.data.mongodb.core.query.Query.query;

@ChangeUnit(order = "036", id = "add-recently-used-entities-for-user")
public class Migration036AddRecentlyUsedEntitiesForUserData {

private final MongoTemplate mongoTemplate;

public Migration036AddRecentlyUsedEntitiesForUserData(MongoTemplate mongoTemplate) {
this.mongoTemplate = mongoTemplate;
}

@RollbackExecution
public void rollbackExecution() {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider implementing a rollback mechanism for this migration.


@Execution
public void addRecentlyUsedEntitiesForUserData() {

final Query userDataQuery =
query(where(fieldName(QUserData.userData.deleted)).ne(true));

// We are not migrating the applicationIds, to avoid long-running migration. Also, as user starts using the
// instance these fields should auto-populate.
userDataQuery.fields().include(fieldName(QUserData.userData.recentlyUsedWorkspaceIds));

List<UserData> userDataList = mongoTemplate.find(userDataQuery, UserData.class);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider using a cursor or pagination if the dataset is large to avoid memory issues.

for (UserData userData : userDataList) {
final Update update = new Update();
if (CollectionUtils.isNullOrEmpty(userData.getRecentlyUsedWorkspaceIds())) {
break;
abhvsn marked this conversation as resolved.
Show resolved Hide resolved
}
List<RecentlyUsedEntityDTO> recentlyUsedEntityDTOS = new ArrayList<>();
for (String workspaceId : userData.getRecentlyUsedWorkspaceIds()) {
RecentlyUsedEntityDTO recentlyUsedEntityDTO = new RecentlyUsedEntityDTO();
recentlyUsedEntityDTO.setWorkspaceId(workspaceId);
recentlyUsedEntityDTOS.add(recentlyUsedEntityDTO);
}
update.set(fieldName(QUserData.userData.recentlyUsedEntityIds), recentlyUsedEntityDTOS);
mongoTemplate.updateFirst(
query(where(fieldName(QUserData.userData.id)).is(userData.getId())), update, UserData.class);
}
}
}
Loading
Loading