NIFI-7188 Extending UI search with filters and refactoring existing s…

…olution

This closes apache#4123.

Signed-off-by: Peter Turcsanyi <turcsanyi@apache.org>

nifi-docs/src/main/asciidoc/user-guide.adoc

@@ -161,8 +161,7 @@ The Operate Palette sits to the left-hand side of the screen. It consists of but
 used by DFMs to manage the flow, as well as by administrators who manage user access
 and configure system properties, such as how many system resources should be provided to the application.
 
-On the right side of the canvas is Search, and the Global Menu. You can use Search to easily find components on the
-canvas and to search by component name, type, identifier, configuration properties, and their values. The Global Menu
+On the right side of the canvas is Search, and the Global Menu. For more information on search refer to <<search>>. The Global Menu
 contains options that allow you to manipulate existing components on the canvas:
 
 image::global-menu.png[NiFi Global Menu]
@@ -1652,6 +1651,106 @@ and select "Align horizontally" to achieve these results:
 
 image:align-horizontally-after.png["Align Horizontally Example Before"]
 
+[[search]]
+== Search Components in DataFlow
+
+NiFi UI provides searching functionality in order to help easily find components on the canvas. You can use search to find components by name, type, identifier, configuration properties, and their values. Search also makes it possible to refine and narrow the search result based on certain conditions using Filters and Keywords.
+
+[caption="Example 1: "]
+.The result will contain components that match for "processor1".
+=====================================================================
+processor1
+=====================================================================
+
+=== Filters
+
+Filters can be added to the search box as key-value pairs where the keys are predefined and check certain conditions based on the given value. The syntax is "key:value".
+
+[caption="Example 2: "]
+.The search will be executed under Process Groups (directly or via contained Process Groups) containing the string "myGroup" in their name or id. The result will contain components that match for "processor1".
+=====================================================================
+group:myGroup processor1
+=====================================================================
+
+Filters can be used together with other search terms and multiple filters can be used. The only constraint is that the search must start with the filters. Unknown filters or known filters with unknown values are ignored. If the same filter key appears multiple times, the first will be used. The order of different filters has no effect on the result.
+
+[caption="Example 3: "]
+.Search will be restricted to the currently active process group (and process groups within that). The result will contain components that match for "import" but property matches will be excluded.
+=====================================================================
+scope:here properties:exclude import
+=====================================================================
+
+The supported filters are the following:
+
+*scope*: This filter narrows the scope of the search based on the user's currently active Process Group. The only valid value is "here". The usage of this filter looks like "scope:here". Any other value is considered as invalid, thus the filter will be ignored during search.
+
+*group*: This filter narrows the scope of the search based on the provided Process Group name or id. Search will be restricted to groups (and their components - including subgroups and their components) the names or ids of which match the filter value. If no group matches the filter, the result list will be empty.
+
+*properties*: With this, users can prevent property matches to appear in the search result. Valid values are: "no", "none", "false", "exclude" and "0".
+
+=== Keywords
+
+Users can use pre-defined (case-insensitive) keywords in the search box that will check certain conditions.
+
+[caption="Example 4: "]
+."disabled" will be treated both as keyword and regular search term. The result will contain disabled Ports and Processors as all other components that match for "disabled" in any way.
+=====================================================================
+disabled
+=====================================================================
+
+Keywords can be used with filters (see below) but not with other search terms (otherwise they won't be treated as keywords) and only one keyword can be used at a time. Note however that keywords will also be treated as general search terms at the same time.
+
+[caption="Example 5: "]
+.Search will be restricted to the currently selected process group (and its sub process groups). "invalid" here (as it is alone after the filter) will be treated both as a keyword and a regular search term. The result will contain invalid Processors and Ports as well as all other components that match for "invalid" in any way.
+=====================================================================
+scope:here invalid
+=====================================================================
+
+The supported keywords are the following:
+
+- *Scheduled state*
+
+** *disabled*: Adds disabled Ports and Processors to the result list.
+
+** *invalid*: Adds Ports and Processors to the result list where the component is invalid.
+
+** *running*: Adds running Ports and Processors to the result list.
+
+** *stopped*: Adds stopped Ports and Processors to the result list.
+
+** *validating*: Adds Processors to the result list that are validating at the time.
+
+- *Scheduling strategy*
+
+** *event*: Adds Processors to the result list where the Scheduling Strategy is "Event Driven".
+
+** *timer*: Adds Processors to the result list where the Scheduling Strategy is "Timer Driven".
+
+- *Execution*
+
+** *primary:* Adds Processors to the result list that are set to run on the primary node only (whether if the Processor is currently running or not).
+
+- *Back pressure*
+
+** *back pressure*: Adds Connections to the result list that are applying back pressure at the time.
+
+** *pressure*: See "back pressure".
+
+- *Expiration*
+
+** *expiration*: Adds Connections to the result list that contain expired Flow Files.
+
+** *expires*: See "expiration".
+
+- *Transmission*
+
+** *not transmitting*: Adds Remote Process Groups to the result list that are not transmitting data at the time.
+
+** *transmitting*: Adds Remote Process Groups to the result list that are transmitting data at the time.
+
+** *transmission disabled*: See "not transmitting".
+
+** *transmitting enabled*: See "transmitting".
 
 [[monitoring]]
 == Monitoring of DataFlow

nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/pom.xml

@@ -395,10 +395,22 @@
             <version>1.0.1.RELEASE</version>
             <scope>provided</scope>
         </dependency>
+        <dependency>
+            <groupId>org.springframework</groupId>
+            <artifactId>spring-test</artifactId>
+            <version>4.3.26.RELEASE</version>
+            <scope>test</scope>
+        </dependency>
         <dependency>
             <groupId>org.spockframework</groupId>
             <artifactId>spock-core</artifactId>
             <scope>test</scope>
         </dependency>
+        <dependency>
+            <groupId>org.hamcrest</groupId>
+            <artifactId>hamcrest-all</artifactId>
+            <version>1.3</version>
+            <scope>test</scope>
+        </dependency>
     </dependencies>
 </project>

nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/NiFiServiceFacade.java

@@ -197,9 +197,10 @@ public interface NiFiServiceFacade {
      * Searches the controller for the specified query string.
      *
      * @param query query
+     * @param activeGroupId the id of the group currently selected in the editor
      * @return results
      */
-    SearchResultsDTO searchController(String query);
+    SearchResultsDTO searchController(String query, String activeGroupId);
 
     /**
      * Submits a provenance request.

nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/StandardNiFiServiceFacade.java

@@ -3106,8 +3106,8 @@ public ProvenanceEventDTO submitReplay(final Long eventId) {
     // -----------------------------------------
 
     @Override
-    public SearchResultsDTO searchController(final String query) {
-        return controllerFacade.search(query);
+    public SearchResultsDTO searchController(final String query, final String activeGroupId) {
+        return controllerFacade.search(query, activeGroupId);
     }
 
     @Override

nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/FlowResource.java

@@ -884,11 +884,14 @@ public Response activateControllerServices(
                     @ApiResponse(code = 409, message = "The request was valid but NiFi was not in the appropriate state to process it. Retrying the same request later may be successful.")
             }
     )
-    public Response searchFlow(@QueryParam("q") @DefaultValue(StringUtils.EMPTY) String value) throws InterruptedException {
+    public Response searchFlow(
+            @QueryParam("q") @DefaultValue(StringUtils.EMPTY) String value,
+            @QueryParam("a") @DefaultValue(StringUtils.EMPTY) String activeGroupId
+    ) throws InterruptedException {
         authorizeFlow();
 
         // query the controller
-        final SearchResultsDTO results = serviceFacade.searchController(value);
+        final SearchResultsDTO results = serviceFacade.searchController(value, activeGroupId);
 
         // create the entity
         final SearchResultsEntity entity = new SearchResultsEntity();

nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/controller/ControllerFacade.java

@@ -78,7 +78,6 @@
 import org.apache.nifi.provenance.search.SearchTerm;
 import org.apache.nifi.provenance.search.SearchTerms;
 import org.apache.nifi.provenance.search.SearchableField;
-import org.apache.nifi.registry.VariableRegistry;
 import org.apache.nifi.registry.flow.VersionedProcessGroup;
 import org.apache.nifi.remote.PublicPort;
 import org.apache.nifi.remote.RemoteGroupPort;
@@ -109,6 +108,8 @@
 import org.apache.nifi.web.api.dto.status.ControllerStatusDTO;
 import org.apache.nifi.web.api.dto.status.StatusHistoryDTO;
 import org.apache.nifi.web.api.entity.ControllerServiceEntity;
+import org.apache.nifi.web.search.query.SearchQuery;
+import org.apache.nifi.web.search.query.SearchQueryParser;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
@@ -147,7 +148,7 @@ public class ControllerFacade implements Authorizable {
     // properties
     private NiFiProperties properties;
     private DtoFactory dtoFactory;
-    private VariableRegistry variableRegistry;
+    private SearchQueryParser searchQueryParser;
     private ControllerSearchService controllerSearchService;
 
     private ProcessGroup getRootGroup() {
@@ -1614,15 +1615,22 @@ private void setComponentDetails(final ProvenanceEventDTO dto) {
     /**
      * Searches this controller for the specified term.
      *
-     * @param search search
+     * @param searchLiteral search string specified by the user
+     * @param activeGroupId the identifier of the currently visited group
      * @return result
      */
-    public SearchResultsDTO search(final String search) {
+    public SearchResultsDTO search(final String searchLiteral, final String activeGroupId) {
         final ProcessGroup rootGroup = getRootGroup();
+        final ProcessGroup activeGroup = (activeGroupId == null)
+                ? rootGroup
+                : flowController.getFlowManager().getGroup(activeGroupId);
         final SearchResultsDTO results = new SearchResultsDTO();
+        final SearchQuery searchQuery = searchQueryParser.parse(searchLiteral, NiFiUserUtils.getNiFiUser(), rootGroup, activeGroup);
 
-        controllerSearchService.search(results, search, rootGroup);
-        controllerSearchService.searchParameters(results, search);
+        if (!StringUtils.isEmpty(searchQuery.getTerm())) {
+            controllerSearchService.search(searchQuery, results);
+            controllerSearchService.searchParameters(searchQuery, results);
+        }
 
         return results;
     }
@@ -1631,7 +1639,6 @@ public void verifyComponentTypes(VersionedProcessGroup versionedFlow) {
         flowController.verifyComponentTypesInSnippet(versionedFlow);
     }
 
-
     public ProcessorDiagnosticsDTO getProcessorDiagnostics(final ProcessorNode processor, final ProcessorStatus processorStatus, final BulletinRepository bulletinRepository,
             final Function<String, ControllerServiceEntity> serviceEntityFactory) {
         return dtoFactory.createProcessorDiagnosticsDto(processor, processorStatus, bulletinRepository, flowController, serviceEntityFactory);
@@ -1640,28 +1647,29 @@ public ProcessorDiagnosticsDTO getProcessorDiagnostics(final ProcessorNode proce
     /*
      * setters
      */
+
     public void setFlowController(FlowController flowController) {
         this.flowController = flowController;
     }
 
-    public void setProperties(NiFiProperties properties) {
-        this.properties = properties;
+    public void setFlowService(FlowService flowService) {
+        this.flowService = flowService;
     }
 
     public void setAuthorizer(Authorizer authorizer) {
         this.authorizer = authorizer;
     }
 
-    public void setFlowService(FlowService flowService) {
-        this.flowService = flowService;
+    public void setProperties(NiFiProperties properties) {
+        this.properties = properties;
     }
 
     public void setDtoFactory(DtoFactory dtoFactory) {
         this.dtoFactory = dtoFactory;
     }
 
-    public void setVariableRegistry(VariableRegistry variableRegistry) {
-        this.variableRegistry = variableRegistry;
+    public void setSearchQueryParser(SearchQueryParser searchQueryParser) {
+        this.searchQueryParser = searchQueryParser;
     }
 
     public void setControllerSearchService(ControllerSearchService controllerSearchService) {