Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improved database querying for individuals profiles #2360

Merged

Conversation

Omri-Levy
Copy link
Contributor

@Omri-Levy Omri-Levy commented May 7, 2024

User description

Description

Elaborate on the subject, motivation, and context.

Related issues

  • Provide a link to each related issue.

Breaking changes

  • Describe the breaking changes that this pull request introduces.

How these changes were tested

  • Describe the tests that you ran to verify your changes, including devices, operating systems, browsers and versions.

Examples and references

  • Links, screenshots, and other resources related to this change.

Checklist

  • [] I have read the contribution guidelines of this project
  • [] I have read the style guidelines of this project
  • [] I have performed a self-review of my own code
  • [] I have commented my code, particularly in hard-to-understand areas
  • [] I have made corresponding changes to the documentation
  • [] My changes generate no new warnings and errors
  • [] New and existing tests pass locally with my changes

PR Type

enhancement


Description

  • Enhanced data fetching efficiency in CaseManagementController by directly including workflowRuntimeData in the initial query.
  • Removed outdated methods findFirstByEntityId and getByEntityId from WorkflowRuntimeDataRepository and WorkflowService respectively, simplifying the codebase and potentially improving performance.

Changes walkthrough 📝

Relevant files
Enhancement
case-management.controller.ts
Optimize end user data fetching by including workflowRuntimeData

services/workflows-service/src/case-management/controllers/case-management.controller.ts

  • Added workflowRuntimeData to the selection set in the query for
    fetching end users.
  • Replaced fetching workflowRuntimeData via a separate service call with
    direct access from the end user object.
  • +9/-2     
    workflow-runtime-data.repository.ts
    Remove unused method from WorkflowRuntimeDataRepository   

    services/workflows-service/src/workflow/workflow-runtime-data.repository.ts

  • Removed findFirstByEntityId method which was previously used to fetch
    workflow runtime data for a specific entity.
  • +0/-26   
    workflow.service.ts
    Remove deprecated workflow data fetching method                   

    services/workflows-service/src/workflow/workflow.service.ts

  • Removed getByEntityId method which was used to fetch workflow runtime
    data.
  • +0/-12   

    💡 PR-Agent usage:
    Comment /help on the PR to get a list of all available PR-Agent tools and their descriptions

    Summary by CodeRabbit

    • New Features

      • Added a new tags property to workflow runtime data for enhanced case management.
    • Refactor

      • Simplified data retrieval processes in workflow services by removing redundant methods and optimizing existing functionalities.

    Copy link

    changeset-bot bot commented May 7, 2024

    ⚠️ No Changeset found

    Latest commit: f8bb260

    Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

    This PR includes no changesets

    When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

    Click here to learn what changesets are, and how to add one.

    Click here if you're a maintainer who wants to add a changeset to this PR

    Copy link
    Contributor

    coderabbitai bot commented May 7, 2024

    Walkthrough

    This update primarily enhances how workflow runtime data is managed in the case management system. It includes adding a new field to store tags directly within the CaseManagementController, and simplifying data retrieval by removing redundant methods in the WorkflowService and WorkflowRuntimeDataRepository. This refactoring aims to streamline data handling and reduce complexity in the workflow service.

    Changes

    File Path Change Summary
    .../case-management.controller.ts Added workflowRuntimeData field with tags. Refactored to use endUser.workflowRuntimeData?.tags.
    .../workflow-runtime-data.repository.ts & .../workflow.service.ts Removed findFirstByEntityId and getByEntityId methods.

    🐰✨🌟
    In the land of code and bits,
    A rabbit hopped and made some fits.
    Tags in fields, so bright and new,
    Old methods out, less fetching to do.
    Hop, skip, a refactor spree,
    Clean and swift, as code should be! 🌼🚀
    🐰✨🌟


    Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

    Share
    Tips

    Chat

    There are 3 ways to chat with CodeRabbit:

    • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
      • I pushed a fix in commit <commit_id>.
      • Generate unit testing code for this file.
      • Open a follow-up GitHub issue for this discussion.
    • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
      • @coderabbitai generate unit testing code for this file.
      • @coderabbitai modularize this function.
    • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
      • @coderabbitai generate interesting stats about this repository and render them as a table.
      • @coderabbitai show all the console.log statements in this repository.
      • @coderabbitai read src/utils.ts and generate unit testing code.
      • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
      • @coderabbitai help me debug CodeRabbit configuration file.

    Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

    CodeRabbit Commands (invoked as PR comments)

    • @coderabbitai pause to pause the reviews on a PR.
    • @coderabbitai resume to resume the paused reviews.
    • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
    • @coderabbitai full review to do a full review from scratch and review all the files again.
    • @coderabbitai summary to regenerate the summary of the PR.
    • @coderabbitai resolve resolve all the CodeRabbit review comments.
    • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
    • @coderabbitai help to get help.

    Additionally, you can add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.

    CodeRabbit Configration File (.coderabbit.yaml)

    • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
    • Please see the configuration documentation for more information.
    • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

    Documentation and Community

    • Visit our Documentation for detailed information on how to use CodeRabbit.
    • Join our Discord Community to get help, request features, and share feedback.
    • Follow us on X/Twitter for updates and announcements.

    @Omri-Levy Omri-Levy changed the title Improved database querying for individuals profiles **untested** Improved database querying for individuals profiles May 7, 2024
    @github-actions github-actions bot added the enhancement New feature or request label May 7, 2024
    Copy link
    Contributor

    github-actions bot commented May 7, 2024

    PR Description updated to latest commit (02501a1)

    @github-actions github-actions bot changed the title **untested** Improved database querying for individuals profiles Improved database querying for individuals profiles May 7, 2024
    Copy link
    Contributor

    github-actions bot commented May 7, 2024

    PR Review 🔍

    ⏱️ Estimated effort to review [1-5]

    3, because the PR involves changes across multiple files and layers (controller, service, and repository). It requires understanding the data flow and the impact of removing methods, which could affect other parts of the application if not properly checked.

    🧪 Relevant tests

    No

    ⚡ Possible issues

    Possible Bug: The removal of getByEntityId and findFirstByEntityId might cause issues if other parts of the application still depend on these methods. This needs verification across the entire codebase.

    Performance Concern: While the new method of fetching workflowRuntimeData directly in the query is more efficient, it could lead to performance issues if the tags data is large and not properly indexed.

    🔒 Security concerns

    No

    Code feedback:
    relevant fileservices/workflows-service/src/case-management/controllers/case-management.controller.ts
    suggestion      

    Consider adding error handling for the new direct data fetching logic in CaseManagementController. If the workflowRuntimeData or tags are not available or fail to load, the application should handle this gracefully. [important]

    relevant lineworkflowRuntimeData: {

    relevant fileservices/workflows-service/src/case-management/controllers/case-management.controller.ts
    suggestion      

    Ensure that the tags array is properly indexed in the database to prevent potential performance bottlenecks due to the new direct query approach. [important]

    relevant linetags: true,

    Copy link
    Contributor

    github-actions bot commented May 7, 2024

    PR Code Suggestions ✨

    CategorySuggestions                                                                                                                                                       
    Best practice
    Add error handling for asynchronous operations to prevent crashes.

    Consider adding error handling for the asynchronous operation when mapping typedEndUsers.
    This will ensure that any exceptions thrown during the mapping process are caught and
    handled appropriately, preventing potential runtime errors that could crash the
    application.

    services/workflows-service/src/case-management/controllers/case-management.controller.ts [131-136]

     const formattedEndUsers = await Promise.all(
       typedEndUsers.map(async endUser => {
    -    const tag = endUser.workflowRuntimeData?.tags?.find(
    -      tag => !!tagToKyc[tag as keyof typeof tagToKyc],
    -    );
    -    const alerts = await this.alertsService.getAlertsByEntityId(endUser.id, projectId);
    +    try {
    +      const tag = endUser.workflowRuntimeData?.tags?.find(
    +        tag => !!tagToKyc[tag as keyof typeof tagToKyc],
    +      );
    +      const alerts = await this.alertsService.getAlertsByEntityId(endUser.id, projectId);
    +    } catch (error) {
    +      console.error('Error processing endUser:', error);
    +      // Handle the error appropriately
    +    }
       });
     
    Maintainability
    Ensure all usages of the removed method are refactored or replaced.

    Since the method findFirstByEntityId was removed, ensure that all its usages are
    refactored or replaced throughout the project. This is crucial to avoid runtime errors due
    to missing method calls.

    services/workflows-service/src/workflow/workflow-runtime-data.repository.ts [358-361]

    -async findFirstByEntityId<T extends Prisma.WorkflowRuntimeDataFindFirstArgs>(
    -  entityId: string,
    -  projectIds: TProjectIds,
    -  args?: Prisma.SelectSubset<T, Prisma.WorkflowRuntimeDataFindFirstArgs>,
    -) {
    -  return await this.prisma.workflowRuntimeData.findFirst(
    -    this.scopeService.scopeFindFirst(
    -      {
    -        ...args,
    -        where: {
    -          ...args?.where,
    -          OR: [
    -            {
    -              endUserId: entityId,
    -            },
    -            {
    -              businessId: entityId,
    -            },
    -          ],
    -        },
    -      },
    -      projectIds,
    -    ),
    -  );
    -}
    +// Ensure all usages of findFirstByEntityId are refactored or replaced.
    +// Example replacement or refactoring code should be implemented here.
     

    Copy link
    Contributor

    @coderabbitai coderabbitai bot left a comment

    Choose a reason for hiding this comment

    The reason will be displayed to describe this comment to others. Learn more.

    Actionable comments posted: 1

    Out of diff range and nitpick comments (1)
    services/workflows-service/src/workflow/workflow.service.ts (1)

    Line range hint 1-1: Ensure all necessary imports are maintained and unused imports are removed to keep the code clean.

    Review Details

    Configuration used: CodeRabbit UI
    Review profile: CHILL

    Commits Files that changed from the base of the PR and between 64fd9e0 and 02501a1.
    Files selected for processing (3)
    • services/workflows-service/src/case-management/controllers/case-management.controller.ts (3 hunks)
    • services/workflows-service/src/workflow/workflow-runtime-data.repository.ts (1 hunks)
    • services/workflows-service/src/workflow/workflow.service.ts (1 hunks)
    Files skipped from review due to trivial changes (1)
    • services/workflows-service/src/workflow/workflow-runtime-data.repository.ts
    Additional comments not posted (26)
    services/workflows-service/src/case-management/controllers/case-management.controller.ts (1)

    95-99: LGTM! Ensure the integration of workflowRuntimeData with tags does not affect other functionalities.

    Verification successful

    The search results indicate extensive usage of workflowRuntimeData across various files in the workflows-service directory. This includes multiple interactions with the WorkflowRuntimeDataRepository for operations like findById, findMany, updateById, and others. The integration of the workflowRuntimeData with tags in the CaseManagementController seems consistent with the usage patterns observed in other parts of the system, particularly in how data is handled and queried.

    Given the widespread and consistent handling of workflowRuntimeData across the service, it appears that the integration of the tags field in the CaseManagementController is aligned with existing practices and should not adversely affect other functionalities. This suggests that the original review comment's concern about verifying the integration has been addressed by the evidence provided in the script outputs.

    Conclusion:
    The integration of workflowRuntimeData with tags in the CaseManagementController is consistent with the handling of similar data across the system. There are no indications of potential issues from the current integration based on the script outputs.

    • File: services/workflows-service/src/case-management/controllers/case-management.controller.ts
    • Lines: 95-99
    Scripts Executed

    The following scripts were executed for the analysis:


    Script:

    #!/bin/bash
    # Description: Verify the integration of `workflowRuntimeData` with `tags` in other parts of the system.
    
    # Test: Search for usage of `workflowRuntimeData`. Expect: Proper handling in all occurrences.
    rg --type typescript $'workflowRuntimeData'
    

    Length of output: 77



    Script:

    #!/bin/bash
    # Description: Verify the integration of `workflowRuntimeData` with `tags` in other parts of the system.
    
    # Test: Search for usage of `workflowRuntimeData`. Expect: Proper handling in all occurrences.
    rg --type ts 'workflowRuntimeData'
    

    Length of output: 24650

    services/workflows-service/src/workflow/workflow.service.ts (25)

    Line range hint 95-104: Function createWorkflowDefinition is implemented correctly and follows good practices.


    Line range hint 106-123: Function cloneWorkflowDefinition is implemented correctly and follows good practices.


    Line range hint 125-128: Function getWorkflowRuntimeDataById is implemented correctly and follows good practices.


    Line range hint 130-135: Function getWorkflowRuntimeDataByIdAndLockUnscoped is implemented correctly and follows good practices.


    Line range hint 137-160: Function getWorkflowByIdWithRelations is implemented correctly and handles complex queries efficiently.


    Line range hint 162-245: Function formatWorkflow is well-implemented, handling complex formatting and transformations efficiently.


    Line range hint 247-297: Function persistChildEvent is well-implemented, handling complex event persistence and context management efficiently.


    Line range hint 299-302: Function getWorkflowDefinitionById is implemented correctly and follows good practices.


    Line range hint 304-309: Function getWorkflowsByState is implemented correctly and handles state-based filtering efficiently.


    Line range hint 311-321: Function listActiveWorkflowsRuntimeStates is implemented correctly and efficiently lists active workflows with their states.


    Line range hint 323-394: Function listWorkflowRuntimeDataWithRelations is well-implemented, handling complex queries and pagination efficiently.


    Line range hint 396-423: Function formatWorkflowsRuntimeData is implemented correctly and efficiently formats workflow data based on type.


    Line range hint 425-429: Function listWorkflowRuntimeDataByUserId is implemented correctly and follows good practices.


    Line range hint 431-439: Function listFullWorkflowDataByUserId is implemented correctly and efficiently fetches full workflow data with necessary details.


    Line range hint 441-474: Function listRuntimeData is well-implemented, handling complex queries, pagination, and sorting efficiently.


    Line range hint 476-497: Function _resolveOrderByParams is implemented correctly and efficiently constructs order by clauses based on parameters.


    Line range hint 499-515: Function workflowsRuntimeListItemsFactory is implemented correctly and efficiently transforms query results into models.


    Line range hint 517-522: Function listWorkflowDefinitions is implemented correctly and follows good practices.


    Line range hint 524-582: Function updateDecisionAndSendEvent is well-implemented, handling complex decision updating and event sending efficiently.


    Line range hint 584-670: Function updateDocumentDecisionById is well-implemented, handling complex document decision updating efficiently.


    Line range hint 672-774: Function updateDocumentById is well-implemented, handling complex document updating efficiently.


    Line range hint 776-798: Function updateDocumentInContext is implemented correctly and efficiently updates documents in the context based on the method.


    Line range hint 800-812: Function getDocuments is implemented correctly and efficiently retrieves documents from the context based on the method.


    Line range hint 814-827: Function updateDirectorDocument is implemented correctly and efficiently updates director documents in the context.


    Line range hint 829-837: Function getDirectorsDocuments is implemented correctly and efficiently retrieves director documents from the context.

    Copy link

    vercel bot commented Jul 9, 2024

    The latest updates on your projects. Learn more about Vercel for Git ↗︎

    Name Status Preview Comments Updated (UTC)
    ballerine-docs ✅ Ready (Inspect) Visit Preview 💬 Add feedback Jul 9, 2024 9:26am

    @tomer-shvadron tomer-shvadron enabled auto-merge (squash) July 9, 2024 09:30
    @tomer-shvadron tomer-shvadron merged commit 304062e into dev Jul 9, 2024
    11 checks passed
    @tomer-shvadron tomer-shvadron deleted the omri-levy/refactor/individuals-profiles-performance branch July 9, 2024 09:46
    tomer-shvadron added a commit that referenced this pull request Jul 9, 2024
    * feat: setting isCaseOverviewEnabled default to false (#2505)
    
    * Improved database querying for individuals profiles (#2360)
    
    * refactor(workflows-service): now fetching workflows via endusers instead of using repository
    
    * refactor(workflows-service): improved performance on profiles query
    
    * refactor(workflow-service): renamed iterable name
    
    ---------
    
    Co-authored-by: Omri Levy <61207713+Omri-Levy@users.noreply.github.com>
    Omri-Levy added a commit that referenced this pull request Jul 9, 2024
    * fix(backoffice): increase profile timeout to 30s
    
    * Dev --> SB (#2506)
    
    * feat: setting isCaseOverviewEnabled default to false (#2505)
    
    * Improved database querying for individuals profiles (#2360)
    
    * refactor(workflows-service): now fetching workflows via endusers instead of using repository
    
    * refactor(workflows-service): improved performance on profiles query
    
    * refactor(workflow-service): renamed iterable name
    
    ---------
    
    Co-authored-by: Omri Levy <61207713+Omri-Levy@users.noreply.github.com>
    
    ---------
    
    Co-authored-by: Alon Peretz <Alonp99@gmail.com>
    Co-authored-by: Tomer Shvadron <tomers@ballerine.com>
    @Omri-Levy Omri-Levy mentioned this pull request Nov 4, 2024
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    Projects
    None yet
    Development

    Successfully merging this pull request may close these issues.

    2 participants