Implement logic to embed topics and content

[akolson]

Overview

This task involves implementing the logic that interacts with the backend to perform topic and content embedding capabilities within the recommendations adapter

Description and outcomes

• The BackendRequest class
  • Locate the class. It resides in contentcuration/automation/utils/appnexus/base.py
  • The constructor accepts
    • headers: Request headers
    • params: Query parameters for the request.
    • data: Request data
    • json: Request json.
    • timeout: Max. time of response
    • kwargs: Additional keyword arguments.
  • Add the following parameters to the constructor:
    • path: The endpoint path for the request.
    • method: HTTP method for the request (e.g., GET, POST).
• Create new classes in contentcuration/contentcuration/utils/recommendations.py.
  • EmbedTopicsRequest and EmbedContentRequest that inherit from RecommendationsBackendRequest. RecommendationsBackendRequest inherits from BackendRequest
  • Set path and method with appropriate endpoint paths and methods
    • Embedding topics: /embed-topics, POST
    • Embedding content: /embed-content, POST
• The RecommendationsAdapter class
  • Locate the class. It resides in contentcuration/contentcuration/utils/recommendations.py.
  • Create new methods to embed topics and content
    • embed_topics(topics: Dict[str, Any]) -> EmbeddingsResponse: Embeds topics based on provided topic(s).
    • embed_content(self, nodes: List[ContentNode]) -> EmbeddingsResponse: Embeds content based on provided list of content nodes.
  • Inside embed_topics:
    • Attempt a connection to the backend using Backend.connect()
    • Use EmbedTopicsRequest to construct a specific request object based on the provided topic. For example;

    embed_topics_request = EmbedTopicsRequest(
            headers={}, 
            params={}, 
            json=topics,
    )

    • Return the request using Backend.make_request().

    return self.backend.make_request(embed_topics_request)

  • Inside embed_content:
    • Attempt a connection to the backend using Backend.connect()
    • Create a new method extract_content(node: ContentNode) -> Dict[str, Any]] that extracts content from the provided node (see assumptions and scope).
    • For each node, extract required content, and finally build the request json body.

      resources = []
       for node in nodes:
           resource = self.extract_content(node)
           resources.append(resource)
      
       body = {
           'resources': resources,
           'metadata': {}
       }

    • Use EmbedContentRequest to construct a specific request object based on the body. For example;

    embed_content_request = EmbedContentRequest(
            headers={},
            params={},
            json=body
    )

    • Return the request using Backend.make_request().

    return self.backend.make_request(embed_content_request)

• Error handling
  • The make_requests method raises exceptions, based on categories implemented in Implement the core functionality for the connect and make_request methods of the Backend abstract class. #4447
  • Implement the error handling based on the categorizations above. if necessary(See Error handling below)
  • Return the EmbeddingsResponse for the raised exception.

    try:
        ....
    except Exception as e:
        return EmbeddingsResponse(e)

Acceptance Criteria

• The BackendRequest class is updated with arguments(headers, params, json, data, timeout) necessary to make HTTP requests. Additionally, kwargs have been added.
• EmbedTopicsRequest and EmbedContentRequest classes inherit from RecommendationsBackendRequest and appropriate paths and methods are set.
• The RecommendationsAdapter class has methods embed_topics and embed_content that:
  • Attempt a connection to the backend using Backend.connect().
  • Construct appropriate request objects (EmbedTopicsRequest or EmbedContentRequest) based on the provided request data.
  • Call the make_request method with the constructed request objects.
• Error handling is implemented accordingly.
• Tests are written to verify correctness of added code

Assumptions and Dependencies

• extract_content method returns an empty dictionary.
• Whereas some variable and method names already exist in the code, others are presented as suggestions. Feel free to adapt these names as necessary, for clarity.
• Is blocked
  • Implement the core functionality for the connect and make_request methods of the Backend abstract class. #4447
• Blocks
  • Update the embed_content in RecommendationsAdapter #4455

Scope

The scope of this task is limited to;

• Implementing the logic to embed topics and content

This task doesn’t include;

• Implementing the extract_content method logic.

Accessibility Requirements

NA

Resources

• Embed topic request
• Embed content request

[bjester]

There's a search-recommendations branch which this can be targeted against

[akolson]

Closed by #4588