Skip to content

Using ChatGPT, Embeddings, and HyDE to Improve Search Results

Jump to bottom
Richard Hightower edited this page Jul 11, 2023 · 2 revisions

Using ChatGPT, Embeddings, and HyDE to Improve Search Results

Introduction

In today's fast-paced business environment, staying ahead of the competition is imperative. An efficient search engine that can provide accurate information to your customers or employees can make all the difference. However, building and maintaining a robust search engine can be a challenge. In this dev notebook, let’s explore how ChatGPT, Embeddings, and HyDE can help you improve your search results.

Combining ChatGPT with retrieval and re-ranking methods, you can achieve accurate, relevant, and fast search results that will set you apart from your competitors. This approach also facilitates seamless integration with existing search engines, making it an ideal way to improve search engine performance for businesses of all sizes. As CTOs, CIOs, engineering managers, and software engineers, you have much to gain by implementing this approach and making your search engine more efficient.

Overview

ChatGPT is a deep learning model that we will use to rank content returned from a search engine on a hypothetical answer. It creates a hypothetical document based on a given query and then embeddings for this hypothetical answer. The system then sorts the results using the dot calculation to get the cosine similarities to score and sort articles from the search service. Articles are ranked by relevance as determined by the cosine similarity of their embeddings versus the hypothetical answer. Using this approach, ChatGPT can assist in improving search systems.

There are two ways to retrieve information using GPT: Mimicking Human Navigation and Retrieval with embedding. Retrieval with Embeddings can be done by calculating embeddings for query results and an ideal user answer. Then, the most related content, as measured by cosine similarity to the hypothetical answer (HyDE), is retrieved. Combining these approaches and drawing inspiration from re-ranking methods can improve search accuracy.

One of the key benefits of this approach is its ability to be implemented on top of any existing search systems. Combining this with your current system can improve your search engine's accuracy and speed. This approach can be applied to search engines that use Elasticsearch, Solr, or any custom search engine application.

Dive In

This dev notebook shows how ChatGPT employs the Hypothetical Document Embeddings [HyDE](https://arxiv.org/abs/2212.10496) to rank content on a hypothetical answer. We use the HyDE by having ChatGPT create a hypothetical document based on a given query.

Then we use ChatGPT to create embeddings hypothetical answer, then sort the results using the dot calculation to get the cosine_similarities and rank the articles by relevance as determined by the cosine_similarities of their embeddings vs. the hypothetical answer.

Looking for relevant information can be a challenging task, but fear not. This dev notebook adapts a [ChatGPT cookbook for improved search](https://github.com/openai/openai-cookbook/blob/main/examples/Question_answering_using_a_search_API.ipynb) to Java, then breaks the steps down and explains each step. In this developer notebook, we explore a way to improve existing search systems with various AI techniques that will help us. We explore a way to filter through the data using various AI techniques to help us put our data through Boot camp so the data becomes information.

There are two ways to retrieve information using GPT:

  1. Mimicking Human Navigation: GPT initiates a search, scores the results, and adjusts the query. It can also follow up on specific search results to form a chain of thought, much like a human user would do. We can even ask GPT to generate sample queries and sample answers.
  2. Retrieval with Embeddings: We can calculate embeddings for our query and an ideal user answer and then retrieve the most related content as measured by cosine similarity to the hypothetical answer ([HyDE](https://arxiv.org/abs/2212.10496) ).

Combining these approaches and drawing inspiration from re-ranking methods gives us a simple pattern to improve search. This approach can be implemented on top of any existing search systems.

Steps:

  • The first step is to search by asking the user a question.
  • Then we use GPT to generate a list of potential queries based on the question.
  • Then we execute the search queries and retrieve relevant articles.
  • We score each article based on the embeddings for each article compared to the hypothetical ideal answer using dot to calculate cosine_similarities
  • Then we sort and filter the articles based on the similarity obtained from the embeddings of the articles vs. the embedding of the ideal answer.
  • Finally, generate an answer to the user's question, including references and links.

This technique is fast and can be added to a search function you already have without managing a vector database. We will use the [News API](https://newsapi.org/) as our search engine stand-in to show how this fits together. Since our examples are all in Java, we [use JAI, the Java Open AI API client lib](https://github.com/RichardHightower/jai).

💡 **News API - search engine for searching the news**

[News API](https://newsapi.org/) is a simple and easy-to-use API that returns JSON metadata for headlines and articles from various news sources. It provides live breaking news headlines and articles from over 70,000 news sources and blogs worldwide, including CNN, BBC News, The New York Times, and more. The API is free for non-commercial purposes and offers a range of endpoints to query news data, including top headlines, everything, sources, and more.

To use the News API, you must sign up for an API key, a unique identifier that allows you to access the API. Once you have an API key, you can request HTTP to the API endpoints to retrieve news data in JSON format. The API supports a range of parameters that allow you to filter and sort news data based on various criteria, such as language, country, category, and more.

Main Method is the main flow.

Let’s dive right in.

I created a Java class named WhoWonUFC290, incorporating question-answering functionality using a search API and re-ranking techniques. The question we will answer is “Who won Main card fights in UFC 290? Tell us who won. Are there any new champions? Where are they from?”

The example aims to demonstrate a method of augmenting existing search systems with AI techniques to improve information retrieval.

Notice it covers all of the steps we talked about thus far. Here is the main method of doing all the steps. We will break this down step by step.

public static void main(String... args) {
    try {
        // Generating queries based on user question
        String queriesJson = jsonGPT(QUERIES_INPUT.replace("{USER_QUESTION}", USER_QUESTION));
        List<String> queries = JsonParserBuilder.builder().build().parse(queriesJson)
            .getObjectNode().getArrayNode("queries")
            .filter(node -> node instanceof StringNode)
            .stream().map(Object::toString).collect(Collectors.toList());

        // Generating a hypothetical answer and its embedding
        var hypotheticalAnswer = hypotheticalAnswer();
        System.out.println(hypotheticalAnswer);
        var hypotheticalAnswerEmbedding = embeddings(hypotheticalAnswer);

        // Searching news articles based on the queries
        List<ArrayNode> results = queries.subList(0, 10).stream()
            .map(WhoWonUFC290::searchNews).collect(Collectors.toList());

        // Extracting relevant information from the articles
        List<ObjectNode> articles = results.stream().map(arrayNode ->
            arrayNode.map(on -> on.asCollection().asObject()))
            .flatMap(List::stream).collect(Collectors.toList());

        // Extracting article content and generating embeddings for each article
        List<String> articleContent = articles.stream().map(article ->
            String.format("%s %s %s", article.getString("title"),
            article.getString("description"), article.getString("content").substring(0, 100)))
            .collect(Collectors.toList());
        List<float[]> articleEmbeddings = embeddings(articleContent);

        // Calculating cosine similarities between the hypothetical answer embedding and article embeddings
        List<Float> cosineSimilarities = articleEmbeddings.stream()
            .map(articleEmbedding -> dot(hypotheticalAnswerEmbedding, articleEmbedding))
            .collect(Collectors.toList());

        // Creating a set of scored articles based on cosine similarities
        Set<ScoredArticle> articleSet = IntStream.range(0,
            Math.min(cosineSimilarities.size(), articleContent.size()))
            .mapToObj(i -> new ScoredArticle(articles.get(i), cosineSimilarities.get(i)))
            .collect(Collectors.toSet());

        // Sorting the articles based on their scores
        List<ScoredArticle> sortedArticles = new ArrayList<>(articleSet);
        Collections.sort(sortedArticles, (o1, o2) -> Float.compare(o2.getScore(), o1.getScore()));

        // Printing the top 5 scored articles
        sortedArticles.subList(0, 5).forEach(s -> System.out.println(s));

        // Formatting the top results as JSON strings
        String formattedTopResults = String.join(",\n", sortedArticles.stream().map(sa -> sa.getContent())
            .map(article -> String.format(Json.niceJson("{'title':'%s', 'url':'%s', 'description':'%s', 'content':'%s'}\n"),
            article.getString("title"), article.getString("url"), article.getString("description"),
            getArticleContent(article))).collect(Collectors.toList()).subList(0, 10));

        // Generating the final answer with the formatted top results
        String finalAnswer = jsonGPT(ANSWER_INPUT.replace("{USER_QUESTION}", USER_QUESTION)
            .replace("{formatted_top_results}", formattedTopResults));

        System.out.println(finalAnswer);
    } catch (Exception e) {
        e.printStackTrace();
    }
}

The main method does the following, which we will explain in detail in the next few sections:

  1. Generates queries based on the user's question using a JSON template (QUERIES_INPUT) and the USER_QUESTION variable.
  2. Generates a hypothetical answer and its embedding.
  3. Searches for news articles based on the generated queries using the searchNews method, which hits the news API.
  4. Extracts relevant information from the retrieved articles (title, description, and partial content).
  5. Generates embeddings for each article's relevant information.
  6. Calculates cosine similarities between the hypothetical answer embedding and the article embeddings.
  7. Creates a set of scored articles based on the cosine similarities to the hypothetical answer embedding.
  8. Sorts the articles based on their scores.
  9. Prints the top 5 scored articles.
  10. Format the top results as JSON strings so the content can be included in the final question.
  11. Generates the final answer using a JSON template (ANSWER_INPUT) with the user's question and the formatted top results.
  12. Prints the final answer.

This is the basics of the technique to sort and rank search results (articles) using ChatGPT based on the relevance of their content to a hypothetical answer. It utilizes embeddings and cosine similarities to determine the similarity between the articles and the hypothetical answer. The result is a list of top-ranked articles that can be used to provide an informative answer to the user's question.

Please note that the code references certain variables (QUERIES_INPUT, USER_QUESTION, hypotheticalAnswer, jsonGPT, embeddings, etc.) and methods (searchNews, dot, getArticleContent) that are not shown in the provided code snippet. We will show these as we cover the basics.

The general flow of the app

Untitled

Queries Generation

This part of the example generates queries based on the user's question using a JSON template (QUERIES_INPUT) and the USER_QUESTION variable.

public class WhoWonUFC290 {

   private static final String QUERIES_INPUT =  "Generate an array of search queries that are " +
            "relevant to this question.\n" +
            "Use a variation of related keywords for the queries, trying to be as general as possible.\n" +
            "Include as many queries as you can think of, including and excluding terms.\n" +
            "For example, include queries like ['keyword_1 keyword_2', 'keyword_1', 'keyword_2'].\n" +
            "Be creative. The more queries you include, the more likely you are to find relevant results.\n" +
            "\n" +
            "User question: {USER_QUESTION}\n" +
            "\n" +
            "Format: {{\"queries\": [\"query_1\", \"query_2\", \"query_3\"]}}";

    private static final String USER_QUESTION = "Who won Main card fights in UFC 290? Tell us who won. " +
            "Are there any new champions? Where are they from?";

    public static void main(String... args) {
        try {
            // Generating queries based on user question
            String queriesJson = jsonGPT(QUERIES_INPUT.replace("{USER_QUESTION}", USER_QUESTION));
            List<String> queries = JsonParserBuilder.builder().build().parse(queriesJson)
                    .getObjectNode().getArrayNode("queries")
                    .filter(node -> node instanceof StringNode)
                    .stream().map(Object::toString).collect(Collectors.toList());

Let’s walk through this:

  1. Constants:
    • QUERIES_INPUT: A string template containing instructions on generating search queries based on a user's question. It includes placeholders for the user question.
    • USER_QUESTION: A string representing the user's question.
  2. main Method:
    • This method is the entry point of the program.
    • It begins by generating queries based on the user's question using the QUERIES_INPUT template. The user question placeholder is replaced with the actual user question.
    • The generated queries are parsed from JSON format and stored in a list.
    • The method jsonGPT is called to perform some JSON-related operations that are not shown in the provided code snippet.
    • The list of queries is filtered and transformed into a list of strings.

Since we will use jsonGPT a lot, let’s specify how the jsonGPT is implemented. jsonGPT calls GPT and returns a JSON string.

The provided code snippet defines a method named jsonGPT that interacts with the OpenAI API to generate responses in a chat-based manner. Here's a breakdown of the code:

public static String jsonGPT(String input) {

    final var client = OpenAIClient.builder().setApiKey(System.getenv("OPENAI_API_KEY")).build();

    final var chatRequest = ChatRequest.builder()
                .addMessage(Message.builder().role(Role.SYSTEM)
                                       .content("All output shall be JSON").build())
                .addMessage(Message.builder().role(Role.USER).content(input).build())
                .build();

    final var chat = client.chat(chatRequest);

    if (chat.getResponse().isPresent()) {
            return chat.getResponse().get().getChoices().get(0)
                                     .getMessage().getContent();
    } else {
            System.out.println("" + chat.getStatusCode().orElse(666) + "" + chat.getStatusMessage().orElse(""));
            throw new IllegalStateException();
    }
}
  1. The jsonGPT starts by building an instance of the OpenAIClient using the provided API key from the system environment variables.
  • It creates a ChatRequest object, which represents the request to the Open AI API, by adding two Message objects to it.
  • The first Message object represents the system message with the role set to Role.SYSTEM and the content set to "All output shall be JSON".
  • The second Message object represents the user message with the role set to Role.USER and the content set to the input passed to the jsonGPT method.
  • The ChatRequest object is then passed to the client.chat() method to make the API call and obtain a response.
  • The response is checked if it is present using chat.getResponse().isPresent().
  • If the response is present, it retrieves the first choice from the response and gets the content of the message within the choice using chat.getResponse().get().getChoices().get(0).getMessage().getContent().
  • If the response is not present, it outputs an error message including the status code and status message and throws an IllegalStateException.

The jsonGPT method sends a chat request to the OpenAI API and retrieves the response. It takes an input string as a parameter, sends the input to the API, and returns the content of the first choice in the response as a string.

Please note that the code snippet does not show the imports or the class where the jsonGPT method resides. Additionally, it assumes the availability of the OpenAI API client and relevant dependencies. We are [using JAI, the Java Open AI API client lib](https://github.com/RichardHightower/jai). The full code listing is below and this example is included with [JAI](https://github.com/RichardHightower/jai).

Sample Queries Generated by ChatGPT

UFC 290 Main card winners
Who were the winners in Main card fights UFC 290?
UFC 290 Main card champions
List of winners in UFC 290 Main card fights
UFC 290 Main card results
UFC 290 Main card winners nationality
UFC 290 Main card champions country
Who won the fights in Main card of UFC 290?
Tell me the winners of Main card fights in UFC 290
UFC 290 Main card winners and their countries
UFC 290 Main card fight results
New champions in UFC 290 Main card fights
Who are the winners in Main card of UFC 290?
UFC 290 Main card winners list
UFC 290 Main card champions nationality

Hypothetical Answer Generation

This part of the example uses ChatGPT to generate a hypothetical answer and then calculate its embedding. The example integrates with the OpenAI API to generate a hypothetical answer using a chat-based language model. It replaces placeholders in a template with the user's question and retrieves the generated answer as a response. The hypothetical answer embedding is later used to filter articles we retrieve from the new search.

public class WhoWonUFC290 {

  private static final String HA_INPUT ="Generate a hypothetical answer to the user's question. " +
            "This answer will be used to rank search results. \n" +
            "Pretend you have all the information you need to answer, but don't use any actual facts. " +
            "Instead, use placeholders\n" +
            "like NAME did something, or NAME said something at PLACE. \n" +
            "\n" +
            "User question: {USER_QUESTION}\n" +
            "\n" +
            "Format: {{\"hypotheticalAnswer\": \"hypothetical answer text\"}}";
   
   private static final String USER_QUESTION = "Who won Main card fights in UFC 290? Tell us who won. " +
            "Are there any new champions? Where are they from?";

   public static String jsonGPT(String input) {

        final var client = OpenAIClient.builder().setApiKey(System.getenv("OPENAI_API_KEY")).build();

        final var chatRequest = ChatRequest.builder()
                .addMessage(Message.builder().role(Role.SYSTEM).content("All output shall be JSON").build())
                .addMessage(Message.builder().role(Role.USER).content(input).build())
                .build();

        final var chat = client.chat(chatRequest);

        if (chat.getResponse().isPresent()) {
            return chat.getResponse().get().getChoices().get(0).getMessage().getContent();
        } else {
            System.out.println("" + chat.getStatusCode().orElse(666) + "" + chat.getStatusMessage().orElse(""));
            return "";
        }
    }

    public static String hypotheticalAnswer() {
        final var input = HA_INPUT.replace("{USER_QUESTION}",
                USER_QUESTION );
        return jsonGPT(input);
    }

    public static void main(String... args) {
        try {
            // Generating queries based on user question
						...            

            // Generating a hypothetical answer and its embedding
            var hypotheticalAnswer = hypotheticalAnswer();
            System.out.println(hypotheticalAnswer);
            var hypotheticalAnswerEmbedding = embeddings(hypotheticalAnswer);
           ...

The above example represents a Hypothetical Answer Generation functionality within the WhoWonUFC290 class. It generates a hypothetical answer and its embedding. Here's a breakdown of the code:

  1. Constants:
    • HA_INPUT: A string template that provides instructions for generating a hypothetical answer to the user's question. It includes placeholders for the user question.
    • USER_QUESTION: A string representing the user's question.
  2. jsonGPT Method:
    • This method utilizes an OpenAI API client to interact with a chat-based language model.
    • It constructs a ChatRequest object containing system and user messages.
    • The user message includes the input string passed to the method.
    • The method sends the chat request to the OpenAI API and retrieves the response, which includes a JSON-formatted message content.
    • The retrieved message content is returned as a string.
  3. hypotheticalAnswer Method:
    • This method generates a hypothetical answer by replacing the user question placeholder in the HA_INPUT template with the actual user question.
    • It calls the jsonGPT method, passing the modified template as input.
    • The jsonGPT method retrieves the response from the chat-based language model and returns the hypothetical answer as a string.
  4. main Method:
    • This method is the entry point of the program.
    • It starts by generating queries based on the user's question (code not shown).
    • Next, it calls the hypotheticalAnswer method to generate a hypothetical answer.
    • The hypothetical answer is printed to the console.
    • The code does not provide the implementation of the embeddings method, which is likely responsible for generating embeddings for the hypothetical answer.

hypotheticalAnswer

There were some thrilling battles in the Main card fights of UFC 290. 
NAME emerged victorious in the first fight, showcasing impressive 
skills and defeating their opponent with a spectacular knockout. 
In the second fight, NAME came out on top, 
displaying dominant grappling techniques that led to a submission victory. 
In the third fight, NAME showed incredible striking abilities, 
outclassing their opponent and securing a unanimous decision win. 
However, the winners of the Main card fights hailed from diverse backgrounds. 
NAME, the winner of the first fight, is from PLACE1. NAME2, 
who triumphed in the second fight, hails from PLACE2. Lastly, 
the winner of the third fight, NAME3, is from PLACE3.
💡 **Text embedding**

Text embedding is a technique used in natural language processing to represent words, phrases, or documents as vectors of numerical values. These vectors suit various machine learning algorithms, including clustering, classification, and information retrieval. The embedding process maps each word, phrase, or document to a vector in a high-dimensional space, where the distance between vectors represents the degree of semantic similarity between the corresponding words, phrases, or documents. The embedding process is usually performed using deep neural networks trained on large collections of text data. These networks learn to map similar words or phrases to similar vectors in the high-dimensional space, allowing for efficient representation and processing of text data.

ChatGPT’s Open AI API has an embedding API to get text embeddings from your text. Here is an example of getting an embedding which we will use later to filter the articles returned from the new search.

public class WhoWonUFC290 {    
    ...
    public static float[] embeddings(String input) {
        return embeddings(List.of(input)).get(0);
    }
    public static List<float[]> embeddings(List<String> input) {
        final var client = OpenAIClient.builder()
                      .setApiKey(System.getenv("OPENAI_API_KEY")).build();
        var embedding = client.embedding(EmbeddingRequest.builder()
                    .model("text-embedding-ada-002").input(input).build());

        if (embedding.getResponse().isPresent()) {
            return embedding.getResponse().get().getData().stream()
              .map(Embedding::getEmbedding).collect(Collectors.toList());
        } else {
            System.out.println("" + embedding.getStatusCode().orElse(666) + "" + embedding.getStatusMessage().orElse(""));
            throw new IllegalStateException();
        }
    }
    ...

    public static void main(String... args) {
        try {
            // Generating queries based on user question
            ...
            // Generating a hypothetical answer and its embedding
            var hypotheticalAnswer = hypotheticalAnswer();
            System.out.println(hypotheticalAnswer);
            var hypotheticalAnswerEmbedding = embeddings(hypotheticalAnswer);

Let’s break these down a bit.

  1. embeddings Method (Overloaded):
    • The embeddings method is overloaded to accept either a single input string or a list of strings.
    • When a single input string is provided, the method converts it into a list and calls the overloaded version.
    • This ensures consistency in handling input and allows the method to return a single embedding.
  2. embeddings Method (List Version):
    • This method generates embeddings for a list of input strings using an OpenAI API client.
    • It constructs an EmbeddingRequest object, specifying the model and input strings.
    • The method sends the embedding request to the OpenAI API and retrieves the response.
    • If the response is present, the method extracts the embeddings and returns them as a list of float arrays.
    • If the response is not present, an error message is printed, and an IllegalStateException is thrown.
  3. main Method:
    • This method is the entry point of the program.
    • It calls the embeddings method after generating queries and the hypothetical answer (as shown in the previous example).
    • The hypotheticalAnswer string is passed to the embeddings method, which generates embeddings for the hypothetical answer.
    • The embeddings are assigned to the hypotheticalAnswerEmbedding variable.

The embeddings methods utilize the OpenAI API client to generate embeddings for input strings. The embeddings capture the semantic representations of the text. The code handles both single and multiple input strings, returning the corresponding embeddings as float arrays or a list of float arrays.

News Article Search

Next, we need some results to filter. We will use the News Search. The example searches for news articles based on the generated queries using the searchNews method as follows.

public class WhoWonUFC290 {    
    ...
    public static void main(String... args) {
        try {
            // Generating queries based on user question
            ...

            // Generating a hypothetical answer and its embedding
						...

            // Searching news articles based on the queries

            List<ArrayNode> results = queries.subList(0, 10).stream()
                    .map(WhoWonUFC290::searchNews).collect(Collectors.toList());

           ...
  1. main Method (Continued):
    • After generating queries and the hypothetical answer with its embedding (as shown in the previous example), the code searches for news articles.
    • It creates an empty list, results, to store the search results.
  2. Searching News Articles:
    • The code utilizes the stream function on the queries list to perform a series of operations for each query.
    • Each query in the list calls the searchNews method from the WhoWonUFC290 class.
    • The searchNews method is not shown in the provided code snippet but is likely responsible for performing the actual search operation.
    • The search results are collected into a list of ArrayNode objects, which are likely representations of the news articles.

This part of the example focuses on the step of searching news articles based on the generated queries. It uses Java’s stream function and the map operation to apply the searchNews method to each query. The resulting articles are collected into a list of ArrayNode objects stored in the results variable.

The searchNews method is defined as follows:

 public class WhoWonUFC290 {    
    ...
    
    public static ArrayNode searchNews(final String query) {
        final var end = Instant.now();
        final var start = end.minus(java.time.Duration.ofDays(5));
        return searchNews(query, start, end, 5);
    }

    public static ArrayNode searchNews(final String query, final Instant start, final Instant end, final int pageSize) {
        System.out.println(query);
        try {

            String url = "https://newsapi.org/v2/everything?q=" + 
										URLEncoder.encode(query, StandardCharsets.UTF_8)
                    + "&apiKey=" + System.getenv("NEWS_API_KEY") 
										+ "&language=en" + "&sortBy=relevancy"
                    + "&from=" + dateStr(start) + "&to=" + dateStr(end) 
										+ "&pageSize=" + pageSize;

            HttpClient httpClient = HttpClient.newHttpClient();
            HttpResponse<String> response = httpClient.send(
										HttpRequest.newBuilder().uri(URI.create(url))
                    .GET().setHeader("Content-Type", "application/json").build(), 
										HttpResponse.BodyHandlers.ofString());

            if (response.statusCode() >= 200 && response.statusCode() < 299) {
                return JsonParserBuilder.builder().build().parse(response.body()).atPath("articles").asCollection().asArray();
            } else {
                throw new IllegalStateException(" status code " + response.statusCode() + " " + response.body());
            }
        } catch (Exception ex) {
            throw new IllegalStateException(ex);
        }
    }

This part of the example introduces the searchNews method within the WhoWonUFC290 class. This method searches news articles based on a given query and a specified time range. Here's a breakdown of the code:

  1. searchNews Method (First Overload):
    • The first overload of the searchNews method takes a single parameter, query, representing the search query.
    • It obtains the current timestamp (Instant.now()) and sets the start timestamp to five days before the current time.
    • The method then calls the second overload of searchNews, passing the query, start timestamp, end timestamp, and a default page size of 5.
  2. searchNews Method (Second Overload):
    • The second overload of the searchNews method takes four parameters: query, start, end, and pageSize.
    • It constructs the URL for the news API using the provided parameters.
    • The URL includes the query, API key, language, sorting criteria, time range, and page size.
    • An HTTP client is created using HttpClient.newHttpClient().
    • An HTTP GET request is sent to the news API with the constructed URL, and the response is obtained using HttpResponse.BodyHandlers.ofString().
    • If the response status code indicates success (between 200 and 299), the response body is parsed as JSON using JsonParserBuilder.
    • The parsed JSON response is navigated to the "articles" path and returned as an ArrayNode.
    • If the response status code is outside the success range, an IllegalStateException is thrown.

The searchNews method enables searching news articles based on a query and a time range. It interacts with the News API by constructing the appropriate URL, sending an HTTP request, and handling the response. The method parses the JSON response and returns the "articles" section as an ArrayNode.

Please note that the code relies on certain environment variables such as "NEWS_API_KEY" and utilizes classes and methods (JsonParserBuilder, URLEncoder, HttpClient, HttpResponse, etc.).

Relevant Information Extraction

Next, we need to extract relevant information from the retrieved articles (title, description, and partial content) that we can use to create embeddings to score the articles.

public static void main(String... args) {
     ...
        // Generating queries based on user question
        ...
	          
       // Generating a hypothetical answer and its embedding
       ...
       // Searching news articles based on the queries
       ...
       // Extracting relevant information from the articles
       List<ObjectNode> articles = results.stream().map(arrayNode ->
                            arrayNode.map(on -> on.asCollection().asObject()))
                    .flatMap(List::stream).collect(Collectors.toList());

       // Extracting article content and generating embeddings for each article
       List<String> articleContent = articles.stream().map(article ->
                            String.format("%s %s %s", article.getString("title"),
                                    article.getString("description"), 
                                    article.getString("content")
                                             .substring(0, 100)))
                    .collect(Collectors.toList());

After generating queries, a hypothetical answer, and searching for news articles, the code extracts relevant information from the retrieved articles and generates embeddings for each article. Here's a breakdown of the code:

  1. main Method (Continued):
    • After searching for news articles (as shown in the previous example), the code extracts relevant information and generates embeddings.
    • The code creates an empty list, articles, to store the relevant information extracted from the news articles.
  2. Extracting Relevant Information:
    • The code utilizes the stream function on the results list, which contains the retrieved news articles, to perform a series of operations for each ArrayNode object in the list.
    • Each ArrayNode uses the map operation to transform each element to an ObjectNode by mapping each element to its collection representation as an object.
    • The flatMap operation is then used to flatten the stream of ObjectNode elements into a single stream.
    • The flattened ObjectNode elements are collected into a list of ObjectNode objects, stored in the articles variable.
  3. Extracting Article Content and Generating Embeddings:
    • The code uses the stream function on the articles list to perform a series of operations for each ObjectNode object in the list.
    • For each ObjectNode, it uses the map operation to transform each article into a formatted string containing the title, description, and a substring of the content (limited to the first 100 characters).
    • The transformed strings are collected into a list of String objects, stored in the articleContent variable.

Embeddings Generation

Next, we need to generate embeddings for each article's content.

public static void main(String... args) {
     ...
        // Generating queries based on user question
        ...
	          
       // Generating a hypothetical answer and its embedding
       ...
       // Searching news articles based on the queries
       ...
       // Extracting relevant information from the articles
       ...

       // Extracting article content and generating embeddings for each article
       List<String> articleContent = articles.stream().map(article ->
                            String.format("%s %s %s", article.getString("title"),
                                    article.getString("description"), article.getString("content").substring(0, 100)))
                    .collect(Collectors.toList());

       List<float[]> articleEmbeddings = embeddings(articleContent);

After generating queries, a hypothetical answer, searching for news articles, and extracting relevant information from the articles, the example proceeds with extracting article content and generating embeddings for each article. Here's a breakdown of the code:

  1. main Method (Continued):
    • After extracting relevant information from the articles (as shown in the previous example), the code extracts article content and generates embeddings for each article.
  2. Extracting Article Content:
    • The code uses the stream function on the articles list, which contains the relevant information extracted from the news articles, to perform a series of operations for each ObjectNode object in the list.
    • For each ObjectNode, it uses the map operation to transform each article into a formatted string containing the title, description, and a substring of the content (limited to the first 100 characters).
    • The transformed strings are collected into a list of String objects, stored in the articleContent variable.
  3. Generating Embeddings:
    • The code calls the embeddings method, passing the articleContent list as input.
    • The embeddings method is likely responsible for generating embeddings for the given list of strings.
    • It returns a list of float arrays stored in the articleEmbeddings variable.

The code focuses on extracting article content from the relevant information obtained from the news articles. It generates a list of formatted strings that include the title, description, and a substring of the content. Additionally, it calls the embeddings method to generate embeddings for the article content strings, resulting in a list of float arrays representing the embeddings.

Please note that the provided example snippet is a continuation of the previous example, and we will continue to the next step, the cosine similarities calculation.

💡 **Cosine Similarities Calculation**

Cosine similarity measures the similarity between two vectors in a multi-dimensional space. It measures the cosine of the angle between two vectors and provides a value between -1 and 1. A value of 1 indicates that the vectors are pointing in the same direction, 0 indicates that the vectors are orthogonal (perpendicular), and -1 indicates that the vectors are pointing in opposite directions.

The cosine similarity is widely used in various fields, including information retrieval, natural language processing, recommendation systems, and machine learning. It is particularly useful when comparing documents or texts represented as vectors in a high-dimensional space.

To calculate the cosine similarity between two vectors, you can use the following formula:

cosine similarity = (A · B) / (||A|| * ||B||)

Here, A and B represent the two vectors being compared, "·" denotes the dot product of the vectors, and "|| ||" denotes the Euclidean norm or magnitude of a vector.

In natural language processing, cosine similarity is often used for document clustering, retrieval, and sentence similarity tasks. By representing documents or sentences as numerical vectors (e.g., using techniques like TF-IDF or word embeddings), cosine similarity can identify related or similar pieces of text.

Cosine similarity is a similarity measure between two non-zero vectors in an inner product space. It quantifies how similar the directions of two vectors are, regardless of their magnitudes. This concept is essential in many areas of data analysis and machine learning, as it quantifies similarity between vectors and identifies relationships in high-dimensional spaces.

The cosine similarity is a popular metric in text analysis, where it is used to measure the similarity of documents. In text analysis, each word is assigned a different coordinate, and a document is represented by the vector of the number of occurrences of each word in the document. Cosine similarity then gives a useful measure of how similar two documents are likely to be in terms of their subject matter and independently of the length of the documents.

For example, consider the following two documents:

Document 1: The quick brown fox jumps over the lazy dog. Document 2: The dog saw the fox jump over the lazy brown fence.

The cosine similarity between two documents would be high if they contain the same words in the same order, such as "quick", "brown", "fox", "jumps", "over", and "lazy". This metric can also be used to measure the similarity of groups of objects, such as products, customers, or users.

Cosine similarity is a simple and effective metric that can be used to measure the similarity of two or more vectors. It has several advantages, such as being independent of the magnitudes of the vectors and being a popular metric in machine learning algorithms like recommendation systems and clustering algorithms.

However, cosine similarity has some disadvantages. It does not consider the order of the elements in the vectors and is not sensitive to differences in the magnitudes of the elements in the vectors.

Overall, cosine similarity is a versatile and effective metric that can be used in text analysis and other types of data analysis to measure similarity between vectors.

Cosine Similarities Calculation

Next, calculate cosine similarities between the hypothetical answer embedding and the article embeddings.

public static float dot(float[] v1, float[] v2) {
        assert v1.length == v2.length;
        float result = 0;
        for (int i = 0; i < v1.length; i++) {
            result += v1[i] * v2[i];
        }
        return result;
}

public static void main(String... args) {
     ...
        // Generating queries based on user question
        ...
	          
       // Generating a hypothetical answer and its embedding
       ...
       // Searching news articles based on the queries
       ...
       // Extracting relevant information from the articles
       ...

       // Extracting article content and generating embeddings for each article
       ...

       List<float[]> articleEmbeddings = embeddings(articleContent);

			 // Calculating cosine similarities between the hypothetical answer embedding and article embeddings
       List<Float> cosineSimilarities = articleEmbeddings.stream()
                    .map(articleEmbedding -> 
                           dot(hypotheticalAnswerEmbedding, articleEmbedding))
                    .collect(Collectors.toList());

After generating queries, a hypothetical answer, searching for news articles, extracting relevant information from the articles, and generating embeddings for each article, the code proceeds with calculating cosine similarities between the hypothetical answer embedding and the article embeddings, which we will later use to score and sort the articles. Here's a breakdown of the code:

  1. dot Method:
    • This method calculates the dot product between two float arrays, v1 and v2.
    • It ensures that the lengths of v1 and v2 are the same.
    • The method iterates over the elements of v1 and v2, multiplying corresponding elements and summing the results.
    • The final result is returned.
  2. main Method (Continued):
    • After generating article embeddings (as shown in the previous example), the code proceeds to calculate cosine similarities.
  3. Calculating Cosine Similarities:
    • The code uses the stream function on the articleEmbeddings list, which contains the embeddings of the articles, to perform a series of operations for each float array in the list.
    • Using the dot method, each float array uses the map operation to calculate the dot product between the hypotheticalAnswerEmbedding and the current article embedding.
    • The calculated dot product represents the cosine similarity between the hypothetical answer embedding and the current article embedding.
    • The cosine similarities are collected into a list of Float objects, stored in the cosineSimilarities variable.

The example focuses on calculating the cosine similarities between the hypothetical answer embedding and the embeddings of the articles. It utilizes the dot method to compute the dot product and maps the resulting similarities to a list of Float objects.

Please note that the provided code snippet is a continuation of the previous example, and we are off to the next section, where we explain the article score and sort.

Scored Article Creation

Now let’s create a set of scored articles based on the cosine similarities and also get rid of duplicates.

public class WhoWonUFC290 {

    public static class ScoredArticle {
        private final ObjectNode  content;
        private final float  score;

        public ScoredArticle(ObjectNode content, float score) {
            this.content = content;
            this.score = score;
        }

        public ObjectNode getContent() {
            return content;
        }

        public float getScore() {
            return score;
        }

        @Override
        public String toString() {
            return "ScoredArticle{" +
                    "content='" + content.getString("title") + '\'' +
                    ", score=" + score +
                    '}';
        }

        @Override
        public boolean equals(Object o) {
            if (this == o) return true;
            if (!(o instanceof ScoredArticle)) return false;
            ScoredArticle that = (ScoredArticle) o;
            return Float.compare(that.score, score) == 0 &&
                    Objects.equals(content.getString("title"), that.content.getString("title"));
        }

        @Override
        public int hashCode() {
            return Objects.hash(content.getString("title"), score);
        }
    }
    public static void main(String... args) {
       ...
       // Generating queries based on user question
       ...
	          
       // Generating a hypothetical answer and its embedding
       ...
       // Searching news articles based on the queries
       ...
       // Extracting relevant information from the articles
       ...

       // Extracting article content and generating embeddings for each article
       ...

       List<float[]> articleEmbeddings = embeddings(articleContent);

			 // Calculating cosine similarities between the hypothetical answer embedding and article embeddings
       List<Float> cosineSimilarities = ...

       // Creating a set of scored articles based on cosine similarities
       Set<ScoredArticle> articleSet = IntStream.range(0,
                            Math.min(cosineSimilarities.size(), articleContent.size()))
                    .mapToObj(i -> new ScoredArticle(articles.get(i), cosineSimilarities.get(i)))
                    .collect(Collectors.toSet());

       // Sorting the articles based on their scores
       List<ScoredArticle> sortedArticles = new ArrayList<>(articleSet);
            Collections.sort(sortedArticles, (o1, o2) -> Float.compare(o2.getScore(), o1.getScore()));

       // Printing the top 5 scored articles
            sortedArticles.subList(0, 5).forEach(s -> System.out.println(s));

After generating queries, a hypothetical answer, searching for news articles, extracting relevant information from the articles, generating embeddings for each article, and calculating cosine similarities, the code proceeds with creating a set of scored articles, sorting them based on their scores, and printing the top 5 scored articles. Here's a breakdown of the code:

  1. ScoredArticle Class:
    • This class is a nested static class within the WhoWonUFC290 class.
    • It represents a scored article containing the content (as an ObjectNode) and the score (as a float).
    • It provides getter methods for accessing the content and score values.
    • It overrides the toString(), equals(), and hashCode() methods for customized string representation, equality comparison, and hash code calculation, respectively.
    • Overriding equals(), and hashCode() is important because we will put ScoredArticle in a set to get rid of duplicates before we score and sort them. Since we generated multiple queries, we get back duplicate articles.
  2. main Method (Continued):
    • After generating article embeddings and calculating cosine similarities (as shown in the previous example), the code creates a set of scored articles based on the cosine similarities.
  3. Creating a Set of Scored Articles:
    • The code uses the IntStream.range function to iterate over the indices within the range of the minimum between the sizes of cosineSimilarities and articleContent. This is the less elegant Java way to do a zip operation like Scala and Python has.
    • For each index, it creates a new ScoredArticle object by passing the corresponding article (articles.get(i)) and cosine similarity (cosineSimilarities.get(i)) as arguments.
    • The created ScoredArticle objects are collected into a Set using the Collectors.toSet() method, stored in the articleSet variable.
  4. Sorting the Articles:
    • The code creates a new ArrayList called sortedArticles and initializes it with the articleSet.
    • It sorts the sortedArticles list based on the scores of the scored articles, using the Collections.sort method and a comparator that compares the scores in descending order.
  5. Printing the Top 5 Scored Articles:
    • The code uses the subList method to extract a sublist of the top 5 scored articles from the sortedArticles list.
    • It iterates over the sublist using the forEach method and prints each scored article using System.out.println().

This part of the example focuses on creating a set of scored articles, sorting them based on their scores, and printing the top 5 scored articles. It utilizes the ScoredArticle class to encapsulate the content and filter duplicates. Then it scores each article and leverages the Collections.sort method to perform the sorting operation.

The score is basically how similar the article is to the ideal answer.

Please note that the provided code snippet is a continuation of the previous example, and now we are down to sorting based on the score.

Articles Sorting

Next, the example sorts the articles based on their scores, which we calculated in the previous step.

    public static void main(String... args) {
       ...
       // Generating queries based on user question
       ...
	          
       // Generating a hypothetical answer and its embedding
       ...
       // Searching news articles based on the queries
       ...
       // Extracting relevant information from the articles
       ...

       // Extracting article content and generating embeddings for each article
       ...

       List<float[]> articleEmbeddings = embeddings(articleContent);

			 // Calculating cosine similarities between the hypothetical answer embedding and article embeddings
       List<Float> cosineSimilarities = ...

       // Creating a set of scored articles based on cosine similarities
       Set<ScoredArticle> articleSet = ...

       // Sorting the articles based on their scores
       List<ScoredArticle> sortedArticles = new ArrayList<>(articleSet);
            Collections.sort(sortedArticles, 
              (o1, o2) -> Float.compare(o2.getScore(), o1.getScore()));

       // Printing the top 5 scored articles
       sortedArticles.subList(0, 5).forEach(s -> System.out.println(s));

After generating queries, a hypothetical answer, searching for news articles, extracting relevant information from the articles, generating embeddings for each article, calculating cosine similarities, creating a set of scored articles, and sorting them based on their scores, the code proceeds with printing the top 5 scored articles. Here's a breakdown of the code:

  1. main Method (Continued):
    • After sorting the articles based on their scores (as shown in the previous example), the code prints the top 5 scored articles.
  2. Printing the Top 5 Scored Articles:
    • The code uses the subList method on the sortedArticles list to extract a sublist representing the top 5 scored articles.
    • It iterates over the sublist using the forEach method, which takes a lambda expression as a parameter.
    • For each scored article s, the lambda expression calls System.out.println(s) to print the scored article.

The code focuses on printing the top 5 scored articles. It utilizes the subList method to extract the desired sublist from the sorted list and the forEach method to iterate over the sublist and print each scored article.

Top Results Formatting

Next, the example formats the top results as JSON strings that we can embed in our final question to get the final answer.

    public static void main(String... args) {
       ...
       // Generating queries based on user question
       ...
	          
       // Generating a hypothetical answer and its embedding
       ...
       // Searching news articles based on the queries
       ...
       // Extracting relevant information from the articles
       ...

       // Extracting article content and generating embeddings for each article
       ...

       List<float[]> articleEmbeddings = embeddings(articleContent);

			 // Calculating cosine similarities between the hypothetical answer embedding and article embeddings
       List<Float> cosineSimilarities = ...

       // Creating a set of scored articles based on cosine similarities
       Set<ScoredArticle> articleSet = ...

       // Sorting the articles based on their scores
       List<ScoredArticle> sortedArticles = new ArrayList<>(articleSet);
            Collections.sort(sortedArticles, 
              (o1, o2) -> Float.compare(o2.getScore(), o1.getScore()));

       ...
			 
       // Formatting the top results as JSON strings
       String formattedTopResults = String.join(",\n", 
           sortedArticles.stream()
              .map(sa -> sa.getContent())
              .map(article -> String.format(
                             Json.niceJson("{'title':'%s', 'url':'%s', 'description':'%s', 'content':'%s'}\n"),
                                  article.getString("title"),
                                  article.getString("url"), article.getString("description"),
                                 getArticleContent(article)
                              ))
              .collect(Collectors.toList()).subList(0, 10));

After generating queries, a hypothetical answer, searching for news articles, extracting relevant information from the articles, generating embeddings for each article, calculating cosine similarities, creating a set of scored articles, and sorting them based on their scores, the code proceeds with formatting the top results as JSON strings so we can embed them into our final question. Here's a breakdown of the code:

  1. main Method (Continued):
    • After sorting the articles based on their scores (as shown in the previous example), the code proceeds with formatting the top results as JSON strings.
  2. Formatting the Top Results as JSON Strings:
    • The code uses the stream function on the sortedArticles list to perform a series of operations for each ScoredArticle object in the list.
    • Each ScoredArticle uses the map operation to transform each article's content into a formatted JSON string.
    • The map operation applies a lambda expression that uses String.format to create a JSON string, using the provided template format and the relevant fields from the article's content, such as title, URL, description, and content.
    • The map operation also invokes the getArticleContent method to retrieve the truncated content of the article.
    • The transformed JSON strings are collected into a list of String objects, stored in the formattedTopResults variable.
    • The collect operation includes a subList call to limit the collected strings to the top 10 results.
    • The String.join method joins the formatted JSON strings with a separator ",\n" to create a single string.

The code focuses on formatting the top results as JSON strings using the stream function, map operation, and String.format. It collects the transformed strings into a list and joins them using String.join.

Please note that the provided code snippet is a continuation of the previous example and now the final part. The grand finale! The final answer!

Final Answer Generation

Next we generate the final answer using a JSON template (ANSWER_INPUT) with the user's question and the formatted top results.

After generating queries, a hypothetical answer, searching for news articles, extracting relevant information from the articles, generating embeddings for each article, calculating cosine similarities, creating a set of scored articles, sorting them based on their scores, and formatting the top results as JSON strings, the example proceeds with generating the final answer using a ANSWER_INPUT template.

public class WhoWonUFC290 {

private static final String ANSWER_INPUT ="Generate an answer to the user's question " +
            "based on the given search results. \n" +
            "TOP_RESULTS: {formatted_top_results}\n" +
            "USER_QUESTION: {USER_QUESTION}\n" +
            "\n" +
            "Include as much information as possible in the answer. Reference the " +
            "relevant search result urls as markdown links";

public static void main(String... args) {
       ...
       // Generating queries based on user question
       ...
	          
       // Generating a hypothetical answer and its embedding
       ...
       // Searching news articles based on the queries
       ...
       // Extracting relevant information from the articles
       ...

       // Extracting article content and generating embeddings for each article
       ...

       List<float[]> articleEmbeddings = embeddings(articleContent);

			 // Calculating cosine similarities between the hypothetical answer embedding and article embeddings
       List<Float> cosineSimilarities = ...

       // Creating a set of scored articles based on cosine similarities
       Set<ScoredArticle> articleSet = ...

       // Sorting the articles based on their scores
       List<ScoredArticle> sortedArticles = ...

       ...
			 
       // Formatting the top results as JSON strings
       String formattedTopResults = ...

       // Generating the final answer with the formatted top results
       String finalAnswer = jsonGPT(ANSWER_INPUT
                    .replace("{USER_QUESTION}", USER_QUESTION)
                    .replace("{formatted_top_results}", formattedTopResults));

       System.out.println(finalAnswer);

Here's a breakdown of the code:

  1. main Method (Continued):
    • After formatting the top results as JSON strings (as shown in the previous example), the code generates the final answer using a JSON template.
  2. Final Answer Generation:
    • The code defines a ANSWER_INPUT string that represents the JSON template for generating the final answer.
    • The ANSWER_INPUT string contains placeholders {formatted_top_results} and {USER_QUESTION} that will be replaced with the actual values.
    • The replace method is used to replace the placeholders with the corresponding values: USER_QUESTION and formattedTopResults.
    • The updated ANSWER_INPUT string is passed as input to the jsonGPT method, generating the final answer based on the input.
    • The generated final answer is stored in the finalAnswer variable.
    • Finally, the code prints the finalAnswer using System.out.println(finalAnswer).

The code focuses on generating the final answer by replacing the placeholders in the JSON template with the actual values and passing it to the jsonGPT method.

Final Answer

{
  "answer": "In the main card fights of UFC 290, the winners were:\n\n1. 
             Alexander Volkanovski from Australia defeated Yair Rodriguez 
             from Mexico by TKO (punches) in Round 3.\n2. Alexandre Pantoja 
             from Brazil defeated Brandon Moreno from Mexico by split decision.
              \n\nAs a result, Alexander Volkanovski retained the 
             featherweight championship title. He is from Australia.",
  "links": [
    {
      "title": "Alexander Volkanovski And the Real Winners and Losers from UFC 290",
      "url": "https://bleacherreport.com/articles/10082051-alexander-volkanovski-and-the-real-winners-and-losers-from-ufc-290"
    }
  ]
}

Conclusion

In summary, leveraging ChatGPT, Embeddings, and HyDE to enhance search results is an effective strategy for swiftly retrieving accurate information, leading to increased customer and employee satisfaction. By combining ChatGPT with retrieval and re-ranking methods, businesses can achieve precise, relevant, and expedient search results, setting themselves apart from competitors. Furthermore, this approach seamlessly integrates with existing search engines, making it an ideal solution for improving search engine performance across organizations of all sizes. As CTOs, CIOs, Engineering Managers, and Software Engineers, implementing this approach will yield substantial benefits, elevating the efficiency of your search engine.

Please check out [JAI, the Java Open AI API client lib](https://github.com/RichardHightower/jai), and if you liked this article, give it a star on GitHub.

Complete listing

package com.cloudurable.jai.examples;

import com.cloudurable.jai.OpenAIClient;
import com.cloudurable.jai.model.ClientResponse;
import com.cloudurable.jai.model.text.completion.chat.ChatRequest;
import com.cloudurable.jai.model.text.completion.chat.ChatResponse;
import com.cloudurable.jai.model.text.completion.chat.Message;
import com.cloudurable.jai.model.text.completion.chat.Role;
import com.cloudurable.jai.model.text.embedding.Embedding;
import com.cloudurable.jai.model.text.embedding.EmbeddingRequest;
import io.nats.jparse.Json;
import io.nats.jparse.node.ArrayNode;
import io.nats.jparse.node.ObjectNode;
import io.nats.jparse.node.StringNode;
import io.nats.jparse.parser.JsonParserBuilder;

import java.net.URI;
import java.net.URLEncoder;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.time.Instant;
import java.time.LocalDate;
import java.time.ZoneId;
import java.time.format.DateTimeFormatter;
import java.util.*;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

public class WhoWonUFC290 {

    public static class ScoredArticle {
        private final ObjectNode  content;
        private final float  score;

        public ScoredArticle(ObjectNode content, float score) {
            this.content = content;
            this.score = score;
        }

        public ObjectNode getContent() {
            return content;
        }

        public float getScore() {
            return score;
        }

        @Override
        public String toString() {
            return "ScoredArticle{" +
                    "content='" + content.getString("title") + '\'' +
                    ", score=" + score +
                    '}';
        }

        @Override
        public boolean equals(Object o) {
            if (this == o) return true;
            if (!(o instanceof ScoredArticle)) return false;
            ScoredArticle that = (ScoredArticle) o;
            return Float.compare(that.score, score) == 0 &&
                    Objects.equals(content.getString("title"), that.content.getString("title"));
        }

        @Override
        public int hashCode() {
            return Objects.hash(content.getString("title"), score);
        }
    }

    private static final String QUERIES_INPUT =  "Generate an array of search queries that are " +
            "relevant to this question.\n" +
            "Use a variation of related keywords for the queries, trying to be as general as possible.\n" +
            "Include as many queries as you can think of, including and excluding terms.\n" +
            "For example, include queries like ['keyword_1 keyword_2', 'keyword_1', 'keyword_2'].\n" +
            "Be creative. The more queries you include, the more likely you are to find relevant results.\n" +
            "\n" +
            "User question: {USER_QUESTION}\n" +
            "\n" +
            "Format: {{\"queries\": [\"query_1\", \"query_2\", \"query_3\"]}}";

    private static final String HA_INPUT ="Generate a hypothetical answer to the user's question. " +
            "This answer will be used to rank search results. \n" +
            "Pretend you have all the information you need to answer, but don't use any actual facts. " +
            "Instead, use placeholders\n" +
            "like NAME did something, or NAME said something at PLACE. \n" +
            "\n" +
            "User question: {USER_QUESTION}\n" +
            "\n" +
            "Format: {{\"hypotheticalAnswer\": \"hypothetical answer text\"}}";

    private static final String USER_QUESTION = "Who won Main card fights in UFC 290? Tell us who won. " +
            "Are there any new champions? Where are they from?";


    private static final String ANSWER_INPUT ="Generate an answer to the user's question " +
            "based on the given search results. \n" +
            "TOP_RESULTS: {formatted_top_results}\n" +
            "USER_QUESTION: {USER_QUESTION}\n" +
            "\n" +
            "Include as much information as possible in the answer. Reference the " +
            "relevant search result urls as markdown links";

    public static String jsonGPT(String input) {

        final var client = OpenAIClient.builder().setApiKey(System.getenv("OPENAI_API_KEY")).build();

        final var chatRequest = ChatRequest.builder()
                .addMessage(Message.builder().role(Role.SYSTEM).content("All output shall be JSON").build())
                .addMessage(Message.builder().role(Role.USER).content(input).build())
                .build();

        final var chat = client.chat(chatRequest);

        if (chat.getResponse().isPresent()) {
            return chat.getResponse().get().getChoices().get(0).getMessage().getContent();
        } else {
            System.out.println("" + chat.getStatusCode().orElse(666) + "" + chat.getStatusMessage().orElse(""));
            throw new IllegalStateException();
        }
    }
    public static String hypotheticalAnswer() {
        final var input = HA_INPUT.replace("{USER_QUESTION}",
                USER_QUESTION );
        return jsonGPT(input);
    }

    public static float[] embeddings(String input) {
        return embeddings(List.of(input)).get(0);
    }
    public static List<float[]> embeddings(List<String> input) {
        final var client = OpenAIClient.builder().setApiKey(System.getenv("OPENAI_API_KEY")).build();
        var embedding = client.embedding(EmbeddingRequest.builder().model("text-embedding-ada-002").input(input).build());
        if (embedding.getResponse().isPresent()) {
            return embedding.getResponse().get().getData().stream().map(Embedding::getEmbedding).collect(Collectors.toList());
        } else {
            System.out.println("" + embedding.getStatusCode().orElse(666) + "" + embedding.getStatusMessage().orElse(""));
            throw new IllegalStateException();
        }
    }


    public static void main(String... args) {
        try {
            // Generating queries based on user question
            String queriesJson = jsonGPT(QUERIES_INPUT.replace("{USER_QUESTION}", USER_QUESTION));
            List<String> queries = JsonParserBuilder.builder().build().parse(queriesJson)
                    .getObjectNode().getArrayNode("queries")
                    .filter(node -> node instanceof StringNode)
                    .stream().map(Object::toString).collect(Collectors.toList());

            // Generating a hypothetical answer and its embedding
            var hypotheticalAnswer = hypotheticalAnswer();
            System.out.println(hypotheticalAnswer);
            var hypotheticalAnswerEmbedding = embeddings(hypotheticalAnswer);

            // Searching news articles based on the queries
            List<ArrayNode> results = queries.subList(0, 10).stream()
                    .map(WhoWonUFC290::searchNews).collect(Collectors.toList());

            // Extracting relevant information from the articles
            List<ObjectNode> articles = results.stream().map(arrayNode ->
                            arrayNode.map(on -> on.asCollection().asObject()))
                    .flatMap(List::stream).collect(Collectors.toList());

            // Extracting article content and generating embeddings for each article
            List<String> articleContent = articles.stream().map(article ->
                            String.format("%s %s %s", article.getString("title"),
                                    article.getString("description"), article.getString("content").substring(0, 100)))
                    .collect(Collectors.toList());
            List<float[]> articleEmbeddings = embeddings(articleContent);

            // Calculating cosine similarities between the hypothetical answer embedding and article embeddings
            List<Float> cosineSimilarities = articleEmbeddings.stream()
                    .map(articleEmbedding -> dot(hypotheticalAnswerEmbedding, articleEmbedding))
                    .collect(Collectors.toList());

            // Creating a set of scored articles based on cosine similarities
            Set<ScoredArticle> articleSet = IntStream.range(0,
                            Math.min(cosineSimilarities.size(), articleContent.size()))
                    .mapToObj(i -> new ScoredArticle(articles.get(i), cosineSimilarities.get(i)))
                    .collect(Collectors.toSet());

            // Sorting the articles based on their scores
            List<ScoredArticle> sortedArticles = new ArrayList<>(articleSet);
            Collections.sort(sortedArticles, (o1, o2) -> Float.compare(o2.getScore(), o1.getScore()));

            // Printing the top 5 scored articles
            sortedArticles.subList(0, 5).forEach(s -> System.out.println(s));

            // Formatting the top results as JSON strings
            String formattedTopResults = String.join(",\n", sortedArticles.stream().map(sa -> sa.getContent())
                    .map(article -> String.format(Json.niceJson("{'title':'%s', 'url':'%s', 'description':'%s', 'content':'%s'}\n"),
                            article.getString("title"), article.getString("url"), article.getString("description"),
                            getArticleContent(article))).collect(Collectors.toList()).subList(0, 10));

            // Generating the final answer with the formatted top results
            String finalAnswer = jsonGPT(ANSWER_INPUT.replace("{USER_QUESTION}", USER_QUESTION)
                    .replace("{formatted_top_results}", formattedTopResults));

            System.out.println(finalAnswer);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

            /* OUTPUT
            {
  "answer": "In the main card fights of UFC 290, the winners were:\n\n1. Alexander Volkanovski from Australia defeated Yair Rodriguez from Mexico by TKO (punches) in Round 3.\n2. Alexandre Pantoja from Brazil defeated Brandon Moreno from Mexico by split decision.\n\nAs a result, Alexander Volkanovski retained the featherweight championship title. He is from Australia.",
  "links": [
    {
      "title": "Alexander Volkanovski And the Real Winners and Losers from UFC 290",
      "url": "https://bleacherreport.com/articles/10082051-alexander-volkanovski-and-the-real-winners-and-losers-from-ufc-290"
*/

    private static Object getArticleContent(ObjectNode article) {
        String content = article.getString("content");
        if (content.length() < 250) {
            return content;
        } else {
            return content.substring(0, 250);
        }
    }

    public static float dot(float[] v1, float[] v2) {
        assert v1.length == v2.length;
        float result = 0;
        for (int i = 0; i < v1.length; i++) {
            result += v1[i] * v2[i];
        }
        return result;
    }

    private static void handleArticle(ObjectNode node) {
        System.out.println("# " + node.getString("title"));

        if (node.get("author") != null) {
            System.out.println("#### author: " + node.getString("author"));
        }

        final var client = OpenAIClient.builder().setApiKey(System.getenv("OPENAI_API_KEY")).build();


        final var chatRequest = ChatRequest.builder()
                .addMessage(Message.builder().role(Role.SYSTEM).content("All output shall be Markdown").build())
                .addMessage(Message.builder().role(Role.USER).content("First Summarize this article to less than 500 words while " +
                        "getting all of the main points. Also Extract the author, publication, publication date, and link to the article ||| " + node).build())
                .addMessage(Message.builder().role(Role.USER).content("Then create 10 high level bullet points from the article").build())
                .build();

        ClientResponse<ChatRequest, ChatResponse> chat = client.chat(chatRequest);
        if (chat.getResponse().isPresent()) {


            System.out.println(chat.getResponse().get().getChoices().get(0).getMessage().getContent());
        } else {
            System.out.println("" + chat.getStatusCode().orElse(666) + "" + chat.getStatusMessage().orElse(""));
        }

        System.out.println("------");


    }

    public static String dateStr(Instant instant) {
        LocalDate localDate = instant.atZone(ZoneId.systemDefault()).toLocalDate();
        DateTimeFormatter formatter = DateTimeFormatter.ofPattern("'yyyy-MM-dd'");
        return localDate.format(formatter);
    }

    public static ArrayNode searchNews(final String query) {
        final var end = Instant.now();
        final var start = end.minus(java.time.Duration.ofDays(5));
        return searchNews(query, start, end, 5);
    }

    public static ArrayNode searchNews(final String query, final Instant start, final Instant end, final int pageSize) {
        System.out.println(query);
        try {

            String url = "https://newsapi.org/v2/everything?q=" + URLEncoder.encode(query, StandardCharsets.UTF_8)
                    + "&apiKey=" + System.getenv("NEWS_API_KEY") + "&language=en" + "&sortBy=relevancy"
                    + "&from=" + dateStr(start) + "&to=" + dateStr(end) + "&pageSize=" + pageSize;

            HttpClient httpClient = HttpClient.newHttpClient();
            HttpResponse<String> response = httpClient.send(HttpRequest.newBuilder().uri(URI.create(url))
                    .GET().setHeader("Content-Type", "application/json").build(), HttpResponse.BodyHandlers.ofString());

            if (response.statusCode() >= 200 && response.statusCode() < 299) {
                return JsonParserBuilder.builder().build().parse(response.body()).atPath("articles").asCollection().asArray();
            } else {
                throw new IllegalStateException(" status code " + response.statusCode() + " " + response.body());
            }
        } catch (Exception ex) {
            throw new IllegalStateException(ex);
        }
    }


}

Related links

Clone this wiki locally