chore: optimize http-template

[viraatc]

What does this PR do?

address pending comment in #162
(forgot to push after pre-commit reformatted files)

[github-actions]

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request focuses on enhancing the performance of HTTP request generation within the HttpRequestTemplate class. By pre-computing and caching common parts of the HTTP request headers, it minimizes the computational cost associated with constructing each request. This optimization is particularly beneficial for scenarios involving frequently used headers, leading to a more streamlined and efficient request creation process.

Highlights

• Performance Optimization: Optimized the HttpRequestTemplate class by introducing pre-calculated HTTP request prefixes, significantly reducing the overhead of building requests at runtime.
• New Attributes: Added _prefix_streaming and _prefix_non_streaming to __slots__ for storing pre-built request components, improving memory efficiency and access speed.
• Refactored Request Building: Refactored the build_request method to leverage these new pre-calculated prefixes, simplifying the logic and reducing string concatenations for common request paths.
• Prefix Rebuilding Logic: Introduced a new private method _rebuild_prefixes to manage the construction and update of these prefixes, ensuring they are correctly updated when cached headers change.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Changelog

• src/inference_endpoint/endpoint_client/http.py
  • Added _prefix_streaming and _prefix_non_streaming to the __slots__ of HttpRequestTemplate.
  • Introduced a new private method _rebuild_prefixes to combine static and cached headers into ready-to-use prefixes.
  • Modified the __init__ method to call _rebuild_prefixes upon initialization.
  • Updated the cache_headers method to trigger _rebuild_prefixes when new headers are added to the cache.
  • Refactored the build_request method to utilize the newly introduced pre-calculated prefixes, simplifying its logic and improving performance.
  • Updated docstrings for the cache_headers method to reflect its new behavior and purpose.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[Copilot]

Pull request overview

This PR optimizes HttpRequestTemplate request building by precomputing reusable HTTP header prefixes, reducing per-request concatenation work at runtime in the endpoint client transport layer.

Changes:

• Precompute and cache streaming/non-streaming request prefixes (static_prefix + cached_headers + base headers) and reuse them in build_request().
• Rebuild cached prefixes when cache_headers() mutates cached_headers.
• Minor docstring/comment updates to reflect the new behavior.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[gemini-code-assist]

Code Review

The pull request optimizes HTTP request building in the HttpRequestTemplate class by introducing pre-computed request prefixes (_prefix_streaming and _prefix_non_streaming). These prefixes, which combine static, cached, and content-type headers, are initialized during object creation and rebuilt when cached headers are updated, allowing the build_request method to use them directly and reduce runtime string concatenation. However, a security vulnerability was noted where the build_request method constructs HTTP headers from extra_headers without validation, potentially allowing an attacker to inject arbitrary headers and cause issues like request smuggling or security control bypass.