Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

avoid client generation if already exist #324

Merged
merged 2 commits into from
Oct 7, 2024
Merged

avoid client generation if already exist #324

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ac10022
fix client generation
aniketmaurya Oct 7, 2024
41c5c76
update test
aniketmaurya Oct 7, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions src/litserve/python_client.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
client_template = """
# Copyright The Lightning AI team.
#
# Licensed under the Apache License, Version 2.0 (the "License");
Expand All @@ -13,5 +14,6 @@
# limitations under the License.
import requests

response = requests.post("http://127.0.0.1:{{PORT}}/predict", json={"input": 4.0})
print(f"Status: {response.status_code}\nResponse:\n {response.text}")
response = requests.post("http://127.0.0.1:{PORT}/predict", json={{"input": 4.0}})
print(f"Status: {{response.status_code}}\\nResponse:\\n {{response.text}}")
"""
aniketmaurya marked this conversation as resolved.
Show resolved Hide resolved
18 changes: 7 additions & 11 deletions src/litserve/server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
from contextlib import asynccontextmanager
from queue import Empty
from typing import Callable, Dict, Optional, Sequence, Tuple, Union, List

from litserve.python_client import client_template
import uvicorn
from fastapi import Depends, FastAPI, HTTPException, Request, Response
from fastapi.responses import StreamingResponse
Expand Down Expand Up @@ -390,23 +390,19 @@ async def stream_predict(request: self.request_type) -> self.response_type:

@staticmethod
def generate_client_file(port: Union[str, int] = 8000):
src_path = os.path.join(os.path.dirname(__file__), "python_client.py")
dest_path = os.path.join(os.getcwd(), "client.py")

try:
# Read the template code
with open(src_path) as f:
client_code = f.read()

# Replace the default port number with the user-specified port number
client_code = client_code.replace("{{PORT}}", str(port))
if os.path.exists(dest_path):
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But is it a correct behaviour?
If I changed the port I expect that the file should be regenerated with a new, correct one.

Or maybe I am missing something?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if you already have a client.py that you updated based on your server (let's say image classification). You won't want LitServe to replace the file. The generate client is supposed to be static template that people can quickly edit.

logger.debug("client.py already exists in the current directory. Skipping generation.")
return

# Write client code template w/ updated port number to destination
try:
client_code = client_template.format(PORT=port)
with open(dest_path, "w") as f:
f.write(client_code)

except Exception as e:
print(f"Error copying file: {e}")
logger.exception(f"Error copying file: {e}")

def run(
self,
Expand Down
4 changes: 4 additions & 0 deletions tests/test_lit_server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -451,3 +451,7 @@ def test_generate_client_file(tmp_path, monkeypatch):
LitServer.generate_client_file(8123)
with open(tmp_path / "client.py") as fr:
assert expected in fr.read(), f"Expected {expected} in client.py"

LitServer.generate_client_file(8000)
with open(tmp_path / "client.py") as fr:
assert expected in fr.read(), "Shouldn't replace existing client.py"
Loading