Skip to content

Commit

Permalink
Add disclaimer around application state and multiprocessing
Browse files Browse the repository at this point in the history
  • Loading branch information
rmccorm4 committed Aug 22, 2024
1 parent ae2fcd6 commit fc4c15a
Showing 1 changed file with 6 additions and 1 deletion.
7 changes: 6 additions & 1 deletion python/openai/openai/src/api_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,14 @@ def add_cors_middleware(app: FastAPI):

@asynccontextmanager
async def lifespan(app: FastAPI):
print("Starting FastAPI app lifespan...")
# Start the tritonserver on FastAPI app startup
print("Starting FastAPI app lifespan...")
server, model_metadatas = init_tritonserver()

# NOTE: These are meant for read-only access by routes handling requests
# with a single process, and should generally not be modified for the
# lifetime of the application. If multiple uvicorn workers are instantiated,
# then multiple triton servers would be started, one per worker process.
app.server = server
app.models = {metadata.name: metadata for metadata in model_metadatas}

Expand Down

0 comments on commit fc4c15a

Please sign in to comment.