Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโ€™ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

๐ŸŒ [i18n-KO] Translated guides/inference_endpoints.md to Korean #2164

Merged
merged 21 commits into from
Apr 25, 2024
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/source/ko/_toctree.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,8 @@
title: (๋ฒˆ์—ญ ์ค‘) Search
- local: in_translation
title: (๋ฒˆ์—ญ ์ค‘) Inference
- local: guides/inference_endpoints
title: ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ
- local: in_translation
title: (๋ฒˆ์—ญ ์ค‘) Community Tab
- local: in_translation
Expand Down
257 changes: 257 additions & 0 deletions docs/source/ko/guides/inference_endpoints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,257 @@
# ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ[[inference-endpoints]]

์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋Š” ์‰ฝ๊ฒŒ `transformers`, `sentence-transformers` ๋ฐ `diffusers` ๋ชจ๋ธ์„ ๊ธฐ๋ฐ˜์œผ๋กœ ์ „์šฉ ๋ฐ ์ž๋™ ํ™•์žฅ ์ธํ”„๋ผ์— ๋ฐฐํฌํ•  ์ˆ˜ ์žˆ๋Š” ์•ˆ์ „ํ•œ ํ”„๋กœ๋•์…˜ ์†”๋ฃจ์…˜์„ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋Š” [Hub](https://huggingface.co/models)์˜ ๋ชจ๋ธ๋กœ ๊ตฌ์ถ•๋ฉ๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved
์ด ๊ฐ€์ด๋“œ์—์„œ๋Š” `huggingface_hub`๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ํ”„๋กœ๊ทธ๋ž˜๋ฐ ๋ฐฉ์‹์œผ๋กœ ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋ฅผ ๊ด€๋ฆฌํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ๋ฐฐ์›๋‹ˆ๋‹ค. ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ ์ œํ’ˆ ์ž์ฒด์— ๋Œ€ํ•œ ์ž์„ธํ•œ ๋‚ด์šฉ์€ [๊ณต์‹ ๋ฌธ์„œ](https://huggingface.co/docs/inference-endpoints/index)๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

์ด ๊ฐ€์ด๋“œ์—์„œ๋Š” `huggingface_hub`๊ฐ€ ์˜ฌ๋ฐ”๋ฅด๊ฒŒ ์„ค์น˜๋˜๊ณ  ๊ธฐ๊ณ„๊ฐ€ ๋กœ๊ทธ์ธ๋˜์–ด ์žˆ๋‹ค๊ณ  ๊ฐ€์ •ํ•ฉ๋‹ˆ๋‹ค. ์•„์ง ๊ทธ๋ ‡์ง€ ์•Š์€ ๊ฒฝ์šฐ [๋น ๋ฅธ ์‹œ์ž‘ ๊ฐ€์ด๋“œ](https://huggingface.co/docs/huggingface_hub/quick-start#quickstart)๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”. ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ API๋ฅผ ์ง€์›ํ•˜๋Š” ์ตœ์†Œ ๋ฒ„์ „์€ `v0.19.0`์ž…๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

## ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ ์ƒ์„ฑ[[create-an-inference-endpoint]]

์ฒซ ๋ฒˆ์งธ ๋‹จ๊ณ„๋Š” [`create_inference_endpoint`]๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋ฅผ ์ƒ์„ฑํ•˜๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค:

```py
>>> from huggingface_hub import create_inference_endpoint

>>> endpoint = create_inference_endpoint(
... "my-endpoint-name",
... repository="gpt2",
... framework="pytorch",
... task="text-generation",
... accelerator="cpu",
... vendor="aws",
... region="us-east-1",
... type="protected",
... instance_size="medium",
... instance_type="c6i"
... )
```

์ด ์˜ˆ์—์„œ๋Š” `"my-endpoint-name"`๋ผ๋Š” `protected` ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋ฅผ ์ƒ์„ฑํ•˜์—ฌ `text-generation`์„ ์œ„ํ•ด [gpt2](https://huggingface.co/gpt2)๋ฅผ ์ œ๊ณตํ•ฉ๋‹ˆ๋‹ค. `protected` ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋Š” API์— ์•ก์„ธ์Šคํ•˜๋ ค๋ฉด ํ† ํฐ์ด ํ•„์š”ํ•ฉ๋‹ˆ๋‹ค. ๋˜ํ•œ ๋ฒค๋”, ์ง€์—ญ, ๊ฐ€์†๊ธฐ, ์ธ์Šคํ„ด์Šค ์œ ํ˜•, ํฌ๊ธฐ์™€ ๊ฐ™์€ ํ•˜๋“œ์›จ์–ด ์š”๊ตฌ ์‚ฌํ•ญ์„ ๊ตฌ์„ฑํ•˜๊ธฐ ์œ„ํ•œ ์ถ”๊ฐ€ ์ •๋ณด๋ฅผ ์ œ๊ณตํ•ด์•ผ ํ•ฉ๋‹ˆ๋‹ค. ์‚ฌ์šฉ ๊ฐ€๋Šฅํ•œ ๋ฆฌ์†Œ์Šค ๋ชฉ๋ก์€ [์—ฌ๊ธฐ](https://api.endpoints.huggingface.cloud/#/v2%3A%3Aprovider/list_vendors)์—์„œ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋˜๋Š” [์›น ์ธํ„ฐํŽ˜์ด์Šค](https://ui.endpoints.huggingface.co/new)๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ํŽธ๋ฆฌํ•˜๊ฒŒ ์ˆ˜๋™์œผ๋กœ ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋ฅผ ์ƒ์„ฑํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๊ณ ๊ธ‰ ์„ค์ • ๋ฐ ์‚ฌ์šฉ๋ฒ•์— ๋Œ€ํ•œ ์ž์„ธํ•œ ๋‚ด์šฉ์€ [์ด ๊ฐ€์ด๋“œ](https://huggingface.co/docs/inference-endpoints/guides/advanced)๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

[`create_inference_endpoint`]์—์„œ ๋ฐ˜ํ™˜๋œ ๊ฐ’์€ [`InferenceEndpoint`] ๊ฐœ์ฒด์ž…๋‹ˆ๋‹ค:

```py
>>> endpoint
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='pending', url=None)
```

์ด๊ฒƒ์€ `name`, `repository`, `status`, `task`, `created_at`, `updated_at` ๋“ฑ๊ณผ ๊ฐ™์€ ์ค‘์š”ํ•œ ์†์„ฑ์— ์ ‘๊ทผํ•  ์ˆ˜ ์žˆ๋Š” ๋ฐ์ดํ„ฐ ํด๋ž˜์Šค์ž…๋‹ˆ๋‹ค. ํ•„์š”ํ•œ ๊ฒฝ์šฐ `endpoint.raw`๋ฅผ ํ†ตํ•ด ์„œ๋ฒ„์—์„œ ์˜จ ์›์‹œ ์‘๋‹ต์—๋„ ์ ‘๊ทผํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๊ฐ€ ์ƒ์„ฑ๋˜๋ฉด [๊ฐœ์ธ ๋Œ€์‹œ๋ณด๋“œ](https://ui.endpoints.huggingface.co/)์—์„œ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.

![](https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/huggingface_hub/inference_endpoints_created.png)

#### ์‚ฌ์šฉ์ž ์ •์˜ ์ด๋ฏธ์ง€ ์‚ฌ์šฉ[[using-a-custom-image]]

๊ธฐ๋ณธ์ ์œผ๋กœ ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋Š” Hugging Face์—์„œ ์ œ๊ณตํ•˜๋Š” ๋„์ปค ์ด๋ฏธ์ง€๋กœ ๊ตฌ์ถ•๋ฉ๋‹ˆ๋‹ค. ๊ทธ๋Ÿฌ๋‚˜ `custom_image` ๋งค๊ฐœ๋ณ€์ˆ˜๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ชจ๋“  ๋„์ปค ์ด๋ฏธ์ง€๋ฅผ ์ง€์ •ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ผ๋ฐ˜์ ์ธ ์‚ฌ์šฉ ์‚ฌ๋ก€๋Š” [text-generation-inference](https://github.com/huggingface/text-generation-inference) ํ”„๋ ˆ์ž„์›Œํฌ๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ LLM์„ ์‹คํ–‰ํ•˜๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค. ๋‹ค์Œ๊ณผ ๊ฐ™์ด ์ˆ˜ํ–‰ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค:

```python
# TGI์—์„œ Zephyr-7b-beta๋ฅผ ์‹คํ–‰ํ•˜๋Š” ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ ์‹œ์ž‘ํ•˜๊ธฐ
>>> from huggingface_hub import create_inference_endpoint
>>> endpoint = create_inference_endpoint(
... "aws-zephyr-7b-beta-0486",
... repository="HuggingFaceH4/zephyr-7b-beta",
... framework="pytorch",
... task="text-generation",
... accelerator="gpu",
... vendor="aws",
... region="us-east-1",
... type="protected",
... instance_size="medium",
... instance_type="g5.2xlarge",
... custom_image={
... "health_route": "/health",
... "env": {
... "MAX_BATCH_PREFILL_TOKENS": "2048",
... "MAX_INPUT_LENGTH": "1024",
... "MAX_TOTAL_TOKENS": "1512",
... "MODEL_ID": "/repository"
... },
... "url": "ghcr.io/huggingface/text-generation-inference:1.1.0",
... },
... )
```

`custom_image`์— ์ „๋‹ฌํ•  ๊ฐ’์€ ๋„์ปค ์ปจํ…Œ์ด๋„ˆ์˜ URL๊ณผ ์ด๋ฅผ ์‹คํ–‰ํ•˜๊ธฐ ์œ„ํ•œ ๊ตฌ์„ฑ์ด ํฌํ•จ๋œ ๋”•์…”๋„ˆ๋ฆฌ์ž…๋‹ˆ๋‹ค. ์ž์„ธํ•œ ๋‚ด์šฉ์€ [Swagger ๋ฌธ์„œ](https://api.endpoints.huggingface.cloud/#/v2%3A%3Aendpoint/create_endpoint)๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

### ๊ธฐ์กด ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ ๊ฐ€์ ธ์˜ค๊ธฐ ๋˜๋Š” ๋‚˜์—ด[[get-or-list-existing-inference-endpoints]]
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

๊ฒฝ์šฐ์— ๋”ฐ๋ผ ์ด์ „์— ์ƒ์„ฑํ•œ ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋ฅผ ๊ด€๋ฆฌํ•ด์•ผ ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด๋ฆ„์„ ์•Œ๊ณ  ์žˆ๋Š” ๊ฒฝ์šฐ [`get_inference_endpoint`]๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ [`InferenceEndpoint`] ๊ฐœ์ฒด๋ฅผ ๊ฐ€์ ธ์˜ฌ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋˜๋Š” [`list_inference_endpoints`]๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ชจ๋“  ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ ๋ชฉ๋ก์„ ๊ฒ€์ƒ‰ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋‘ ๋ฉ”์„œ๋“œ ๋ชจ๋‘ ์„ ํƒ์  `namespace` ๋งค๊ฐœ๋ณ€์ˆ˜๋ฅผ ํ—ˆ์šฉํ•ฉ๋‹ˆ๋‹ค. ์†ํ•ด ์žˆ๋Š” ์กฐ์ง์˜ `namespace`๋ฅผ ์„ค์ •ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๊ทธ๋ ‡์ง€ ์•Š์œผ๋ฉด ๊ธฐ๋ณธ์ ์œผ๋กœ ์‚ฌ์šฉ์ž ์ด๋ฆ„์ด ์‚ฌ์šฉ๋ฉ๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

```py
>>> from huggingface_hub import get_inference_endpoint, list_inference_endpoints

# ์—”๋“œํฌ์ธํŠธ ๊ฐœ์ฒด ๊ฐ€์ ธ์˜ค๊ธฐ
>>> get_inference_endpoint("my-endpoint-name")
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='pending', url=None)

# ์กฐ์ง์˜ ๋ชจ๋“  ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ ๋‚˜์—ด
>>> list_inference_endpoints(namespace="huggingface")
[InferenceEndpoint(name='aws-starchat-beta', namespace='huggingface', repository='HuggingFaceH4/starchat-beta', status='paused', url=None), ...]

# ์‚ฌ์šฉ์ž๊ฐ€ ์†ํ•ด์žˆ๋Š” ๋ชจ๋“  ์กฐ์ง์˜ ์—”๋“œํฌ์ธํŠธ ๋‚˜์—ด
>>> list_inference_endpoints(namespace="*")
[InferenceEndpoint(name='aws-starchat-beta', namespace='huggingface', repository='HuggingFaceH4/starchat-beta', status='paused', url=None), ...]
```

## ๋ฐฐํฌ ์ƒํƒœ ํ™•์ธ[[check-deployment-status]]

์ด ๊ฐ€์ด๋“œ์˜ ๋‚˜๋จธ์ง€ ๋ถ€๋ถ„์—์„œ๋Š” `endpoint`๋ผ๋Š” ์ด๋ฆ„์˜ [`InferenceEndpoint`] ๊ฐ์ฒด๋ฅผ ๊ฐ€์ง€๊ณ  ์žˆ๋‹ค๊ณ  ๊ฐ€์ •ํ•ฉ๋‹ˆ๋‹ค. ์—”๋“œํฌ์ธํŠธ์— `status` ์†์„ฑ์ด [`InferenceEndpointStatus`] ์œ ํ˜•์ด๋ผ๋Š” ๊ฒƒ์„ ์•Œ ์ˆ˜ ์žˆ์—ˆ์Šต๋‹ˆ๋‹ค. ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๊ฐ€ ๋ฐฐํฌ๋˜๊ณ  ์ ‘๊ทผ ๊ฐ€๋Šฅํ•˜๋ฉด ์ƒํƒœ๊ฐ€ `"running"`์ด ๋˜๊ณ  `url` ์†์„ฑ์ด ์„ค์ •๋ฉ๋‹ˆ๋‹ค:

```py
>>> endpoint
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='running', url='https://jpj7k2q4j805b727.us-east-1.aws.endpoints.huggingface.cloud')
```

`"running"` ์ƒํƒœ์— ๋„๋‹ฌํ•˜๊ธฐ ์ „์— ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋Š” ์ผ๋ฐ˜์ ์œผ๋กœ `"initializing"` ๋˜๋Š” `"pending"` ๋‹จ๊ณ„๋ฅผ ๊ฑฐ์นฉ๋‹ˆ๋‹ค. [`~InferenceEndpoint.fetch`]๋ฅผ ์‹คํ–‰ํ•˜์—ฌ ์—”๋“œํฌ์ธํŠธ์˜ ์ƒˆ ์ƒํƒœ๋ฅผ ๊ฐ€์ ธ์˜ฌ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. [`InferenceEndpoint`]์˜ ๋‹ค๋ฅธ ๋ฉ”์„œ๋“œ์™€ ๋งˆ์ฐฌ๊ฐ€์ง€๋กœ ์ด ๋ฉ”์„œ๋“œ๋Š” ์„œ๋ฒ„์— ์š”์ฒญ์„ ํ•˜๋ฉฐ, `endpoint`์˜ ๋‚ด๋ถ€ ์†์„ฑ์ด ๊ทธ ์ž๋ฆฌ์—์„œ ๋ณ€๊ฒฝ๋ฉ๋‹ˆ๋‹ค:
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

```py
>>> endpoint.fetch()
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='pending', url=None)
```

์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๊ฐ€ ์‹คํ–‰๋  ๋•Œ๊นŒ์ง€ ๊ธฐ๋‹ค๋ฆฌ๋ฉด์„œ ์ƒํƒœ๋ฅผ ๊ฐ€์ ธ์˜ค๋Š” ๋Œ€์‹  [`~InferenceEndpoint.wait`]๋ฅผ ์ง์ ‘ ํ˜ธ์ถœํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด ํ—ฌํผ๋Š” `timeout`๊ณผ `fetch_every` ๋งค๊ฐœ๋ณ€์ˆ˜(์ดˆ)๋ฅผ ์ž…๋ ฅ์œผ๋กœ ๋ฐ›์•„ ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๊ฐ€ ๋ฐฐํฌ๋  ๋•Œ๊นŒ์ง€ ์Šค๋ ˆ๋“œ๋ฅผ ์ฐจ๋‹จํ•ฉ๋‹ˆ๋‹ค. ๊ธฐ๋ณธ๊ฐ’์€ ๊ฐ๊ฐ `None`(์ œํ•œ ์‹œ๊ฐ„ ์—†์Œ)๊ณผ `5`์ดˆ์ž…๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

```py
# ์—”๋“œํฌ์ธํŠธ ํฌ๋ฅ˜์ค‘
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved
>>> endpoint
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='pending', url=None)

# 10์ดˆ ๋Œ€๊ธฐ => InferenceEndpointTimeoutError ๋ฐœ์ƒ
>>> endpoint.wait(timeout=10)
raise InferenceEndpointTimeoutError("Timeout while waiting for Inference Endpoint to be deployed.")
huggingface_hub._inference_endpoints.InferenceEndpointTimeoutError: Timeout while waiting for Inference Endpoint to be deployed.

# ์ถ”๊ฐ€ ๋Œ€๊ธฐ
>>> endpoint.wait()
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='running', url='https://jpj7k2q4j805b727.us-east-1.aws.endpoints.huggingface.cloud')
```

`timeout`์ด ์„ค์ •๋˜์–ด ์žˆ๊ณ  ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๊ฐ€ ๋กœ๋“œ๋˜๋Š” ๋ฐ ๋„ˆ๋ฌด ์˜ค๋ž˜ ๊ฑธ๋ฆฌ๋ฉด [`InferenceEndpointTimeoutError`] ์ œํ•œ ์‹œ๊ฐ„ ์ดˆ๊ณผ ์˜ค๋ฅ˜๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

## ์ถ”๋ก  ์‹คํ–‰[[run-inference]]

์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๊ฐ€ ์‹คํ–‰ ์ค‘์ด๋ฉด ๋งˆ์ง€๋ง‰์œผ๋กœ ์ถ”๋ก ์„ ์‹คํ–‰ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค!
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

[`InferenceEndpoint`]์—๋Š” ๊ฐ๊ฐ [`InferenceClient`]์™€ [`AsyncInferenceClient`]๋ฅผ ๋ฐ˜ํ™˜ํ•˜๋Š” `client`์™€ `async_client` ์†์„ฑ์ด ์žˆ์Šต๋‹ˆ๋‹ค.

```py
# ํ…์ŠคํŠธ ์ƒ์„ฑ ์ž‘์—… ์‹คํ–‰:
>>> endpoint.client.text_generation("I am")
' not a fan of the idea of a "big-budget" movie. I think it\'s a'

# ๋น„๋™๊ธฐ ์ปจํ…์ŠคํŠธ์—์„œ๋„ ๋งˆ์ฐฌ๊ฐ€์ง€:
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved
>>> await endpoint.async_client.text_generation("I am")
```

์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๊ฐ€ ์‹คํ–‰ ์ค‘์ด ์•„๋‹ˆ๋ฉด [`InferenceEndpointError`] ์˜ˆ์™ธ๊ฐ€ ๋ฐœ์ƒํ•ฉ๋‹ˆ๋‹ค:
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

```py
>>> endpoint.client
huggingface_hub._inference_endpoints.InferenceEndpointError: Cannot create a client for this Inference Endpoint as it is not yet deployed. Please wait for the Inference Endpoint to be deployed using `endpoint.wait()` and try again.
```

[`InferenceClient`]๋ฅผ ์‚ฌ์šฉํ•˜๋Š” ๋ฐฉ๋ฒ•์— ๋Œ€ํ•œ ์ž์„ธํ•œ ๋‚ด์šฉ์€ [์ถ”๋ก  ๊ฐ€์ด๋“œ](../guides/inference)๋ฅผ ์ฐธ์กฐํ•˜์„ธ์š”.

## ๋ผ์ดํ”„์‚ฌ์ดํด ๊ด€๋ฆฌ[[manage-lifecycle]]

์ด์ œ ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋ฅผ ์ƒ์„ฑํ•˜๊ณ  ์ถ”๋ก ์„ ์‹คํ–‰ํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์‚ดํŽด๋ณด์•˜์œผ๋‹ˆ, ๋ผ์ดํ”„์‚ฌ์ดํด์„ ๊ด€๋ฆฌํ•˜๋Š” ๋ฐฉ๋ฒ•์„ ์‚ดํŽด๋ด…์‹œ๋‹ค.

<Tip>

์ด ์„น์…˜์—์„œ๋Š” [`~InferenceEndpoint.pause`], [`~InferenceEndpoint.resume`], [`~InferenceEndpoint.scale_to_zero`], [`~InferenceEndpoint.update`] ๋ฐ [`~InferenceEndpoint.delete`] ๋“ฑ์˜ ๋ฉ”์„œ๋“œ๋ฅผ ์‚ดํŽด๋ณผ ๊ฒƒ์ž…๋‹ˆ๋‹ค. ๋ชจ๋“  ๋ฉ”์„œ๋“œ๋Š” ํŽธ์˜๋ฅผ ์œ„ํ•ด [`InferenceEndpoint`]์— ์ถ”๊ฐ€๋œ ๋ณ„์นญ์ž…๋‹ˆ๋‹ค. ์›ํ•œ๋‹ค๋ฉด `HfApi`์— ์ •์˜๋œ ์ผ๋ฐ˜ ๋ฉ”์„œ๋“œ [`pause_inference_endpoint`], [`resume_inference_endpoint`], [`scale_to_zero_inference_endpoint`], [`update_inference_endpoint`] ๋ฐ [`delete_inference_endpoint`]๋ฅผ ์‚ฌ์šฉํ•  ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

</Tip>

### ์ผ์‹œ ์ค‘์ง€ ๋˜๋Š” 0์œผ๋กœ ํ™•์žฅ[[pause-or-scale-to-zero]]

์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋ฅผ ์‚ฌ์šฉํ•˜์ง€ ์•Š์„ ๋•Œ ๋น„์šฉ์„ ์ ˆ๊ฐํ•˜๊ธฐ ์œ„ํ•ด [`~InferenceEndpoint.pause`]๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์ผ์‹œ ์ค‘์ง€ํ•˜๊ฑฐ๋‚˜ [`~InferenceEndpoint.scale_to_zero`]๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ 0์œผ๋กœ ํ™•์žฅํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

<Tip>

*์ผ์‹œ ์ค‘์ง€* ๋˜๋Š” *0์œผ๋กœ ํ™•์žฅ*๋œ ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋Š” ๋น„์šฉ์ด ๋“ค์ง€ ์•Š์Šต๋‹ˆ๋‹ค. ์ด ๋‘ ๊ฐ€์ง€์˜ ์ฐจ์ด์ ์€ *์ผ์‹œ ์ค‘์ง€* ์—”๋“œํฌ์ธํŠธ๋Š” [`~InferenceEndpoint.resume`]๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ๋ช…์‹œ์ ์œผ๋กœ *์žฌ๊ฐœ*ํ•ด์•ผ ํ•œ๋‹ค๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค. ๋ฐ˜๋Œ€๋กœ *0์œผ๋กœ ํ™•์žฅ*๋œ ์—”๋“œํฌ์ธํŠธ๋Š” ์ถ”๋ก  ํ˜ธ์ถœ์ด ์žˆ์œผ๋ฉด ์ถ”๊ฐ€ ์ฝœ๋“œ ์Šคํƒ€ํŠธ ์ง€์—ฐ ์‹œ๊ฐ„๊ณผ ํ•จ๊ป˜ ์ž๋™์œผ๋กœ ์‹œ์ž‘๋ฉ๋‹ˆ๋‹ค. ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋Š” ์ผ์ • ๊ธฐ๊ฐ„ ๋น„ํ™œ์„ฑํ™”๋œ ํ›„ ์ž๋™์œผ๋กœ 0์œผ๋กœ ํ™•์žฅ๋˜๋„๋ก ๊ตฌ์„ฑํ•  ์ˆ˜๋„ ์žˆ์Šต๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

</Tip>

```py
# ์—”๋“œํฌ์ธํŠธ ์ผ์‹œ์ค‘์ง€ ๋ฐ ์žฌ์‹œ์ž‘
>>> endpoint.pause()
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='paused', url=None)
>>> endpoint.resume()
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='pending', url=None)
>>> endpoint.wait().client.text_generation(...)
...

# 0์œผ๋กœ ์Šค์ผ€์ผ๋ง
>>> endpoint.scale_to_zero()
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2', status='scaledToZero', url='https://jpj7k2q4j805b727.us-east-1.aws.endpoints.huggingface.cloud')
# Endpoint is not 'running' but still has a URL and will restart on first call.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved
```

### ๋ชจ๋ธ ๋˜๋Š” ํ•˜๋“œ์›จ์–ด ์š”๊ตฌ ์‚ฌํ•ญ ์—…๋ฐ์ดํŠธ[[update-model-or-hardware-requirements]]

๊ฒฝ์šฐ์— ๋”ฐ๋ผ ์ƒˆ๋กœ์šด ์—”๋“œํฌ์ธํŠธ๋ฅผ ์ƒ์„ฑํ•˜์ง€ ์•Š๊ณ  ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋ฅผ ์—…๋ฐ์ดํŠธํ•˜๊ณ  ์‹ถ์„ ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ํ˜ธ์ŠคํŒ…๋œ ๋ชจ๋ธ์ด๋‚˜ ๋ชจ๋ธ ์‹คํ–‰์— ํ•„์š”ํ•œ ํ•˜๋“œ์›จ์–ด ์š”๊ตฌ ์‚ฌํ•ญ์„ ์—…๋ฐ์ดํŠธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ์ด๋ ‡๊ฒŒ ํ•˜๋ ค๋ฉด [`~InferenceEndpoint.update`]๋ฅผ ์‚ฌ์šฉํ•ฉ๋‹ˆ๋‹ค:

```py
# ํƒ€๊ฒŸ ๋ชจ๋ธ ๋ณ€๊ฒฝ
>>> endpoint.update(repository="gpt2-large")
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2-large', status='pending', url=None)

# ๋ณต์ œ๋ณธ ๊ฐฏ์ˆ˜ ์—…๋ฐ์ดํŠธ
>>> endpoint.update(min_replica=2, max_replica=6)
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2-large', status='pending', url=None)

# ๋” ํฐ ์ธ์Šคํ„ด์Šค๋กœ ์—…๋ฐ์ดํŠธ
>>> endpoint.update(accelerator="cpu", instance_size="large", instance_type="c6i")
InferenceEndpoint(name='my-endpoint-name', namespace='Wauplin', repository='gpt2-large', status='pending', url=None)
```

### ์—”๋“œํฌ์ธํŠธ ์‚ญ์ œ[[delete-the-endpoint]]

๋งˆ์ง€๋ง‰์œผ๋กœ ๋” ์ด์ƒ ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋ฅผ ์‚ฌ์šฉํ•˜์ง€ ์•Š์„ ๊ฒฝ์šฐ [`~InferenceEndpoint.delete()`]๋ฅผ ํ˜ธ์ถœํ•˜๊ธฐ๋งŒ ํ•˜๋ฉด ๋ฉ๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

<Tip warning={true}>

์ด๊ฒƒ์€ ๊ตฌ์„ฑ, ๋กœ๊ทธ ๋ฐ ์‚ฌ์šฉ ๋ฉ”ํŠธ๋ฆญ์„ ํฌํ•จํ•˜์—ฌ ์—”๋“œํฌ์ธํŠธ๋ฅผ ์™„์ „ํžˆ ์ œ๊ฑฐํ•˜๋Š” ๋น„๊ฐ€์—ญ์ ์ธ ์ž‘์—…์ž…๋‹ˆ๋‹ค. ์‚ญ์ œ๋œ ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๋Š” ๋ณต์›ํ•  ์ˆ˜ ์—†์Šต๋‹ˆ๋‹ค.
usr-bin-ksh marked this conversation as resolved.
Show resolved Hide resolved

</Tip>

## ์—”๋“œ ํˆฌ ์—”๋“œ ์˜ˆ์ œ[an-end-to-end-example]

์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ์˜ ์ผ๋ฐ˜์ ์ธ ์‚ฌ์šฉ ์‚ฌ๋ก€๋Š” ํ•œ ๋ฒˆ์— ์—ฌ๋Ÿฌ ๊ฐœ์˜ ์ž‘์—…์„ ์ฒ˜๋ฆฌํ•˜์—ฌ ์ธํ”„๋ผ ๋น„์šฉ์„ ์ œํ•œํ•˜๋Š” ๊ฒƒ์ž…๋‹ˆ๋‹ค. ์ด ๊ฐ€์ด๋“œ์—์„œ ๋ณธ ๊ฒƒ์„ ์‚ฌ์šฉํ•˜์—ฌ ์ด ํ”„๋กœ์„ธ์Šค๋ฅผ ์ž๋™ํ™”ํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค:

```py
>>> import asyncio
>>> from huggingface_hub import create_inference_endpoint

# ์—”๋“œํฌ์ธํŠธ ์‹œ์ž‘ + ์ดˆ๊ธฐํ™”๋  ๋•Œ๊นŒ์ง€ ๋Œ€๊ธฐ
>>> endpoint = create_inference_endpoint(name="batch-endpoint",...).wait()

# ์ถ”๋ก  ์‹คํ–‰
>>> client = endpoint.client
>>> results = [client.text_generation(...) for job in jobs]

# ๋น„๋™๊ธฐ ์ถ”๋ก  ์‹คํ–‰
>>> async_client = endpoint.async_client
>>> results = asyncio.gather(*[async_client.text_generation(...) for job in jobs])

# ์—”๋“œํฌ์ธํŠธ ์ค‘์ง€
>>> endpoint.pause()
```

๋˜๋Š” ์ถ”๋ก  ์—”๋“œํฌ์ธํŠธ๊ฐ€ ์ด๋ฏธ ์กด์žฌํ•˜๊ณ  ์ผ์‹œ ์ค‘์ง€๋œ ๊ฒฝ์šฐ:

```py
>>> import asyncio
>>> from huggingface_hub import get_inference_endpoint

# ์—”๋“œํฌ์ธํŠธ ๊ฐ€์ ธ์˜ค๊ธฐ + ์ดˆ๊ธฐํ™”๋  ๋•Œ๊นŒ์ง€ ๋Œ€๊ธฐ
>>> endpoint = get_inference_endpoint("batch-endpoint").resume().wait()

# ์ถ”๋ก  ์‹คํ–‰
>>> async_client = endpoint.async_client
>>> results = asyncio.gather(*[async_client.text_generation(...) for job in jobs])

# ์—”๋“œํฌ์ธํŠธ ์ค‘์ง€
>>> endpoint.pause()
```
Loading