feat: allow to pass str or dict to LLM.generate() #286

piotrrojek · 2025-01-21T08:53:52Z

Implements #279

github-actions · 2025-01-21T09:41:15Z

Code Coverage Summary

Filename                                                                                                      Stmts    Miss  Cover    Missing
----------------------------------------------------------------------------------------------------------  -------  ------  -------  --------------------------------------------
packages/__init__.py                                                                                              0       0  100.00%
packages/ragbits-cli/src/ragbits/cli/__init__.py                                                                 31       4  87.10%   73-74, 81-82
packages/ragbits-cli/src/ragbits/cli/_utils.py                                                                   23       4  82.61%   45, 62-64
packages/ragbits-cli/src/ragbits/cli/state.py                                                                    55       8  85.45%   48-49, 58, 61-62, 105, 112-113
packages/ragbits-conversations/src/ragbits/conversations/__init__.py                                              0       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/__init__.py                                      0       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/compressors/__init__.py                          3       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/compressors/base.py                             10       0  100.00%
packages/ragbits-conversations/src/ragbits/conversations/history/compressors/llm.py                              29       1  96.55%   79
packages/ragbits-conversations/tests/unit/history/test_llm_compressor.py                                         64       0  100.00%
packages/ragbits-core/src/ragbits/core/__init__.py                                                                6       2  66.67%   8-9
packages/ragbits-core/src/ragbits/core/cli.py                                                                     6       0  100.00%
packages/ragbits-core/src/ragbits/core/config.py                                                                 17       0  100.00%
packages/ragbits-core/src/ragbits/core/options.py                                                                17       0  100.00%
packages/ragbits-core/src/ragbits/core/types.py                                                                   9       0  100.00%
packages/ragbits-core/src/ragbits/core/audit/__init__.py                                                         74       5  93.24%   42-45, 52-54
packages/ragbits-core/src/ragbits/core/audit/base.py                                                             44       1  97.73%   119
packages/ragbits-core/src/ragbits/core/audit/cli.py                                                              67       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/__init__.py                                                     4       0  100.00%
packages/ragbits-core/src/ragbits/core/embeddings/base.py                                                        20       2  90.00%   56, 69
packages/ragbits-core/src/ragbits/core/embeddings/exceptions.py                                                  17       7  58.82%   7-8, 17, 26-27, 36, 45
packages/ragbits-core/src/ragbits/core/embeddings/litellm.py                                                     38      18  52.63%   78-112
packages/ragbits-core/src/ragbits/core/embeddings/noop.py                                                         8       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/__init__.py                                                           4       0  100.00%
packages/ragbits-core/src/ragbits/core/llms/base.py                                                              71       8  88.73%   67, 79, 200, 219-227, 231
packages/ragbits-core/src/ragbits/core/llms/exceptions.py                                                        17       7  58.82%   7-8, 17, 26-27, 36, 45
packages/ragbits-core/src/ragbits/core/llms/factory.py                                                           12       3  75.00%   30, 41, 51
packages/ragbits-core/src/ragbits/core/llms/litellm.py                                                           95      29  69.47%   94, 100-121, 165, 175, 204-229, 251-256, 267
packages/ragbits-core/src/ragbits/core/llms/local.py                                                             56      27  51.79%   9-12, 66-74, 86-87, 108-119, 140-156
packages/ragbits-core/src/ragbits/core/llms/mock.py                                                              29       9  68.97%   63-74
packages/ragbits-core/src/ragbits/core/metadata_stores/__init__.py                                                3       0  100.00%
packages/ragbits-core/src/ragbits/core/metadata_stores/base.py                                                   11       0  100.00%
packages/ragbits-core/src/ragbits/core/metadata_stores/exceptions.py                                              4       0  100.00%
packages/ragbits-core/src/ragbits/core/metadata_stores/in_memory.py                                              16       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/__init__.py                                                         2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/_cli.py                                                            44      21  52.27%   25-33, 47-49, 63-65, 73-75, 89-97
packages/ragbits-core/src/ragbits/core/prompt/base.py                                                            28       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/parsers.py                                                         35       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/prompt.py                                                         126       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/__init__.py                                               2       0  100.00%
packages/ragbits-core/src/ragbits/core/prompt/discovery/prompt_discovery.py                                      33       2  93.94%   54-55
packages/ragbits-core/src/ragbits/core/utils/__init__.py                                                          0       0  100.00%
packages/ragbits-core/src/ragbits/core/utils/_pyproject.py                                                       38       1  97.37%   113
packages/ragbits-core/src/ragbits/core/utils/config_handling.py                                                  72       8  88.89%   16, 54-55, 62-63, 151-153
packages/ragbits-core/src/ragbits/core/utils/decorators.py                                                       29       0  100.00%
packages/ragbits-core/src/ragbits/core/utils/dict_transformations.py                                             72       3  95.83%   24, 27, 108
packages/ragbits-core/src/ragbits/core/vector_stores/__init__.py                                                  3       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/_cli.py                                                     51       4  92.16%   63, 85, 91, 121
packages/ragbits-core/src/ragbits/core/vector_stores/base.py                                                     40       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/chroma.py                                                   59       1  98.31%   77
packages/ragbits-core/src/ragbits/core/vector_stores/in_memory.py                                                38       0  100.00%
packages/ragbits-core/src/ragbits/core/vector_stores/qdrant.py                                                   66       1  98.48%   81
packages/ragbits-core/tests/cli/__init__.py                                                                       0       0  100.00%
packages/ragbits-core/tests/cli/test_cli_trace_handler.py                                                        48       3  93.75%   30, 43, 56
packages/ragbits-core/tests/cli/test_vector_store.py                                                            103       0  100.00%
packages/ragbits-core/tests/integration/vector_stores/test_vector_store.py                                       31       0  100.00%
packages/ragbits-core/tests/unit/__init__.py                                                                      0       0  100.00%
packages/ragbits-core/tests/unit/test_options.py                                                                 21       0  100.00%
packages/ragbits-core/tests/unit/audit/__init__.py                                                                0       0  100.00%
packages/ragbits-core/tests/unit/audit/test_cli.py                                                              102       0  100.00%
packages/ragbits-core/tests/unit/audit/test_trace.py                                                             93       3  96.77%   14, 17, 20
packages/ragbits-core/tests/unit/embeddings/test_from_config.py                                                  14       0  100.00%
packages/ragbits-core/tests/unit/llms/__init__.py                                                                 0       0  100.00%
packages/ragbits-core/tests/unit/llms/test_base.py                                                               69       3  95.65%   34-37
packages/ragbits-core/tests/unit/llms/test_from_config.py                                                        16       0  100.00%
packages/ragbits-core/tests/unit/llms/test_litellm.py                                                            70       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/__init__.py                                                         0       0  100.00%
packages/ragbits-core/tests/unit/llms/factory/test_get_default_llm.py                                            12       0  100.00%
packages/ragbits-core/tests/unit/metadata_stores/__init__.py                                                      0       0  100.00%
packages/ragbits-core/tests/unit/metadata_stores/test_from_config.py                                             11       0  100.00%
packages/ragbits-core/tests/unit/metadata_stores/test_in_memory.py                                               22       0  100.00%
packages/ragbits-core/tests/unit/prompts/__init__.py                                                              0       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_parsers.py                                                         65       0  100.00%
packages/ragbits-core/tests/unit/prompts/test_prompt.py                                                         165       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/__init__.py                                                    0       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/prompt_classes_for_tests.py                                   30       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/test_prompt_discovery.py                                      18       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/__init__.py                     2       1  50.00%   3
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/__init__.py             3       2  33.33%   2-4
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt1.py        14       0  100.00%
packages/ragbits-core/tests/unit/prompts/discovery/ragbits_tests_pkg_with_prompts/prompts/temp_prompt2.py        14       0  100.00%
packages/ragbits-core/tests/unit/utils/__init__.py                                                                0       0  100.00%
packages/ragbits-core/tests/unit/utils/test_config_handling.py                                                   65       2  96.92%   27-28
packages/ragbits-core/tests/unit/utils/test_decorators.py                                                        26       2  92.31%   17, 39
packages/ragbits-core/tests/unit/utils/test_dict_transformations.py                                              69       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_find.py                                                    13       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_config.py                                               9       0  100.00%
packages/ragbits-core/tests/unit/utils/pyproject/test_get_instace.py                                             37       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/__init__.py                                                        0       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_chroma.py                                                    61       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_from_config.py                                               42       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_in_memory.py                                                 84       0  100.00%
packages/ragbits-core/tests/unit/vector_stores/test_qdrant.py                                                    40       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/__init__.py                                          2       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/_main.py                                           115       4  96.52%   257-264, 267
packages/ragbits-document-search/src/ragbits/document_search/documents/__init__.py                                0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/documents/document.py                               61       2  96.72%   99, 146
packages/ragbits-document-search/src/ragbits/document_search/documents/element.py                                78      12  84.62%   87, 167-174, 183-185
packages/ragbits-document-search/src/ragbits/document_search/documents/exceptions.py                             11       5  54.55%   7-8, 17, 26-27
packages/ragbits-document-search/src/ragbits/document_search/documents/sources.py                               122      16  86.89%   131, 214-219, 256-259, 263-264, 289-291
packages/ragbits-document-search/src/ragbits/document_search/ingestion/__init__.py                                0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/document_processor.py                     33       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/__init__.py           5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/base.py              25       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/batched.py           18       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/distributed.py       30       8  73.33%   8-9, 36, 64-71
packages/ragbits-document-search/src/ragbits/document_search/ingestion/processor_strategies/sequential.py        13       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/__init__.py                      3       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/base.py                         19       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/dummy.py                        20       7  65.00%   33, 54-60
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/__init__.py         4       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/default.py         46       4  91.30%   98, 103-104, 137
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/images.py          50      19  62.00%   73-80, 87-99, 111, 124
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/pdf.py             19       6  68.42%   23, 35-43
packages/ragbits-document-search/src/ragbits/document_search/ingestion/providers/unstructured/utils.py           38      11  71.05%   71, 82-83, 98-101, 110, 121-123
packages/ragbits-document-search/src/ragbits/document_search/retrieval/__init__.py                                0       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/__init__.py                     5       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/base.py                         9       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/llm.py                         25       4  84.00%   47-50
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/noop.py                         6       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rephrasers/prompts.py                     16       1  93.75%   52
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/__init__.py                      3       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/base.py                         17       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/litellm.py                      16       0  100.00%
packages/ragbits-document-search/src/ragbits/document_search/retrieval/rerankers/noop.py                          9       0  100.00%
packages/ragbits-document-search/tests/__init__.py                                                                0       0  100.00%
packages/ragbits-document-search/tests/helpers.py                                                                 3       0  100.00%
packages/ragbits-document-search/tests/integration/__init__.py                                                    0       0  100.00%
packages/ragbits-document-search/tests/integration/test_rerankers.py                                             15       6  60.00%   18-38
packages/ragbits-document-search/tests/integration/test_sources.py                                               23      10  56.52%   22-32, 40-45
packages/ragbits-document-search/tests/integration/test_unstructured.py                                          48      10  79.17%   52-58, 71-77
packages/ragbits-document-search/tests/unit/test_config.py                                                       57       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_processor.py                                           17       0  100.00%
packages/ragbits-document-search/tests/unit/test_document_search.py                                              88       0  100.00%
packages/ragbits-document-search/tests/unit/test_documents.py                                                    13       0  100.00%
packages/ragbits-document-search/tests/unit/test_elements.py                                                     19       0  100.00%
packages/ragbits-document-search/tests/unit/test_local_file_source.py                                            13       0  100.00%
packages/ragbits-document-search/tests/unit/test_processing_strategies.py                                        25       0  100.00%
packages/ragbits-document-search/tests/unit/test_providers.py                                                    41       0  100.00%
packages/ragbits-document-search/tests/unit/test_rephrasers.py                                                   26       0  100.00%
packages/ragbits-document-search/tests/unit/test_rerankers.py                                                    51       1  98.04%   23
packages/ragbits-document-search/tests/unit/test_source_discriminator.py                                         35       0  100.00%
packages/ragbits-document-search/tests/unit/test_sources.py                                                      25       0  100.00%
packages/ragbits-document-search/tests/unit/testprojects/project_with_instance_factory/__init__.py                0       0  100.00%
packages/ragbits-document-search/tests/unit/testprojects/project_with_instance_factory/factories.py              11       0  100.00%
packages/ragbits-guardrails/src/ragbits/guardrails/__init__.py                                                    0       0  100.00%
packages/ragbits-guardrails/src/ragbits/guardrails/base.py                                                       15       0  100.00%
packages/ragbits-guardrails/src/ragbits/guardrails/openai_moderation.py                                          19       5  73.68%   29-33
packages/ragbits-guardrails/tests/unit/test_openai_moderation.py                                                 35       0  100.00%
TOTAL                                                                                                          4364     323  92.60%

Diff against main

Filename                                                 Stmts    Miss  Cover
-----------------------------------------------------  -------  ------  --------
packages/ragbits-core/src/ragbits/core/llms/base.py         +5       0  +0.85%
packages/ragbits-core/src/ragbits/core/prompt/base.py       +8       0  +100.00%
packages/ragbits-core/tests/unit/llms/test_base.py         +69      +3  +95.65%
TOTAL                                                      +82      +3  +0.07%

Results for commit: 76a0ca8

Minimum allowed coverage is 60%

♻️ This comment has been updated with latest results

github-actions · 2025-01-21T09:47:24Z

Trivy scanning results.

.venv/lib/python3.10/site-packages/litellm/llms/huggingface/huggingface_llms_metadata/hf_text_generation_models.txt (secrets)

Total: 1 (MEDIUM: 0, HIGH: 0, CRITICAL: 1)

CRITICAL: HuggingFace (hugging-face-access-token)
════════════════════════════════════════
Hugging Face Access Token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/llms/huggingface/huggingface_llms_metadata/hf_text_generation_models.txt:36162
────────────────────────────────────────
36160 mncai/Llama2-7B-Active_3rd-floor-LoRA-dim64_epoch4
36161 ajcdp/CM
36162 [ Nagharjun17/*************************************
36163 BigSalmon/InformalToFormalLincoln114Paraphrase
────────────────────────────────────────

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: Slack (slack-web-hook)
════════════════════════════════════════
Slack Webhook
────────────────────────────────────────
.venv/lib/python3.10/site-packages/litellm/proxy/_types.py:1314
────────────────────────────────────────
1312 alert_to_webhook_url: Optional[Dict] = Field(
1313 None,
1314 [ bhook_url: {'budget_alerts': '*****************************************************************************'}`",
1315 )
────────────────────────────────────────

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

Total: 1 (MEDIUM: 1, HIGH: 0, CRITICAL: 0)

MEDIUM: JWT (jwt-token)
════════════════════════════════════════
JWT token
────────────────────────────────────────
.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA:80
────────────────────────────────────────
78 >>> encoded = jwt.encode({"some": "payload"}, "secret", algorithm="HS256")
79 >>> print(encoded)
80 [ *********************************************************************************************************
81 >>> jwt.decode(encoded, "secret", algorithms=["HS256"])
────────────────────────────────────────

…or standalone functions

…ielding directly

ludwiktrammer · 2025-01-21T15:47:14Z

packages/ragbits-core/tests/unit/llms/test_base.py

+    pass
+
+
+class DummyLLM(LLM[DummyOptions]):


nit: you can just use the built-in MockLLM class from ragbits.core.llms.mock

nice, I haven't noticed it before. will change it to the built-in one.

micpst · 2025-01-22T11:56:43Z

i think we can also add this functionality for generate_with_metadata method

ludwiktrammer · 2025-01-29T17:40:02Z

We could also add this to the generate_streaming method

Allow to pass str or dict to LLM.generate()

f076d57

piotrrojek changed the title ~~Allow to pass str or dict to LLM.generate()~~ feat: allow to pass str or dict to LLM.generate() Jan 21, 2025

change "content" to "response" key in LLM response

65cfd6c

piotrrojek added 2 commits January 21, 2025 10:42

make linter happy: isinstance using | operator instead of a tuple

98e41bd

add simple unit test to cover all 4 ways to call LLM.generate()

52262e0

piotrrojek added 3 commits January 21, 2025 10:48

make linters happy again: don't use TestLLM class, use staticmethods …

b3bd418

…or standalone functions

fix _call_streaming in unit test by returning generate() instead if y…

28cdae5

…ielding directly

format the codebase

76a0ca8

ludwiktrammer approved these changes Jan 21, 2025

View reviewed changes

micpst linked an issue Jan 22, 2025 that may be closed by this pull request

feat: allow to pass str or dict into LLM.generate() #279

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: allow to pass str or dict to LLM.generate() #286

feat: allow to pass str or dict to LLM.generate() #286

piotrrojek commented Jan 21, 2025 •

edited

Loading

github-actions bot commented Jan 21, 2025 •

edited

Loading

github-actions bot commented Jan 21, 2025 •

edited

Loading

ludwiktrammer Jan 21, 2025 •

edited

Loading

piotrrojek Jan 22, 2025

micpst commented Jan 22, 2025

ludwiktrammer commented Jan 29, 2025

feat: allow to pass str or dict to LLM.generate() #286

Are you sure you want to change the base?

feat: allow to pass str or dict to LLM.generate() #286

Conversation

piotrrojek commented Jan 21, 2025 • edited Loading

github-actions bot commented Jan 21, 2025 • edited Loading

Code Coverage Summary

Diff against main

github-actions bot commented Jan 21, 2025 • edited Loading

.venv/lib/python3.10/site-packages/litellm/llms/huggingface/huggingface_llms_metadata/hf_text_generation_models.txt (secrets)

.venv/lib/python3.10/site-packages/litellm/proxy/_types.py (secrets)

.venv/lib/python3.10/site-packages/PyJWT-2.9.0.dist-info/METADATA (secrets)

ludwiktrammer Jan 21, 2025 • edited Loading

Choose a reason for hiding this comment

piotrrojek Jan 22, 2025

Choose a reason for hiding this comment

micpst commented Jan 22, 2025

ludwiktrammer commented Jan 29, 2025

piotrrojek commented Jan 21, 2025 •

edited

Loading

github-actions bot commented Jan 21, 2025 •

edited

Loading

github-actions bot commented Jan 21, 2025 •

edited

Loading

ludwiktrammer Jan 21, 2025 •

edited

Loading