Merge pull request SylphAI-Inc#369 from SylphAI-Inc/fix2

[Documentation v2] Greatly simplify the onboarding doc.

.gitignore

@@ -29,11 +29,9 @@ storage/
 *.pkl
 # /*.png
 /*.dot
-/*.svg
 /*.csv
 index.faiss
 *.json
-*.svg
 # ignore the softlink to adalflow cache
 *.adalflow
 .idea

README.md

@@ -11,26 +11,27 @@
 
 <h2>
     <p align="center">
-     ⚡ The Library to Build and Auto-optimize LLM Applications ⚡
+     ⚡ Say Goodbye to Manual Prompting and Vendor Lock-In ⚡
     </p>
 </h2>
 
 
 
+
 <p align="center">
-    <a href="https://colab.research.google.com/drive/1TKw_JHE42Z_AWo8UuRYZCO2iuMgyslTZ?usp=sharing">
+    <a href="https://colab.research.google.com/drive/1_YnD4HshzPRARvishoU4IA-qQuX9jHrT?usp=sharing">
         <img alt="Try Quickstart in Colab" src="https://colab.research.google.com/assets/colab-badge.svg">
     </a>
 </p>
 
 <h4 align="center">
     <p>
-        <a href="https://adalflow.sylph.ai/">All Documentation</a> |
-        <a href="https://adalflow.sylph.ai/apis/components/components.model_client.html">Models</a> |
+        <a href="https://adalflow.sylph.ai/">View Documentation</a>
+        <!-- <a href="https://adalflow.sylph.ai/apis/components/components.model_client.html">Models</a> |
         <a href="https://adalflow.sylph.ai/apis/components/components.retriever.html">Retrievers</a> |
         <a href="https://adalflow.sylph.ai/apis/components/components.agent.html">Agents</a> |
         <a href="https://adalflow.sylph.ai/tutorials/evaluation.html"> LLM evaluation</a> |
-        <a href="https://adalflow.sylph.ai/use_cases/question_answering.html">Trainer & Optimizers</a>
+        <a href="https://adalflow.sylph.ai/use_cases/question_answering.html">Trainer & Optimizers</a> -->
     <p>
 </h4>
 <p align="center">
@@ -57,14 +58,21 @@
     </a>
 </p>
 
-<h4>
+<!-- <h4>
 <p align="center">
 For AI researchers, product teams, and software engineers who want to learn the AI way.
 </p>
+</h4> -->
+
+<h4>
+<p align="center">
+AdalFlow is a PyTorch-like library to build and auto-optimize any LM workflows, from Chatbots, RAG,  to Agents.
+</p>
 </h4>
 
 
 
+
 <!-- <a href="https://colab.research.google.com/drive/1PPxYEBa6eu__LquGoFFJZkhYgWVYE6kh?usp=sharing">
         <img alt="Try Quickstart in Colab" src="https://colab.research.google.com/assets/colab-badge.svg">
     </a> -->
@@ -73,7 +81,24 @@ For AI researchers, product teams, and software engineers who want to learn the
 <img alt="PyPI Downloads" src="https://img.shields.io/pypi/dm/lightRAG?style=flat-square">
 </a> -->
 
+# Why AdalFlow
+
+1. **Say goodbye to manual prompting**: AdalFlow provides a unified auto-differentiative framework for both zero-shot optimization and few-shot prompt optimization. Our research, ``LLM-AutoDiff`` and ``Learn-to-Reason Few-shot In Context Learning``, achieve the highest accuracy among all auto-prompt optimization libraries.
+2. **Switch your LLM app to any model via a config**:  AdalFlow provides `Model-agnostic` building blocks for LLM task pipelines, ranging from RAG, Agents to classical NLP tasks.
+
+<p align="center" style="background-color: #f0f0f0;">
+  <img src="https://raw.githubusercontent.com/SylphAI-Inc/AdalFlow/main/docs/source/_static/images/classification_training_map.png" style="width: 80%;" alt="AdalFlow Auto-optimization">
+</p>
+
+<p align="center" style="background-color: #f0f0f0;">
+  <img src="https://raw.githubusercontent.com/SylphAI-Inc/AdalFlow/main/docs/source/_static/images/classification_opt_prompt.png" alt="AdalFlow Optimized Prompt" style="width: 80%;">
+</p>
+<!-- Among all libraries, AdalFlow achieved the highest accuracy with manual prompting (starting at 82%) and the highest accuracy after optimization. -->
+<!-- <p align="center" style="background-color: #f0f0f0;">
+  <img src="https://raw.githubusercontent.com/SylphAI-Inc/AdalFlow/main/docs/source/_static/images/classification_opt_prompt.png" alt="AdalFlow Optimized Prompt" style="width: 80%;">
+</p> -->
 
+View [Documentation](https://adalflow.sylph.ai)
 
 
 # Quick Start
@@ -85,12 +110,13 @@ Install AdalFlow with pip:
 pip install adalflow
 ```
 
-Please refer to the [full installation guide](https://adalflow.sylph.ai/get_started/installation.html) for more details.
-[Package changelog](https://github.com/SylphAI-Inc/AdalFlow/blob/main/adalflow/CHANGELOG.md).
+<!-- Please refer to the [full installation guide](https://adalflow.sylph.ai/get_started/installation.html) for more details.
+[Package changelog](https://github.com/SylphAI-Inc/AdalFlow/blob/main/adalflow/CHANGELOG.md). -->
+View [Quickstart](https://colab.research.google.com/drive/1_YnD4HshzPRARvishoU4IA-qQuX9jHrT?usp=sharing): Learn AdalFlow end-to-end experience in 15 mins.
 
 
-* Try the [Building Quickstart](https://colab.research.google.com/drive/1TKw_JHE42Z_AWo8UuRYZCO2iuMgyslTZ?usp=sharing) in Colab to see how AdalFlow can build the task pipeline, including Chatbot, RAG, agent, and structured output.
-* Try the [Optimization Quickstart](https://colab.research.google.com/github/SylphAI-Inc/AdalFlow/blob/main/notebooks/qas/adalflow_object_count_auto_optimization.ipynb) to see how AdalFlow can optimize the task pipeline.
+<!-- * Try the [Building Quickstart](https://colab.research.google.com/drive/1TKw_JHE42Z_AWo8UuRYZCO2iuMgyslTZ?usp=sharing) in Colab to see how AdalFlow can build the task pipeline, including Chatbot, RAG, agent, and structured output.
+* Try the [Optimization Quickstart](https://colab.research.google.com/github/SylphAI-Inc/AdalFlow/blob/main/notebooks/qas/adalflow_object_count_auto_optimization.ipynb) to see how AdalFlow can optimize the task pipeline. -->
 
 
 
@@ -107,47 +133,9 @@ We work closely with the [**VITA Group** at University of Texas at Austin](https
 For collaboration, contact [Li Yin](https://www.linkedin.com/in/li-yin-ai/).
 
 
-# Why AdalFlow
-
-1. **Say goodbye to manual prompting**: AdalFlow provides a unified auto-differentiative framework for both zero-shot optimization and few-shot prompt optimization. Our research, ``LLM-AutoDiff`` and ``Learn-to-Reason Few-shot In Context Learning``, achieve the highest accuracy among all auto-prompt optimization libraries.
-2. **Switch your LLM app to any model via a config**:  AdalFlow provides `Model-agnostic` building blocks for LLM task pipelines, ranging from RAG, Agents to classical NLP tasks.
-
-
-<!-- It advances existing auto-optimization research, including Text-Grad and DsPy. Through our research, Text-Grad 2.0, and Learn-to-Reason Few-shot In-Context Learning, AdalFlow Trainer achieves the highest accuracy while being the most token-efficient. -->
 
-<!-- AdalFlow not only helps developers build model-agnostic LLM task pipelines with full control over prompts and output processing, but it also auto-optimizes these pipelines to achieve SOTA accuracy. -->
-<!-- Embracing a design pattern similar to PyTorch, AdalFlow is powerful, light, modular, and robust. -->
 
-<!-- Here is an optimization demonstration on a text classification task: -->
-<!-- <p align="center">
-  <img src="docs/source/_static/images/classification_training_map.png" alt="AdalFlow Auto-optimization" style="width: 80%;">
-</p>
-
-<p align="center">
-  <img src="docs/source/_static/images/classification_opt_prompt.png" alt="AdalFlow Auto-optimization" style="width: 80%;">
-</p> -->
-
-<!-- <p align="center" style="background-color: #f0f0f0;">
-  <img src="https://raw.githubusercontent.com/SylphAI-Inc/AdalFlow/main/docs/source/_static/images/adalflow_rag_benchmarks.png" style="width: 80%;" alt="AdalFlow Auto-optimization">
-</p> -->
-
-<!-- A trained DeepSeek R1 LLaMA70B(r1 distilled) is even better than GPT-o1 without training. -->
-
-<p align="center" style="background-color: #f0f0f0;">
-  <img src="https://raw.githubusercontent.com/SylphAI-Inc/AdalFlow/main/docs/source/_static/images/classification_training_map.png" style="width: 80%;" alt="AdalFlow Auto-optimization">
-</p>
-
-<p align="center" style="background-color: #f0f0f0;">
-  <img src="https://raw.githubusercontent.com/SylphAI-Inc/AdalFlow/main/docs/source/_static/images/classification_opt_prompt.png" alt="AdalFlow Optimized Prompt" style="width: 80%;">
-</p>
-<!-- Among all libraries, AdalFlow achieved the highest accuracy with manual prompting (starting at 82%) and the highest accuracy after optimization. -->
-<!-- <p align="center" style="background-color: #f0f0f0;">
-  <img src="https://raw.githubusercontent.com/SylphAI-Inc/AdalFlow/main/docs/source/_static/images/classification_opt_prompt.png" alt="AdalFlow Optimized Prompt" style="width: 80%;">
-</p> -->
-
-Further reading: [Use Cases](https://adalflow.sylph.ai/use_cases/)
-
-## Light, Modular, and Model-Agnostic Task Pipeline
+<!-- ## Light, Modular, and Model-Agnostic Task Pipeline
 
 LLMs are like water; AdalFlow help you quickly shape them into any applications, from GenAI applications such as chatbots, translation, summarization, code generation, RAG, and autonomous agents to classical NLP tasks like text classification and named entity recognition.
 
@@ -170,53 +158,19 @@ Many providers and models accessible via the same interface:
 [All available model providers](https://adalflow.sylph.ai/apis/components/components.model_client.html)
 
 
-<!-- LLMs are like water; they can be shaped into anything, from GenAI applications such as chatbots, translation, summarization, code generation, and autonomous agents to classical NLP tasks like text classification and named entity recognition. They interact with the world beyond the model’s internal knowledge via retrievers, memory, and tools (function calls). Each use case is unique in its data, business logic, and user experience.
-
-Because of this, no library can provide out-of-the-box solutions. Users must build towards their own use case. This requires the library to be modular, robust, and have a clean, readable codebase. The only code you should put into production is code you either 100% trust or are 100% clear about how to customize and iterate. -->
 
-<!-- This is what AdalFlow is: light, modular, and robust, with a 100% readable codebase. -->
 
+Further reading: [How We Started](https://www.linkedin.com/posts/li-yin-ai_both-ai-research-and-engineering-use-pytorch-activity-7189366364694892544-Uk1U?utm_source=share&utm_medium=member_desktop),[Design Philosophy](https://adalflow.sylph.ai/tutorials/lightrag_design_philosophy.html) and [Class hierarchy](https://adalflow.sylph.ai/tutorials/class_hierarchy.html).
 
-Further reading: [How We Started](https://www.linkedin.com/posts/li-yin-ai_both-ai-research-and-engineering-use-pytorch-activity-7189366364694892544-Uk1U?utm_source=share&utm_medium=member_desktop), <!-- [Introduction](https://adalflow.sylph.ai/),  -->[Design Philosophy](https://adalflow.sylph.ai/tutorials/lightrag_design_philosophy.html) and [Class hierarchy](https://adalflow.sylph.ai/tutorials/class_hierarchy.html).
 
 
-<!--
-
-**PyTorch**
-
-```python
-import torch
-import torch.nn as nn
-
-class Net(nn.Module):
-   def __init__(self):
-      super(Net, self).__init__()
-      self.conv1 = nn.Conv2d(1, 32, 3, 1)
-      self.conv2 = nn.Conv2d(32, 64, 3, 1)
-      self.dropout1 = nn.Dropout2d(0.25)
-      self.dropout2 = nn.Dropout2d(0.5)
-      self.fc1 = nn.Linear(9216, 128)
-      self.fc2 = nn.Linear(128, 10)
-
-   def forward(self, x):
-      x = self.conv1(x)
-      x = self.conv2(x)
-      x = self.dropout1(x)
-      x = self.dropout2(x)
-      x = self.fc1(x)
-      return self.fc2(x)
-``` -->
 ## Unified Framework for Auto-Optimization
 
-<!-- AdalFlow provides token-efficient and high-performing prompt optimization within a unified framework. -->
+
 To optimize your pipeline, simply define a ``Parameter`` and pass it to AdalFlow's ``Generator``.
 You use `PROMPT` for prompt tuning via textual gradient descent and `DEMO` for few-shot demonstrations.
-<!-- Whether you need to optimize task instructions or run some few-shot demonstrations, -->
 We let you **diagnose**, **visualize**, **debug**, and **train** your pipeline.
 
-<!-- This [Dynamic Computation Graph](https://adalflow.sylph.ai/tutorials/trace_graph.html) demonstrates how our auto-differentiation and the dynamic computation graph work.
-
-No need to manually defined nodes and edges, AdalFlow will automatically trace the computation graph for you. -->
 
 ### **Trainable Task Pipeline**
 
@@ -234,23 +188,21 @@ Just define it as a ``Parameter`` and pass it to AdalFlow's ``Generator``.
     <img src="https://raw.githubusercontent.com/SylphAI-Inc/AdalFlow/main/docs/source/_static/images/trainer.png" alt="AdalFlow AdalComponent & Trainer">
 
 </p>
+ -->
 
 
-
-#
-
 # Documentation
 
 AdalFlow full documentation available at [adalflow.sylph.ai](https://adalflow.sylph.ai/):
-- [How We Started](https://www.linkedin.com/posts/li-yin-ai_both-ai-research-and-engineering-use-pytorch-activity-7189366364694892544-Uk1U?utm_source=share&utm_medium=member_desktop)
+<!-- - [How We Started](https://www.linkedin.com/posts/li-yin-ai_both-ai-research-and-engineering-use-pytorch-activity-7189366364694892544-Uk1U?utm_source=share&utm_medium=member_desktop)
 - [Introduction](https://adalflow.sylph.ai/)
 - [Full installation guide](https://adalflow.sylph.ai/get_started/installation.html)
 - [Design philosophy](https://adalflow.sylph.ai/tutorials/lightrag_design_philosophy.html)
 - [Class hierarchy](https://adalflow.sylph.ai/tutorials/class_hierarchy.html)
 - [Tutorials](https://adalflow.sylph.ai/tutorials/index.html)
 - [Supported Models](https://adalflow.sylph.ai/apis/components/components.model_client.html)
 - [Supported Retrievers](https://adalflow.sylph.ai/apis/components/components.retriever.html)
-- [API reference](https://adalflow.sylph.ai/apis/index.html)
+- [API reference](https://adalflow.sylph.ai/apis/index.html) -->
 
 
 # AdalFlow: A Tribute to Ada Lovelace

adalflow/CHANGELOG.md

@@ -1,3 +1,11 @@
+
+## [1.0.4] - 2025-02-13
+
+### Modified
+- `Embedder` and `BatchEmbedder` changed to `DataComponent`.
+
+### model_client (added)
+- `list_models` method.
 ## [1.0.2] - 2025-02-02
 ### Added
 - Added `TogetherClient` to support the Together API.

adalflow/adalflow/__init__.py

@@ -1,4 +1,4 @@
-__version__ = "1.0.3"
+__version__ = "1.0.4"
 
 from adalflow.core.component import (
     Component,
@@ -21,7 +21,7 @@
     Document,
 )
 from adalflow.core.model_client import ModelClient
-from adalflow.core.embedder import Embedder
+from adalflow.core.embedder import Embedder, BatchEmbedder
 
 # parser
 from adalflow.core.string_parser import (
@@ -65,16 +65,15 @@
     GroqAPIClient,
     OllamaClient,
     TransformersClient,
-    AnthropicAPIClient,
     CohereAPIClient,
     BedrockAPIClient,
     DeepSeekClient,
     TogetherClient,
+    AnthropicAPIClient,
 )
 
 # data pipeline
 from adalflow.components.data_process.text_splitter import TextSplitter
-from adalflow.components.data_process.data_components import ToEmbeddings
 
 __all__ = [
     "Component",
@@ -96,6 +95,7 @@
     "ModelClient",
     "Generator",
     "Embedder",
+    "BatchEmbedder",
     "Retriever",
     "Parameter",
     "AdalComponent",
@@ -146,4 +146,5 @@
     "CohereAPIClient",
     "BedrockAPIClient",
     "TogetherClient",
+    "AnthropicAPIClient",
 ]

adalflow/adalflow/components/agent/react.py

@@ -111,7 +111,6 @@
 "kwargs": {{history.action.kwargs}}",
 {% endif %}
 "Observation": "{{history.observation}}"
-
 ------------------------
 {% endfor %}
 </STEPS>

adalflow/adalflow/components/data_process/__init__.py

@@ -1,11 +1,11 @@
 """Components here are used for data processing/transformation."""
 
 from .text_splitter import TextSplitter
-from .data_components import ToEmbeddings, RetrieverOutputToContextStr
+from .data_components import RetrieverOutputToContextStr, ToEmbeddings
 from adalflow.utils.registry import EntityMapping
 
 
-__all__ = ["TextSplitter", "ToEmbeddings", "RetrieverOutputToContextStr"]
+__all__ = ["TextSplitter", "RetrieverOutputToContextStr", "ToEmbeddings"]
 
 for name in __all__:
     EntityMapping.register(name, globals()[name])

adalflow/adalflow/components/data_process/data_components.py

@@ -88,7 +88,7 @@ def __call__(self, input: ToEmbeddingsInputType) -> ToEmbeddingsOutputType:
         # convert documents to a list of strings
         embedder_input: BatchEmbedderInputType = [chunk.text for chunk in output]
         outputs: BatchEmbedderOutputType = self.batch_embedder(input=embedder_input)
-        # n them back to the original order along with its query
+        # put them back to the original order along with its query
         for batch_idx, batch_output in tqdm(
             enumerate(outputs), desc="Adding embeddings to documents from batch"
         ):

adalflow/adalflow/components/model_client/__init__.py

@@ -81,6 +81,10 @@
     "adalflow.components.model_client.together_client.TogetherClient",
     OptionalPackages.TOGETHER,
 )
+AzureAIClient = LazyImport(
+    "adalflow.components.model_client.azureai_client.AzureAIClient",
+    OptionalPackages.AZURE,
+)
 get_first_message_content = LazyImport(
     "adalflow.components.model_client.openai_client.get_first_message_content",
     OptionalPackages.OPENAI,

adalflow/adalflow/components/model_client/groq_client.py

@@ -163,3 +163,16 @@ def to_dict(self) -> Dict[str, Any]:
         ]  # unserializable object
         output = super().to_dict(exclude=exclude)
         return output
+
+    def list_models(self):
+        return self.sync_client.models.list()
+
+
+if __name__ == "__main__":
+
+    from adalflow.utils import setup_env
+
+    setup_env()
+
+    client = GroqAPIClient()
+    print(client.list_models())

adalflow/adalflow/components/model_client/ollama_client.py

@@ -64,6 +64,13 @@ class OllamaClient(ModelClient):
 
     - [Download Ollama app] Go to https://github.com/ollama/ollama?tab=readme-ov-file to download the Ollama app (command line tool).
       Choose the appropriate version for your operating system.
+      One way to do is to run the following command:
+
+    .. code-block:: shell
+
+        curl -fsSL https://ollama.com/install.sh | sh
+        ollama serve
+
 
     - [Pull a model] Run the following command to pull a model:
 

adalflow/adalflow/components/model_client/openai_client.py

@@ -574,3 +574,18 @@ def _prepare_image_content(
     # model_client_dict = model_client.to_dict()
     # from_dict_model_client = OpenAIClient.from_dict(model_client_dict)
     # assert model_client_dict == from_dict_model_client.to_dict()
+
+
+if __name__ == "__main__":
+    import adalflow as adal
+
+    # setup env or pass the api_key
+    from adalflow.utils import setup_env
+
+    setup_env()
+
+    openai_llm = adal.Generator(
+        model_client=adal.OpenAIClient(), model_kwargs={"model": "gpt-3.5-turbo"}
+    )
+    resopnse = openai_llm(prompt_kwargs={"input_str": "What is LLM?"})
+    print(resopnse)