TypeError("'NoneType' object is not callable")

[RamishSiddiqui]

System Info

OS: Redhat 8
Python: 3.9
Langchain: 0.0.246

Who can help?

No response

Information

• The official example notebooks/scripts
• My own modified scripts

Related Components

• LLMs/Chat Models
• Embedding Models
• Prompts / Prompt Templates / Prompt Selectors
• Output Parsers
• Document Loaders
• Vector Stores / Retrievers
• Memory
• Agents / Agent Executors
• Tools / Toolkits
• Chains
• Callbacks/Tracing
• Async

Reproduction

So i initially reported this bug with mlflow, bug upon further investigation it is related to langchain.

Following code is simple representation of bigger code. I will also post that in bottom.

===================================================================

Code:

import os
from langchain import PromptTemplate, LLMChain
from langchain.llms import HuggingFacePipeline


os.environ["CURL_CA_BUNDLE"] = ""
if True:  # run the following code to download the model flan-t5-small from huggingface.co
  from transformers import pipeline
  
  model = pipeline(model="google/flan-t5-small")  #'text2text-generation'
  model.save_pretrained("/tmp/model/flan-t5-small")

llm = HuggingFacePipeline.from_model_id(
    model_id="/tmp/model/flan-t5-small",
    task="text2text-generation",
    model_kwargs={"temperature": 1e-10},
)

template = """Translate everything you see after this into French:

{input}
"""

prompt = PromptTemplate(template=template, input_variables=["input"])

llm_chain = LLMChain(prompt=prompt, llm=llm)

print(llm_chain("my name is John"))  # works

llm_chain.save("llm_chain.json")

from langchain.chains import load_chain

m = load_chain("llm_chain.json")

print(m("my name is John"))

Error trace:

{'input': 'my name is John', 'text': " toutefois, je suis en uvre à l'heure"}

Traceback (most recent call last):

File "a.py", line 37, in 

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/chains/base.py", line 258, in call

raise e

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/chains/base.py", line 252, in call

self._call(inputs, run_manager=run_manager)

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/chains/llm.py", line 92, in _call

response = self.generate([inputs], run_manager=run_manager)

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/chains/llm.py", line 102, in generate

return self.llm.generate_prompt(

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/base.py", line 451, in generate_prompt

return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs)

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/base.py", line 582, in generate

output = self._generate_helper(

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/base.py", line 488, in _generate_helper

raise e

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/base.py", line 475, in _generate_helper

self._generate(

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/base.py", line 961, in _generate

self._call(prompt, stop=stop, run_manager=run_manager, **kwargs)

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/huggingface_pipeline.py", line 168, in _call

response = self.pipeline(prompt)

TypeError: 'NoneType' object is not callable

===================================================================

Original Code where this bug occurred (MLFlow):

import mlflow
from datetime import datetime
import logging

logging.getLogger("mlflow").setLevel(logging.DEBUG)

from langchain import PromptTemplate, LLMChain, HuggingFaceHub
from langchain.llms import HuggingFacePipeline
import os

def now_str():
return datetime.now().strftime("%Y%m%d%H%M%S")

os.environ["CURL_CA_BUNDLE"] = ""
if True:
from transformers import pipeline

model = pipeline(model="google/flan-t5-small") #'text2text-generation'
model.save_pretrained("/tmp/model/flan-t5-small")

llm = HuggingFacePipeline.from_model_id(
model_id="/tmp/model/flan-t5-small",
task="text2text-generation",
model_kwargs={"temperature": 1e-10},
)

template = """Translate everything you see after this into French:

{input}
"""

prompt = PromptTemplate(template=template, input_variables=["input"])

llm_chain = LLMChain(prompt=prompt, llm=llm)

print(llm_chain.run("my name is John")) # This is working !!

Output: {'input': 'my name is John', 'text': "j'ai le nom de John"}

experiment_id = mlflow.create_experiment(f"HF_LLM_{now_str()}")

with mlflow.start_run(experiment_id=experiment_id) as run:
logged_model = mlflow.langchain.log_model(
lc_model=llm_chain,
artifact_path="HF_LLM",
)

m = mlflow.langchain.load_model(logged_model.model_uri)
m.run("my name is John")


Error Trace:

Traceback (most recent call last):

File "a.py", line 51, in 

m.run("my name is John")

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/chains/base.py", line 451, in run

return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/chains/base.py", line 258, in call

raise e

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/chains/base.py", line 252, in call

self._call(inputs, run_manager=run_manager)

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/chains/llm.py", line 92, in _call

response = self.generate([inputs], run_manager=run_manager)

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/chains/llm.py", line 102, in generate

return self.llm.generate_prompt(

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/base.py", line 451, in generate_prompt

return self.generate(prompt_strings, stop=stop, callbacks=callbacks, **kwargs)

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/base.py", line 582, in generate

output = self._generate_helper(

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/base.py", line 488, in _generate_helper

raise e

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/base.py", line 475, in _generate_helper

self._generate(

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/base.py", line 961, in _generate

self._call(prompt, stop=stop, run_manager=run_manager, **kwargs)

File "/home/haru/miniconda3/envs/mlflow-dev-env/lib/python3.8/site-packages/langchain/llms/huggingface_pipeline.py", line 168, in _call

response = self.pipeline(prompt)

TypeError: 'NoneType' object is not callable

===================================================================

Another Example:

from datetime import datetime

import logging

logging.getLogger("mlflow").setLevel(logging.DEBUG)

from langchain import PromptTemplate, LLMChain, HuggingFaceHub

from langchain.llms import HuggingFacePipeline

import os
def now_str():

return datetime.now().strftime("%Y%m%d%H%M%S")
os.environ["CURL_CA_BUNDLE"] = ''

if (True): # run the following code to download the model flan-t5-large from huggingface.co

from transformers import pipeline

model= pipeline(model="google/flan-t5-large") #'text2text-generation'

model.save_pretrained("/tmp/model/flan-t5-large")
llm = HuggingFacePipeline.from_model_id(model_id="/tmp/model/flan-t5-large", task="text2text-generation", model_kwargs={"temperature":1e-10})
template = """Translate everything you see after this into French:
{input}

"""
prompt = PromptTemplate(template=template, input_variables=["input"])
llm_chain = LLMChain(

prompt=prompt,

llm=llm

)
llm_chain("my name is John") # This is working !!

#Output: {'input': 'my name is John', 'text': "j'ai le nom de John"}
experiment_id = mlflow.create_experiment(f'HF_LLM_{now_str()}')
with mlflow.start_run(experiment_id=experiment_id) as run:

logged_model = mlflow.langchain.log_model(

lc_model=llm_chain,

artifact_path="HF_LLM",

)
=============== This throws error ==============

input_str = "My name is John"
loaded_model = mlflow.pyfunc.load_model(logged_model.model_uri)

output = loaded_model.predict(

[

{

"input": input_str

},

{

"input": "Do you like coffee?"

}

]

)
print(output)

Error Trace:

2023/08/07 08:57:08 WARNING mlflow.langchain.api_request_parallel_processor: Request #0 failed with TypeError("'NoneType' object is not callable")

2023/08/07 08:57:08 WARNING mlflow.langchain.api_request_parallel_processor: Request #1 failed with TypeError("'NoneType' object is not callable")

MlflowException Traceback (most recent call last)
in <cell line: 3>()
3 with project.setup_mlflow(mf) as mlflow:
4 loaded_model = mlflow.pyfunc.load_model(logged_model.model_uri)
----> 5 output = loaded_model.predict(
6 [
7 {

/hadoopfs/fs1/dataiku/data_dir/code-envs/python/mlflow_25_python_39/lib/python3.9/site-packages/mlflow/pyfunc/init.py in predict(self, data)
426 raise
427
--> 428 return self._predict_fn(data)
429
430 @experimental

/hadoopfs/fs1/dataiku/data_dir/code-envs/python/mlflow_25_python_39/lib/python3.9/site-packages/mlflow/langchain/init.py in predict(self, data)
654 "Input must be a pandas DataFrame or a list of strings or a list of dictionaries",
655 )
--> 656 return process_api_requests(lc_model=self.lc_model, requests=messages)
657
658

/hadoopfs/fs1/dataiku/data_dir/code-envs/python/mlflow_25_python_39/lib/python3.9/site-packages/mlflow/langchain/api_request_parallel_processor.py in process_api_requests(lc_model, requests, max_workers)
138 # after finishing, log final status
139 if status_tracker.num_tasks_failed > 0:
--> 140 raise mlflow.MlflowException(
141 f"{status_tracker.num_tasks_failed} tasks failed. See logs for details."
142 )

MlflowException: 2 tasks failed. See logs for details.


Looks like langchain doesn't restore the pipeline.

Expected behavior

Langhain loads the chain successfully.

[dosubot]

Answer generated by a 🤖

Answer

Hello,

Thank you for reaching out and providing a detailed description of the issue you're facing. The error you're encountering, TypeError: 'NoneType' object is not callable, typically occurs when you're trying to call a function or method that doesn't exist or hasn't been properly initialized. In your case, it seems like the issue is with the pipeline in the HuggingFacePipeline class not being callable.

From the code snippet you've shared, it appears that you're saving a model using the save_pretrained method and then trying to load it using the HuggingFacePipeline.from_model_id method. However, the from_model_id method expects a model ID that corresponds to a model on the HuggingFace Model Hub, not a local file path. This could be the reason why the pipeline is not being properly initialized, leading to the error you're seeing.

To resolve this issue, you could try uploading your model to the HuggingFace Model Hub and then using the model ID to load it. Alternatively, you could modify the HuggingFacePipeline class to support loading models from a local file path.

Regarding the load_chain function, it's a unified method for loading a chain from LangChainHub or local filesystem. It first tries to load the chain from LangChainHub, and if it fails, it loads the chain from a local file. You can find more details about its implementation in the LangChain codebase.

If you're still encountering the error, please ensure that the path you're providing to the load_chain function is correct and the chain exists either on LangChainHub or your local filesystem. Also, check if the file you're trying to load is of the correct format. The _load_chain_from_file function supports only json and yaml files. If the file is of any other format, it will raise a ValueError.

Lastly, ensure that the chain you're trying to load is supported by the LangChain framework. The load_chain_from_config function uses a dictionary type_to_loader_dict to map the type of the chain to its corresponding loader function. If the type of the chain you're trying to load is not in this dictionary, it will raise a ValueError.

I hope this helps! If you have any other questions or need further clarification, please don't hesitate to ask.

Best,
Dosu

Sources

• libs/langchain/langchain/callbacks/mlflow_callback.py
• libs/langchain/langchain/chains/loading.py
• docs/extras/modules/chains/how_to/serialization.ipynb
• libs/langchain/langchain/chains/qa_with_sources/loading.py

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

[RamishSiddiqui]

Hi, this solution is not working for me.

[dosubot]

Answer generated by a 🤖

Answer

Hello,

The error you're encountering, TypeError: 'NoneType' object is not callable, typically occurs when you're trying to call a None object as if it were a function or method. In the context of the HuggingFacePipeline.from_model_id method, this error could be due to several reasons:

1. The transformers library might not be installed or properly installed. The from_model_id method tries to import several classes and functions from the transformers library. If the library is not installed or not properly installed, these imports will fail, and the imported names will be None. When the method later tries to call these None objects as if they were functions or classes, the TypeError will occur. You can install it with pip install transformers.

2. The model_id you provided does not correspond to a valid model in the HuggingFace Model Hub. The from_model_id method tries to load a model and a tokenizer from the HuggingFace Model Hub using the provided model_id. If the model_id does not correspond to a valid model, the loading will fail, and the loaded model and tokenizer will be None. When the method later tries to call these None objects as if they were functions or classes, the TypeError will occur.

3. The task you provided is not supported. The from_model_id method supports only three tasks: "text-generation", "text2text-generation", and "summarization". If you provided a different task, the method will not know how to load the corresponding model, and the loaded model will be None. When the method later tries to call this None object as if it were a function or class, the TypeError will occur.

4. The device you provided is not valid. The _load_transformer function checks if the device is within the range of available CUDA devices. If the device is not within this range, the function will raise a ValueError. If this ValueError is not properly caught and handled, it could cause the function to return None, leading to the TypeError when the returned None object is later called as if it were a function or class.

5. The huggingface_hub python package might not be installed or properly installed. The HuggingFaceHub class in LangChain uses this package. If this package is not installed or not properly installed, the ImportError will be raised when trying to import the InferenceApi class from the huggingface_hub.inference_api module. This could cause the client attribute of the HuggingFaceHub instance to be None, leading to the TypeError when the client attribute is later called as if it were a function or method. You can install it with pip install huggingface_hub.

Please check these points and let me know if the issue persists. If it does, could you please provide the exact code snippet that is causing the error? This will help me understand the issue better and provide a more accurate solution.

Best regards,
Dosu

Sources

• libs/langchain/langchain/llms/huggingface_pipeline.py
• libs/langchain/langchain/llms/self_hosted_hugging_face.py
• libs/langchain/langchain/llms/huggingface_hub.py

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

[dosubot]

Hi, @RamishSiddiqui. I'm Dosu, and I'm helping the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, you encountered a TypeError when using the Langchain library, specifically a 'NoneType' object not callable. Dosubot provided some suggestions, such as uploading the model to the HuggingFace Model Hub or modifying the HuggingFacePipeline class. However, it seems that these solutions didn't work for you. Dosubot then asked for the exact code snippet causing the error to further troubleshoot the issue.

Before we close this issue, we wanted to check if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding.