Notebook samples outdated??

[biggikalli]

I tried Autogen with a Github Codespace, straight from the repo and a fresh Openai API key for GPT-4.
Every single notebook I tried resulted in errors while it burned through $20 in API calls.

What is the most likley reason for this?

See below sample results from two of the notebooks I ran.

The "agentchat_teaching.ipynb" notebook:

`--------------------------------------------------------------------------------

EXECUTING CODE BLOCK 0 (inferred language is python)...
user_proxy (to assistant):

exitcode: 0 (execution succeeded)
Code output:

---

assistant (to user_proxy):

I apologize for the inconvenience. It seems like there are no papers on arXiv that match the search queries "trust calibration in AI" or "trust in AI".

This could be due to a few reasons:

1. The topic is very specific and there might not be many papers on it yet.
2. The papers on this topic might not be on arXiv. They could be on other databases or journals.
3. The terminology used in the papers might be different. For example, they might use terms like "confidence in AI" or "reliability of AI" instead of "trust".

I recommend trying to search for papers on other databases or journals, or using different search queries.

TERMINATE
`

The "agentchat_auto_feedback_from_code_execution" notebook:

`user_proxy (to assistant):

exitcode: 1 (execution failed)
Code output:
Traceback (most recent call last):
File "", line 20, in
meta_start_price = get_stock_price("FB", start_date)
File "", line 16, in get_stock_price
return float(data["Time Series (Daily)"][date.strftime("%Y-%m-%d")]["4. close"])
KeyError: 'Time Series (Daily)'

---

assistant (to user_proxy):

I apologize for the confusion. The error message suggests that the Alpha Vantage API response does not contain the "Time Series (Daily)" key. This could be due to a variety of reasons, such as an invalid API key, an invalid stock symbol, or issues with the Alpha Vantage API.

Unfortunately, without more information, it's difficult to determine the exact cause of the problem. However, I can suggest a modification to the code to print the API response. This can help us understand what's going wrong.

Here's the modified code:

# python code
import datetime
import requests
import json

# Get the current date
current_date = datetime.date.today()

# Define the Alpha Vantage API key
api_key = "YOUR_ALPHA_VANTAGE_API_KEY"

# Define the function to get the stock price
def get_stock_price(symbol, date):
    response = requests.get(f"https://www.alphavantage.co/query?function=TIME_SERIES_DAILY&symbol={symbol}&apikey={api_key}")
    data = json.loads(response.text)
    print(data)  # Print the API response
    return float(data["Time Series (Daily)"][date.strftime("%Y-%m-%d")]["4. close"])

# Get the stock prices at the start of the year
start_date = datetime.date(current_date.year, 1, 1)
meta_start_price = get_stock_price("FB", start_date)
tesla_start_price = get_stock_price("TSLA", start_date)

# Get the current stock prices
meta_current_price = get_stock_price("FB", current_date)
tesla_current_price = get_stock_price("TSLA", current_date)

# Calculate the YTD gain
meta_ytd_gain = ((meta_current_price - meta_start_price) / meta_start_price) * 100
tesla_ytd_gain = ((tesla_current_price - tesla_start_price) / tesla_start_price) * 100

print("META (FB) YTD gain:", meta_ytd_gain, "%")
print("TESLA YTD gain:", tesla_ytd_gain, "%")

Please replace "YOUR_ALPHA_VANTAGE_API_KEY" with your actual Alpha Vantage API key. If you don't have an API key, you can get one for free by signing up on the Alpha Vantage website.

Please note that the Alpha Vantage free API has a limit of 5 API requests per minute and 500 API requests per day. If you exceed these limits, you may need to wait for a while before you can make more API requests.

---

EXECUTING CODE BLOCK 0 (inferred language is python)...
user_proxy (to assistant):

exitcode: 1 (execution failed)
Code output:
Traceback (most recent call last):
File "", line 21, in
meta_start_price = get_stock_price("FB", start_date)
File "", line 17, in get_stock_price
return float(data["Time Series (Daily)"][date.strftime("%Y-%m-%d")]["4. close"])
KeyError: 'Time Series (Daily)'

--------------------------------------------------------------------------------`

[qingyun-wu]

Hi @biggikalli, It's true that the results from the notebook can differ across runs. One potential factor is the inherent stochastic nature of the language models like GPT-4, which means results may vary across different runs. Additionally, if any notebooks involve tasks that fetch information from the web, the outcome could also be influenced by real-time data changes or connectivity issues.

Regarding the first notebook example you mentioned, it seems more like an unsatisfactory solution rather than an outright error. To enhance the results, you might consider setting the "human_input_mode" to "ALWAYS" and instructing the assistant (via user_proxy) not to strictly adhere to the exact search queries like "trust calibration in AI" or "trust in AI".

The cost is indeed a potential concern if you use GPT-4 in all the use cases. For some of the tasks, cheaper models, e.g., GPT-3.5 or even free OSS models could give decent results as well.

Are there specific application domains or task scenarios you're focusing on? This could help in providing more tailored suggestions on how to construct a multi-agent system to yield satisfactory results and save costs.

Thank you!

@sonichi @LittleLittleCloud @afourney feel free to chime in if you have thoughts or suggestions on the issues raised here.

[sonichi]

For the second notebook, I got similar error during the conversation, though the code is fixed eventually. Can others test and report if it works?