Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Errors in visualwebarena #217

Open
recursix opened this issue Oct 28, 2024 · 0 comments
Open

Errors in visualwebarena #217

recursix opened this issue Oct 28, 2024 · 0 comments

Comments

@recursix
Copy link
Collaborator

16x : Exception uncaught by agent or environment in task <task_name>.
TimeoutError:
Timeout 10000ms exceeded.
========================

  • visualwebarena.282 seed: 22
  • visualwebarena.305 seed: 18
  • visualwebarena.314 seed: 0
  • visualwebarena.316 seed: 33
  • visualwebarena.317 seed: 31
  • visualwebarena.318 seed: 24
  • visualwebarena.319 seed: 0
  • visualwebarena.322 seed: 21
  • visualwebarena.403 seed: 16
  • visualwebarena.410 seed: 18
  • visualwebarena.411 seed: 11
  • visualwebarena.427 seed: 4
  • visualwebarena.436 seed: 6
  • visualwebarena.437 seed: 12
  • visualwebarena.438 seed: 8
  • visualwebarena.440 seed: 1

Showing Max 2 stack traces:

2024-10-28 01:38:58,428 - 2180615 - browsergym.experiments.loop - INFO - Running experiment GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.282_22 in:
  /home/toolkit/agentlab_results/2024-10-24_21-33-14_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena/2024-10-28_01-38-10_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.282_22
2024-10-28 01:38:58,431 - 2180615 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:38:58,432 - 2180615 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:38:58,458 - 2180615 - browsergym.experiments.loop - DEBUG - Agent created.
2024-10-28 01:38:58,547 - 2180615 - browsergym.experiments.loop - DEBUG - Environment created.
2024-10-28 01:38:58,965 - 2180615 - asyncio - DEBUG - Using selector: EpollSelector
2024-10-28 01:39:01,707 - 2180615 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:39:01,708 - 2180615 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:39:01,717 - 2180615 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
...
...truncated middle of the log
...
2024-10-28 01:39:09,115 - 2180615 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/'>
2024-10-28 01:39:12,031 - 2180615 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/f/dataisbeautiful'>
2024-10-28 01:39:21,144 - 2180615 - browsergym.experiments.loop - WARNING - Exception uncaught by agent or environment in task visualwebarena.282.
TimeoutError:
Timeout 10000ms exceeded.
=========================== logs ===========================
navigating to "http://visualwebarena.eastus.cloudapp.azure.com:8080/f/dataisbeautiful", waiting until "load"
============================================================
Traceback (most recent call last):
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 216, in run
    step_info.from_reset(
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 405, in from_reset
    self.obs, env_info = env.reset(seed=seed)
                         ^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/time_limit.py", line 75, in reset
    return self.env.reset(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/order_enforcing.py", line 61, in reset
    return self.env.reset(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 285, in reset
    task_goal, task_info = self.task.setup(page=self.page)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/visualwebarena/src/browsergym/visualwebarena/task.py", line 204, in setup
    page.goto(url)
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/sync_api/_generated.py", line 9303, in goto
    self._sync(
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
    return task.result()
           ^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_page.py", line 473, in goto
    return await self._main_frame.goto(**locals_to_params(locals()))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_frame.py", line 138, in goto
    await self._channel.send("goto", locals_to_params(locals()))
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 490, in wrap_api_call
    return await cb()
           ^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 99, in inner_send
    result = next(iter(done)).result()
             ^^^^^^^^^^^^^^^^^^^^^^^^^
playwright._impl._api_types.TimeoutError: Timeout 10000ms exceeded.
=========================== logs ===========================
navigating to "http://visualwebarena.eastus.cloudapp.azure.com:8080/f/dataisbeautiful", waiting until "load"
============================================================


2024-10-28 01:38:58,366 - 2180618 - browsergym.experiments.loop - INFO - Running experiment GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.305_18 in:
  /home/toolkit/agentlab_results/2024-10-24_21-33-14_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena/2024-10-28_01-38-11_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.305_18
2024-10-28 01:38:58,368 - 2180618 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:38:58,370 - 2180618 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:38:58,394 - 2180618 - browsergym.experiments.loop - DEBUG - Agent created.
2024-10-28 01:38:58,455 - 2180618 - browsergym.experiments.loop - DEBUG - Environment created.
2024-10-28 01:38:58,879 - 2180618 - asyncio - DEBUG - Using selector: EpollSelector
2024-10-28 01:39:01,629 - 2180618 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:39:01,629 - 2180618 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:39:01,638 - 2180618 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
...
...truncated middle of the log
...
2024-10-28 01:39:08,417 - 2180618 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/'>
2024-10-28 01:39:09,799 - 2180618 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/f/memes/new'>
2024-10-28 01:39:20,434 - 2180618 - browsergym.experiments.loop - WARNING - Exception uncaught by agent or environment in task visualwebarena.305.
TimeoutError:
Timeout 10000ms exceeded.
=========================== logs ===========================
navigating to "http://visualwebarena.eastus.cloudapp.azure.com:8080/f/memes/new", waiting until "load"
============================================================
Traceback (most recent call last):
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 216, in run
    step_info.from_reset(
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 405, in from_reset
    self.obs, env_info = env.reset(seed=seed)
                         ^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/time_limit.py", line 75, in reset
    return self.env.reset(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/order_enforcing.py", line 61, in reset
    return self.env.reset(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 285, in reset
    task_goal, task_info = self.task.setup(page=self.page)
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/visualwebarena/src/browsergym/visualwebarena/task.py", line 204, in setup
    page.goto(url)
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/sync_api/_generated.py", line 9303, in goto
    self._sync(
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
    return task.result()
           ^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_page.py", line 473, in goto
    return await self._main_frame.goto(**locals_to_params(locals()))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_frame.py", line 138, in goto
    await self._channel.send("goto", locals_to_params(locals()))
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 490, in wrap_api_call
    return await cb()
           ^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 99, in inner_send
    result = next(iter(done)).result()
             ^^^^^^^^^^^^^^^^^^^^^^^^^
playwright._impl._api_types.TimeoutError: Timeout 10000ms exceeded.
=========================== logs ===========================
navigating to "http://visualwebarena.eastus.cloudapp.azure.com:8080/f/memes/new", waiting until "load"
============================================================

8x : Exception uncaught by agent or environment in task <task_name>.
TypeError:
'NoneType' object is not callabl

  • visualwebarena.597 seed: 12
  • visualwebarena.598 seed: 10
  • visualwebarena.602 seed: 10
  • visualwebarena.608 seed: 2
  • visualwebarena.610 seed: 2
  • visualwebarena.730 seed: 1
  • visualwebarena.739 seed: 29
  • visualwebarena.741 seed: 4

Showing Max 2 stack traces:

2024-10-28 01:40:47,810 - 2180614 - browsergym.experiments.loop - INFO - Running experiment GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.597_12 in:
  /home/toolkit/agentlab_results/2024-10-24_21-33-14_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena/2024-10-28_01-38-11_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.597_12
2024-10-28 01:40:47,811 - 2180614 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:40:47,813 - 2180614 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:40:47,835 - 2180614 - browsergym.experiments.loop - DEBUG - Agent created.
2024-10-28 01:40:47,836 - 2180614 - browsergym.experiments.loop - DEBUG - Environment created.
2024-10-28 01:40:49,014 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='about:blank'>
2024-10-28 01:40:49,615 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/customer/account/login/'>
2024-10-28 01:40:50,545 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/customer/account/login/'>
2024-10-28 01:40:50,549 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/customer/account/login/'>
...
...truncated middle of the log
...
action:
click('1625')  # This link appears to be for the Microsoft Xbox One Console with Accessories, 1TB HDD - Black.

2024-10-28 01:41:39,838 - 2180614 - browsergym.experiments.loop - DEBUG - Chat info sent.
2024-10-28 01:41:39,838 - 2180614 - browsergym.experiments.loop - DEBUG - Sending action to environment.
2024-10-28 01:41:39,838 - 2180614 - browsergym.core.env - DEBUG - Executing action
2024-10-28 01:41:39,943 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/catalogsearch/result/?q=xbox+one+console'>
2024-10-28 01:41:39,946 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/catalogsearch/result/?q=xbox+one+console'>
2024-10-28 01:41:39,948 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/catalogsearch/result/?q=xbox+one+console'>
2024-10-28 01:41:39,949 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/catalogsearch/result/?q=xbox+one+console'>
2024-10-28 01:41:39,951 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/catalogsearch/result/?q=xbox+one+console'>
2024-10-28 01:41:40,979 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/microsoft-xbox-one-console-with-accessories-1tb-hdd-black.html'>
2024-10-28 01:41:41,189 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/microsoft-xbox-one-console-with-accessories-1tb-hdd-black.html'>
2024-10-28 01:41:41,192 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/microsoft-xbox-one-console-with-accessories-1tb-hdd-black.html'>
2024-10-28 01:41:42,957 - 2180614 - browsergym.core.env - DEBUG - Action executed
2024-10-28 01:41:43,512 - 2180614 - browsergym.core.env - DEBUG - Active page checked
2024-10-28 01:41:43,513 - 2180614 - browsergym.core.env - DEBUG - User message done
2024-10-28 01:41:43,513 - 2180614 - browsergym.core.env - DEBUG - Initiating task validation
2024-10-28 01:41:43,691 - 2180614 - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): visualwebarena.eastus.cloudapp.azure.com:8082
2024-10-28 01:41:43,792 - 2180614 - urllib3.connectionpool - DEBUG - http://visualwebarena.eastus.cloudapp.azure.com:8082 "GET /media/catalog/product/cache/829a59e57f886f8cf0598ffca4f8a940/B/0/B00KL3WBBC.0.jpg HTTP/11" 200 14830
2024-10-28 01:41:43,860 - 2180614 - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): visualwebarena.eastus.cloudapp.azure.com:8082
2024-10-28 01:41:43,947 - 2180614 - urllib3.connectionpool - DEBUG - http://visualwebarena.eastus.cloudapp.azure.com:8082 "GET /media/catalog/product/cache/829a59e57f886f8cf0598ffca4f8a940/B/0/B00KL3WBBC.1.jpg HTTP/11" 200 17349
2024-10-28 01:41:43,976 - 2180614 - browsergym.experiments.loop - WARNING - Exception uncaught by agent or environment in task visualwebarena.597.
TypeError:
'NoneType' object is not callable
Traceback (most recent call last):
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 246, in run
    step_info.from_step(env, action, obs_preprocessor=agent.obs_preprocessor)
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 379, in from_step
    self.obs, self.reward, self.terminated, self.truncated, env_info = env.step(action)
                                                                       ^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/time_limit.py", line 57, in step
    observation, reward, terminated, truncated, info = self.env.step(action)
                                                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/order_enforcing.py", line 56, in step
    return self.env.step(action)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 415, in step
    reward, done, user_message, task_info = self._task_validate()
                                            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 440, in _task_validate
    reward, done, user_message, info = self.task.validate(self.page, self.chat.messages)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/visualwebarena/src/browsergym/visualwebarena/task.py", line 259, in validate
    score = self.evaluator(
            ^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/visualwebarena/evaluation_harness/evaluators.py", line 636, in __call__
    cur_score = evaluator(trajectory, config_file, page)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<@beartype(visualwebarena.evaluation_harness.evaluators.PageImageEvaluator.__call__) at 0x7fb75721b2e0>", line 92, in __call__
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/visualwebarena/evaluation_harness/evaluators.py", line 576, in __call__
    pred_ans = self.captioning_fn(
               ^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable


2024-10-28 01:41:03,079 - 2186124 - browsergym.experiments.loop - INFO - Running experiment GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.598_10 in:
  /home/toolkit/agentlab_results/2024-10-24_21-33-14_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena/2024-10-28_01-38-11_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.598_10
2024-10-28 01:41:03,081 - 2186124 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:41:03,082 - 2186124 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:41:03,110 - 2186124 - browsergym.experiments.loop - DEBUG - Agent created.
2024-10-28 01:41:03,171 - 2186124 - browsergym.experiments.loop - DEBUG - Environment created.
2024-10-28 01:41:03,720 - 2186124 - asyncio - DEBUG - Using selector: EpollSelector
2024-10-28 01:41:07,604 - 2186124 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:41:07,605 - 2186124 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:41:07,615 - 2186124 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
...
...truncated middle of the log
...
action:
click('1528')  # Click on the first headphone option to view details.

2024-10-28 01:41:58,377 - 2186124 - browsergym.experiments.loop - DEBUG - Chat info sent.
2024-10-28 01:41:58,377 - 2186124 - browsergym.experiments.loop - DEBUG - Sending action to environment.
2024-10-28 01:41:58,377 - 2186124 - browsergym.core.env - DEBUG - Executing action
2024-10-28 01:41:58,494 - 2186124 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/catalogsearch/result/?q=Sony+headphones'>
2024-10-28 01:41:58,498 - 2186124 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/catalogsearch/result/?q=Sony+headphones'>
2024-10-28 01:41:58,499 - 2186124 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/catalogsearch/result/?q=Sony+headphones'>
2024-10-28 01:41:58,502 - 2186124 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/catalogsearch/result/?q=Sony+headphones'>
2024-10-28 01:41:58,503 - 2186124 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/catalogsearch/result/?q=Sony+headphones'>
2024-10-28 01:41:59,328 - 2186124 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/sony-zx110nc-noise-cancelling-headphones.html'>
2024-10-28 01:41:59,547 - 2186124 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/sony-zx110nc-noise-cancelling-headphones.html'>
2024-10-28 01:41:59,550 - 2186124 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8082/sony-zx110nc-noise-cancelling-headphones.html'>
2024-10-28 01:42:01,497 - 2186124 - browsergym.core.env - DEBUG - Action executed
2024-10-28 01:42:02,052 - 2186124 - browsergym.core.env - DEBUG - Active page checked
2024-10-28 01:42:02,052 - 2186124 - browsergym.core.env - DEBUG - User message done
2024-10-28 01:42:02,052 - 2186124 - browsergym.core.env - DEBUG - Initiating task validation
2024-10-28 01:42:02,222 - 2186124 - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): visualwebarena.eastus.cloudapp.azure.com:8082
2024-10-28 01:42:02,328 - 2186124 - urllib3.connectionpool - DEBUG - http://visualwebarena.eastus.cloudapp.azure.com:8082 "GET /media/catalog/product/cache/829a59e57f886f8cf0598ffca4f8a940/B/0/B01K5CTWPG.0.jpg HTTP/11" 200 17096
2024-10-28 01:42:02,417 - 2186124 - urllib3.connectionpool - DEBUG - Starting new HTTP connection (1): visualwebarena.eastus.cloudapp.azure.com:8082
2024-10-28 01:42:02,487 - 2186124 - urllib3.connectionpool - DEBUG - http://visualwebarena.eastus.cloudapp.azure.com:8082 "GET /media/catalog/product/cache/829a59e57f886f8cf0598ffca4f8a940/B/0/B01K5CTWPG.1.jpg HTTP/11" 200 18600
2024-10-28 01:42:02,518 - 2186124 - browsergym.experiments.loop - WARNING - Exception uncaught by agent or environment in task visualwebarena.598.
TypeError:
'NoneType' object is not callable
Traceback (most recent call last):
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 246, in run
    step_info.from_step(env, action, obs_preprocessor=agent.obs_preprocessor)
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 379, in from_step
    self.obs, self.reward, self.terminated, self.truncated, env_info = env.step(action)
                                                                       ^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/time_limit.py", line 57, in step
    observation, reward, terminated, truncated, info = self.env.step(action)
                                                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/order_enforcing.py", line 56, in step
    return self.env.step(action)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 415, in step
    reward, done, user_message, task_info = self._task_validate()
                                            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 440, in _task_validate
    reward, done, user_message, info = self.task.validate(self.page, self.chat.messages)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/visualwebarena/src/browsergym/visualwebarena/task.py", line 259, in validate
    score = self.evaluator(
            ^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/visualwebarena/evaluation_harness/evaluators.py", line 636, in __call__
    cur_score = evaluator(trajectory, config_file, page)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<@beartype(visualwebarena.evaluation_harness.evaluators.PageImageEvaluator.__call__) at 0x7f2f38f731a0>", line 92, in __call__
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/visualwebarena/evaluation_harness/evaluators.py", line 576, in __call__
    pred_ans = self.captioning_fn(
               ^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

2x : Exception uncaught by agent or environment in task <task_name>.
Error:
Execution context was destroyed, most likely because of a navigatio

  • visualwebarena.248 seed: 30
  • visualwebarena.249 seed: 32

Showing Max 2 stack traces:

2024-10-28 01:38:58,353 - 2180614 - browsergym.experiments.loop - INFO - Running experiment GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.248_30 in:
  /home/toolkit/agentlab_results/2024-10-24_21-33-14_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena/2024-10-28_01-38-10_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.248_30
2024-10-28 01:38:58,355 - 2180614 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:38:58,356 - 2180614 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:38:58,381 - 2180614 - browsergym.experiments.loop - DEBUG - Agent created.
2024-10-28 01:38:58,440 - 2180614 - browsergym.experiments.loop - DEBUG - Environment created.
2024-10-28 01:38:58,841 - 2180614 - asyncio - DEBUG - Using selector: EpollSelector
2024-10-28 01:39:01,588 - 2180614 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:39:01,589 - 2180614 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:39:01,598 - 2180614 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
...
...truncated middle of the log
...
action:
click('324')

2024-10-28 01:39:35,218 - 2180614 - browsergym.experiments.loop - DEBUG - Chat info sent.
2024-10-28 01:39:35,218 - 2180614 - browsergym.experiments.loop - DEBUG - Sending action to environment.
2024-10-28 01:39:35,218 - 2180614 - browsergym.core.env - DEBUG - Executing action
2024-10-28 01:39:35,334 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums'>
2024-10-28 01:39:35,337 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums'>
2024-10-28 01:39:35,340 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums'>
2024-10-28 01:39:35,343 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums'>
2024-10-28 01:39:35,345 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums'>
2024-10-28 01:39:38,340 - 2180614 - browsergym.core.env - DEBUG - Action executed
2024-10-28 01:39:38,896 - 2180614 - browsergym.core.env - DEBUG - Active page checked
2024-10-28 01:39:38,896 - 2180614 - browsergym.core.env - DEBUG - User message done
2024-10-28 01:39:38,896 - 2180614 - browsergym.core.env - DEBUG - Initiating task validation
2024-10-28 01:39:41,532 - 2180614 - browsergym.experiments.loop - WARNING - Exception uncaught by agent or environment in task visualwebarena.248.
Error:
Execution context was destroyed, most likely because of a navigation
Traceback (most recent call last):
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 246, in run
    step_info.from_step(env, action, obs_preprocessor=agent.obs_preprocessor)
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 379, in from_step
    self.obs, self.reward, self.terminated, self.truncated, env_info = env.step(action)
                                                                       ^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/time_limit.py", line 57, in step
    observation, reward, terminated, truncated, info = self.env.step(action)
                                                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/order_enforcing.py", line 56, in step
    return self.env.step(action)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 415, in step
    reward, done, user_message, task_info = self._task_validate()
                                            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 440, in _task_validate
    reward, done, user_message, info = self.task.validate(self.page, self.chat.messages)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/visualwebarena/src/browsergym/visualwebarena/task.py", line 259, in validate
    score = self.evaluator(
            ^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/visualwebarena/evaluation_harness/evaluators.py", line 636, in __call__
    cur_score = evaluator(trajectory, config_file, page)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<@beartype(visualwebarena.evaluation_harness.evaluators.PageImageEvaluator.__call__) at 0x7fb75721b2e0>", line 92, in __call__
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/visualwebarena/evaluation_harness/evaluators.py", line 532, in __call__
    elements = page.query_selector_all(locator)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/sync_api/_generated.py", line 8237, in query_selector_all
    self._sync(self._impl_obj.query_selector_all(selector=selector))
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
    return task.result()
           ^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_page.py", line 337, in query_selector_all
    return await self._main_frame.query_selector_all(selector)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_frame.py", line 304, in query_selector_all
    await self._channel.send("querySelectorAll", dict(selector=selector)),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 490, in wrap_api_call
    return await cb()
           ^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 99, in inner_send
    result = next(iter(done)).result()
             ^^^^^^^^^^^^^^^^^^^^^^^^^
playwright._impl._api_types.Error: Execution context was destroyed, most likely because of a navigation

2024-10-28 01:39:41,563 - 2180614 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/f/pics'>

2024-10-28 01:38:58,350 - 2180616 - browsergym.experiments.loop - INFO - Running experiment GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.249_32 in:
  /home/toolkit/agentlab_results/2024-10-24_21-33-14_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena/2024-10-28_01-38-10_GenericAgent-gpt-4o-mini-2024-07-18_on_visualwebarena.249_32
2024-10-28 01:38:58,351 - 2180616 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:38:58,352 - 2180616 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:38:58,375 - 2180616 - browsergym.experiments.loop - DEBUG - Agent created.
2024-10-28 01:38:58,434 - 2180616 - browsergym.experiments.loop - DEBUG - Environment created.
2024-10-28 01:38:58,841 - 2180616 - asyncio - DEBUG - Using selector: EpollSelector
2024-10-28 01:39:01,545 - 2180616 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
2024-10-28 01:39:01,546 - 2180616 - httpx - DEBUG - load_verify_locations cafile='/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/certifi/cacert.pem'
2024-10-28 01:39:01,555 - 2180616 - httpx - DEBUG - load_ssl_context verify=True cert=None trust_env=True http2=False
...
...truncated middle of the log
...
action:
click('110')

2024-10-28 01:39:17,341 - 2180616 - browsergym.experiments.loop - DEBUG - Chat info sent.
2024-10-28 01:39:17,341 - 2180616 - browsergym.experiments.loop - DEBUG - Sending action to environment.
2024-10-28 01:39:17,341 - 2180616 - browsergym.core.env - DEBUG - Executing action
2024-10-28 01:39:17,451 - 2180616 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums/all'>
2024-10-28 01:39:17,454 - 2180616 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums/all'>
2024-10-28 01:39:17,457 - 2180616 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums/all'>
2024-10-28 01:39:17,459 - 2180616 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums/all'>
2024-10-28 01:39:17,461 - 2180616 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums/all'>
2024-10-28 01:39:20,472 - 2180616 - browsergym.core.env - DEBUG - Action executed
2024-10-28 01:39:21,026 - 2180616 - browsergym.core.env - DEBUG - Active page checked
2024-10-28 01:39:21,026 - 2180616 - browsergym.core.env - DEBUG - User message done
2024-10-28 01:39:21,026 - 2180616 - browsergym.core.env - DEBUG - Initiating task validation
2024-10-28 01:39:21,149 - 2180616 - browsergym.experiments.loop - WARNING - Exception uncaught by agent or environment in task visualwebarena.249.
Error:
Execution context was destroyed, most likely because of a navigation
Traceback (most recent call last):
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 246, in run
    step_info.from_step(env, action, obs_preprocessor=agent.obs_preprocessor)
  File "/home/toolkit/dev/BrowserGym/browsergym/experiments/src/browsergym/experiments/loop.py", line 379, in from_step
    self.obs, self.reward, self.terminated, self.truncated, env_info = env.step(action)
                                                                       ^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/time_limit.py", line 57, in step
    observation, reward, terminated, truncated, info = self.env.step(action)
                                                       ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/gymnasium/wrappers/order_enforcing.py", line 56, in step
    return self.env.step(action)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 415, in step
    reward, done, user_message, task_info = self._task_validate()
                                            ^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/core/src/browsergym/core/env.py", line 440, in _task_validate
    reward, done, user_message, info = self.task.validate(self.page, self.chat.messages)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/dev/BrowserGym/browsergym/visualwebarena/src/browsergym/visualwebarena/task.py", line 259, in validate
    score = self.evaluator(
            ^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/visualwebarena/evaluation_harness/evaluators.py", line 636, in __call__
    cur_score = evaluator(trajectory, config_file, page)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<@beartype(visualwebarena.evaluation_harness.evaluators.PageImageEvaluator.__call__) at 0x7f52afb932e0>", line 92, in __call__
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/visualwebarena/evaluation_harness/evaluators.py", line 532, in __call__
    elements = page.query_selector_all(locator)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/sync_api/_generated.py", line 8237, in query_selector_all
    self._sync(self._impl_obj.query_selector_all(selector=selector))
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_sync_base.py", line 109, in _sync
    return task.result()
           ^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_page.py", line 337, in query_selector_all
    return await self._main_frame.query_selector_all(selector)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_frame.py", line 304, in query_selector_all
    await self._channel.send("querySelectorAll", dict(selector=selector)),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 61, in send
    return await self._connection.wrap_api_call(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 490, in wrap_api_call
    return await cb()
           ^^^^^^^^^^
  File "/home/toolkit/micromamba/envs/ui-assist/lib/python3.12/site-packages/playwright/_impl/_connection.py", line 99, in inner_send
    result = next(iter(done)).result()
             ^^^^^^^^^^^^^^^^^^^^^^^^^
playwright._impl._api_types.Error: Execution context was destroyed, most likely because of a navigation

2024-10-28 01:39:21,177 - 2180616 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums'>
2024-10-28 01:39:21,236 - 2180616 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums'>
2024-10-28 01:39:21,238 - 2180616 - browsergym.core.env - DEBUG - _activate_page_from_js(page) called, page=<Page url='http://visualwebarena.eastus.cloudapp.azure.com:8080/forums'>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@recursix