Web Agent Contribution

Web Agent Contribution

1. Prepare

Dev env see Installation.

2. Run Web-Agent

Complete Configuration and run:

rush dev:eval

3. Web-Agent Guide

Main structure of web-bench could see Evaluator

├─ tools
│ ├─ bench-agent
│ │ ├─ src
│ │ │ ├─ llm
│ │ │ ├─ prompt
│ │ │ ├─ agent.ts
│ │ │ ├─ schedule.ts

Web-Agent is inspired by Continue.

• agent: Agent entry, process task description, file content, and error passed by the Evaluator into LLMs messages.
• schedule: Provide WebAgent with scheduling capabilities to implement rate limiting based on each LLM's constraints.
• llm：Provider of LLMs.If you want to add provider, you can refer to the files in this folder.

4. Test

Execute the following command to run evaluations and view the results in apps/eval/report:

rush eval

5. Tips

In the development environment, configure parameters in apps/eval/src/config.json5:

• logLevel: 'debug', get more information.
• projects: ['@web-bench/xxxx'], not process all projects.

More details in Config Parameters.