Open
Description
crawl4ai version
0.4.3b3
Expected Behavior
API return ssl certification
Current Behavior
client:
Internal Server Error
server:
"POST /crawl_direct HTTP/1.1" 500 Internal Server Error
Is this reproducible?
Yes
Inputs Causing the Bug
curl -X 'POST' \
'http://192.168.1.2:11235/crawl_direct' \
-H 'accept: application/json' \
-H 'Content-Type: application/json' \
-d '{
"urls": "https://example.com",
"magic": true,
"extra": {
"page_timeout": 30000,
"proxy": "",
"scan_full_page": true,
"fetch_ssl_certificate": true,
"user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:92.0) Gecko/20100101 Firefox/92.0"
},
"cache_mode": "disabled"
}'
Steps to Reproduce
Code snippets
OS
Linux
Python version
3.12
Browser
No response
Browser version
No response
Error logs & Screenshots (if applicable)
INFO: 192.168.1.2:56052 - "POST /crawl_direct HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/uvicorn/protocols/http/h11_impl.py", line 403, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/fastapi/applications.py", line 1054, in __call__
await super().__call__(scope, receive, send)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/applications.py", line 112, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/middleware/errors.py", line 187, in __call__
raise exc
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/middleware/errors.py", line 165, in __call__
await self.app(scope, receive, _send)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/middleware/cors.py", line 85, in __call__
await self.app(scope, receive, send)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/routing.py", line 715, in __call__
await self.middleware_stack(scope, receive, send)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/routing.py", line 735, in app
await route.handle(scope, receive, send)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/routing.py", line 288, in handle
await self.app(scope, receive, send)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/routing.py", line 76, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/starlette/routing.py", line 73, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/fastapi/routing.py", line 327, in app
content = await serialize_response(
^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/fastapi/routing.py", line 181, in serialize_response
return field.serialize(
^^^^^^^^^^^^^^^^
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/fastapi/_compat.py", line 151, in serialize
return self._type_adapter.dump_python(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/iboomw/mambaforge/envs/crawl4ai/lib/python3.12/site-packages/pydantic/type_adapter.py", line 527, in dump_python
return self.serializer.to_python(
^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.PydanticSerializationError: Unable to serialize unknown type: <class 'crawl4ai.ssl_certificate.SSLCertificate'>
Activity