Skip to content

APM agent crashes if the SQL query is too large #827

Closed
@lfrodrigues

Description

@lfrodrigues

Describe the bug:

When running in celery (I guess it happens elsewhere also) if the query is too large APM server will reject the event.

TransportException: HTTP 400: {"accepted":0,"errors":[{"message":"event exceeded the permitted size.","document":"{\"span\": {\"id\": \"332c20cdb36c0b03\", \"transaction_id\": \"26dfc86436ea1c9c\", \"trace_id\": \"8b3a7bd7e830fe4800bd6ff98af24ecd\", \"parent_id\": \"26dfc86436ea1c9c\", \"name\": ... MY_DATABASE_FIELDS_HERE... ) VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s::integer[], %s::text[], %s, %s, %s, %s, %s, %s, %s, %s, %s::varchar(100)[], %s, %s, %s, %s, %s::integer[], %s, %s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s::integer[], %s::text[], %s, %s, %s, %s, %s, %s, %s, %s, %s::varchar(100)[], %s, %s, %s, %s, %s::integer[], %s, %s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s::integer[], %s::text[], %s, %s, %s, %s, %s, %s, %s, %s, %s::varchar(100)[], %s, %s, %s, %s, %s::integer[], %s, %s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s::integer[], %s::text[], %s, %s, %s, %s, %s, %s, %s, %s, %s::varchar(100)[], %s, %s, %s, %s, %s::integer[], %s, %s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s::integer[], %s::text[], %s, %s, %s, %s, %s, %s, %s, %s, %s::varchar(100)[], %s, %s, %s, %s, %s::integer[], %s, %s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s::integer[], %s::text[], %s, %s, %s, %s, %s, %s, %s, %s, %s::varchar(100)[], %s, %s, %s, %s, %s::integer[], %s, %s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s::integer[], %s::text[], %s, %s, %s, %s, %s, %s, %s, %s, %s::varchar(100)[], %s, %s, %s, %s, %s::integer[], %s, %s, %s, %s, %s, %s, %s, %s, %s, %s), (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s, %s::integer[], %s::text[], %s, %s, %s, %s, %s, %s, %s, %s, %s::var...
  File "elasticapm/transport/base.py", line 227, in _flush
    self.send(data)
  File "elasticapm/transport/http.py", line 105, in send
    raise TransportException(message, data, print_trace=print_trace)

To Reproduce

  1. Run a bulk_create in Django with a large list.
    In our code, we're creating 450k rows in a task at night.

Environment (please complete the following information)

  • OS: Ubuntu 18.04
  • Python version: 3.6
  • Framework and version: Django 2.2
  • APM Server version: 5.6.0
  • Agent version: 7.6.0

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions