Return HTTP 413 (Request Entity Too Large) when http.max_content_length exceeded

[dbertram]

Currently elasticsearch drops the connection if http.max_content_length is exceeded. While this is acceptable behavior based on RFC2616 (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.14), it's not particularly friendly to client libraries.

Depending on the library being used, it can be difficult to determine the exact size of the HTTP request prior to actually sending it. Additionally, when the connection is simply closed, it leaves the underlying cause of the problem somewhat ambiguous without also inspecting the elasticsearch logs.

Proposed change: add an option to return HTTP 413 when http.max_content_length is exceeded instead of just dropping the connection.

Steps to repro the current behavior:

1. Set http.max_content_length = 1kb in elasticsearch.yml
2. Create a test index:
   curl -XPUT 'http://localhost:9200/testindex/'
3. Index a case that exceeds http.max_content_length:
   curl -v -XPUT 'http://localhost:9200/testindex/testtype/1' -d '{
   "message": "large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message large message"
   }'

Expected: HTTP status code 413 (Request Entity Too Large)

Actual: Dropped connection client-side, and a TooLongFrameException in elasticsearch log

Here's the output from curl:

* About to connect() to localhost port 9200 (#0)
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 9200 (#0)
> PUT /testindex/testtype/1 HTTP/1.1
> User-Agent: curl/7.20.1 (i686-pc-cygwin) libcurl/7.20.1 OpenSSL/0.9.8o zlib/1.2.5 libidn/1.18 libssh2/1.2.5
> Host: localhost:9200
> Accept: */*
> Content-Length: 1026
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* Empty reply from server
* Connection #0 to host localhost left intact
curl: (52) Empty reply from server
* Closing connection #0

[mike-wade]

+1 - This is causing me trouble at the moment (Have a specific list of indexes to search), specifically the Pyes python library raises NoServerAvailable exception when this happens, not very helpful!

[spinscale]

Looks like a dupe of #2137 (or am I missing something) - You need to do some extra work to do this with netty3 IIRC, so maybe netty4 will help here.

[seti123]

+1 Version 0.90.10

org.elasticsearch.common.netty.handler.codec.frame.TooLongFrameException: HTTP content length exceeded 104857600 bytes.

Update: I increased size in /etc/elasticsearch/elasticsearch.yml and it works again

    # Set a custom allowed content length:
    # 
    http.max_content_length: 500mb

[denispeplin]

The exception with same name can be caused by large header, and it should be fixed differently: #5665

[wflanagan]

Is there a workaround for this? We are regularly getting this error on some larger documents.

[jasontedor]

Is there a workaround for this? We are regularly getting this error on some larger documents.

@wflanagan Did you increase http.max_content_length?

The issue here is about how the connection is terminated and does not return HTTP status code 413 when the configured max content length is exceeded (by the way, this is an underlying issue with Netty).