Not a valid DAV response

[idolpx]

I'm trying to use this in a project with an SD card. I've added some debug code and can see that sendPropResponse() is sending the data in chunks. I get the error in the attached screenshot though. In wireshark it looks like the last chunk is never sent. Even though I'm seeing it being sent in the serial monitor while running. I also tried just sending one small chunk and calling closeChunk() and I don't see any of the data come across in wireshark. This leads me to believe that the buffer is not being flushed when httpd_resp_send_chunk() in esp-idf is being called.

I'm using esp-idf 5.1.0 btw. Can someone confirm that things work for them? Or that they are having the same issue?
I'd really like to get this working. Great job on the code. It is very well thought out and easy to follow.

[zonque]

It's been a while since I last tried it, but in my tests things worked very well. There were some corner cases with moving files between folders, but plain read and write was not an issue. I used edpídf 4.x tho.

You seem to have dug deep enough already to find the root cause :) If you do and the bug is in this library, please file a PR!

[zonque]

You could also give the libsoup example a spin. It uses the exact same DAV implementation and was quite helpful during development on a desktop PC.

[idolpx]

Will do. If you get a moment and can check it on esp32 to confirm it is or not working that would be great.
Maybe I'm doing something wrong.

I'm not familiar with libsoup. I will try that too.

I did find that I have to strip the trailing slash off of paths to get a directory listing from the sd card mounted into the vfs at "/sd". That was easy enough though.

[zonque]

What might be relevant is that I only tested this with the Linux gvfs implementation. I didn't have a Mac around when I hacked on that. Could you try a Linux machine?

[idolpx]

I could try that but am most interested in getting it working on ESP32.

I do have a Linux VM on my macbook though.

[zonque]

No what I meant was: I had an esp32 as server, but used Linux as DAV client.

[idolpx]

Oh... ok. I did try cadaver but it does not like the chunked encoding.

What client did you use on Linux?

[idolpx]

Oh.. I see gvfs? I will check and see if that is on my Linux server. I run Linux Mint on my home dev server.

[idolpx]

I did try mounting it with the file manager and it didn't like that either. I will try again.

[idolpx]

Well my theory about the bug with httpd_resp_send_chunk() was wrong. I wrote a small test program and it works just fine. It doesn't do anything except return a list of 20 fake files but it works without issue. I will update when I have more info.

Here's the test program btw.

#include <driver/timer.h>
#include <freertos/FreeRTOS.h>
#include <freertos/task.h>
#include <nvs_flash.h>
#include <stdio.h>
#include <esp_log.h>

#include <esp_http_server.h>
#include <esp_wifi.h>

#include <string>
#include <sstream>

#define CONFIG_WIFI_SSID       "WIFI_SSID"
#define CONFIG_WIFI_PASSWORD   "password"

static const char *TAG = "httpd chunk test";

static httpd_handle_t server = NULL;

static esp_err_t sendHEAD(httpd_req_t *req)
{
    httpd_resp_set_status(req, "200 OK");

    httpd_resp_send_chunk(req, NULL, 0);
    return ESP_OK;
}

static esp_err_t sendPROPFIND(httpd_req_t *req)
{
    httpd_resp_set_status(req, "207 Multi-Status");
    httpd_resp_set_type(req, "application/xml; charset=\"utf-8\"");
    httpd_resp_set_hdr(req, "DAV", "2");
    httpd_resp_set_hdr(req, "Allow", "PROPFIND,OPTIONS,DELETE,COPY,MOVE,HEAD,POST,PUT,GET");

    httpd_resp_send_chunk(req, "<?xml version=\"1.0\" encoding=\"utf-8\" ?>\n", HTTPD_RESP_USE_STRLEN);
    httpd_resp_send_chunk(req, "<multistatus xmlns=\"DAV:\">\n", HTTPD_RESP_USE_STRLEN);

    // s << "<response>\n";
    // s << "<href>/</href>";
    // s << "<propstat><prop>\n";

    // s << "<displayname></displayname>";
    // s << "<getlastmodified>Wed, 16 Nov 02:14:28 GMT</getlastmodified>";
    // s << "<getetag>0</getetag>";
    // s << "<resourcetype>";
    // s << "<collection/>";
    // s << "</resourcetype>";

    // s << "</prop>\n";
    // s << "<status>HTTP/1.1 200 OK</status>";
    // s << "</propstat></response>\n";

    // httpd_resp_send_chunk(req, s.str().c_str(), HTTPD_RESP_USE_STRLEN);

    int i = 0;
    while ( i++ < 20)
    {
        //httpd_resp_send_chunk(req, ("This is chunck #" + std::to_string(i) + "<br/>\n").c_str(), -1);

        std::ostringstream s;
        s << "<response>\n";
        s << "<href>file" << std::to_string(i) << ".txt</href>";
        s << "<propstat><prop>\n";

        s << "<displayname>file" << std::to_string(i) << ".txt</displayname>";
        s << "<getlastmodified>Wed, 16 Nov 02:14:28 GMT</getlastmodified>";
        s << "<getetag>0</getetag>";
        s << "<resourcetype>";
        // s << "<collection/>";
        s << "</resourcetype>";

        s << "</prop>\n";
        s << "<status>HTTP/1.1 200 OK</status>";
        s << "</propstat></response>\n";

        httpd_resp_send_chunk(req, s.str().c_str(), HTTPD_RESP_USE_STRLEN);
    }

    httpd_resp_send_chunk(req, "</multistatus>\n", HTTPD_RESP_USE_STRLEN);

    httpd_resp_send_chunk(req, NULL, 0);
    return ESP_OK;
}

static httpd_uri_t http_head = {
    .uri        = "/",
    .method     = HTTP_HEAD,
    .handler    = sendHEAD,
    .user_ctx   = NULL
};

static httpd_uri_t http_propfind = {
    .uri        = "/",
    .method     = HTTP_PROPFIND,
    .handler    = sendPROPFIND,
    .user_ctx   = NULL
};

httpd_handle_t http_start_webserver(void)
{
    httpd_handle_t server = NULL;
    httpd_config_t config = HTTPD_DEFAULT_CONFIG();

    // Start the httpd server
    ESP_LOGI(TAG, "Starting server on port: '%d'", config.server_port);
    if (httpd_start(&server, &config) == ESP_OK) {
        // Set URI handlers
        ESP_LOGI(TAG, "Registering URI handlers");
        httpd_register_uri_handler(server, &http_head);
        httpd_register_uri_handler(server, &http_propfind);
        return server;
    }

    ESP_LOGI(TAG, "Error starting server!");
    return NULL;
}

void http_stop_webserver(httpd_handle_t server)
{
    // Stop the httpd server
    httpd_stop(server);
}

static esp_err_t wifi_event_handler(void *ctx, system_event_t *event)
{
    switch(event->event_id) {
    case SYSTEM_EVENT_STA_START:
        ESP_LOGI(TAG, "SYSTEM_EVENT_STA_START");
        ESP_ERROR_CHECK(esp_wifi_connect());
        break;
    case SYSTEM_EVENT_STA_GOT_IP:

        ip4_addr_t ip4;
        ip4.addr = event->event_info.got_ip.ip_info.ip.addr;

        ESP_LOGI(TAG, "SYSTEM_EVENT_STA_GOT_IP");
        ESP_LOGI(TAG, "Got IP: '%s'", ip4addr_ntoa(&ip4));

        /* Start the web server */
        if (server == NULL) {
            server = http_start_webserver();
        }
        break;
    case SYSTEM_EVENT_STA_DISCONNECTED:
        ESP_LOGI(TAG, "SYSTEM_EVENT_STA_DISCONNECTED");
        ESP_ERROR_CHECK(esp_wifi_connect());

        /* Stop the web server */
        if (server) {
            http_stop_webserver(server);
            server = NULL;
        }
        break;
    default:
        break;
    }
    return ESP_OK;
}

void wifi_initialise()
{
    tcpip_adapter_init();
    ESP_ERROR_CHECK(esp_event_loop_init(wifi_event_handler, NULL));
    wifi_init_config_t cfg = WIFI_INIT_CONFIG_DEFAULT();
    ESP_ERROR_CHECK(esp_wifi_init(&cfg));
    ESP_ERROR_CHECK(esp_wifi_set_storage(WIFI_STORAGE_RAM));

    wifi_config_t wifi_config;
    memset(&wifi_config, 0, sizeof(wifi_config));

    strlcpy((char *)wifi_config.sta.ssid, CONFIG_WIFI_SSID, sizeof(wifi_config.sta.ssid));
    strlcpy((char *)wifi_config.sta.password, CONFIG_WIFI_PASSWORD, sizeof(wifi_config.sta.password));

    ESP_LOGI(TAG, "Setting WiFi configuration SSID %s...", wifi_config.sta.ssid);
    ESP_ERROR_CHECK(esp_wifi_set_mode(WIFI_MODE_STA));
    ESP_ERROR_CHECK(esp_wifi_set_config(WIFI_IF_STA, &wifi_config));
    ESP_ERROR_CHECK(esp_wifi_start());
}

extern "C"
{
    void app_main()
    {
        ESP_ERROR_CHECK(nvs_flash_init());
        wifi_initialise();
    }
}

[idolpx]

[idolpx]

Finally revisiting this and discovered what was going on. In your example "webdav_handler()" was returning the http status code rather than ESP_OK. That was causing the connection to immediately close and the last few chunks didn't get sent.

Fixing that resolves the issue I was describing here.

I found some other issues as well but I will bundle them up in a PR to submit later.
I still haven't quite got it working correctly when mounting it as a network location in Windows Explorer.
The date format didn't include the year and that was throwing it off initially but there are other inconsistencies.
Your code gave me a starting point. Thanks. :)

You can see it in action here. https://github.com/idolpx/meatloaf-specialty