[pixels-cpp] Implementation of dynamic bufferpool

[whzruc]

Description

Earlier versions of pixels-cpp employed a static buffer pool design, in which buffer sizes and column IDs were fixed during initialization, limiting flexibility once the system was running.

    else
    {
        // check if resize the buffer is needed
        assert(colIds.size() == BufferPool::colCount);
        for (int i = 0; i < colIds.size(); i++)
        {
            uint32_t colId = colIds.at(i);
            uint64_t byte = bytes.at(i);
            std::string columnName = columnNames[colId];
            if (BufferPool::nrBytes.find(colId) == BufferPool::nrBytes.end())
            {
                throw InvalidArgumentException("BufferPool::Initialize: no such the column id.");
            }
            // Note: this code should never happen in the pixels scenario
            if (BufferPool::nrBytes[colId] < byte)
            {
                throw InvalidArgumentException("the new buffer byte cannot larger than the previous buffer byte. ");
            }
        }
    }

The current design has a fundamental limitation: the buffer size is fixed, so it cannot support queries such as the one shown below.

[whzruc]

Solution

• DynamicBufferPool: Uses sparse buffer registration with io_uring and dynamically manages the buffer pool. The core API is io_uring_register_buffers_update_tag.
• DirectUringRandomAccessFileDynamic: A dynamic file access implementation based on io_uring.
  There are functional tests shown below:

[whzruc]

Evaluation