Skip to content

Heap leak in h5str_convert() for array and complex datatypes #6511

Description

@K-ANOY

I found a possible heap leak in h5str_convert(). The H5T_ARRAY andH5T_COMPLEX cases allocate a temporary buffer with calloc(), but the buffer is never released.

File: java/src-jni/jni/h5util.c

Function: h5str_convert

Relevant code:

case H5T_ARRAY: {
    hsize_t i, dims[H5S_MAX_RANK], total_elmts;
    size_t  baseTypeSize;
    int     rank = 0;

    ...

    if (NULL == (cptr = (char *)calloc((size_t)total_elmts, baseTypeSize)))
        H5_OUT_OF_MEMORY_ERROR(ENVONLY, "h5str_convert: failed to allocate array buffer");

    for (i = 0; i < total_elmts; i++) {
        if (!(h5str_convert(ENVONLY, &this_str, container, mtid, out_buf, i * baseTypeSize))) {
            CHECK_JNI_EXCEPTION(ENVONLY, JNI_FALSE);
            goto done;
        }
        ...
    }

    ...
    break;
}

and:

case H5T_COMPLEX: {
    size_t baseTypeSize;

    ...

    if (NULL == (cptr = calloc(1, typeSize)))
        H5_OUT_OF_MEMORY_ERROR(ENVONLY, "h5str_convert: failed to allocate array buffer");

    /* Convert real part */
    if (!(h5str_convert(ENVONLY, &this_str, container, mtid, out_buf, 0))) {
        CHECK_JNI_EXCEPTION(ENVONLY, JNI_FALSE);
        goto done;
    }

    ...
    break;
}

The done block only closes mtid; it does not free the buffer allocated into
cptr:

done:
    if (mtid >= 0)
        H5Tclose(mtid);

    return retVal;

This happens on the ordinary success path for array and complex datatype
conversion, not only on an error path. The recursive calls write directly into
out_buf, so the allocated buffer also appears to be unused after allocation.

The function is reachable from JNI string write paths such as
H5AwriteVL_asstr() and H5DwriteVL_asstr(), which call h5str_convert() for
each string element before writing the converted buffer to HDF5.

Suggested fix: remove the unused calloc() allocations if they are not needed,
or keep them in a separate temporary pointer and release that pointer on every
exit path. Since cptr can also point into out_buf, the fix should avoid
blindly freeing cptr without tracking whether it owns heap memory.

Metadata

Metadata

Assignees

Labels

Type

Fields

No fields configured for Bug.

Projects

Status
Planning

Relationships

None yet

Development

No branches or pull requests

Issue actions