[SPARK-53755][CORE] Add log support in BlockManager

[ivoson]

What changes were proposed in this pull request?

Add log support in BlockManager to help collecting and analyzing logs for live debug. A few new components have been introduced to do that.

1. LogBlockId to represent log data, which can be used for log data filter. And LogBlockType to identify different kinds of logs.
2. LogLine to define the log structure.
3. LogBlockWriter to help to write a single log block.
4. RollingLogWriter to help to write logs in a rolling manner, split blocks by size(approximately).

Usage Example:

// Define a blockId generator which can help to generate unique block ids.
val logBlockIdGenerator = new LogBlockIdGenerator {
    override def logBlockType: LogBlockType = LogBlockType.TEST

    override protected def genUniqueBlockId(
        lastLogTime: Long, executorId: String): LogBlockId = {
      TestLogBlockId(lastLogTime, executorId)
    }
}

// Get a log writer and write logs.
val logBlockWriter = store.getRollingLogWriter[LogLine](logBlockIdGenerator)
logBlockWriter.writeLog(LogLine(0L, 1, "Log message 1"))
logBlockWriter.writeLog(LogLine(1L, 2, "Log message 2"))
logBlockWriter.writeLog(LogLine(2L, 3, "Log message 3"))
logBlockWriter.writeLog(LogLine(3L, 4, "Log message 4"))

// Close writer after writing all logs.
logBlockWriter.close()

Why are the changes needed?

Collect and analyze logs with spark jobs to make live debug easier.

Does this PR introduce any user-facing change?

No

How was this patch tested?

UT added.

Was this patch authored or co-authored using generative AI tooling?

No

[ivoson]

cc @ueshin

[ueshin]

LGTM. Thanks!
cc @cloud-fan for another look.