Skip to content

Conversation

@swamirishi
Copy link

Adding a Jni layer for table Iterator. Also changing the table iterator's interface preparse internal key so that SSTFileReaderIterator can be reused.

@swamirishi
Copy link
Author

swamirishi commented Apr 5, 2024

@cbi42 @jowlyzhang Can you please review this? In ozone we depend on jni for rocksdb apis. I have just added JNI layer for the table_iterator written in your PR.

@swamirishi swamirishi force-pushed the JniReaderForTableIterator branch from 52a2255 to 6b83480 Compare April 5, 2024 01:03
@swamirishi swamirishi force-pushed the JniReaderForTableIterator branch from 6b83480 to dcd5135 Compare April 5, 2024 01:04
@adamretter adamretter self-requested a review April 5, 2024 07:03
Copy link
Contributor

@adamretter adamretter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR. I have left some feedback on the Java changes which need some amendment - also a Java test needs to be added please for this new API. I will leave @ajkr @pdillinger and the RocksDB team to comment on the internal C++ changes to RocksDB table... as I cannot myself say if they are desirable or not for the RocksDB team.

*/
class RawSstFileReaderIterator extends SstFileReaderIterator {

protected RawSstFileReaderIterator(SstFileReader reader, long nativeHandle) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please set all arguments and variables to final where possible.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

*/
class RawSstFileReaderIterator extends SstFileReaderIterator {

protected RawSstFileReaderIterator(SstFileReader reader, long nativeHandle) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This class needs to implement some distinct form of disposeInternal to avoid leaking memory when it is finished with.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It extends SstFileReaderIterator which clears off the required handle.

return nullptr;
}
return std::make_unique<TableIterator>(internal_iter);
return std::make_unique<TableIterator>(internal_iter, &rep_->options);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ajkr @pdillinger Can you comment on this change in the internals for RocksDB please?


public:
explicit TableIterator(InternalIterator* iter) : iter_(iter) {}
explicit TableIterator(InternalIterator* iter, Options* options) : iter_(iter), options_(options) {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ajkr @pdillinger Can you comment on this change in the internals for RocksDB please?

void SeekToFirst() override { return iter_->SeekToFirst(); }
void SeekToLast() override { return iter_->SeekToLast(); }
void Seek(const Slice& target) override { return iter_->Seek(target); }
void Seek(const Slice& target) override {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ajkr @pdillinger Can you comment on this change in the internals for RocksDB please?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will review this. Thanks!

}
void SeekForPrev(const Slice& target) override {
return iter_->SeekForPrev(target);
std::string seek_key_buf;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ajkr @pdillinger Can you comment on this change in the internals for RocksDB please?

void SeekToFirst() override { return iter_->SeekToFirst(); }
void SeekToLast() override { return iter_->SeekToLast(); }
void Seek(const Slice& target) override { return iter_->Seek(target); }
void Seek(const Slice& target) override {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will review this. Thanks!

void SeekToFirst() override { return iter_->SeekToFirst(); }
void SeekToLast() override { return iter_->SeekToLast(); }
void Seek(const Slice& target) override { return iter_->Seek(target); }
void Seek(const Slice& target) override {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The type of key taken as input for Seek and SeekForPrev should be consistent with the type of key produced by Iterator::key. Can you add jni APIs for the util methods GetInternalKeyForSeek etc instead?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we change the definition of the TableIterator to seek and get only on the user_key instead? Isn't this a better way to define this class?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The name TableIterator indicates it's iterating the raw table, and that makes internal key a better format of key for these APIs.

return Status::NotSupported("TableIterator does not support GetProperty.");
}

uint64_t SequenceNumber() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The public API is Iterator in rocksdb/includes/iterator.h, the TableIterator is an internal class that is not supposed to be publicly accessible. These SequenceNumber, and type APIs are not defined for Iterator.

We have added ParseEntry util methods to parse sequence number and type out of an internal key. You can have you own wrapper iterator class wrapping a RocksDB Iterator, and add the parsing logic in to the wrapper iterator.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need all of these infos separately and storing the parsed entry would be much more optimal. Otherwise for each and every call we would have to parse the key over and over again. All I am doing here is changing the definition of the table iter which would seek over the same user key and return user key when getKey() is called. I have added 2 more functions on top of iterator, I believe that should be fine, since JNI would be called only when TableIterator is initialized.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jowlyzhang TableIterator itself is a wrapper for InternalIterator why have 2 wrappers? Should we consider making this public?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's worth having 2 wrappers because it will be your own wrapper iterator that you don't need to check into RocksDB codebase, you own those logic and you can modify it however you want. Parsing an entry every time over and over again isn't computationally heavy or performance wise concerning, you can just wrap it in your own wrapper iterator. It's beset for the RocksDB TableIterator to just do the most vanilla things of iterating raw entries. Others wishing to add their own iterating constraints can just add on top of it.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jowlyzhang I have updated the code to add another jni functionality to convert internal Key to user key and vice versa

@swamirishi
Copy link
Author

@jowlyzhang Can you check this PR?

@jowlyzhang
Copy link
Contributor

@jowlyzhang Can you check this PR?

Thanks for making the changes. Since this PR now mostly have java changes, I'll leave it to @adamretter to make the call here.

Copy link
Contributor

@adamretter adamretter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The issues I think I have identified repeat throughout and should be addressed please

kEntryWideColumnEntity((byte)0x7),
kEntryOther((byte)0x8);
private final byte value;
private static Map<Byte, EntryType> reverseLookupMap = new HashMap<>();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need a map here? The collection is small enough to iterate I would have thought?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed the map and falling back to iterating through the values.

private final byte value;
private static Map<Byte, EntryType> reverseLookupMap = new HashMap<>();

EntryType(byte value) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be final param.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

jint len) {
const std::unique_ptr<char[]> target(new char[len]);
if (target == nullptr) {
jclass oom_class = env->FindClass("/lang/java/OutOfMemoryError");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this has been tested as the class name is incorrect. Also there are util methods in portal.h for throwing errors that could be used.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added util method to also do operations on indirect byte array buffers

"Memory allocation failed in RocksDB JNI function");
return;
}
env->GetByteArrayRegion(jtarget, off, len,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs error checking

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

auto slice_size = key_slice.size();
jsize copy_size = std::min(static_cast<uint32_t>(slice_size),
static_cast<uint32_t>(jlen));
env->SetByteArrayRegion(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs error checking

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

auto slice_size = key_slice.size();
jsize copy_size = std::min(static_cast<uint32_t>(slice_size),
static_cast<uint32_t>(int_key_len));
env->SetByteArrayRegion(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs error checking.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

reinterpret_cast<const ROCKSDB_NAMESPACE::Options *>(options_handle);
const std::unique_ptr<char[]> target(new char[user_key_len]);
if (target == nullptr) {
jclass oom_class = env->FindClass("/lang/java/OutOfMemoryError");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Class name and testing?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

"Memory allocation failed in RocksDB JNI function");
return static_cast<jsize>(0);
}
env->GetByteArrayRegion(user_key, user_key_off, user_key_len,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs error checking.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

auto slice_size = key_slice.size();
jsize copy_size = std::min(static_cast<uint32_t>(slice_size),
static_cast<uint32_t>(int_key_len));
env->SetByteArrayRegion(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs error checking.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

// exception thrown: OutOfMemoryError
return nullptr;
}
env->SetByteArrayRegion(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Needs error checking.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@swamirishi
Copy link
Author

@adamretter Apologies for the late response and not following through addressing review comments on time. I have addressed all the review comments. Can you please take a look at it? I have added a Java test for the same as well and also added required util functions as well in portal.h. FYI these util functions can be used in other JNI classes as well.

@swamirishi swamirishi requested a review from adamretter May 28, 2024 16:34
@swamirishi
Copy link
Author

@adamretter Any updates on this?

@swamirishi
Copy link
Author

@adamretter Can you review this PR or is there anyone else who can review this PR if you are loaded with other stuff

Copy link
Contributor

@adamretter adamretter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still some missing error handling that needs to be implemented.

}
env->GetByteArrayRegion(jkey, jkey_off, jkey_len,
reinterpret_cast<jbyte*>(target.get()));
ROCKSDB_NAMESPACE::Slice target_slice(target.get(), jkey_len);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing error check for the preceding JNI call, e.g. if (env->ExceptionOccurred())

reinterpret_cast<const ROCKSDB_NAMESPACE::Options *>(options_handle);
auto *parsed_entry_info =
reinterpret_cast<ROCKSDB_NAMESPACE::ParsedEntryInfo *>(handle);
jbyte *target = env->GetByteArrayElements(jtarget, nullptr);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to consider the lifetime of target... should there be a corresponding call to ReleaseByteArrayElements?

Copy link
Author

@swamirishi swamirishi Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah you are right thanks for pointing out the issue. I have added a boolean in ParsedEntryInfo struct to clear up on every parseEntry function or close

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it would be much easier to use env->GetByteArrayElements to copy directly into charCopy and avoid having to use the GetByteArrayElements/ReleaseByteArrayElements pair.

/* Helper for copying value in source into a byte array.
*/
template <class T>
static jint copyToByteArray(JNIEnv* env, T& source, jbyteArray jtarget,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are already some copyBytes functions in portal.h that are very similar apart from they allocate the jbyteArray. Could you create an overload of copyBytes that takes the jtarget, jtarget_off, jtarget_len, and then rename this copyBytes and refactor to remove code duplication?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, renamed the function copyBytes. I don't think we can really reuse any of the existing function to reuse. I have called the existing copyBytes to create a new byte array instead of calling the copyToByteArray function

case ROCKSDB_NAMESPACE::EntryType::kEntryOther:
return 0x8;
default:
return 0x9;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel uncomfortable with this approach that raises an exception for an invalid enum value. Is there a better approach perhaps?

};
ROCKSDB_NAMESPACE::JniUtil::k_op_indirect(getInternalKeySeek, env, user_key,
user_key_off, user_key_len);
ROCKSDB_NAMESPACE::Slice key_slice = seek_key_buf;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing an error check to see if k_op_indirect raised a Java exception.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in k_op_indirect function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same principle. Respect the k_op_indirect and check the status of ROCKSDB_NAMESPACE::GetInternalKeyForSeekForPrev, raising an exception if the status is bad.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

};
ROCKSDB_NAMESPACE::JniUtil::k_op_indirect(getInternalKeySeek, env, user_key,
user_key_off, user_key_len);
ROCKSDB_NAMESPACE::Slice key_slice = seek_key_buf;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing an error check to see if k_op_indirect raised a Java exception.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed in k_op_indirect function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, please respect the contract for k_op_indirect which allows a Java exception to be raised, and check the result of ROCKSDB_NAMESPACE::GetInternalKeyForSeekForPrev for a possible failure status, raising an exception if the status is an error.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added an exception check here

}
ROCKSDB_NAMESPACE::JniUtil::copyToByteArray(
env, key_slice, jkey, 0, static_cast<jint>(key_slice.size()));
return jkey;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing an error check to see if copyToByteArray raised a Java exception.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added a check in copyBytes function

}
ROCKSDB_NAMESPACE::JniUtil::copyToByteArray(
env, key_slice, jkey, 0, static_cast<jint>(key_slice.size()));
return jkey;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing an error check to see if copyToByteArray raised a Java exception.

}
ROCKSDB_NAMESPACE::JniUtil::copyToByteArray(
env, key_slice, jkey, 0, static_cast<jint>(key_slice.size()));
return jkey;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing an error check to see if copyToByteArray raised a Java exception.

@adamretter
Copy link
Contributor

@swamirishi Are you still working on this?

@swamirishi
Copy link
Author

@swamirishi Are you still working on this?
@adamretter Apologies for not addressing the issues. I have been busy working on other stuff. But I am working on it now and would be able to address all the review comments by EOD.

@swamirishi
Copy link
Author

@adamretter I have addressed most of the review comments to the best of my knowledge. I had a couple of doubts for few review comments and have left comments inline.

@swamirishi
Copy link
Author

@adamretter Can this be merged?

@swamirishi
Copy link
Author

@adamretter Do you think this can be merged?

@adamretter
Copy link
Contributor

@swamirishi I have asked my colleague here @alanpaxton to take a look at this for you. He will come back to you shortly...

Copy link
Contributor

@alanpaxton alanpaxton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have pointed out a few issues around memory management and lifetime, as well as some coding style/encapsulation issues. Happy to discuss further if that will be of assistance.

};
ROCKSDB_NAMESPACE::JniUtil::k_op_indirect(getInternalKeySeek, env, user_key,
user_key_off, user_key_len);
ROCKSDB_NAMESPACE::Slice key_slice = seek_key_buf;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As above, please respect the contract for k_op_indirect which allows a Java exception to be raised, and check the result of ROCKSDB_NAMESPACE::GetInternalKeyForSeekForPrev for a possible failure status, raising an exception if the status is an error.

};
ROCKSDB_NAMESPACE::JniUtil::k_op_direct(getInternalKeySeek, env, user_key,
user_key_off, user_key_len);
ROCKSDB_NAMESPACE::Slice key_slice = seek_key_buf;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contract for k_op_direct allows for a Java exception. Therefore you should check for it, to ensure encapsulation. Notice in fact that ROCKSDB_NAMESPACE::GetInternalKeyForSeekForPrev returns a status, so your lambda getInternalKeySeek should check that status and raise a Java exception from it, using a RocksDBExceptionJni::ThrowNew

};
ROCKSDB_NAMESPACE::JniUtil::k_op_indirect(getInternalKeySeek, env, user_key,
user_key_off, user_key_len);
ROCKSDB_NAMESPACE::Slice key_slice = seek_key_buf;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same principle. Respect the k_op_indirect and check the status of ROCKSDB_NAMESPACE::GetInternalKeyForSeekForPrev, raising an exception if the status is bad.

};
ROCKSDB_NAMESPACE::JniUtil::k_op_direct(getInternalKeySeek, env, user_key,
user_key_off, user_key_len);
ROCKSDB_NAMESPACE::Slice key_slice = seek_key_buf;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

k_op_direct contract etc..

};
ROCKSDB_NAMESPACE::JniUtil::k_op_indirect(getInternalKeySeek, env, user_key,
user_key_off, user_key_len);
ROCKSDB_NAMESPACE::Slice key_slice = seek_key_buf;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

k_op_direct contract etc..

jbyteArray JNICALL Java_org_rocksdb_TypeUtil_getInternalKeyForPrevJni(
JNIEnv *env, jclass /*cls*/, jbyteArray user_key, jint user_key_len,
jlong options_handle) {
jbyte *target = env->GetByteArrayElements(user_key, nullptr);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There should also be a corresponding call to ReleaseByteArrayElements here - see https://docs.oracle.com/javase/8/docs/technotes/guides/jni/spec/functions.html

reinterpret_cast<const ROCKSDB_NAMESPACE::Options *>(options_handle);
auto *parsed_entry_info =
reinterpret_cast<ROCKSDB_NAMESPACE::ParsedEntryInfo *>(handle);
jbyte *target = env->GetByteArrayElements(jtarget, nullptr);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here it would be much easier to use env->GetByteArrayElements to copy directly into charCopy and avoid having to use the GetByteArrayElements/ReleaseByteArrayElements pair.

char* charCopy = new char[len];
std::memcpy(charCopy, target, len);
env->ReleaseByteArrayElements(jtarget, target, JNI_ABORT);
ROCKSDB_NAMESPACE::Slice target_slice(reinterpret_cast<char *>(target), len);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be charCopy and not target in target_slice(... I think.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed the implementation to copy directly to target by using GetByteArrayRegion to avoid unnecessary copy from GetByteArray and release since we are going to be copy anyhow at the end.

ROCKSDB_NAMESPACE::ParseEntry(target_slice, options->comparator,
parsed_entry_info);
};
ROCKSDB_NAMESPACE::JniUtil::k_op_indirect(parse, env, jtarget, joff, jlen);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The slice data constructed by k_op_indirect has a lifetime which ends with the k_op_indirect call (std::unique_ptr), so the parsed_entry_info cannot be guaranteed to persist when it is returned. I wonder if it would be easier to make the ParsedEntryInfo a complete Java object, with members for key , type, sequence number, and fully instantiate this using JNI from a temporary C++ ParsedEntryInfo.

I also don't like the copied_user_key flag in ParsedEntryInfo, and I think if you refactored in the way I suggest, it would allow you to remove copied_user_key.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be another GC overhead on the jvm for each and every key we might end up creating an object which might be unoptimal if the number of SstFiles are going to be too many. With the byte buffer api we would have all the entries passed as it is. One possible way would be to have different byteBuffers being passed for different fields but this would mean we won't be able to reuse the SstFileReaderIterator object similar to the c++ implementation.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed the k_op_indirect callback function to return a boolean value to indicate if the target array can be released. This way we can avoid unnecessary copies.
https://github.com/swamirishi/rocksdb/blob/3a7bd0a7eee27dd18c116597246a9afac75e769e/java/rocksjni/portal.h#L2511-L2514

}
}

private void assertInternalKey(final Options options, final ParsedEntryInfo parsedEntryInfo,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of the parseEntry() variants are not being tested. Please ensure in particular that all the JNI methods you write get exercised.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done added a new test class SstTableReaderIteratorTest to test out all possible combinations.

@swamirishi
Copy link
Author

swamirishi commented Dec 21, 2025

@alanpaxton Apologies for the delay in reverting back I got held up working on other items in Ozone. I have addressed all the review comments. Please check and let me know if this looks good. Actually this is a critical piece for Apache Ozone's rocksdb upgrade plan for performing efficient snapshot diff & Ozone Snapshot Defragmentation it would great if we can merge this change as soon as possible.

@swamirishi
Copy link
Author

@alanpaxton I have also added the api to read the table iterator in range. This would be a good api to have in JNI wherein a java code can do range iteration without having to do unnecessary copies to directByteBuffer/Heap

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants