Skip to content

Use PutVarInt instead of PutFixed while encoding stream entry value#791

Merged
ShooterIT merged 7 commits intoapache:unstablefrom
torwig:streams_fixed_to_varint
Aug 27, 2022
Merged

Use PutVarInt instead of PutFixed while encoding stream entry value#791
ShooterIT merged 7 commits intoapache:unstablefrom
torwig:streams_fixed_to_varint

Conversation

@torwig
Copy link
Contributor

@torwig torwig commented Aug 24, 2022

As @ShooterIT recommended here: #745
I changed PutFixed32 to PutVarint32 and GetFixed32 to GetVarint32.
Encoding/decoding functions were 'borrowed' from RocksDB codebase.
Obviously, this change makes it impossible to decode entries if entries were encoded with PutFixed32 (an error will be returned).
Is it too late to introduce such a change?

@torwig torwig force-pushed the streams_fixed_to_varint branch from 5843935 to fab3152 Compare August 24, 2022 20:43
@ShooterIT
Copy link
Member

Is it too late to introduce such a change?

I don't think it is late, since now it is only on unstable branch which we may make changes on it.

one problem we need to concern is about open source license about these codes copied from leveldb/rocksdb.
should we put its license header. And is xxFixed32 copied from leveldb/rocksdb, should we describe this? @git-hulk

@git-hulk
Copy link
Member

@torwig Thanks for your contribution. As @ShooterIT mentioned that we don't promise stability and compatibility before releasing, so feel free to do that.

@ShooterIT Those Put|GetFixedXXX are NOT copied from rocksdb/leveldb indeed, we rewritten them into BIGENDIAN while the rocksdb is little endian, so I think it's good to keep Apache 2.0 statement only. https://github.com/facebook/rocksdb/blob/0050a73a4fb1ba52d78655b271a0f09ba1fbf7fc/util/coding_lean.h#L25

Copy link
Member

@PragmaTwice PragmaTwice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Other code looks good to me. Thanks for your contribution!

And you could add some unit test for *Varint functions.

git-hulk
git-hulk previously approved these changes Aug 25, 2022
Copy link
Member

@git-hulk git-hulk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks

@git-hulk git-hulk requested a review from ShooterIT August 25, 2022 12:16
Co-authored-by: Twice <twice@apache.org>
Co-authored-by: Twice <twice@apache.org>
PragmaTwice
PragmaTwice previously approved these changes Aug 25, 2022
Copy link
Member

@PragmaTwice PragmaTwice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@torwig
Copy link
Contributor Author

torwig commented Aug 26, 2022

@git-hulk BTW, after reverting snappy version I have Cmake error on Fedora 35

CMake Error at /usr/lib64/cmake/GTest/GTestTargets.cmake:37 (message):
  Some (but not all) targets in this export set were already defined.

  Targets Defined: GTest::gtest;GTest::gtest_main

  Targets not yet defined: GTest::gmock;GTest::gmock_main

Call Stack (most recent call first):
  /usr/lib64/cmake/GTest/GTestConfig.cmake:42 (include)
  /usr/share/cmake/Modules/FindGTest.cmake:187 (find_package)
  build/_deps/snappy-src/CMakeLists.txt:40 (find_package)

However, I see that all builds you have in your CI are successful.

@PragmaTwice
Copy link
Member

PragmaTwice commented Aug 26, 2022

@git-hulk BTW, after reverting snappy version I have Cmake error on Fedora 35

CMake Error at /usr/lib64/cmake/GTest/GTestTargets.cmake:37 (message):
  Some (but not all) targets in this export set were already defined.

  Targets Defined: GTest::gtest;GTest::gtest_main

  Targets not yet defined: GTest::gmock;GTest::gmock_main

Call Stack (most recent call first):
  /usr/lib64/cmake/GTest/GTestConfig.cmake:42 (include)
  /usr/share/cmake/Modules/FindGTest.cmake:187 (find_package)
  build/_deps/snappy-src/CMakeLists.txt:40 (find_package)

However, I see that all builds you have in your CI are successful.

There are some cmake changes from Snappy 1.1.7 to 1.1.9, find_package(gtest) always exists in 1.1.7 regardless of SNAPPY_BUILD_TESTS being set to OFF, which is fixed in 1.1.9.

@torwig Could you open an issue for this? I will fix it.

@PragmaTwice
Copy link
Member

And if you want to workaround it now, you can just remove find_package(GTest QUIET) in <build dir>/_deps/snappy-src/CMakeLists.txt.

@torwig
Copy link
Contributor Author

torwig commented Aug 26, 2022

@PragmaTwice I've created an issue: #792

@git-hulk
Copy link
Member

@ShooterIT Can have a look again and merge if have no comment.

@ShooterIT ShooterIT merged commit 694d561 into apache:unstable Aug 27, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants