Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generated protobuf files with protobuf 3.12.4 on Ubuntu 21.04 #104

Merged

Conversation

baziorek
Copy link

@baziorek baziorek commented Jul 7, 2022

Some time ago I updated python files from protobufs (*.proto) files. It can be done easy with attached scripts:

python3 download-schema.py
python3 scripts/compile-proto.py

Everything was working, I was happy and the iroha-python 1.4.1 was released. Then in our servers with Ubuntu 21.04 I tried to use iroha 1.4.1 instead of iroha 1.0.0, but unfortunately I noticed errors described here: #102

After some investigation I found why it is not working - I have Manjaro, which has almost always newest packages, so generated protobufs was generated with really new version (currently I have protoc --version libprotoc 3.21.1). Everything was working in my OS - Manjaro. But debian-based servers usually have not everything as new as Manjaro, so those files was not compatible with versions in Ubuntu 21.04 (which has protoc 3.12.4).


What is done in the PR is just generating python files from *.proto in Ubuntu 20.04 LTS:

python3 download-schema.py
python3 scripts/compile-proto.py

Tests:

  1. pip3 install iroha==1.0.0 && python3 -c 'import iroha' - works OK, but we want new version
pip3 install iroha==1.4.1 && python3 -c 'import iroha'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/devops/.local/lib/python3.9/site-packages/iroha/__init__.py", line 7, in <module>
    from .iroha import *
  File "/home/devops/.local/lib/python3.9/site-packages/iroha/iroha.py", line 17, in <module>
    from . import endpoint_pb2
  File "/home/devops/.local/lib/python3.9/site-packages/iroha/endpoint_pb2.py", line 38, in <module>
    _descriptor.EnumValueDescriptor(
  File "/home/devops/.local/lib/python3.9/site-packages/google/protobuf/descriptor.py", line 755, in __new__
    _message.Message._CheckCalledFromGeneratedFile()
TypeError: Descriptors cannot not be created directly.
If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
If you cannot immediately regenerate your protos, some other possible workarounds are:
 1. Downgrade the protobuf package to 3.20.x or lower.
 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).

More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates
  1. pip3 install iroha==1.4.1 && PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python python3 -c 'import iroha' - works OK, but is may looks strange for new users
  2. pip3 install "git+https://github.com/baziorek/iroha-python.git@generate_protobufs_again" && python3 -c 'import iroha' - works OK

Summary:

So IMO we should downgrade generated protobuf files to be in lower version (I've generated in Manjaro which has protoc --version libprotoc 3.21.1), but rather users would have earlier versions (currently Ubuntu 21.04 has protobuf libprotoc 3.12.4). For users would be easier if they don't need extra flag to run simplest example PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python.

Signed-off-by: G.Bazior <bazior@agh.edu.pl>
@baziorek baziorek added bug dependencies Pull requests that update a dependency file iroha1 labels Jul 7, 2022
@baziorek baziorek self-assigned this Jul 7, 2022
@baziorek baziorek marked this pull request as ready for review July 18, 2022 14:52
@baziorek baziorek merged commit 30b76e9 into hyperledger-iroha:develop Jul 27, 2022
@baziorek baziorek deleted the generate_protobufs_again branch July 27, 2022 10:11
baziorek added a commit to baziorek/iroha-python that referenced this pull request Aug 8, 2022
baziorek added a commit that referenced this pull request Aug 22, 2022
* Extending TxPaginationMeta in queries (#77)

* quries extended
* example added
* docs comment added
* schema changed
* formatting fixed
* ordering added
* edge case fix, now 0 passed as height or timestamp is passing
* PaginationMeta in GetPendingTransactions
* examples/query_transactions.py

Signed-off-by: Piotr Pawlowski <ppiotru@gmail.com>
Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Update proto files and generated python files to support Iroha 1.4 (#96)

* Update proto files with script `download-schema.py`

Signed-off-by: Grzegorz Bazior <g.bazior@yodiss.pl>

* Generated python files from protobuf files with script `compile-proto.py`

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

Co-authored-by: Grzegorz Bazior <g.bazior@yodiss.pl>
Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Corrected merge - rerun scripts: download-schema.py and compile-proto.py

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Tab -> spaces

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Add ability to provide custom TLS cert (#63)

Signed-off-by: Stepan Lavrentev <lawrentievsv@gmail.com>
Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Add ordering sequence to params in query (#73)

* Add ordering sequence to params

Makes possible to add ordering sequence to Query. Fixes #72

Signed-off-by: Rafik Naccache <rafik@fekr.tech>

* Add :param ordering_sequence:

Signed-off-by: Rafik Naccache <rafik@fekr.tech>

* Fix wrong example for :param ordering_sequence:

Signed-off-by: Rafik Naccache <rafik@fekr.tech>

* fix wrong ordering message construction in query

After more documentation I got to the way to construct to ordering object

Signed-off-by: Rafik Naccache <rafik@fekr.tech>

* remove redundant OR clause to manage ordering_sequence

Signed-off-by: Rafik Naccache <rafik@fekr.tech>
Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Extending TxPaginationMeta in queries (#77)

* quries extended
* example added
* docs comment added
* schema changed
* formatting fixed
* ordering added
* edge case fix, now 0 passed as height or timestamp is passing
* PaginationMeta in GetPendingTransactions
* examples/query_transactions.py

Signed-off-by: Piotr Pawlowski <ppiotru@gmail.com>
Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Added missing changes from Pawlak00@b6d7f42 corrected in #77

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Examples refactor + added example of MST developed by Leo (#103)

* Refactoring of example tx-example.py

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Added sample of MST transactions. The code was originally developed by Leo: https://github.com/iptelephony/jubilant-engine but he agreed to add his code to examples

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Generated protobuf files with protobuf 3.12.4 on Ubuntu 21.04 (#104)

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Serious refactoring + improvements + adding new examples for Iroha 1 (#107)

* Update README
* Refactoring in tls-example.py
* A little refactoring of batch-example.py
* Small refactoring of blocks-query.py
* Small refactoring of infinite-blocks-stream.py
* Added sample how to use ordering of result transactions
* Refacroting in query_transactions.py
* Renamed file with convention used in other files
* Added examples in comments
* Some refactoring in examples + added docstring in each file
* chmod +x
* Updated iroha version from 1.4 -> 1.5

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Add autorestart to iroha

Signed-off-by: Alexey Rodionov <rodionov12@gmail.com>

* Cherry-pick fix

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Generated python files from protobuf on Ubuntu:20.04

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Dependencies fix

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Added space in logging after review

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

* Made import in code more readable after review

Signed-off-by: G.Bazior <bazior@agh.edu.pl>

Signed-off-by: Piotr Pawlowski <ppiotru@gmail.com>
Signed-off-by: G.Bazior <bazior@agh.edu.pl>
Signed-off-by: Stepan Lavrentev <lawrentievsv@gmail.com>
Signed-off-by: Rafik Naccache <rafik@fekr.tech>
Signed-off-by: Alexey Rodionov <rodionov12@gmail.com>
Co-authored-by: Piotr Pawłowski <68233055+Pawlak00@users.noreply.github.com>
Co-authored-by: Grzegorz Bazior <g.bazior@yodiss.pl>
Co-authored-by: Stepan Lavrentev <40560660+stepanLav@users.noreply.github.com>
Co-authored-by: Rafik NACCACHE <rafik@fekr.tech>
Co-authored-by: Alexey Rodionov <rodionov12@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug dependencies Pull requests that update a dependency file iroha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants