Error while starting server docker image

[Yosbnwpq]

After I built the docker image from the server code and attempt to run it, I'm getting an error complaining about an expected argument to the obasparql.QueryManager class. The init.py file on the latest obasparql github does not list queries_types as one of the supported arguments:

obasparql/query_manager.py

def init(self,
endpoint,
named_graph_base,
uri_prefix,
queries_dir,
context_dir,
endpoint_username=None,
endpoint_password=None):

init.py in oba generated server code

query_manager = QueryManager(queries_dir=QUERY_DIRECTORY,
context_dir=CONTEXT_DIRECTORY,
queries_types=QUERIES_TYPES,
endpoint=ENDPOINT,
endpoint_username=ENDPOINT_USERNAME,
endpoint_password=ENDPOINT_PASSWORD,
graph_base=ENDPOINT_GRAPH_BASE,
prefix=ENDPOINT_RESOURCE_PREFIX)

In addition, the arguments "uri_prefix" and "named_graph_base" may be called "prefix" and "graph_base" in the oba generated server code, respectively.

Here are the commands I'm running to build and start the docker container

docker build -t openapi_server .
...
docker run -p 8080:8080 openapi_server
...
Traceback (most recent call last):
File "/usr/local/lib/python3.10/runpy.py", line 187, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/local/lib/python3.10/runpy.py", line 146, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/usr/local/lib/python3.10/runpy.py", line 110, in _get_module_details
import(pkg_name)
File "/usr/src/app/openapi_server/init.py", line 13, in
query_manager = QueryManager(queries_dir=QUERY_DIRECTORY,
TypeError: QueryManager.init() got an unexpected keyword argument 'queries_types'

[dgarijo]

@mosoriob has been changing the obasparql package recently.

Is there a chance something broke?

@Yosbnwpq have you given a try to the latest release? Last time I tried everything worked fine, but that was a while ago.

[Yosbnwpq]

my obasparql version is 4.0.5 and oba is 3.6.0

[mosoriob]

Hi @Yosbnwpq

OBAsparql has been changed and I didn't update the templates files. Sorry 😢.

[Yosbnwpq]

Thanks, Maximiliano, can you give me the updated template files until the next build is ready?

[jphi545]

Hi @mosoriob

I am also having the same issue with obasparql and OBA.

When can I expect the next build to ensure this doesn't continue?

[DougalW]

Hi @mosoriob, I'm also having the same issue as @Yosbnwpq and @jphi545.

Here's two test runs I performed with the Restrictions Example.

Run 1. - followed documented process

Step 1: ran java -jar oba-3.6.0-jar-with-dependencies.jar -c config-restrict.yaml
Output: completes successfully
Step 2: in the outputs/RestrictionsExample directory I ran generate-server.sh
Output: completes successfully
Step 3: changed to server directory and ran docker build -t openapi_server .
Output: fails on "Error: Please make sure the libxml2 and libxslt development packages are installed."

Here's the log file:

[+] Building 19.5s (10/11)
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 35B 0.0s
=> [internal] load metadata for docker.io/library/python:3-alpine 0.0s
=> [1/7] FROM docker.io/library/python:3-alpine 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 8.89kB 0.0s
=> CACHED [2/7] RUN apk add build-base libffi-dev openssl-dev git 0.0s
=> CACHED [3/7] RUN mkdir -p /usr/src/app 0.0s
=> CACHED [4/7] WORKDIR /usr/src/app 0.0s
=> [5/7] COPY requirements.txt /usr/src/app/ 0.0s
=> ERROR [6/7] RUN pip3 install --no-cache-dir -r requirements.txt 19.3s

[6/7] RUN pip3 install --no-cache-dir -r requirements.txt:
#9 1.397 Collecting pythonql3==0.9.61
#9 1.505 Downloading pythonql3-0.9.61.tar.gz (65 kB)
#9 2.635 Collecting connexion>=2.6.0
#9 2.661 Downloading connexion-2.12.0-py2.py3-none-any.whl (94 kB)
#9 2.766 Collecting obasparql>=3.4.2
#9 2.786 Downloading obasparql-4.0.7-py3-none-any.whl (21 kB)
#9 2.893 Collecting werkzeug==0.16.1
#9 2.917 Downloading Werkzeug-0.16.1-py2.py3-none-any.whl (327 kB)
#9 3.052 Collecting swagger-ui-bundle>=0.0.2
#9 3.072 Downloading swagger_ui_bundle-0.0.9-py3-none-any.whl (6.2 MB)
#9 4.288 Collecting python_dateutil>=2.6.0
#9 4.308 Downloading python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
#9 4.359 Requirement already satisfied: setuptools>=21.0.0 in /usr/local/lib/python3.8/site-packages (from -r requirements.txt (line 7)) (50.3.0)
#9 4.419 Collecting validators>=0.14.2
#9 4.436 Downloading validators-0.18.2-py3-none-any.whl (19 kB)
#9 4.490 Collecting python-jose>=3.0.1
#9 4.513 Downloading python_jose-3.3.0-py2.py3-none-any.whl (33 kB)
#9 4.567 Collecting ply>=3.9
#9 4.587 Downloading ply-3.11-py2.py3-none-any.whl (49 kB)
#9 4.672 Collecting jsonschema<5,>=2.5.1
#9 4.693 Downloading jsonschema-4.4.0-py3-none-any.whl (72 kB)
#9 4.780 Collecting flask<3,>=1.0.4
#9 4.805 Downloading Flask-2.0.3-py3-none-any.whl (95 kB)
#9 4.868 Collecting clickclick<21,>=1.2
#9 4.887 Downloading clickclick-20.10.2-py2.py3-none-any.whl (7.4 kB)
#9 5.005 Collecting requests<3,>=2.9.1
#9 5.031 Downloading requests-2.27.1-py2.py3-none-any.whl (63 kB)
#9 5.099 Collecting inflection<0.6,>=0.3.1
#9 5.119 Downloading inflection-0.5.1-py2.py3-none-any.whl (9.5 kB)
#9 5.165 Collecting itsdangerous>=0.24
#9 5.185 Downloading itsdangerous-2.1.0-py3-none-any.whl (15 kB)
#9 5.304 Collecting PyYAML<6,>=5.1
#9 5.325 Downloading PyYAML-5.4.1.tar.gz (175 kB)
#9 5.518 Installing build dependencies: started
#9 8.795 Installing build dependencies: finished with status 'done'
#9 8.796 Getting requirements to build wheel: started
#9 10.92 Getting requirements to build wheel: finished with status 'done'
#9 10.93 Preparing wheel metadata: started
#9 11.73 Preparing wheel metadata: finished with status 'done'
#9 11.88 Collecting simplejson==3.16.0
#9 11.91 Downloading simplejson-3.16.0.tar.gz (81 kB)
#9 12.16 Collecting webencodings==0.5.1
#9 12.18 Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
#9 12.23 Collecting pyaml==18.11.0
#9 12.25 Downloading pyaml-18.11.0-py2.py3-none-any.whl (16 kB)
#9 12.30 Collecting six==1.11.0
#9 12.32 Downloading six-1.11.0-py2.py3-none-any.whl (10 kB)
#9 12.36 Collecting PyLD>=2.0.3
#9 12.39 Downloading PyLD-2.0.3.tar.gz (70 kB)
#9 12.64 Collecting rdflib==6.1.1
#9 12.66 Downloading rdflib-6.1.1-py3-none-any.whl (482 kB)
#9 12.79 Collecting urllib3==1.26.5
#9 12.82 Downloading urllib3-1.26.5-py2.py3-none-any.whl (138 kB)
#9 12.92 Collecting Jinja2>=2.0
#9 12.94 Downloading Jinja2-3.0.3-py3-none-any.whl (133 kB)
#9 13.03 Collecting decorator>=3.4.0
#9 13.05 Downloading decorator-5.1.1-py3-none-any.whl (9.1 kB)
#9 13.10 Collecting ecdsa!=0.15
#9 13.12 Downloading ecdsa-0.17.0-py2.py3-none-any.whl (119 kB)
#9 13.25 Collecting pyasn1
#9 13.27 Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
#9 13.34 Collecting rsa
#9 13.36 Downloading rsa-4.8-py3-none-any.whl (39 kB)
#9 13.46 Collecting attrs>=17.4.0
#9 13.48 Downloading attrs-21.4.0-py2.py3-none-any.whl (60 kB)
#9 13.64 Collecting importlib-resources>=1.4.0; python_version < "3.9"
#9 13.66 Downloading importlib_resources-5.4.0-py3-none-any.whl (28 kB)
#9 13.76 Collecting pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0
#9 13.78 Downloading pyrsistent-0.18.1.tar.gz (100 kB)
#9 13.83 Installing build dependencies: started
#9 15.22 Installing build dependencies: finished with status 'done'
#9 15.22 Getting requirements to build wheel: started
#9 15.97 Getting requirements to build wheel: finished with status 'done'
#9 15.98 Preparing wheel metadata: started
#9 16.74 Preparing wheel metadata: finished with status 'done'
#9 16.81 Collecting click>=7.1.2
#9 16.83 Downloading click-8.0.4-py3-none-any.whl (97 kB)
#9 16.90 Collecting idna<4,>=2.5; python_version >= "3"
#9 16.92 Downloading idna-3.3-py3-none-any.whl (61 kB)
#9 16.99 Collecting certifi>=2017.4.17
#9 17.01 Downloading certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
#9 17.11 Collecting charset-normalizer~=2.0.0; python_version >= "3"
#9 17.13 Downloading charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
#9 17.39 Collecting cachetools
#9 17.41 Downloading cachetools-5.0.0-py3-none-any.whl (9.1 kB)
#9 17.47 Collecting frozendict
#9 17.50 Downloading frozendict-2.3.0-py3-none-any.whl (11 kB)
#9 17.97 Collecting lxml
#9 18.01 Downloading lxml-4.8.0.tar.gz (3.2 MB)
#9 18.93 ERROR: Command errored out with exit status 1:
#9 18.93 command: /usr/local/bin/python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-b1yhh7er/lxml/setup.py'"'"'; file='"'"'/tmp/pip-install-b1yhh7er/lxml/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(file);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-ciaifxcl
#9 18.93 cwd: /tmp/pip-install-b1yhh7er/lxml/
#9 18.93 Complete output (3 lines):
#9 18.93 Building lxml version 4.8.0.
#9 18.93 Building without Cython.
#9 18.93 Error: Please make sure the libxml2 and libxslt development packages are installed.
#9 18.93 ----------------------------------------
#9 18.93 ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
#9 19.11 WARNING: You are using pip version 20.2.3; however, version 22.0.3 is available.
#9 19.11 You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.

---

executor failed running [/bin/sh -c pip3 install --no-cache-dir -r requirements.txt]: exit code: 1

Run 2. - changed the requirements.txt file to use the earliest compatible version of OBASPARQL (obasparql == 3.4.2)

Step 1: in the server directory I ran docker build -t openapi_server .
Output: completes successfully. Here's the log file:

dwatt$ docker build -t openapi_server .
[+] Building 32.3s (12/12) FINISHED
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 37B 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 35B 0.0s
=> [internal] load metadata for docker.io/library/python:3-alpine 0.0s
=> [1/7] FROM docker.io/library/python:3-alpine 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 8.74kB 0.0s
=> CACHED [2/7] RUN apk add build-base libffi-dev openssl-dev git 0.0s
=> CACHED [3/7] RUN mkdir -p /usr/src/app 0.0s
=> CACHED [4/7] WORKDIR /usr/src/app 0.0s
=> [5/7] COPY requirements.txt /usr/src/app/ 0.0s
=> [6/7] RUN pip3 install --no-cache-dir -r requirements.txt 31.5s
=> [7/7] COPY . /usr/src/app 0.1s
=> exporting to image 0.5s
=> => exporting layers 0.5s
=> => writing image sha256:060980d14e3600b3757091f57ea9655ec23413ed75fafa12811102f9784aa43b 0.0s
=> => naming to docker.io/library/openapi_server 0.0s

Use 'docker scan' to run Snyk tests against images to find vulnerabilities and learn how to fix them

Step 2: ran docker run -p 8080:8080 openapi_server
Output: errors with this log trace

Traceback (most recent call last):
File "/usr/local/lib/python3.8/runpy.py", line 185, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "/usr/local/lib/python3.8/runpy.py", line 144, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "/usr/local/lib/python3.8/runpy.py", line 111, in _get_module_details
import(pkg_name)
File "/usr/src/app/openapi_server/init.py", line 1, in
from obasparql import QueryManager
File "/usr/local/lib/python3.8/site-packages/obasparql/init.py", line 1, in
from .query_manager import QueryManager
File "/usr/local/lib/python3.8/site-packages/obasparql/query_manager.py", line 14, in
from obasparql import gquery
File "/usr/local/lib/python3.8/site-packages/obasparql/gquery.py", line 10, in
from rdflib.plugins.sparql.parser import Query, UpdateUnit
File "/usr/local/lib/python3.8/site-packages/rdflib/plugins/sparql/init.py", line 35, in
from . import parser
File "/usr/local/lib/python3.8/site-packages/rdflib/plugins/sparql/parser.py", line 182, in
Param('prefix', PN_PREFIX)) + Suppress(':').leaveWhitespace()
File "/usr/local/lib/python3.8/site-packages/rdflib/plugins/sparql/parserutils.py", line 113, in init
self.name = name
AttributeError: can't set attribute

Interestingly, with both runs if I start the server using "python3 -m openapi_server" it works fine and starts the OpenAPI server.

Seems to be some problem with how it's running the Docker image.

Any advice on how to fix this? I'd really like to be able to build the docker image.

[DougalW]

This seems to be a problem with dependencies/versions in requirements.txt and the lxml library not building / installing correctly on my OSX 12.2.1, M1 Pro (pretty much stock with minimal software installs). The identical problem and error messages also occurs on my Intel OSX 12.2.1 MacBook Pro.

Here's my diagnosis.......

Running through dependencies, I ran the model catalog example, and before building the server I updated the following:

1. modified these files:
   openapi_server/init.py
   openapi_server/controllers/init.py

Change query manager line at the end to:

query_manager = QueryManager(queries_dir=QUERY_DIRECTORY,
context_dir=CONTEXT_DIRECTORY,
endpoint=ENDPOINT,
endpoint_username=ENDPOINT_USERNAME,
endpoint_password=ENDPOINT_PASSWORD,
named_graph_base=ENDPOINT_GRAPH_BASE,
uri_prefix=ENDPOINT_RESOURCE_PREFIX)

2. modified the requirements.txt file to the following more recent versions:

pythonql3==0.9.61
connexion >= 2.6.0
obasparql >= 3.4.2
werkzeug>=2.0
swagger-ui-bundle >= 0.0.2
python_dateutil >= 2.6.0
setuptools >= 21.0.0
validators >= 0.14.2
python-jose >= 3.0.1
markupsafe==2.0.1

3. ran 'docker build -t openapi_server .' which produced the following output error:

...
#8 16.04 Collecting lxml
#8 16.06 Downloading lxml-4.8.0.tar.gz (3.2 MB)
#8 16.43 ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 3.2/3.2 MB 8.7 MB/s eta 0:00:00
#8 16.68 Preparing metadata (setup.py): started
#8 16.77 Preparing metadata (setup.py): finished with status 'error'
#8 16.78 error: subprocess-exited-with-error
#8 16.78
#8 16.78 × python setup.py egg_info did not run successfully.
#8 16.78 │ exit code: 1
#8 16.78 ╰─> [3 lines of output]
#8 16.78 Building lxml version 4.8.0.
#8 16.78 Building without Cython.
#8 16.78 Error: Please make sure the libxml2 and libxslt development packages are installed.
#8 16.78 [end of output]
#8 16.78
#8 16.78 note: This error originates from a subprocess, and is likely not a problem with pip.
#8 16.78 error: metadata-generation-failed
#8 16.78
#8 16.78 × Encountered error while generating package metadata.
#8 16.78 ╰─> See above for output.
#8 16.78
#8 16.78 note: This is an issue with the package mentioned above, not pip.
#8 16.78 hint: See above for details.

executor failed running [/bin/sh -c pip3 install --no-cache-dir -r requirements.txt]: exit code: 1

4. tried to build libxml2 and libxslt using various options, and none worked.

5. tried Mac Ports, which completes successfully and port list shows these libraries are installed, but re-running docker build fails with the same error

6. followed more detailed instructions here: https://lxml.de/build.html#building-lxml-on-macos-x

This failed with an ssl certificate error (but this may be due to installing macports because it downloaded openSSL):

Building lxml version 4.8.0.
https://zlib.net/
Using existing zlib downloaded into libs/zlib-1.2.12.tar.gz (delete this file if you want to re-download the package)
Unpacking zlib-1.2.12.tar.gz into build/tmp
https://ftp.gnu.org/pub/gnu/libiconv/
Using existing libiconv downloaded into libs/libiconv-1.16.tar.gz (delete this file if you want to re-download the package)
Unpacking libiconv-1.16.tar.gz into build/tmp
Traceback (most recent call last):
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1346, in do_open
h.request(req.get_method(), req.selector, req.data, headers,
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1285, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1331, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1280, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1040, in _send_output
self.send(msg)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 980, in send
self.connect()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/http/client.py", line 1454, in connect
self.sock = self._context.wrap_socket(self.sock,
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 500, in wrap_socket
return self.sslsocket_class._create(
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1040, in _create
self.do_handshake()
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/ssl.py", line 1309, in do_handshake
self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Users/dougalwatt/repos/lxml/setup.py", line 270, in
**setup_extra_options()
File "/Users/dougalwatt/repos/lxml/setup.py", line 162, in setup_extra_options
ext_modules = setupinfo.ext_modules(
File "/Users/dougalwatt/repos/lxml/setupinfo.py", line 74, in ext_modules
XML2_CONFIG, XSLT_CONFIG = build_libxml2xslt(
File "/Users/dougalwatt/repos/lxml/buildlibxml.py", line 418, in build_libxml2xslt
libxml2_dir = unpack_tarball(download_libxml2(download_dir, libxml2_version), build_dir)
File "/Users/dougalwatt/repos/lxml/buildlibxml.py", line 249, in download_libxml2
from_location = http_find_latest_version_directory(LIBXML2_LOCATION)
File "/Users/dougalwatt/repos/lxml/buildlibxml.py", line 180, in http_find_latest_version_directory
with closing(urlopen(url)) as res:
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 214, in urlopen
return opener.open(url, data, timeout)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 517, in open
response = self._open(req, data)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 534, in _open
result = self._call_chain(self.handle_open, protocol, protocol +
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 494, in _call_chain
result = func(*args)
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1389, in https_open
return self.do_open(http.client.HTTPSConnection, req,
File "/opt/local/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/urllib/request.py", line 1349, in do_open
raise URLError(err)
urllib.error.URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1129)>

So seems to be lxml causing the final problem.

[tahh1]

@mosoriob Is there any update/fix for this bug please ?

[dgarijo]

It looks like there has not been any updates yet. Sorry @tahh1, but maintaining this stack requires more time/resources than the ones we seem to have right now. The first part of the API generation seems to be working, and that can be reused as is. We'll have to see if we can fix the API implementation in the near future.

[tahh1]

@dgarijo I totally understand. Good luck !

[rhyslewisakl]

I have created a pull request that fixes the issues above.

[dgarijo]

Thanks @rhyslewisakl . I left a small comment. Other than that it looks ready to merge

[mosoriob]

@rhyslewisakl Thanks a lot. I have accepted it

[dgarijo]

Issue addressed in #161 thanks to @rhyslewisakl