the app crach in time of import psycopg2

[Bouchra994]

im working with psycopg2 to connect my app with posstgres data base it work good in laptop but when i try to deloy it crach in time of importing psycopg2

print("before psycopg2")
import psycopg2

print("after psycopg2")

Versions

• Python:3.7
• OS:wsl
• Kivy:1.11.1
• Cython:0.29.19
• OpenJDK:8

Description

// REPLACE ME: i want to know why does that happen

Spec file:

[app]

# (str) Title of your application
title = My Application

# (str) Package name
package.name = myapp

# (str) Package domain (needed for android/ios packaging)
package.domain = org.test

# (str) Source code where the main.py live
source.dir = .

# (list) Source files to include (let empty to include all the files)
source.include_exts = py,png,jpg,kv,atlas,ttf

# (list) List of inclusions using pattern matching
#source.include_patterns = assets/*,images/*.png

# (list) Source files to exclude (let empty to not exclude anything)
#source.exclude_exts = spec

# (list) List of directory to exclude (let empty to not exclude anything)
#source.exclude_dirs = tests, bin

# (list) List of exclusions using pattern matching
#source.exclude_patterns = license,images/*/*.jpg

# (str) Application versioning (method 1)
version = 0.1

# (str) Application versioning (method 2)
# version.regex = __version__ = ['"](.*)['"]
# version.filename = %(source.dir)s/main.py

# (list) Application requirements
# comma separated e.g. requirements = sqlite3,kivy
requirements = android,python3,kivy,kivymd,psycopg2,psycopg2-binary,libpq,pillow

# (str) Custom source folders for requirements
# Sets custom source for any requirements with recipes
# requirements.source.kivy = ../../kivy

# (list) Garden requirements
#garden_requirements =mapview

# (str) Presplash of the application
#presplash.filename = %(source.dir)s/data/presplash.png

# (str) Icon of the application
#icon.filename = %(source.dir)s/data/icon.png

# (str) Supported orientation (one of landscape, sensorLandscape, portrait or all)
orientation = portrait

# (list) List of service to declare
#services = NAME:ENTRYPOINT_TO_PY,NAME2:ENTRYPOINT2_TO_PY

#
# OSX Specific
#

#
# author = © Copyright Info

# change the major version of python used by the app
osx.python_version = 3

# Kivy version to use
osx.kivy_version = 1.11.1

#
# Android specific
#

# (bool) Indicate if the application should be fullscreen or not
fullscreen = 0

# (string) Presplash background color (for new android toolchain)
# Supported formats are: #RRGGBB #AARRGGBB or one of the following names:
# red, blue, green, black, white, gray, cyan, magenta, yellow, lightgray,
# darkgray, grey, lightgrey, darkgrey, aqua, fuchsia, lime, maroon, navy,
# olive, purple, silver, teal.
#android.presplash_color = #FFFFFF

# (list) Permissions
android.permissions = INTERNET

# (int) Target Android API, should be as high as possible.
android.api = 27

# (int) Minimum API your APK will support.
android.minapi = 21

# (int) Android SDK version to use
#android.sdk = 20

# (str) Android NDK version to use
android.ndk = 19c

# (int) Android NDK API to use. This is the minimum API your app will support, it should usually match android.minapi.
#android.ndk_api = 21

# (bool) Use --private data storage (True) or --dir public storage (False)
#android.private_storage = True

# (str) Android NDK directory (if empty, it will be automatically downloaded.)
#android.ndk_path =

# (str) Android SDK directory (if empty, it will be automatically downloaded.)
#android.sdk_path =

# (str) ANT directory (if empty, it will be automatically downloaded.)
#android.ant_path =

# (bool) If True, then skip trying to update the Android sdk
# This can be useful to avoid excess Internet downloads or save time
# when an update is due and you just want to test/build your package
# android.skip_update = False

# (bool) If True, then automatically accept SDK license
# agreements. This is intended for automation only. If set to False,
# the default, you will be shown the license when first running
# buildozer.
# android.accept_sdk_license = False

# (str) Android entry point, default is ok for Kivy-based app
#android.entrypoint = org.renpy.android.PythonActivity

# (str) Android app theme, default is ok for Kivy-based app
# android.apptheme = "@android:style/Theme.NoTitleBar"

# (list) Pattern to whitelist for the whole project
#android.whitelist =

# (str) Path to a custom whitelist file
#android.whitelist_src =

# (str) Path to a custom blacklist file
#android.blacklist_src =

# (list) List of Java .jar files to add to the libs so that pyjnius can access
# their classes. Don't add jars that you do not need, since extra jars can slow
# down the build process. Allows wildcards matching, for example:
# OUYA-ODK/libs/*.jar
#android.add_jars = foo.jar,bar.jar,path/to/more/*.jar

# (list) List of Java files to add to the android project (can be java or a
# directory containing the files)
#android.add_src =

# (list) Android AAR archives to add (currently works only with sdl2_gradle
# bootstrap)
#android.add_aars =

# (list) Gradle dependencies to add (currently works only with sdl2_gradle
# bootstrap)
#android.gradle_dependencies =

# (list) add java compile options
# this can for example be necessary when importing certain java libraries using the 'android.gradle_dependencies' option
# see https://developer.android.com/studio/write/java8-support for further information
# android.add_compile_options = "sourceCompatibility = 1.8", "targetCompatibility = 1.8"

# (list) Gradle repositories to add {can be necessary for some android.gradle_dependencies}
# please enclose in double quotes 
# e.g. android.gradle_repositories = "maven { url 'https://kotlin.bintray.com/ktor' }"
#android.add_gradle_repositories =

# (list) packaging options to add 
# see https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.PackagingOptions.html
# can be necessary to solve conflicts in gradle_dependencies
# please enclose in double quotes 
# e.g. android.add_packaging_options = "exclude 'META-INF/common.kotlin_module'", "exclude 'META-INF/*.kotlin_module'"
#android.add_gradle_repositories =

# (list) Java classes to add as activities to the manifest.
#android.add_activities = com.example.ExampleActivity

# (str) OUYA Console category. Should be one of GAME or APP
# If you leave this blank, OUYA support will not be enabled
#android.ouya.category = GAME

# (str) Filename of OUYA Console icon. It must be a 732x412 png image.
#android.ouya.icon.filename = %(source.dir)s/data/ouya_icon.png

# (str) XML file to include as an intent filters in <activity> tag
#android.manifest.intent_filters =

# (str) launchMode to set for the main activity
#android.manifest.launch_mode = standard

# (list) Android additional libraries to copy into libs/armeabi
#android.add_libs_armeabi = libs/android/*.so
#android.add_libs_armeabi_v7a = libs/android-v7/*.so
#android.add_libs_arm64_v8a = libs/android-v8/*.so
#android.add_libs_x86 = libs/android-x86/*.so
#android.add_libs_mips = libs/android-mips/*.so

# (bool) Indicate whether the screen should stay on
# Don't forget to add the WAKE_LOCK permission if you set this to True
#android.wakelock = False

# (list) Android application meta-data to set (key=value format)
#android.meta_data =

# (list) Android library project to add (will be added in the
# project.properties automatically.)
#android.library_references =

# (list) Android shared libraries which will be added to AndroidManifest.xml using <uses-library> tag
#android.uses_library =

# (str) Android logcat filters to use
android.logcat_filters = *:S python:D

# (bool) Copy library instead of making a libpymodules.so
#android.copy_libs = 1

# (str) The Android arch to build for, choices: armeabi-v7a, arm64-v8a, x86, x86_64
android.arch = armeabi-v7a

# (int) overrides automatic versionCode computation (used in build.gradle)
# this is not the same as app version and should only be edited if you know what you're doing
# android.numeric_version = 1

#
# Python for android (p4a) specific
#

# (str) python-for-android fork to use, defaults to upstream (kivy)
#p4a.fork = kivy

# (str) python-for-android branch to use, defaults to master
#p4a.branch = master

# (str) python-for-android git clone directory (if empty, it will be automatically cloned from github)
#p4a.source_dir =

# (str) The directory in which python-for-android should look for your own build recipes (if any)
#p4a.local_recipes =

# (str) Filename to the hook for p4a
#p4a.hook =

# (str) Bootstrap to use for android builds
# p4a.bootstrap = sdl2

# (int) port number to specify an explicit --port= p4a argument (eg for bootstrap flask)
#p4a.port =


#
# iOS specific
#

# (str) Path to a custom kivy-ios folder
#ios.kivy_ios_dir = ../kivy-ios
# Alternately, specify the URL and branch of a git checkout:
ios.kivy_ios_url = https://github.com/kivy/kivy-ios
ios.kivy_ios_branch = master

# Another platform dependency: ios-deploy
# Uncomment to use a custom checkout
#ios.ios_deploy_dir = ../ios_deploy
# Or specify URL and branch
ios.ios_deploy_url = https://github.com/phonegap/ios-deploy
ios.ios_deploy_branch = 1.7.0

# (str) Name of the certificate to use for signing the debug version
# Get a list of available identities: buildozer ios list_identities
#ios.codesign.debug = "iPhone Developer: <lastname> <firstname> (<hexstring>)"

# (str) Name of the certificate to use for signing the release version
#ios.codesign.release = %(ios.codesign.debug)s


[buildozer]

# (int) Log level (0 = error only, 1 = info, 2 = debug (with command output))
log_level = 2

# (int) Display warning if buildozer is run as root (0 = False, 1 = True)
warn_on_root = 1

# (str) Path to build artifact storage, absolute or relative to spec file
# build_dir = ./.buildozer

# (str) Path to build output (i.e. .apk, .ipa) storage
# bin_dir = ./bin

#    -----------------------------------------------------------------------------
#    List as sections
#
#    You can define all the "list" as [section:key].
#    Each line will be considered as a option to the list.
#    Let's take [app] / source.exclude_patterns.
#    Instead of doing:
#
#[app]
#source.exclude_patterns = license,data/audio/*.wav,data/images/original/*
#
#    This can be translated into:
#
#[app:source.exclude_patterns]
#license
#data/audio/*.wav
#data/images/original/*
#


#    -----------------------------------------------------------------------------
#    Profiles
#
#    You can extend section / key with a profile
#    For example, you want to deploy a demo version of your application without
#    HD content. You could first change the title to add "(demo)" in the name
#    and extend the excluded directories to remove the HD content.
#
#[app@demo]
#title = My Application (demo)
#
#[app:source.exclude_patterns@demo]
#images/hd/*
#
#    Then, invoke the command line with the "demo" profile:
#
#buildozer --profile demo android debug
' ' '
Command:
buildozer android debug run logcat

 Logs
# Run '/home/bouchra/.buildozer/android/platform/android-sdk/platform-tools/adb install -r "/mnt/c/users/lenovo/desktop/login/bin/myapp-0.1-armeabi-v7a-debug.apk"'
# Cwd /home/bouchra/.buildozer/android/platform
Performing Streamed Install
Success
# Application pushed.
# Run on DEFNW19829001345
# Run '/home/bouchra/.buildozer/android/platform/android-sdk/platform-tools/adb shell am start -n org.test.myapp/org.kivy.android.PythonActivity -a org.kivy.android.PythonActivity'
# Cwd /home/bouchra/.buildozer/android/platform
Starting: Intent { act=org.kivy.android.PythonActivity cmp=org.test.myapp/org.kivy.android.PythonActivity }
# Application started.
# Run 'dpkg --version'
# Cwd None
Debian 'dpkg' package management program version 1.19.7 (amd64).
This is free software; see the GNU General Public License version 2 or
later for copying conditions. There is NO warranty.
# Search for Git (git)
#  -> found at /usr/bin/git
# Search for Cython (cython)
#  -> found at /home/bouchra/.local/bin/cython
# Search for Java compiler (javac)
#  -> found at /usr/lib/jvm/java-8-openjdk-amd64/bin/javac
# Search for Java keytool (keytool)
#  -> found at /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/keytool
# Run '/home/bouchra/.buildozer/android/platform/android-sdk/platform-tools/adb logcat *:S python:D'
# Cwd /home/bouchra/.buildozer/android/platform

--------- beginning of crash
--------- beginning of system
--------- beginning of main
06-22 18:48:14.474 9178 9396 I python : Initializing Python for Android
06-22 18:48:14.474 9178 9396 I python : Setting additional env vars from p4a_env_vars.txt
06-22 18:48:14.474 9178 9396 I python : Changing directory to the one provided by ANDROID_ARGUMENT
06-22 18:48:14.474 9178 9396 I python : /data/user/0/org.test.myapp/files/app
06-22 18:48:14.474 9178 9396 I python : Preparing to initialize python
06-22 18:48:14.474 9178 9396 I python : _python_bundle dir exists
06-22 18:48:14.474 9178 9396 I python : calculated paths to be...
06-22 18:48:14.474 9178 9396 I python : /data/user/0/org.test.myapp/files/app/_python_bundle/stdlib.zip:/data/user/0/org.test.myapp/files/app/_python_bundle/modules
06-22 18:48:14.475 9178 9396 I python : set wchar paths...
06-22 18:48:14.561 9178 9396 I python : Initialized python
06-22 18:48:14.561 9178 9396 I python : AND: Init threads
06-22 18:48:14.563 9178 9396 I python : testing python print redirection
06-22 18:48:14.564 9178 9396 I python : Android path ['.', '/data/user/0/org.test.myapp/files/app/_python_bundle/stdlib.zip', '/data/user/0/org.test.myapp/files/app/_python_bundle/modules', '/data/user/0/org.test.myapp/files/app/_python_bundle/site-packages']
06-22 18:48:14.565 9178 9396 I python : os.environ is environ({'PATH': '/sbin:/system/sbin:/system/bin:/system/xbin:/odm/bin:/vendor/bin:/vendor/xbin', 'DOWNLOAD_CACHE': '/data/cache', 'ANDROID_BOOTLOGO': '1', 'ANDROID_ROOT': '/system', 'ANDROID_ASSETS': '/system/app', 'ANDROID_DATA': '/data', 'ANDROID_STORAGE': '', 'EXTERNAL_STORAGE': '/sdcard', 'ASEC_MOUNTPOINT': '/mnt/asec', 'BOOTCLASSPATH': '/system/framework/core-oj.jar:/system/framework/core-libart.jar:/system/framework/conscrypt.jar:/system/framework/okhttp.jar:/system/framework/bouncycastle.jar:/system/framework/apache-xml.jar:/system/framework/ext.jar:/system/framework/framework.jar:/system/framework/telephony-common.jar:/system/framework/voip-common.jar:/system/framework/ims-common.jar:/system/framework/android.hidl.base-V1.0-java.jar:/system/framework/android.hidl.manager-V1.0-java.jar:/system/framework/framework-oahl-backward-compatibility.jar:/system/framework/android.test.base.jar:/system/framework/featurelayer-widget.jar:/system/framework/hwEmui.jar:/system/framework/hwTelephony-common.jar:/system/framework/hwIms-common.jar:/system/framework/hwframework.jar:/system/framework/com.huawei.nfc.jar:/system/framework/org.ifaa.android.manager.jar:/system/framework/hwperf.jar:/system/framework/hwaps.jar:/system/framework/hwcustEmui.jar:/system/framework/hwcustframework.jar:/system/framework/hwcustTelephony-common.jar:/system/framework/servicehost.jar:/system/framework/hwIAwareAL.jar', 'SYSTEMSERVERCLASSPATH': '/system/framework/services.jar:/system/framework/ethernet-service.jar:/system/framework/wifi-service.jar:/system/framework/com.android.location.provider.jar:/system/framework/hwServices.jar:/system/framework/hwWifi-service.jar:/system/framework/hwcustServices.jar:/system/framework/hwcustwifi-service.jar', 'OEM_ROOT': '/product/hw_oem/JKM-L21', 'ODM_PRODUCT': '/odm/hw_odm/JKM-L21', 'CUST_POLICY_DIRS': '/system/emui/base:/system/emui/oversea:/system/emui/lite:/vendor/etc:/odm/etc:/product/etc:/odm/hw_odm/JKM-L21:/product/region_comm/oversea:/preas/oversea:/product/hw_oem/JKM-L21:/preload/JKM-L21/hw/meafnaf:/cust/global:/cust/hw/meafnaf:/product/special_cust/JKM-L21/hw/meafnaf', 'ANDROID_SOCKET_zygote_secondary': '18', 'ANDROID_ENTRYPOINT': 'main.pyc', 'ANDROID_ARGUMENT': '/data/user/0/org.test.myapp/files/app', 'ANDROID_APP_PATH': '/data/user/0/org.test.myapp/files/app', 'ANDROID_PRIVATE': '/data/user/0/org.test.myapp/files', 'ANDROID_UNPACK': '/data/user/0/org.test.myapp/files/app', 'PYTHONHOME': '/data/user/0/org.test.myapp/files/app', 'PYTHONPATH': '/data/user/0/org.test.myapp/files/app:/data/user/0/org.test.myapp/files/app/lib', 'PYTHONOPTIMIZE': '2', 'P4A_BOOTSTRAP': 'SDL2', 'PYTHON_NAME': 'python', 'P4A_IS_WINDOWED': 'True', 'P4A_ORIENTATION': 'portrait', 'P4A_NUMERIC_VERSION': 'None', 'P4A_MINSDK': '21', 'LC_CTYPE': 'C.UTF-8'})
06-22 18:48:14.565 9178 9396 I python : Android kivy bootstrap done. name is main
06-22 18:48:14.565 9178 9396 I python : AND: Ran string
06-22 18:48:14.565 9178 9396 I python : Run user program, change dir and execute entrypoint
06-22 18:48:14.565 9178 9396 I python : before psycopg2
06-22 18:48:14.568 9178 9396 I python : Traceback (most recent call last):
06-22 18:48:14.568 9178 9396 I python : File "/mnt/c/users/lenovo/desktop/login/.buildozer/android/app/main.py", line 4, in
06-22 18:48:14.568 9178 9396 I python : File "/mnt/c/users/lenovo/desktop/login/.buildozer/android/platform/build-armeabi-v7a/build/python-installs/myapp/psycopg2/init.py", line 50, in
06-22 18:48:14.568 9178 9396 I python : ImportError: dlopen failed: cannot locate symbol "PQencryptPasswordConn" referenced by "/data/data/org.test.myapp/files/app/_python_bundle/site-packages/psycopg2/_psycopg.so"...
06-22 18:48:14.568 9178 9396 I python : Python for android ended.

[RobertFlatt]

From https://pypi.org/project/psycopg2/

Psycopg 2 is mostly implemented in C as a libpq wrapper,

The C needs to be compiled for ARM.
psycopg2-binary only supports Linux/OSX/Windows https://pypi.org/project/psycopg2-binary/

Specifying the uncompiled psycopg2 and the compiled psycopg2-binary in requirements is probably not helpful.

p4a has a recipe mechanism for compiling non pure Python packages, a list of implemented packages is here:
https://github.com/kivy/python-for-android/tree/develop/pythonforandroid/recipes

The alternatives are finding a pure Python package, or building a recipe
https://python-for-android.readthedocs.io/en/latest/recipes/

Not a bug I think.

[AndreMiras]

I didn't realised the psycopg2-binary was used there, thanks for the heads-up.
@Bouchra994 could you try without and try from clean build (buildozer android clean)?

[Bouchra994]

Hi thank u for reply for the psycopg2 - Binary I have already test without it but still the same error

[RobertFlatt]

I think this is the expected result with psycopg2 (psycopg2-binary contains nothing useful)
The compile for ARM is not occuring because there is no ARM compile recipe; so the symbol "PQencryptPasswordConn" is not found.

[Bouchra994]

Thank u but what is the Arm and what do u think I should do

[RobertFlatt]

ARM is a processor architecture, the C code must be compiled to generate binaries for the instruction set of a particular processor architecture (usually ARM or derivatives of intel's x86). Android uses ARM, desktops use Intel based (Apple's announcement aside).

What you should do depends on your needs, not a question I can answer.

As I stated previously the options I know of are: 1) an existing recipe, 2) find a pure Python package, 3) write a recipe.

If you have to ask about ARM, I'd suggest that 3) might be a a challenge. If 2) exists (see below) and if the speed is sufficient then this would be the easiest. And for 1) I don't know if the available recipes meet your need or what app changes would be required.

A quick Google search returns a starting point: https://wiki.postgresql.org/wiki/Python

[AndreMiras]

What do you mean @Ham-Merhead? We do have a recipe for https://github.com/kivy/python-for-android/blob/v2020.06.02/pythonforandroid/recipes/psycopg2/__init__.py right?
So the question to be is with the recipe is not compiling/linking libqa properly https://github.com/kivy/python-for-android/blob/v2020.06.02/pythonforandroid/recipes/libpq/__init__.py

[RobertFlatt]

My error about the existence of the recipe, so yes it looks like a bug.
Workaround is still the same, use pure Python.

[Bouchra994]

thank u but please what do u mean by using pure python

[RobertFlatt]

pure Python - the code is python and no other language.

[Bouchra994]

In the exemple I used just python I didn't even use kivy

[RobertFlatt]

From https://pypi.org/project/psycopg2/

Psycopg 2 is mostly implemented in C as a libpq wrapper,

So the app is not pure Python

The app code may be pure Python, but psycopg2 is not pure Python and the app uses psycopg2. Which is why we talk about compiling C with a recipe. This is what appears to be failing in this case. A pure Python alternative works around this issue (assuming it does work!, and it is fast enough) by replacing psycopg2 with something else.

As I said previously: A quick Google search returns a starting point: https://wiki.postgresql.org/wiki/Python
Look at the "Platforms" column.

If you want to debug the current configuration, simplify the .spec buy removing unnecessary requirements and don't forget the buildozer appclean.
Read the log_level=2 build log carefully (every line), there may be a hint in there.

[RobertFlatt]

I just noticed this is the "develop" branch of p4a
https://github.com/kivy/python-for-android/tree/develop/pythonforandroid/recipes/psycopg2

[AndreMiras]

Yes we got some recent runtime fixes in "develop"

• Bump libpq version #2245
• Fix runtime psycopg2 error #2246

It might be worth giving a try @Bouchra994 change to p4a.branch = develop and do a clean buildozer android clean and a rebuild to see

[AndiEcker]

alternatively you could use this pure-python package for to connect to your postgres database: https://github.com/tlocke/pg8000 Am So., 28. Juni 2020 um 09:46 Uhr schrieb Andre Miras < notifications@github.com>:

…

Yes we got some recent runtime fixes in "develop" - #2245 <#2245> - #2246 <#2246> It might be worth giving a try @Bouchra994 <https://github.com/Bouchra994> change to p4a.branch = develop and do a clean buildozer android clean and a rebuild to see — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2240 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AA4U63RIOHHYIXWDEKKWTZLRY37PFANCNFSM4OE6CXAA> .