Build instructions #18
Comments
|
Hi Bruno, So far, I have used the PhoneGap Desktop App to build the app locally (so downloading DearScholar, and opening this project using in the PhoneGap Desktop App should work). [Also see, "Guide for potential contributors and academic researchers" in the readme file]. When this doesn't work or you want more specific building instructions, please let me know. An option is to write a small program that installs the Desktop App, php, MySQL, and myPhPadmin locally? Best, Peter |
|
Hi Peter, I will have to check how to use phonegap, to remind me how to install and run it. But it would be simpler if there were build instructions. Here's an example of Annif, a Python backend that includes a small JS frontend. There is also a Dockerfile that can be used for both building instructions, or to run the application locally. Or this other project that contains both detailed installation instructions (points to another page) and how to run the application with docker. Perhaps you could just say something like:
Having a Docker container is convenient, but I think the installation instructions are more important. Especially if there are any limitations (OS version, system library, etc). For example, there is at least one PHP script using Cheers |
|
Hi Bruno, Thanks for these detailed suggestions. Very much appreciated. I will give priority to this issue--that is, provide more detailed installation instructions and check docker, which I have never used--and solve issue #20 next. Best, PS. Thanks for point out this inconsistency with the PHP script, will check all this. For your information I use |
|
Perfect Peter! I could be wrong on the PHP/MySQL issue. Very interested to see what DearScholar looks like running it locally while reading the code to understand how it works :-) Cheers |
|
Thanks :) ... All feedback is very welcome. |
|
Hi Bruno, To give you an update on the installation instructions. I am able to build a Docker image of the server (using this repro: https://github.com/jcavat/docker-lamp) and storing it on Docker Hub. However, when I download the image and try to restart it, it does not work (the admin panel does not show up at any port on my Mac and neither can I reach the php file). Once I have solved this issue, I will write clear installation instructions. Best, |
|
Hi Peter! Really impressed! You mentioned your daily job was not writing code. I was going to comment here saying that preparing an image for Docker could be difficult. But looks like you are already doing great progress! If you'd like to get a small container size, take a look at the Docker best practices doc. It's probably a ~20-40 minutes article, but may be helpful in case you ever need to use other images that appear to be too large. Cheers |
|
Hi Bruno, Thanks for your help. Over the weekend I wrote the installation instructions in the README.md file. I learned a lot about Docker, really helpful. Putting everything in the Docker seems ideal to me, but I thought it is best to start with writing out manual installation instructions (but using a Docker image for the server makes things a lot easier already). Could you please have a look at these instructions, and tell me if the instructions are clear and work to launch DearScholar locally? Thanks, (Note. I also corrected some minor errors and updated some procedures in the sample-study set up files.) |
|
Hi Peter! Reading the new documentation, really helpful! Thanks! Prepared a PR #23 with some suggestions, feel free to discard/suggest any edits. I've got
Next, I would run Do you know what I am missing, @pmkruyen, please? Tried that on the other Android directory, but had a similar issue. Thanks! |
|
Thanks for testing. O... I understand, the I have uploaded a sample Thanks, PS. Also thanks for showing me how you build the app on the command line :) |
|
Thanks for the Next error: Would that |
|
Hi Bruno, Thanks for super fast testing! Euhm.. I have not seen this error before (likely because I used the phonegap compiler online?). Yes, it has to do with Apple I found some resources online. I will try to replicate your error tonight, and find out whether the solution proposed in Discussion 1 does the trick. Discussion 1 --> Thanks for your patience and effort. Best, PS. For Android, I realise, it should be possible to drop this Cordova swift plugin from |
|
Thanks @pmkruyen , calling it a day now. But can test it again tomorrow. I forgot to say what's my environment, sorry. I am on Ubuntu 18.04.4 LTS 👍 just in case that's helpful. Cheers |
|
Hi Bruno, good morning ... :), I received the error too. To solve this, I did:
(Note, I used |
|
Can you then test the app now with Best, |
|
I think I will look for alternative documentation for the Android SDK on Ubuntu. Here's what I followed: https://medium.com/michael-wallace/how-to-install-android-sdk-and-setup-avd-emulator-without-android-studio-aeb55c014264 With these instructions: mkdir -p ~/Android/Sdk
export ANDROID_HOME=~/Android/Sdk
export ANDROID_SDK_ROOT=~/Android/Sdk
export ANDROID_AVD_HOME=~/.android/avd
mkdir ~/Android/Sdk/cmdline-tools
mv tools/ ~/Android/Sdk/cmdline-tools/
cd ~/Android/Sdk/cmdline-tools/tools/
export PATH=$PATH:$ANDROID_SDK_ROOT/cmdline-tools/tools:$ANDROID_SDK_ROOT/cmdline-tools/tools/bin:$ANDROID_SDK_ROOT/emulator:$ANDROID_SDK_ROOT/platform-tools
sdkmanager --licenses
sdkmanager --update
[=======================================] 100% Computing updates...
sdkmanager --list
sdkmanager emulator "system-images;android-28;default;x86" "platform-tools" "platforms;android-28" "build-tools;28.0.3"
avdmanager create avd --name android28 --package "system-images;android-28;default;x86"
cd ~/Development/javascript/workspace/dearscholar/App/
yarn add xcode
npx cordova platform rm android
npx cordova platform add android
npx cordova run androidBut got the following: npx: installed 432 in 13.892s
Checking Java JDK and Android SDK versions
ANDROID_SDK_ROOT=/home/kinow/Android/Sdk (recommended setting)
ANDROID_HOME=/home/kinow/Android/Sdk (DEPRECATED)
Subproject Path: CordovaLib
Subproject Path: app
> Configure project :app
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
registerResGeneratingTask is deprecated, use registerGeneratedResFolders(FileCollection)
> Task :app:preBuild UP-TO-DATE
> Task :CordovaLib:preBuild UP-TO-DATE
> Task :CordovaLib:preDebugBuild UP-TO-DATE
> Task :CordovaLib:checkDebugManifest UP-TO-DATE
> Task :CordovaLib:processDebugManifest UP-TO-DATE
> Task :app:preDebugBuild UP-TO-DATE
> Task :CordovaLib:compileDebugAidl NO-SOURCE
> Task :app:compileDebugAidl NO-SOURCE
> Task :CordovaLib:packageDebugRenderscript NO-SOURCE
> Task :app:compileDebugRenderscript UP-TO-DATE
> Task :app:checkDebugManifest UP-TO-DATE
> Task :app:generateDebugBuildConfig UP-TO-DATE
> Task :app:prepareLintJar UP-TO-DATE
> Task :app:generateDebugSources UP-TO-DATE
> Task :CordovaLib:compileDebugRenderscript UP-TO-DATE
> Task :CordovaLib:generateDebugBuildConfig UP-TO-DATE
> Task :CordovaLib:generateDebugResValues UP-TO-DATE
> Task :CordovaLib:generateDebugResources UP-TO-DATE
> Task :CordovaLib:packageDebugResources UP-TO-DATE
> Task :CordovaLib:generateDebugRFile UP-TO-DATE
> Task :CordovaLib:prepareLintJar UP-TO-DATE
> Task :CordovaLib:generateDebugSources UP-TO-DATE
> Task :CordovaLib:javaPreCompileDebug UP-TO-DATE
> Task :CordovaLib:compileDebugJavaWithJavac UP-TO-DATE
> Task :CordovaLib:processDebugJavaRes NO-SOURCE
> Task :CordovaLib:transformClassesAndResourcesWithPrepareIntermediateJarsForDebug UP-TO-DATE
> Task :app:javaPreCompileDebug UP-TO-DATE
> Task :app:mainApkListPersistenceDebug UP-TO-DATE
> Task :app:generateDebugResValues UP-TO-DATE
> Task :app:generateDebugResources UP-TO-DATE
> Task :app:writeDebugApplicationId UP-TO-DATE
> Task :app:processDebugGoogleServices
Parsing json file: /home/kinow/Development/javascript/workspace/dearscholar/Android26/platforms/android/app/google-services.json
> Task :app:mergeDebugResources UP-TO-DATE
> Task :app:createDebugCompatibleScreenManifests UP-TO-DATE
> Task :app:processDebugManifest UP-TO-DATE
> Task :app:processDebugResources UP-TO-DATE
> Task :app:compileDebugJavaWithJavac UP-TO-DATE
> Task :app:compileDebugNdk NO-SOURCE
> Task :app:compileDebugSources UP-TO-DATE
> Task :app:mergeDebugShaders UP-TO-DATE
> Task :app:compileDebugShaders UP-TO-DATE
> Task :app:generateDebugAssets UP-TO-DATE
> Task :CordovaLib:mergeDebugShaders UP-TO-DATE
> Task :CordovaLib:compileDebugShaders UP-TO-DATE
> Task :CordovaLib:generateDebugAssets UP-TO-DATE
> Task :CordovaLib:packageDebugAssets UP-TO-DATE
> Task :app:mergeDebugAssets UP-TO-DATE
> Task :app:validateSigningDebug UP-TO-DATE
> Task :app:signingConfigWriterDebug UP-TO-DATE
> Task :app:transformClassesWithDexBuilderForDebug UP-TO-DATE
> Task :app:transformDexArchiveWithExternalLibsDexMergerForDebug UP-TO-DATE
> Task :app:transformDexArchiveWithDexMergerForDebug UP-TO-DATE
> Task :app:mergeDebugJniLibFolders UP-TO-DATE
> Task :CordovaLib:compileDebugNdk NO-SOURCE
> Task :CordovaLib:mergeDebugJniLibFolders UP-TO-DATE
> Task :CordovaLib:transformNativeLibsWithMergeJniLibsForDebug UP-TO-DATE
> Task :CordovaLib:transformNativeLibsWithIntermediateJniLibsForDebug UP-TO-DATE
> Task :app:transformNativeLibsWithMergeJniLibsForDebug UP-TO-DATE
> Task :app:processDebugJavaRes NO-SOURCE
> Task :app:transformResourcesWithMergeJavaResForDebug UP-TO-DATE
> Task :app:packageDebug UP-TO-DATE
> Task :app:assembleDebug UP-TO-DATE
> Task :app:cdvBuildDebug UP-TO-DATE
BUILD SUCCESSFUL in 1s
44 actionable tasks: 1 executed, 43 up-to-date
Built the following apk(s):
/home/kinow/Development/javascript/workspace/dearscholar/Android26/platforms/android/app/build/outputs/apk/debug/app-debug.apk
Checking Java JDK and Android SDK versions
ANDROID_SDK_ROOT=/home/kinow/Android/Sdk (recommended setting)
ANDROID_HOME=/home/kinow/Android/Sdk (DEPRECATED)
No target specified and no devices found, deploying to emulator
No emulator specified, defaulting to android26
Waiting for emulator to start...
PANIC: Broken AVD system path. Check your ANDROID_SDK_ROOT value [/home/kinow/Android/Sdk]!Tried unsetting the env-var as recommended by some StackOverflow messages, changing to parent directory, but no luck so far. Will keep trying until the weekend, when I should have more time to troubleshoot it. Bruno |
|
Hi Bruno, Thanks for trying and the docs. I am going to give it try as well tonight to fix the Peter What I note is that you are in the |
Same issue. Logged in again to try that as I thought that was a really good guess! But the problem must be somewhere else. Almost! Thanks Peter! |
|
@pmkruyen found a post after I sent this message, saying for the SDK I needed now to have a different directory structure.
So moving the tools under a newly created
The UI is blank, but I assume that's because I don't have the backend running. Next step will be to create a container with PHP + Apache to run these scripts, start them mapping a port like 🤞 I hope you didn't start downloading the SDK (or freeing up space for that). Thanks |
|
Hi Bruno, 👍 super cool! Will try to replicate your steps. I have downloaded the command line tools too; but cannot see the other files. I assume that you generated those other files using the steps in the tutorial you posted above? I am glad the logo is there, but the empty screen worries me somewhat, seems that the app does not pass the splash screen?
We will find out :) |
|
Tried Maybe this is the error? |
|
Hi Bruno, Android emulator Browser Have you installed / use the PhoneGap desk emulator as specified in the installation instructions? When I pull the project from Github, and (re)open the app folder in this emulator, I did the following:
Does this work for you too? |
|
Hi Bruno, Android I set the restrictions because of iOS restrictions, but I don't think that should be the problem because you would still see the login screen. For testing, you can remove it. Browser Thanks for your help. |
|
Removed the domain restriction, using Used the meta entry for CSP: And it loaded most resources. The fingerprint plug-in wouldn't load, but that's not a huge issue. The push plug-in also was giving 404, so used Alas the browser still display a blank page. Also could't load in the emulator, same blank page 😞 |
|
Sorry, we are going to solve this. What I did:
=== Did you try to open the app at the PhoneGap port === When this does not work either, I will install a docker ubuntu container and run above steps too, to find out how to solve this issue. PS. Thanks for adding the |
|
My version of the PhoneGap emulator is v0.4.5 :) |
|
Optionally, when there is still the blank screen, you can remove all plugins and check what happens? Also, what happens when you try to run the hello world app? By the way, I realise, sometimes a blank screen occurs when the javascript is broken. But that should not be the case now, because when the javascript was broken neither I would be able to start the app. Peter |
|
Running the instructions from readme, the MySQL container appears to get stuck and fail to start? kinow@ranma:~/Development/javascript/workspace/dearscholar$ docker run --rm --name dearscholar -it -p "9000:9000" -v ${PWD}/App:/app -v ${PWD}/mysql:/var/lib/mysql mattrayner/lamp:latest-1804
Updating for PHP 7.4
Replacing CLI php.ini values
Editing APACHE_RUN_GROUP environment variable
Editing phpmyadmin config
Setting up MySQL directories
Allowing Apache/PHP to write to the app
Allowing Apache/PHP to write to MySQL
Editing MySQL config
=> An empty or uninitialized MySQL volume is detected in /var/lib/mysql
=> Installing MySQL ...
mysqld: Can't create directory '/var/lib/mysql/' (Errcode: 17 - File exists)
2020-07-31T22:14:57.043288Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2020-07-31T22:14:57.045058Z 0 [ERROR] Aborting
=> Done!
=> Waiting for confirmation of MySQL service startup
=> Waiting for confirmation of MySQL service startup
|
|
Hi Bruno, Thanks. We are going in the right direction. 👍 A quick reply: Backend Browser Android I am now at the point to install and load an emulator. I see that I have to specify the (note. the Android26 folder includes the settings that work for api 26 and 27 [it is a small adjustment in de Best, |
|
By the way, what happens when you delete the folder I now see that the other issue has been solved?
Peter |
Agree! Good progress, we might close this issue very soon 🤞
I used 28. Initially I was trying 26, in the Android26 folder. Then after you mentioned that was only for <=26, I switched to App folder with the 28 version. I also chose 28, because the first time I ran the
My understanding is that that is not a folder on my computer, , but instead a folder in the container. And after reading the container README file, I think that we are mapping the file
Yup! I was trying to run a separate MySQL container. I didn't remember you had written new instructions with Docker :-) |
|
Hi Bruno Android I manually deleted all build files, including those in
When I the app works as expected on Android, I will update the installation instructions. Hope you can verify these instructions (for Linux). PS. I got this warning now Browser Backend Best, |
|
Hi Bruno, Browser Can you verify the new installation instructions for the Browser [Step 6] (https://github.com/pmkruyen/dearscholar#step-6-run-dearscholar-in-a-browser)? I deleted the old Guide for contributors section from the Readme because of the overlap with the new (and better) installation instructions. I cannot verify / do not completely understand how you used the command Best, PS. The Android instructions will follow. |
Looks good!
Good idea. I wonder if the installation instructions would deserve its own page (wiki, INSTALL.md, site page, etc) someday. Just because it seems to be getting quite lengthy (not that it bothers me, on the contrary :)
👍 for now I think the application should work with any browser. If I run into problems using the current version I will try it again and raise a PR. |
|
The
The ! marks where I think we have a problem right now. If we navigate to the container's documentation at https://github.com/mattrayner/docker-lamp/blob/c3f1244fec229deaed61e94d1229ba831b1da465/README.md, it says:
See the part creating a mysql/ folder within your project, which is not what we have here. I think this is the root cause of my current issue. We probably need to find out how we can load the contents of the file |
|
Hi Bruno,
Had the same thought last night. Will do after the whole installation manual does deliver :) Browser Backend Best, PS. Thank you for your PR! |
|
Hi Bruno, Great that Browser works! Thanks for your help! That is the most convenient way of testing the app. Android Backend There are three issues--as in, points of departure for getting the backend running 😄 )--I noticed in your comments. Issue 1 I ran
... and the terminal shows....
.... and all works. Inspecting your code, I see that you changed the ports and you use a capital letter "A" in app. Does that cause the problem? What do you see when you navigate to Issue 2 Issue 3 In short, can you let me know what happens when you re-install the Docker image, preferably also delete the app folder and mysql folder created the image (both folders are at located at Thanks, |
Sounds like a good plan, if we have at least the browser platform working, that should be enough to run the app locally. Later we can work on the Android or iOS platforms, or tell users that they can download the apk or use the store version for the time being.
I don't think so. I tried both ways yesterday. But will try again today to confirm. Just synced the repo. Then executed: Then running
With the container in that state:
and
(slight change in address)
Ah, I see. I thought we were mapping the I believe my container has no services listening on any ports at the moment. Its initialization seems to have failed.
👍 looks like I was thinking too much ahead. Your instructions would have showed me how to use the
Sure. Removed the
|
|
Hi Bruno,
Thanks for your feedback and suggestions.
I am googling your error but have not figured out what’s going wrong.
An idea: have you tried to run the docker command with sudo rights?
Peter
… Op 3 aug. 2020 om 11:06 heeft Bruno P. Kinoshita ***@***.***> het volgende geschreven:
Almost there... but I cannot connect to the backend (receiving 404 errors), drives me crazy. I will give it another try tonight; or else will upload it to the installation instructions as 'work in progress'.
Sounds like a good plan, if we have at least the browser platform working, that should be enough to run the app locally. Later we can work on the Android or iOS platforms, or tell users that they can download the apk or use the store version for the time being.
Inspecting your code, I see that you changed the ports and you use a capital letter "A" in app. Does that cause the problem?
I don't think so. I tried both ways yesterday. But will try again today to confirm.
Just synced the repo. Then executed:
docker system prune -a -f
docker network prune
docker volume prune
Then running docker run -i -t -p "80:80" -v ${PWD}/app:/app -v ${PWD}/mysql:/var/lib/mysql mattrayner/lamp:latest-1804:
What do you see when you navigate to localhost/myphpadmin in your browser? What happens when you wipe the image completely and re-install / re-initialize it?
With the container in that state:
and
(slight change in address)
To be sure: The app folder in docker run does not refer to the app folder in the repo, but to the persistent location of the created webserver by mattrayner/lamp. Here is where you need to put the dearscholar.php file (see the instructions, step 5, this is a bit confusing I now see). It would be great indeed to include DearScholar in the docker too, but that is something on my bucket list.
Ah, I see. I thought we were mapping the App folder within the container. I am stuck at step 1 at the moment, trying to get pass "Step 1: Server setup". I can skip it, but looks like mysql is not running.
I believe my container has no services listening on any ports at the moment. Its initialization seems to have failed.
To be sure: Along similar lines, you cannot directly load the content of my mysql file into the MySQL database, but (at the moment) should use the mysql command line in the docker image (see step 1 and 2 of the manual). Can you open the mysql docker in your terminal (see step 1 of the manual).
👍 looks like I was thinking too much ahead. Your instructions would have showed me how to use the app and mysql folders. Once we manage to fix the initial set up, everything should work 🤞
In short, can you let me know what happens when you re-install the Docker image, preferably also delete the app folder and mysql folder created the image (both folders are at located at Users/peterkruyen/) and run docker run -i -t -p "80:80" -v ${PWD}/app:/app -v ${PWD}/mysql:/var/lib/mysql mattrayner/lamp:latest-1804.
Sure. Removed the app directory (looks like it had been created), and re-executed the exact command docker run -i -t -p "80:80" -v ${PWD}/app:/app -v ${PWD}/mysql:/var/lib/mysql mattrayner/lamp:latest-1804.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Wait a sec... I also see that you are trying to run When trying this too, I got the same error because of the presence of 🤞 Best, |
|
And if your run the docker command, I think you should also rename the When this work, will find other names for the file and folder ... |
|
Hi Bruno, Android Backend Peter |
|
Problem solved @pmkruyen !
Sorry! You were correct here. Just Will continue with the other instructions today 👍 Thanks Peter! |
|
Further progress today @pmkruyen Right now I think I have the backend working. I can see the POST request in the browser going to the Docker container, and being returned as success. However, the log in still fails with
I logged in with user
Any idea what I could be doing wrong? |
|
Hi Bruno,
Happy the Docker works. Will improve the instructions when you/we passed all setup stages.
Ok, let’s start working on the PIN error. Check: how is the password stored on the server (mysql database)?
Has it been hashed too (see the instructions, and the example.php script about how to achieve this)?
Will look at it further tonight!
(Also i will check if I did not make a mistake in setting the correct database rights).
Best,
Peter
… Op 4 aug. 2020 om 01:09 heeft Bruno P. Kinoshita ***@***.***> het volgende geschreven:
Further progress today @pmkruyen
Right now I think I have the backend working. I can see the POST request in the browser going to the Docker container, and being returned as success.
However, the log in still fails with
Er is iets mis gegaan met het opslaan van uw PIN code. Controleer uw gebruikersnaam en wachtwoord en probeer vervolgens opnieuw of neem contact op met Peter Kruyen.
I logged in with user test1, password test1. Then typed 1234 twice as my PIN. It looks like the PIN was actually stored in the database (that pintable table was empty before I tried).
Any idea what I could be doing wrong?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.
|
|
Hi Bruno, Can you also check what happens when you try to access the other tables in Maybe caused by moving around with the Docker image? I see online several solutions, such as stopping and restarting Peter PS. Do you also receive this error when trying to click on the tables in the left panel of phpmyadmin? |
|
Hi Bruno, I removed the All works, indeed the Android emulator works too :) (albeit the rendering is not 100% correct). Will start updating the instructions tonight. Hope we get it working on your computer very soon too 😄 . Peter |
|
Hi Bruno, Following your comments, I have moved all the technical details on the Wiki page (and restructured them a bit). Backend Android Thanks for all your patience and help. Peter |
|
Thanks @pmkruyen ! I had an evening meeting with the rest of my team ($work) yesterday, and today was organizing everything to start a 2 days leave (thu/fri NZ). Planning to use some of this spare time to have another go at reviewing the build instructions from scratch, after cleaning the whole environment. I also believe we should be able to run some command like Without having to perform any extra steps, such as running SQL or PHP commands, or editing files. Just would need some time to try - this is not necessary for the JOSS review, IMO. We can use the build instructions in the Wiki if they work. But I want to practice some Docker fu, so will try to automate the steps later :) Will try to reproduce the build tomorrow I think 👍 Thanks a lot!!! |
|
Hi Bruno, I just uploaded the files (in the newly created
Yes, this would be really welcome when the JOSS review is over and the paper has been accepted for publication (especially because all the testing on emulated Android devices requires quite some steps, from which I learned a lot by the way). Thank you Bruno. Have a nice leave / long weekend and thanks for the effort. Hope that a fresh restart solves the login issues and you get the Android emulator working. Best, PS. I will be on vacation next week (my responses will somewhat be delayed). PPS. When you loaded the tables in DearScholar and you change settings (e.g., put in different questions), it is important to push the reset button in the app otherwise the changes will not be shown (see the 'settings and options' sections too) 😄 |
|
Probably restarting the review tomorrow @pmkruyen 👍 if not the whole process, at least i) cleaning local environment (remove platforms, and the docker containers), and ii) sync repo, and iii) re-read the instructions. Will try to follow the instructions again with care either tomorrow or Sunday. |
|
Hi Bruno, Thank you so much! Hope we got everything working 😄 . Have a good weekend, |
|
Followed new installation instructions from the beginning to end, line by line. Successfully built and ran DearScholar locally 👍
Closing now! 🎉 Thanks @pmkruyen ! |
Hi @pmkruyen
Could you point to the building instructions, please? There are two folders with
package.json, and runningyarn installshould probably download the dependencies. But given these are cordova/phonegap projects, I believe I would need some dependencies installed (node, npm/yarn, cordova, etc).Would you have it available anywhere, please?
Related to JOSS paper review: openjournals/joss-reviews#2506
Thanks
Bruno
The text was updated successfully, but these errors were encountered: