Setup local env fails

[imod]

Hi,

I'm trying to setup a local environment to develop a custom policy, but I fail to get APIcast up and running as described...

Version

master branch

Steps To Reproduce

1. git clone git@github.com:3scale/APIcast.git
2. cd APIcast
3. make development
4. make dependencies --> fails

Current Result

domi 🙈 APIcast $ make development
Running on Darwin
docker compose -f docker-compose-devel.yml -f docker-compose-devel-volmount-mac.yml up -d
[+] Running 2/2
 ⠿ Container apicast_build_0-redis-1        Running                                                                            0.0s
 ⠿ Container apicast_build_0-development-1  Started                                                                            1.5s
docker compose -f docker-compose-devel.yml -f docker-compose-devel-volmount-mac.yml exec -e COLUMNS="`tput cols`" -e LINES="`tput lines`" --user 501:20 development bash
bash-4.4$ make dependencies
/usr/local/openresty/luajit/bin/rover lock --roverfile=/opt/app-root/src/gateway/Roverfile
Resolving /opt/app-root/src/gateway/Roverfile
Cloning into 'lua-resty-repl'...
fatal: unable to connect to github.com:
github.com[0: 140.82.121.3]: errno=Connection refused

/usr/local/openresty/luajit/bin/luajit-2.1.0-beta3: /usr/share/lua/5.1/rover/lock.lua:169: attempt to index local 'rockspec' (a nil value)
stack traceback:
        /usr/share/lua/5.1/rover/lock.lua:169: in function 'expand_dependencies'
        /usr/share/lua/5.1/rover/lock.lua:199: in function 'resolve'
        /usr/share/lua/5.1/rover/cli/lock.lua:23: in function </usr/share/lua/5.1/rover/cli/lock.lua:11>
        /usr/share/lua/5.1/rover/cli.lua:33: in function </usr/share/lua/5.1/rover/cli.lua:27>
        /usr/lib64/luarocks/rocks/lua-rover/scm-1/bin/rover:8: in main chunk
        [C]: at 0x55ad138087a0
make: *** [Makefile:214: /opt/app-root/src/gateway/Roverfile.lock] Error 1
bash-4.4$ ^C
bash-4.4$ exit

Additional Information

• I tried to access github.com from within the running container, it works with git (ssh and https) and curl (https)

[eguzki]

That is because some lua deps have git://github.com/ as source and github, for security reasons removed git plain protocol.

The proper fix should be done in the lua deps repos, replacing git://github.com/ with git@github.com or ssh://github.com. In the meantime, the workaround is to update your git configuration, i.e. in .gitconfig adding

[url "https://github.com/"]
	insteadOf = git://github.com/

That should do it

[imod]

thanks @eguzki ! It does get me further, but now it fails with this...

domi 🙈 APIcast $ make development
Running on Darwin
docker-compose -f docker-compose-devel.yml -f docker-compose-devel-volmount-mac.yml up -d
Creating network "apicast_build_0_default" with the default driver
Creating apicast_build_0_redis_1 ... done
Creating apicast_build_0_development_1 ... done
docker-compose -f docker-compose-devel.yml -f docker-compose-devel-volmount-mac.yml exec -e COLUMNS="`tput cols`" -e LINES="`tput lines`" --user 501:20 development bash
bash-4.4$ git config --global url.https://github.com/.insteadOf git://github.com/
bash-4.4$ cat ~/.gitconfig 
[url "https://github.com/"]
	insteadOf = git://github.com/
bash-4.4$ make dependencies
/usr/local/openresty/luajit/bin/rover lock --roverfile=/opt/app-root/src/gateway/Roverfile
Resolving /opt/app-root/src/gateway/Roverfile
Cloning into 'lua-resty-repl'...
remote: Enumerating objects: 52, done.
remote: Counting objects: 100% (52/52), done.
remote: Compressing objects: 100% (46/46), done.
remote: Total 52 (delta 3), reused 36 (delta 0), pack-reused 0
Unpacking objects: 100% (52/52), 21.30 KiB | 474.00 KiB/s, done.
Note: switching to '3878f41b7e8f97b1c96919db19dbee9496569dda'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by switching back to a branch.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -c with the switch command. Example:

  git switch -c <new-branch-name>

Or undo this operation with:

  git switch -

Turn off this advice by setting config variable advice.detachedHead to false

/usr/local/openresty/luajit/bin/luajit-2.1.0-beta3: /usr/share/lua/5.1/rover/lock.lua:169: attempt to index local 'rockspec' (a nil value)
stack traceback:
	/usr/share/lua/5.1/rover/lock.lua:169: in function 'expand_dependencies'
	/usr/share/lua/5.1/rover/lock.lua:199: in function 'resolve'
	/usr/share/lua/5.1/rover/cli/lock.lua:23: in function </usr/share/lua/5.1/rover/cli/lock.lua:11>
	/usr/share/lua/5.1/rover/cli.lua:33: in function </usr/share/lua/5.1/rover/cli.lua:27>
	/usr/lib64/luarocks/rocks/lua-rover/scm-1/bin/rover:8: in main chunk
	[C]: at 0x55ceb66087a0
make: *** [Makefile:214: /opt/app-root/src/gateway/Roverfile.lock] Error 1
bash-4.4$

[eguzki]

Actually, my hint about the git configuration is not needed, because the Makefile already does that for us (target translate_git_protocol called as dependecy of the target make dependencies. So I do not know why helped a bit.

The error is about trying to load a dependecy that is not found

Which commitID of the master branch are you working on?

Try to run make clean that will remove local docker images so you make sure images being used are latest.

[eguzki]

Ok, this is something related to macos. We will fix that.

[imod]

@eguzki great, many thanks! can you please ping me once this is available?

[imod]

unfortunate I don't understand the issue (yet), but is there anything I can help with?

[eguzki]

I am waiting to have one macOS so I can dig into the issue. Some lua library is not being installed and the lua manager, rover, does not check when a dep is not found and when accessing the name, it crashes. That's all I can say until I can reproduce in a macOS and figure out which dep is not being found and why

[imod]

@eguzki maybe I can help you with some more details? just let me know what information you require and I'm happy to reproduce and provide it.
I really would like to get started with custom policies :)

[guicassolato]

I did some investigation on this today. Here are a couple changes to workaround the issues reported (until a proper fix is sent).

1. "Connection refused"

To avoid Connection refused and the extra git config --global url.https://github.com/.insteadOf git://github.com command, which should not be needed.

Error:

bash-4.4$ make dependencies
/usr/local/openresty/luajit/bin/rover lock --roverfile=/opt/app-root/src/gateway/Roverfile
Resolving /opt/app-root/src/gateway/Roverfile
Cloning into 'lua-resty-repl'...
fatal: unable to connect to github.com:
github.com[0: 140.82.121.3]: errno=Connection refused

/usr/local/openresty/luajit/bin/luajit-2.1.0-beta3: /usr/share/lua/5.1/rover/lock.lua:169: attempt to index local 'rockspec' (a nil value)
stack traceback:
	/usr/share/lua/5.1/rover/lock.lua:169: in function 'expand_dependencies'
	/usr/share/lua/5.1/rover/lock.lua:199: in function 'resolve'
	/usr/share/lua/5.1/rover/cli/lock.lua:23: in function </usr/share/lua/5.1/rover/cli/lock.lua:11>
	/usr/share/lua/5.1/rover/cli.lua:33: in function </usr/share/lua/5.1/rover/cli.lua:27>
	/usr/lib64/luarocks/rocks/lua-rover/scm-1/bin/rover:8: in main chunk
	[C]: at 0x40000087a0
make: *** [Makefile:214: /opt/app-root/src/gateway/Roverfile.lock] Error 1

Fix:

diff --git a/Makefile b/Makefile
index dc8a667d..d1586798 100644
--- a/Makefile
+++ b/Makefile
@@ -216,7 +216,7 @@ $(GATEWAY_CONTEXT)/Roverfile.lock : $(GATEWAY_CONTEXT)/Roverfile $(GATEWAY_CONTE
 translate_git_protocol:
 	@git config --global url.https://github.com/.insteadOf git://github.com/
 
-lua_modules: $(ROVER) $(GATEWAY_CONTEXT)/Roverfile.lock translate_git_protocol
+lua_modules: $(ROVER) translate_git_protocol $(GATEWAY_CONTEXT)/Roverfile.lock
 # This variable is to skip issues with openssl 1.1.1
 # https://github.com/wahern/luaossl/issues/175
 	EXTRA_CFLAGS="-DHAVE_EVP_KDF_CTX=1" $(ROVER) install --roverfile=$(GATEWAY_CONTEXT)/Roverfile > /dev/null

2. "Rockspec format 3.0 not supported" (lua-resty-jwt v0.2.3) – the root cause of the last issue reported by @imod

To avoid attempt to index local 'rockspec' (a nil value), after fixing the git scheme and running make dependencies; caused by lua-resty-jwt with of the addition of lua-resty-openssl in v0.2.3 (see cdbattags/lua-resty-jwt#44).

Error:

bash-4.4$ make dependencies
/usr/local/openresty/luajit/bin/rover lock --roverfile=/opt/app-root/src/gateway/Roverfile
Resolving /opt/app-root/src/gateway/Roverfile
/usr/local/openresty/luajit/bin/luajit-2.1.0-beta3: /usr/share/lua/5.1/rover/lock.lua:169: attempt to index local 'rockspec' (a nil value)
stack traceback:
	/usr/share/lua/5.1/rover/lock.lua:169: in function 'expand_dependencies'
	/usr/share/lua/5.1/rover/lock.lua:199: in function 'resolve'
	/usr/share/lua/5.1/rover/cli/lock.lua:23: in function </usr/share/lua/5.1/rover/cli/lock.lua:11>
	/usr/share/lua/5.1/rover/cli.lua:33: in function </usr/share/lua/5.1/rover/cli.lua:27>
	/usr/lib64/luarocks/rocks/lua-rover/scm-1/bin/rover:8: in main chunk
	[C]: at 0x40000087a0
make: *** [Makefile:214: /opt/app-root/src/gateway/Roverfile.lock] Error 1

Fix:

diff --git a/gateway/apicast-scm-1.rockspec b/gateway/apicast-scm-1.rockspec
index bac8c66a..bc7999e0 100644
--- a/gateway/apicast-scm-1.rockspec
+++ b/gateway/apicast-scm-1.rockspec
@@ -14,7 +14,7 @@ dependencies = {
    'inspect',
    'lyaml',
    'router',
-   'lua-resty-jwt',
+   'lua-resty-jwt == 0.2.0',
    'lua-resty-url',
    'lua-resty-env',
    'lua-resty-execvp',

3. "Could not find header file for PCRE" (jsonschema v0.9.8)

To avoid Could not find header file for PCRE (not reported here), after rover lock passes and rover install is executed by the lua_modules target; caused by jsonschema with the addition of lrexlib-pcre in v0.9.8 (see api7/jsonschema#76).

Error:

bash-4.4$ make dependencies
Resolved Roverfile.lock in 	142	s
EXTRA_CFLAGS="-DHAVE_EVP_KDF_CTX=1" /usr/local/openresty/luajit/bin/rover install --roverfile=/opt/app-root/src/gateway/Roverfile > /dev/null
[...]
/usr/local/openresty/luajit/bin/luajit-2.1.0-beta3: /usr/share/lua/5.1/rover/install.lua:59: Could not find header file for PCRE
  No file pcre.h in /usr/local/include
  No file pcre.h in /usr/include
You may have to install PCRE in your system and/or pass PCRE_DIR or PCRE_INCDIR to the luarocks command.
Example: luarocks install lrexlib-pcre PCRE_DIR=/usr/local
stack traceback:
	[C]: in function 'assert'
	/usr/share/lua/5.1/rover/install.lua:59: in function 'install'
	/usr/share/lua/5.1/rover/install.lua:102: in function 'call'
	/usr/share/lua/5.1/rover/cli/install.lua:42: in function </usr/share/lua/5.1/rover/cli/install.lua:19>
	/usr/share/lua/5.1/rover/cli.lua:33: in function </usr/share/lua/5.1/rover/cli.lua:27>
	/usr/lib64/luarocks/rocks/lua-rover/scm-1/bin/rover:8: in main chunk
	[C]: at 0x40000087a0
make: *** [Makefile:222: lua_modules] Error 1

Fix:

diff --git a/gateway/Roverfile b/gateway/Roverfile
index a331fe06..b0f16e93 100644
--- a/gateway/Roverfile
+++ b/gateway/Roverfile
@@ -9,7 +9,7 @@ luarocks {
     group 'testing' {
         module { 'busted' },
         module { 'luacov' },
-        module { 'jsonschema' },
+        module { 'jsonschema', '0.8' },
     },
 
     group 'development' {

I have tested the above on macOS with M1 chip and the fixes make make dependencies pass. However, neither I have checked for regression on Linux, nor I can tell just yet if this is enough for the development environment to be fully functional on macOS.

After applying all the 3 fixes above and running make dependencies inside the development container, you should see the following output:

Complete! Modules were bundled into /opt/app-root/src/vendor/cache

---

In time: I also observed the following error when trying to compile luaossl at one of my attempts:

bash-4.4$ make dependencies
[...]
Resolved Roverfile.lock in 	142	s
EXTRA_CFLAGS="-DHAVE_EVP_KDF_CTX=1" /usr/local/openresty/luajit/bin/rover install --roverfile=/opt/app-root/src/gateway/Roverfile > /dev/null
Warning: skipping dependency checks.
No existing manifest. Attempting to rebuild...
Warning: skipping dependency checks.
Warning: skipping dependency checks.
Warning: skipping dependency checks.
Hunk 1 found at offset 2...
successfully patched /tmp/luarocks_luarocks-source-luaossl-20220711-0-oCpRgw/luaossl-rel-20220711/src/openssl.c
src/openssl.c:623:10: fatal error: openssl/core_names.h: No such file or directory
 #include <openssl/core_names.h>
          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
/usr/local/openresty/luajit/bin/luajit-2.1.0-beta3: /usr/share/lua/5.1/rover/install.lua:59: Build error: Failed compiling object src/openssl.o
stack traceback:
	[C]: in function 'assert'
	/usr/share/lua/5.1/rover/install.lua:59: in function 'install'
	/usr/share/lua/5.1/rover/install.lua:102: in function 'call'
	/usr/share/lua/5.1/rover/cli/install.lua:42: in function </usr/share/lua/5.1/rover/cli/install.lua:19>
	/usr/share/lua/5.1/rover/cli.lua:33: in function </usr/share/lua/5.1/rover/cli.lua:27>
	/usr/lib64/luarocks/rocks/lua-rover/scm-1/bin/rover:8: in main chunk
	[C]: at 0x40000087a0
make: *** [Makefile:222: lua_modules] Error 1

Exiting the container and running make clean, make development, and then make dependencies again solved the issue for me.

[eguzki]

I have run on Linux with your fixes (in details), but unfortunately I have the issue

src/openssl.c:623:10: fatal error: openssl/core_names.h: No such file or directory
 #include <openssl/core_names.h>
          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

I cannot solve the issue running make clean, make development and then make dependencies.

iff --git a/Makefile b/Makefile
index dc8a667d..d1586798 100644
--- a/Makefile
+++ b/Makefile
@@ -216,7 +216,7 @@ $(GATEWAY_CONTEXT)/Roverfile.lock : $(GATEWAY_CONTEXT)/Roverfile $(GATEWAY_CONTE
 translate_git_protocol:
 	@git config --global url.https://github.com/.insteadOf git://github.com/
 
-lua_modules: $(ROVER) $(GATEWAY_CONTEXT)/Roverfile.lock translate_git_protocol
+lua_modules: $(ROVER) translate_git_protocol $(GATEWAY_CONTEXT)/Roverfile.lock
 # This variable is to skip issues with openssl 1.1.1
 # https://github.com/wahern/luaossl/issues/175
 	EXTRA_CFLAGS="-DHAVE_EVP_KDF_CTX=1" $(ROVER) install --roverfile=$(GATEWAY_CONTEXT)/Roverfile > /dev/null
diff --git a/gateway/Roverfile b/gateway/Roverfile
index a331fe06..b0f16e93 100644
--- a/gateway/Roverfile
+++ b/gateway/Roverfile
@@ -9,7 +9,7 @@ luarocks {
     group 'testing' {
         module { 'busted' },
         module { 'luacov' },
-        module { 'jsonschema' },
+        module { 'jsonschema', '0.8' },
     },
 
     group 'development' {
diff --git a/gateway/Roverfile.lock b/gateway/Roverfile.lock
index 055ca7a6..72e914f6 100644
--- a/gateway/Roverfile.lock
+++ b/gateway/Roverfile.lock
@@ -1,40 +1,40 @@
-argparse 0.6.0-1||production
-busted 2.0.rc12-1||testing
+argparse 0.7.1-1||production
 basexx 0.4.1-1||development
 binaryheap 0.4-1||development
-bit32 5.3.0-1||development
-compat53 0.7-1||development
-cqueues 20190813.51-0||development
-dkjson 2.5-2||testing
+bit32 5.3.5.1-1||development
+busted 2.1.1-1||testing
+compat53 0.8-1||development
+cqueues 20200726.51-0||development
+date 2.2-2||production
+dkjson 2.6-1||testing
 fifo 0.2-0||development
-http 0.3-0||development
-inspect 3.1.1-0||production
+http 0.4-0||development
+inspect 3.1.3-0||production
+jsonschema 0.8-0|c1d72d86bb3dc5b33da57d47febc47657d29ea74|testing
 ldoc 1.4.6-2||development
-date 2.2-2||production
 liquid 0.2.0-2||production
-jsonschema 0.8-0|aa4740624cca4c10585bd7d086b42aa0b9ab14fa|testing
 lpeg 1.0.2-1||development
 lpeg_patterns 0.5-0||development
 lua-resty-env 0.4.0-1||production
 lua-resty-execvp 0.1.1-1||production
-lua-resty-http 0.15-0||production
-lua-resty-iputils 0.3.0-1||production
+lua-resty-http 0.17.0.beta.1-0||production
+lua-resty-iputils 0.3.0-2||production
 lua-resty-jit-uuid 0.0.7-2||production
 lua-resty-jwt 0.2.0-0||production
 lua-resty-repl 0.0.6-0|3878f41b7e8f97b1c96919db19dbee9496569dda|development
 lua-resty-url 0.3.5-1||production
 lua-term 0.7-1||testing
 lua_cliargs 3.0-2||testing
-luacov 0.13.0-1||testing
-luafilesystem 1.7.0-2||production,development,testing
-luaossl 20190731-0||development
-luassert 1.8.0-0||testing
+luacov 0.15.0-1||testing
+luafilesystem 1.8.0-1||testing,production,development
+luaossl 20220711-0||development
+luassert 1.9.0-1||testing
 luasystem 0.2.1-0||testing
-lyaml 6.2.4-1||production
+lyaml 6.2.7-1||production
 markdown 0.33-1||development
 mediator_lua 1.1.2-0||testing
-net-url 0.9-1||testing
-nginx-lua-prometheus 0.20181120-3||production
-penlight 1.7.0-1||production,development,testing
+net-url 1.1-1||testing
+nginx-lua-prometheus 0.20220527-1||production
+penlight 1.13.1-1||testing,production,development
 router 2.1-0||production
-say 1.3-1||testing
+say 1.4.1-3||testing
\ No newline at end of file
diff --git a/gateway/apicast-scm-1.rockspec b/gateway/apicast-scm-1.rockspec
index bac8c66a..bc7999e0 100644
--- a/gateway/apicast-scm-1.rockspec
+++ b/gateway/apicast-scm-1.rockspec
@@ -14,7 +14,7 @@ dependencies = {
    'inspect',
    'lyaml',
    'router',
-   'lua-resty-jwt',
+   'lua-resty-jwt == 0.2.0',
    'lua-resty-url',
    'lua-resty-env',
    'lua-resty-execvp',

[guicassolato]

I have run on Linux with your fixes (in details), but unfortunately I have the issue

src/openssl.c:623:10: fatal error: openssl/core_names.h: No such file or directory
 #include <openssl/core_names.h>
          ^~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

I cannot solve the issue running make clean, make development and then make dependencies.

@eguzki, you're right.

This is caused by the luaossl module version 20220711, specifically wahern/luaossl#199.

It is not a Linux nor macOS specific issue, but triggered whenever trying to compile this version of the module on any system with OpenSSL 1.1.1 while setting HAVE_EVP_KDF_CTX=1.

The root cause is this fix of ours, combined with this requirement from luaossl 20220711 and the fact that OpenSSL 1.1.1 does not include the core_names.h file (see openssl-1.1.1/includes vs openssl-3.0/includes).

I have tried a few things to workaround this issue without bumping to OpenSSL 3.x or having to rebuild the devel image in my local machine, but nothing worked given other constraints. I'll try a different approach tomorrow.