Python error with cgo lambda #64

angelabad · 2017-07-20T08:14:03Z

Hello @mweagle, I am trying to switch to sparta cgo, but allways I get the same error in cloudwatch (I attach it)

Error: unknown command "/var/runtime/awslambda/bootstrap.py" for "python3.6"

I attach the provision -n log too, and you can see my go code in this branch:

https://github.com/angelabad/unsplash-telegram-bot/tree/feature/cgo

cloudwatch_error.txt

Thanks in advance

angelabad · 2017-07-20T08:36:14Z

provision command output with -n:

INFO[0000] ========================================
INFO[0000] Welcome to UnsplashTelegramBot-Devel GoVersion=go1.8.3 LinkFlags= Option=provision SpartaSHA=4a93bd5 SpartaVersion=0.13.0 UTC="2017-07-20T08:33:41Z"
INFO[0000] ========================================
INFO[0000] Provisioning service BuildID=398f4896d1b9301c2adeae538065dafa83acfb7d CodePipelineTrigger= InPlaceUpdates=false NOOP=true Tags=
INFO[0000] Verifying IAM Lambda execution roles
INFO[0000] IAM roles verified Count=1
INFO[0000] Bypassing S3 upload due to -n/-noop command line argument. Bucket=angelsparta VersioningEnabled=false
INFO[0000] Running go generate
INFO[0000] Building cgo library in Docker Args="[run --rm -v /home/angel/devel/golang:/usr/src/gopath -w /usr/src/gopath/src/github.com/angelabad/unsplash-telegram-bot -e GOPATH=/usr/src/gopath -e GOOS=linux -e GOARCH=amd64 golang:1.8.3 go build -o UnsplashTelegramBot_Devel.lambda.so -tags lambdabinary linux -buildmode=c-shared -tags lambdabinary noop ]" Name=UnsplashTelegramBot_Devel.lambda.so
INFO[0022] Executable binary size KB=18481 MB=18
INFO[0022] Creating code ZIP archive for upload TempName=/home/angel/devel/golang/src/github.com/angelabad/unsplash-telegram-bot/.sparta/UnsplashTelegramBot_Devel-code.zip
INFO[0023] Lambda function deployment package size KB=4929 MB=4
INFO[0023] Bypassing S3 upload due to -n/-noop command line argument Bucket=angelsparta File=UnsplashTelegramBot_Devel-code.zip Key=UnsplashTelegramBot-Devel/UnsplashTelegramBot_Devel-code-96f51ac1de427ef7e9f64f8580cfbeee63a30ba0.zip
INFO[0023] Checking current APIGateway stage status APIName=UnsplashTelegramBot-Devel StageName=prod
INFO[0023] Bypassing APIGateway check to -n/-noop command line argument
INFO[0023] Bypassing Stack creation due to -n/-noop command line argument Bucket=angelsparta TemplateName=UnsplashTelegramBot_Devel-cftemplate.json
INFO[0023] ----------------------------------------
INFO[0023] Summary
INFO[0023] ----------------------------------------
INFO[0023] Verifying IAM roles Duration (s)=0
INFO[0023] Verifying AWS preconditions Duration (s)=0
INFO[0023] Creating code bundle Duration (s)=23
INFO[0023] Uploading code Duration (s)=0
INFO[0023] Ensuring CloudFormation stack Duration (s)=0
INFO[0023] Total elapsed time Duration (s)=23
INFO[0023] ----------------------------------------

mweagle · 2017-07-23T05:33:27Z

Taking a look

mweagle · 2017-07-23T17:47:47Z

Hi @angelabad - I'm seeing some odd behavior with the sample app as well. The Lambda function fails to provision, but there is no error cause. Need to investigate further...

angelabad · 2017-07-24T15:14:57Z

Ok, thanks for your work!

mweagle · 2017-07-29T16:10:36Z

I think this was a different manifestation of #63 - working on a release to resolve.

mweagle · 2017-07-29T16:31:42Z

@angelabad - Can you please regress with https://github.com/mweagle/Sparta/blob/master/CHANGES.md#v0131 and LMK if this resolves your issue? TIA.

angelabad · 2017-07-30T15:36:23Z

Hi @mweagle, same error here, I attach cloudwatch error log

Thanks in advance!
error.txt

mweagle · 2017-07-30T17:34:58Z

🤔 - I'm not able to reproduce this. A few questions:

What AWS region are you deploying to?
What OS, docker version (docker -v) are you using?
Are you able to reproduce with the https://github.com/mweagle/SpartaPython/tree/master/cgo sample application?
- Are you able to reproduce if you deploy to us-west-2 (this is the region I test against)
Can you share with me a provision --level debug log?

Thanks!

mweagle · 2017-08-03T12:36:51Z

Hi @angelabad - checking in to see if you've been able to make any progress on this issue. TIA.

angelabad · 2017-08-04T08:19:39Z

Hi @mweagle, sorry for my delay.

What AWS region are you deploying to?
- us-west-2
What OS, docker version (docker -v) are you using?
- Debian Sid
- Linux bangalore 4.11.0-2-amd64 Fix for parsing of existing IAM policy #1 SMP Debian 4.11.11-1 (2017-07-22) x86_64 GNU/Linux
- Docker version 17.06.0-ce, build 02c1d87
Are you able to reproduce with the https://github.com/mweagle/SpartaPython/tree/master/cgo sample application?
- No, I uploaded this app and it worked fine, I think the problem is in my code, but viewing documentation I dont know where the error is :-(
Are you able to reproduce if you deploy to us-west-2 (this is the region I test against)
- I use this region
Can you share with me a provision --level debug log?
- I attach a provision log and 2 update logs, one with dep and another with normal $GOPATH

Thanks in advance!

Ref: #64

mweagle · 2017-08-12T05:25:01Z

Hi @angelabad - I've pushed up https://github.com/mweagle/Sparta/tree/mweagle/0.13.3 that has a bit more error handling to help diagnose. Could you try with that branch and see if there's any more diagnostic information? Thanks!

angelabad · 2017-08-12T07:22:22Z

Hi @mweagle, same error here :-(

I attach logs

Its possible the error be in my code? Here is my cgo branch:

https://github.com/angelabad/unsplash-telegram-bot/tree/feature/cgo

Logs:

Cheers,

mweagle · 2017-08-12T18:09:06Z

Thanks Angel, I’ll take a look & update with results. Appreciate you sharing the source. From: Angel Abad <notifications@github.com> Reply-To: mweagle/Sparta <reply@reply.github.com> Date: Saturday, August 12, 2017 at 12:22 AM To: mweagle/Sparta <Sparta@noreply.github.com> Cc: Matt Weagle <mweagle@gmail.com>, Mention <mention@noreply.github.com> Subject: Re: [mweagle/Sparta] Python error with cgo lambda (#64) Hi @mweagle<https://github.com/mweagle>, same error here :-( I attach logs Its possible the error be in my code? Here is my cgo branch: * https://github.com/angelabad/unsplash-telegram-bot/tree/feature/cgo Logs: * provision.txt<https://github.com/mweagle/Sparta/files/1219564/provision.txt> * error.txt<https://github.com/mweagle/Sparta/files/1219565/error.txt> Cheers, — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#64 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAYr6neAv1FG6o9YN1TepRGz0khcB7Toks5sXVKvgaJpZM4Odv5Y>.

mweagle · 2017-08-15T10:01:21Z

Hi @angelabad - it seems like this code is expecting additional configuration. What command do you use to build? TIA.

angelabad · 2017-08-17T09:19:34Z

Sorry, yo need a config.json in directory project, I attach a fake config.json.txt (rename it to config.json), you can use it to see the error in aws.

I build the project wth

$ go build

and provision it with

$ ./unsplash-telegram-bot provision -s bucket

Thanks for your attention

config.json.txt

mweagle · 2017-08-27T14:31:42Z

Hi @angelabad - I think I've found the issue. When building with cgo, Sparta uses the application's main() function and rewrites that to be the library init() function. You can see this if you build with --noop as in:

go build && ./unsplash-telegram-bot provision -s <BUCKET_NAME> --noop

and look at the ./sparta/main.go source.

In the current configuration, func main() is attempting to parse options that are supplied on the command line. In the Python AWS Lambda execution environment, that panics/os.Exits because those options aren't available.

There are a few ways to manage the build/deploy/execution contexts - can you help me understand what you're trying to accomplish with the command line options?

angelabad · 2017-08-28T08:01:10Z

Hi @mweagle , I understand... I attach the generated .sparta/main.go.

I help it helps us.

Thanks in advance!

main.go.txt

angelabad · 2017-10-01T16:24:31Z

Hi @mweagle I tested this issue with 0.20.0 and I got the same error :-(

Cheers

mweagle · 2017-10-05T14:12:05Z

Hi @angelabad - there isn't really a way for Sparta to better handle this situation. In the cgo world, your main function becomes init() and panics during library load due to the parsing error. What type of functionality are you trying to support? There might be another alternative.

angelabad · 2017-11-14T16:24:44Z

Hi @mweagle, sorry for my delay. But I dont understand why my main function fails to convert in init() function. So I cant fox the error :-(

Cheers

mweagle · 2017-11-15T03:49:40Z

Your main function is properly transformed into an init function - the problem is when the .so is loaded during lambda execution. The library is loaded by a Python handler. When the library is loaded, your transformed init function executes:

func init() {

	err := parseConfig()
	if err != nil {
		log.Panic("Error parsing config ", err.Error())
	}

	parseErrors := sparta.ParseOptions(nil)
	if parseErrors != nil {
		os.Exit(2)
	}

	var StackName string
       ....

Since there isn't a configuration file available to load when running in AWS, the parseConfig function returns an error due to missing configuration information, and the library forcibly exits. This causes the Python handler to exit as well, since it can't load the library.

There are a few ways to make configuration information available to your function, including ArchiveHooks, KMS-wrapped values with SpartaVault or just via environment variables. Will one of those work for your use case?

angelabad · 2017-11-15T17:27:09Z

Hi @mweagle, I understand it, sorry for the inconvenience.

I will try one of your alternatives.

Cheers,

mweagle added a commit that referenced this issue Aug 12, 2017

Improve error handling for cgo launch failures

cab8c31

Ref: #64

angelabad closed this as completed Nov 15, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python error with cgo lambda #64

Python error with cgo lambda #64

angelabad commented Jul 20, 2017

angelabad commented Jul 20, 2017

mweagle commented Jul 23, 2017

mweagle commented Jul 23, 2017 •

edited

Loading

angelabad commented Jul 24, 2017

mweagle commented Jul 29, 2017

mweagle commented Jul 29, 2017

angelabad commented Jul 30, 2017

mweagle commented Jul 30, 2017

mweagle commented Aug 3, 2017

angelabad commented Aug 4, 2017

mweagle commented Aug 12, 2017

angelabad commented Aug 12, 2017

mweagle commented Aug 12, 2017 via email

mweagle commented Aug 15, 2017

angelabad commented Aug 17, 2017

mweagle commented Aug 27, 2017 •

edited

Loading

angelabad commented Aug 28, 2017

angelabad commented Oct 1, 2017

mweagle commented Oct 5, 2017

angelabad commented Nov 14, 2017

mweagle commented Nov 15, 2017

angelabad commented Nov 15, 2017

Python error with cgo lambda #64

Python error with cgo lambda #64

Comments

angelabad commented Jul 20, 2017

angelabad commented Jul 20, 2017

mweagle commented Jul 23, 2017

mweagle commented Jul 23, 2017 • edited Loading

angelabad commented Jul 24, 2017

mweagle commented Jul 29, 2017

mweagle commented Jul 29, 2017

angelabad commented Jul 30, 2017

mweagle commented Jul 30, 2017

mweagle commented Aug 3, 2017

angelabad commented Aug 4, 2017

mweagle commented Aug 12, 2017

angelabad commented Aug 12, 2017

mweagle commented Aug 12, 2017 via email

mweagle commented Aug 15, 2017

angelabad commented Aug 17, 2017

mweagle commented Aug 27, 2017 • edited Loading

angelabad commented Aug 28, 2017

angelabad commented Oct 1, 2017

mweagle commented Oct 5, 2017

angelabad commented Nov 14, 2017

mweagle commented Nov 15, 2017

angelabad commented Nov 15, 2017

mweagle commented Jul 23, 2017 •

edited

Loading

mweagle commented Aug 27, 2017 •

edited

Loading