Automatic testing infrastructure

[neilcsmith-net]

Integrating pull requests is currently time consuming and buggy. One thing needed to make this easier is automatically running tests against baseline (1.8) and latest GStreamer versions on all pull requests. (Travis? Jenkins?)

[woglinde]

Hi,

I suggest to use travis or circle-ci which have nice integrations with github. A Jenkins system needs maintenance.

[neilcsmith-net]

Hi @woglinde - agreed if GitHub remains the primary repository for this project. Jenkins might be an option if we end up being able to use freedesktop resources alongside other GStreamer related projects. That's a whole other topic of conversation! 😄

EDIT : actually not sure it would be Jenkins now they've migrated to GitLab.

[a-morales]

@neilcsmith-net any updates on this? I can take a stab at setting up the testing infrastructure if a service has been chosen, though I don't have much experience with Jenkins.

[neilcsmith-net]

@a-morales as per the mailing list discussion [1] I'm not in a position to put much time in right now. If you want to have a look at a Travis or Circle config that would be great. There may be something useful to look at there in the Rust bindings re. Travis. Main thing is installing / uninstalling both required versions of GStreamer, hence happy with whichever service is easier for that.

[1] https://groups.google.com/forum/?nomobile=true#!topic/gstreamer-java/gDLHP_zSjdc

[a-morales]

@neilcsmith-net

I did take a look at setting up the testing infrastructure and looked at gstreamer-rs to see how they do it. It looks like they host the build artifacts for GStreamer and they just pull it before running any tests[1]. Only a single version of GStreamer is being tested against and it is a pretty new version.

Since testing needs to be done on the baseline version and the latest version of Gstreamer, there are multiple ways that can be done.

• The build artifacts for GStreamer 1.8 and latest can be hosted and just be pulled/installed before running test
• Add a step to the testing process to build Gstreamer, and cache the artifacts so they can be reused and not have to build Gstreamer every time.
• Or if the CI framework supports docker images, just build an image with openjdk and includes GStreamer and just tag the images differently for different versions of GStreamer.

I personally like the third option, but just want to get your input before proceeding.

[1]https://gitlab.freedesktop.org/gstreamer/gstreamer-rs/blob/master/.travis.yml#L33

[neilcsmith-net]

@a-morales agreed, I think option 3 does look the best. Thanks! I had been planning to look at extracting build artefacts in that way. But having to do that for (at least) two different versions, and finding suitable build artefacts or needing to also build them from source, does make that option rather less attractive!

[neilcsmith-net]

Well, now Travis has Xenial we can do a basic test against GStreamer 1.8 easily so added this in pending something better.