Allow use of [edit: 32-bit] GeoTIFF files

[rosiel]

Overview of feature request

What kind of user is the feature intended for?
Collection managers dealing with geographic files

What inspired the request?
I uploaded a sample tiff file (attached, renamed .gif because github) but it failed to create image derivatives. A parallel upload that was a "normal" tiff had the derivatives created just fine.

Any brand new behavior do you want to add to Islandora?

Is there a compiler flag to enable geotiff formats? (or was the file bad?). I'd like for image derivatives to be generated and for the files to be displayable in viewers.

Any related open or closed issues to this feature request?
no

[rosiel]

Hmm, uploaded file isn't working. Here's a link to the file on dropbox. And as Seth said during today's call, if anyone has another geotiff to share we could see if it's just a problem with this file.
https://www.dropbox.com/s/hvjapxk4jgasdx8/output_srtm.tif?dl=0

[seth-shaw-unlv]

Odd. I spun up my current instance of Isle this morning and it produced derivatives without complaint.

That said, this particular generated derivatives that are bitonal, as did the Windows viewer and Dropbox image display, instead of the greyscale I got opening it in IrfanView.

Screen capture of image in IrfanView:

Derivative:

[rosiel]

Interesting. I used the playbook, and I think it's my ImageMagick. Here's the output of identify -version and of an attempted convert on that file:

vagrant@islandora8:~$ identify -version
Version: ImageMagick 6.9.10-23 Q16 x86_64 20190101 https://imagemagick.org
Copyright: © 1999-2019 ImageMagick Studio LLC
License: https://imagemagick.org/script/license.php
Features: Cipher DPC Modules OpenMP 
Delegates (built-in): bzlib djvu fftw fontconfig freetype jbig jng jp2 jpeg lcms lqr ltdl lzma openexr pangocairo png tiff webp wmf x xml zlib

vagrant@islandora8:~$ convert output_srtm.tif output_srtm.png
convert-im6.q16: Unknown field with tag 33550 (0x830e) encountered. `TIFFReadDirectory' @ warning/tiff.c/TIFFWarnings/949.
convert-im6.q16: Unknown field with tag 33922 (0x8482) encountered. `TIFFReadDirectory' @ warning/tiff.c/TIFFWarnings/949.
convert-im6.q16: Unknown field with tag 34735 (0x87af) encountered. `TIFFReadDirectory' @ warning/tiff.c/TIFFWarnings/949.
convert-im6.q16: Unknown field with tag 34736 (0x87b0) encountered. `TIFFReadDirectory' @ warning/tiff.c/TIFFWarnings/949.
convert-im6.q16: Unknown field with tag 34737 (0x87b1) encountered. `TIFFReadDirectory' @ warning/tiff.c/TIFFWarnings/949.
convert-im6.q16: Unknown field with tag 42113 (0xa481) encountered. `TIFFReadDirectory' @ warning/tiff.c/TIFFWarnings/949.
convert-im6.q16: Sorry, can not handle images with 32-bit samples. `output_srtm.tif' @ error/tiff.c/TIFFErrors/604.

[rosiel]

I'll just leave it here that there's a javascript library for viewing Geotiffs in a browser that could potentially become a cool new viewer:

https://geotiffjs.github.io/

[alxp]

I didn't realize this but it looks like the playbook is using a custom ImageMagick package on CrayFish hosted by Lyrasis. Looks like it's so it has JP2 support.

roles/internal/Islandora-Devops.crayfish/tasks/install.yml

- name: Add ImageMagick repo (apt)
  apt_repository:
    repo: 'ppa:lyrasis/imagemagick-jp2'
  when: ansible_os_family == "Debian"

[whikloj]

I think @jonathangreen built that package for us waaaaaay back. But I could be getting confused.

[alxp]

There's documentation about it here. I don't know if it's still needed in ubuntu 20.04

https://islandora.github.io/documentation/technical-documentation/ppa-documentation/

[alxp]

Just checked and libopenjp2-tools does not confer support into ImageMagick so the custom package is still needed unless we update Crayfish to use a different command line tool for JP2. So perhaps one could re-compile the Lyrasis package with GeoTIFF support.

[alxp]

So open question:

Does ISLE's ImageMagick support JP2, if so how?

If not, what does Crayfish do to convert JP2 to other formats, and can / does playbook do the same thing in which case the custom package can be dropped?

[rosiel]

We also have this documentation for "Updating a deb and uploading it to the Lyrasis PPA" in the technical documentation for Islandora.

But as far as I can tell, ISLE compiles their own version of Imagemagick but not in a way that, to my non-expert eyes, adds JP2 support.

Before this gets too much into whether ISLE supports JP2 which ought to be its own ticket, i'll get back to my errors. Only the last message, about 32-bit samples, was actually an error (the rest were warnings). I have the same issue as this poster, and the 'magick' admin user said they'd fix it, back in May 2019. It makes sense that ISLE works on that file since it seems they're getting the newest version of imagemagick 7.1.0, and the lyrasis version I'm using, 6.9.10-23, is timestamped (in identify -version) at 2019-01-01.

Turns out there's no lack of geotiff support, but our imagemagick is out of date and failing on 32-bit tiling or something.

So my request becomes: Can someone please update the imagemagick in the Lyrasis repository?

[rosiel]

BTW, ISLE processes jp2's fine.

• Inside the houdini container, identify -version showed jp2 listed among the built-in "delegates".
• And after 7 minutes that had me convinced something was broken, my JP2's derivatives showed up.
  No worries!

[ruebot]

I've been keeping that package up-to-date for each Ubuntu major version. After each update, I've dropped a note in the #islandora slack whenever I update it (18.04 & 20.04).

I checked out what ISLE is doing with their compile and updated the ppa package. @rosiel when you have time, can you test? Hopefully it addresses your the issue you were having.

[rosiel]

We seem to have removed the use of the lyrasis ppa, back in December. Can anyone confirm? (@whikloj @seth-shaw-asu )

[seth-shaw-asu]

Looks like you are right, @rosiel. I admit that I didn't test JP2 when reviewing the PR.

[whikloj]

Yes, because... 🤔 the default installed ImageMagick already had JP2 support. I'd have to stand it up to confirm again. Are you hitting an issue?

[ruebot]

I yanked the PPA and Grok out of our Playbook, and I replaced it with compiling ImageMagick with JP2 support.

[rosiel]

Closing this issue, we can make derivatives of that file in islandora now.

Note: If anyone's interested in the derivatives being more useful - not black&white but showing the gradients Seth demonstrated that IrfanView can see - then pls open another issue.