Page.widgets() returns a Widget-Annot mixed list

[akira-okumura]

Please provide all mandatory information!

Describe the bug (mandatory)

Calling Page.Widgets() returns a Widget-Annot mixed list

To Reproduce (mandatory)

In [1]: import fitz
In [2]: pdf = fitz.open("exp_phys_test0.pdf")
In [3]: page = pdf[0]
In [4]: for w in page.widgets():
   ...:     print(type(w))
   ...: 
<class 'fitz.fitz.Widget'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>
<class 'fitz.fitz.Annot'>

Expected behavior (optional)

The returned list should contains only Widget instances.

Screenshots (optional)

N/A

Your configuration (mandatory)

• Operating system, potentially version and bitness: macOS 10.15.7, 64 bit
• Python version, bitness: 3.8.5 (64 bit)
• PyMuPDF version, installation method (1.18.0 via the pip3 command)

In [5]: fitz.__doc__
Out[5]: '\nPyMuPDF 1.18.0: Python bindings for the MuPDF 1.18.0 library.\nVersion date: 2020-10-06 07:15:59.\nBuilt for Python 3.8 on darwin (64-bit).\n'

Additional context (optional)

Please see the attached input file, in which forms were created with Acrobat Pro DC for Mac (2020.012.20048).
https://www.dropbox.com/s/pplvp26zd5qqy6t/exp_phys_test0.pdf?dl=0

[JorjMcKie]

Thanks for reporting this!
This is an error new in v1.18.0 - Just check that it works in v1.17.7.
As I did not make changes in that corner at all, I suspect we have an upstream error here - but let me verify.

[JorjMcKie]

Found it!
MuPDF changed the enum list of annotation types in v1.18.0: there are a few more annotation types listed now.
But instead of adding the new values to the end of the list, they inserted them somewhere in the middle. This changed the id number for all annotation types after the insertions.
So I continue to check for PDF_ANNOT_WIDGET, but the value behind this name has been changed ... which lets me conclude, that the next in sequence is not a widget but some other annot type.

The update is very trivial. Please let me know how urgent this is for you.

[JorjMcKie]

Just as a comment:
It is not a bug in MuPDF. It is at best a questionable change in the sense of software engineering.
While one could argue, that my tests should have revealed this, there is the counterargument: Widget are technically also an annotations subtype in the PDF sense, which means that a sub-type check for widgets would entail considerably more than just checking that number PDF_ANNOT_WIDGET.

[akira-okumura]

Thank you for the reply. If you could fix it quickly, it would be very great.
The attached file is an online exam paper for my university class in this semester, and I need to collect the answer fields from 70 students. Actually, I first tried PyPDF2, but it could not read some PDF files saved with iOS Acrobat. That’s why I switched to PyMuPDF.
If you could tell me where I should fix locally (and if it is not very hard), I would do it myself on my Mac.

[JorjMcKie]

Ok, I understand. I will generate a pre-release wheel tomorrow and send you a download link. You can of course also use the version before via “python -m pip install pymupdf==1.17.7”. Von meinem iPhone gesendet Am 12/10/20 um 18:39 schrieb Akira Okumura <notifications@github.com>:  Thank you for the reply. If you could fix it quickly, it would be very great. The attached file is an online exam paper for my university class in this semester, and I need to collect the answer fields from 70 students. Actually, I first tried PyPDF2, but it could not read some PDF files saved with iOS Acrobat. That’s why I switched to PyMuPDF. If you could tell me where I should fix locally (and if it is not very hard), I would do it myself on my Mac. — You are receiving this because you were assigned. Reply to this email directly, view it on GitHub<#686 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AB7IDIS7RJRXHG5KJZV5XTDSKOATFANCNFSM4SM53WUA>.

[akira-okumura]

Thank you again for the prompt reply. 1.17.7 looks to be working OK as expected. I will use it for the moment.

[JorjMcKie]

you can now download the pre-version from here: https://github.com/pymupdf/PyMuPDF-wheels/tree/osx.
Install it like so: python -m pip install -U PyMuPDF-1.18.1-cp38-cp38-macosx_10_9_x86_64.whl
Should work, I have tested it in Windows and Linux.

[akira-okumura]

It works! My environment is macOS 10.15.7 and Python 3.8.5.

[JorjMcKie]

New version 1.18.1 uploaded to PyPI.