Question / Comment: Support for layers/optional content

[Evidlo]

I've browsed through the examples and the source, but I haven't seen any mention of optional content groups. mupdf has this functionality in pdf-layer.c

Is there some lower level way to access these functions or will support need to be added explicitly?

[JorjMcKie]

Is there some lower level way to access these functions or will support need to be added explicitly?

There is "free" access to low level information via manipulation of object definitions and of stream content. Both in the end mean basically dealing with strings and bytes objects.
I cannot tell at the moment though, whether this may achieve your intentions.

[JorjMcKie]

I am willing to deal with this topic, although I cannot promise a timeframe for implementation.
For speeding up and simplifying the process:
Can you help and spare me a some research by providing the following input:

• what is your goal of getting access to OCGs?
• what API features would you need to achieve your goals?
  • information on available and or currently active OCGs?
  • switching to a certain OC?
  • ...?
    Ideally provide pseudo-code descriptions / design of the methods you would like to see ...

[Evidlo]

What is your goal of getting access to OCGs?

I'm working on an application which allows marking up PDFs using the Ink annotation. In order to differentiate these annotations from those created by external tools, I want to put my annotations in an OCG (e.g. 'mycustomapp annots').

In order to achieve this, the minimum needed functionality is

• ability to create OCGs
• ability to set OCG on objects

Although I guess the average user would also want to be able to modify and delete existing OCGs and set their visibility properties.

Examples of other OCG APIs:

• http://www.quickpdflibrary.com/faq/how-to-create-optional-content-groups-ocgs-in-pdf-files.php
• https://www.colorpilot.com/layers-optional-content-groups/
• https://www.gdpicture.com/blog/pdf-layers-ocg/

[JorjMcKie]

I have developed a first cut for this functionality. Don't want to publish it yet, but I am certainly interested in you testing it.
Please tell me your configuration (Py version, bitness, OS), so I can provide a pre-release wheel.

Here is a rough synopsis of the features and implemented API

• create OCG: doc.addOCG(name, config=-1, on=1, intent=None). Creates an optional content group - if required also makes the minimum PDF catalog changes for this kind of support.

  • name: arbitrary name string
  • config: -1 default configuration, else a configuration must have been previously set up.
  • on: 1/True = visible, 0/False = hidden
  • intent: string or list of strings, default "View"

• list OCGs: doc.getOCGs(), a tuple of dictionaries of OCGs

>>> doc.addOCG("hide", on=False)  # default status hidden
>>> for item in doc.getOCGs(): print(item)

{'xref': 132, 'name': 'hide', 'intent': ['View']}
>>> # the on-status is not part of the OCG, so it doesn't show here

• list configuration of modifyable ON / OFF congurations: doc.layerUIConfigs().
• set / unset visibility: doc.setLayerConfigUI(number, action=0).
  • number: number shown in previous list
  • action: 0=set visible, 1=toggle visibility, 2=set invisible

>>> for item in doc.layerUIConfigs(): print(item)

{'number': 0, 'text': 'hide', 'depth': 0, 'type': 'checkbox', 'selected': False, 'locked': False}
>>> # refers to OCG with name "hide", which has default status invisible ==> selected=False.
>>> # let's make it visible:
>>> doc.setLayerConfigUI(0, action=0)
>>> for item in doc.layerUIConfigs(): print(item)

{'number': 0, 'text': 'hide', 'depth': 0, 'type': 'checkbox', 'selected': True, 'locked': False}
>>> # selected=True now!

• images, form XObjects and annotations now support optional content spec:
  • page.insertImage( ..., oc=xref, ...) and page.showPDFpage(..., oc=xref, ...). xref is the xref of an OCG created before (wil be type-checked).
  • annot.setOC(xref): similar to previous, but setOC(0) will remove any optional content reference.

There is more functionality already, but the above should be enough for your immediate needs.

[JorjMcKie]

Please find your wheel either here (Linux) or here (OSX).
Notify me if you have Windows, will need to drop that wheel here.

[Evidlo]

Thanks for the quick turnaround.

images, form XObjects and annotations now support optional content spec:

Can't any element technically be marked as optional content? I'm not familiar with MuPDF so I don't know the implications of trying to implement this.

I should have mentioned that I need to find which annotations belong in a particular content group, maybe with annot.getOC()/annot.oc.

Also, are you implementing OCG support yourself or mostly wrapping functions in pdf-layers.c? I may be writing a C application which uses MuPDF in the long term and I'm interested in seeing what you're doing.

[JorjMcKie]

Can't any element technically be marked as optional content?

No, only those 3 object types. This is a PDF spec - not a MuPDF restriction. In the PDF specification, there is the possibility to also make text and drawings OC-dependent, but that's an even greater effort, which I haven't invested yet.

Support of images, form xobjects and annotations was not that big a deal.

Also, are you implementing OCG support yourself or mostly wrapping functions in pdf-layers.c?

In MuPDF, only informational code (not bug-free!) exists, and I am wrapping it.
Creating OC stuff is my doing - and that was not really trivial - mostly though because I first had to understand the whole concept.

Okay, annot.getOC() will be it. It will return the OCG xref. Information on that OCG can then be extracted from doc.getOCGs(), which I will change to be a Python dict with xref as key (not a list, which it is now).

[JorjMcKie]

There are now new available wheels reflecting my previous post.

[JorjMcKie]

The whole thing works like this:

>>> import fitz
>>> from pprint import pprint
>>> doc=fitz.open("ocg-test.pdf")
>>> page=doc[0]
>>> annot = page.addFreetextAnnot((100,300,300,400), "rect now visible")
>>> pprint(doc.getOCGs())
{13: {'hidden': False,
      'intent': ['View', 'Design'],
      'name': 'Circle',
      'usage': 'Artwork'},
 14: {'hidden': False,
      'intent': ['View', 'Design'],
      'name': 'Square',
      'usage': 'Artwork'},
 15: {'hidden': True, 'intent': ['View'], 'name': 'Square', 'usage': 'Artwork'}}
>>> annot.setOC(15)  # show annot if "Square" is set to on
>>> doc.save(...)
>>> # inquiry of an annot's OC:
>>> xref = annot.getOC()
>>> doc.getOCGs()[xref]
{'name': 'Square', 'intent': ['View'], 'hidden': True, 'usage': 'Artwork'}
>>> 

Next, I will streamline all those status terms "selected", "hidden", "on" into just one: "on".

[canedha]

Dear @JorjMcKie ,

with great interest I have read and followed this topic as I am searching for the possibility of working with layered (OCG) pdfs as well.
I'm working on Windows 64bit with Python 3.8 and would be happy to help with testing and improving the functionalities. If you could also release the wheel for windows, that would be awesome!

my usecase or needed functionality would simply be:

• create OCG layers and fill them

you mentioned that ony:
_

"images, form XObjects and annotations"

_
can be marked as OCG. If I e.g. want to crop a page of an existing pdf and insert into a OCG layer of a new pdf, would that be possible? I have no clue, what type of object this would be. right now I'm using the ".showPDFpage" command from pymupdf and simply define

(rect, docsrc, pno = 0, keep_proportion = True, overlay = True, reuse_xref = 0, clip = None)

rect = area in new pdf, where croppe input is placed
docsrc, pno = define source pdf & page
clip = define area in source pdf to be cropped into rect area

If I read your text and instructions correctly I could simply use your wheel with the new page.showPDFpage(..., oc=xref, ...) command, where oc= xxx defines in which (previously created) OCG layer the content is placed?

Hope to hear back from you,
Toby

[JorjMcKie]

Hi Toby,

overall: yes, that should work like this. Just a few comments to make sure we are in sync:

• OCG must be an optional content group, which you either must create yourself via xref = doc.addOCG("name", on=True/False, ...) or reuse an existing one. In the latter case do pprint(doc.getOCGs()) and pick its xref from that dictionary.
• Argument reuse_xref in showPDFpage() is deprecated and can / should be omitted.
• It sounds like you want to show the copied-over source page under some condition only in the new file. Fine. If you turn the respective OCG to OFF, then the rectangle on the page on which the shown source page lives will become empty. The page does not disappear as a page from the new PDF.

[JorjMcKie]

One more thing:
You are aware, that multiple items in a PDF (annots, images, form xobjects) can be dependent on the same OCG?
So you can switch ON / OFF multiple objects by changing the state of just one OCG.

What I haven't looked at is another potentially useful feature, so-called radio button OCG groups. Each such group (array) switches off all other of its OCGs , if one OCG is switched on - a behaviour similar to grouped radio buttons.

Might be interesting in your case: instead of simply showing an empty page if the OCG is switched to OFF, an alternative content might switched on automatically ...

[canedha]

Hey @JorjMcKie,

thx for your clarification! yes, we are absolutely in sync.
In my case I have a pdf page with content that is always shown and have additional content that I want to be optional (which would be put in the ocg layers).
I understand those layers like overhead projectors we had back in the day. you have a source paper and some transparent foils with additional content. even if you remove all transparent foils, the core page still is present :)
the option to have something switched on by switching off sth else is not sth that I would need, but for sure I'm happy to test it for you!

can you create the 64bit Windows version for me? How long would this take? Don't get me wrong on this, I don't want to hurry/pressure you as I'm really thankful that you are doing it in the first place.
I simply ask as I have small children (therefore rarely free time) and tonight I would have a couple of minutes 😅

Thanks in advance,
Toby