HTTP status code 403 on import

[SpringerJack]

Bug description

Recently I have noticed that when importing acsm files I only get an HTTP status code 403. This does not seem to be book-specific as I have reproduced this with a book that used to work.

I registered with my Adobe account and this activation was reported to be successful. I also confirmed that the same acsm file can be imported with my account on another device with Adobe's own software and also with the utils provided by the libgourou project (which I have registered only afterwards, so this does not seem to be an issue with device limits either).

The default stack trace:

ACSM Input v0.1.0: Try to fulfill ...
Post request caused HTTPError 403
Traceback (most recent call last):
File "calibre_plugins.deacsm.init", line 547, in run_single
success, replyData = fulfill(path_to_ebook, deacsmprefs["notify_fulfillment"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/linus/.config/calibre/plugins/ACSM Input.zip/libadobeFulfill.py", line 442, in fulfill
ret = operatorAuth(fulfillURL)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/linus/.config/calibre/plugins/ACSM Input.zip/libadobeFulfill.py", line 286, in operatorAuth
ret = doOperatorAuth(operatorURL)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/linus/.config/calibre/plugins/ACSM Input.zip/libadobeFulfill.py", line 238, in doOperatorAuth
replyData = sendRequestDocu(auth_req, authURL + "/Auth").decode("utf-8")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'decode'

Operating system

Linux

Which version of Calibre are you running?

7.16

Which version of the ACSM Input plugin are you running?

build from master (6c29c42)

Import type

Clicking the 'Add books' button in the menu bar, Dragging-and-Dropping the ACSM file into the Calibre window

Further information

No response

[Leseratte10]

Interesting. I haven't run into that issue so far. Is this the entire error message? Looking at the code I fail to see how sendRequestDocu can return None without printing an error message.

Can you try restarting Calibre in debug mode (Right-click preferences, then click "Restart in debug mode"), then import the ACSM again, then post the full log?

Does this only happen with ACSMs from this particular source or does it happen with all of them? Can you test ones from Adobe's sample library?

[SpringerJack]

The full log in debug mode:

Fontconfig error: "/etc/fonts/fonts.conf", line 5: not well-formed (invalid token)
Fontconfig error: Cannot load config file from /etc/fonts/fonts.conf
ApplicationPaletteChange event ignored
Using calibre Qt style: True
calibre Debug log
calibre 7.16 embedded-python: False
Linux-6.10.7-x86_64-with-glibc2.39 Linux ('64bit', 'ELF')
('Linux', '6.10.7', '#1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 29 15:36:13 UTC 2024')
Python 3.12.5
Interface language: None
EXE path: /nix/store/als1frcnwrfhy1nj6r8w4j69my8zlmmp-python3-3.12.5-env/bin/python3.12
Successfully initialized third party plugins: ACSM Input (0, 1, 0) && DeDRM (10, 0, 3) && ACSM Input Plugin GUI Extension (0, 1, 0)
calibre 7.16 embedded-python: False
Linux-6.10.7-x86_64-with-glibc2.39 Linux ('64bit', 'ELF')
('Linux', '6.10.7', '#1-NixOS SMP PREEMPT_DYNAMIC Thu Aug 29 15:36:13 UTC 2024')
Python 3.12.5
Interface language: None
EXE path: /nix/store/als1frcnwrfhy1nj6r8w4j69my8zlmmp-python3-3.12.5-env/bin/python3.12
Successfully initialized third party plugins: ACSM Input (0, 1, 0) && DeDRM (10, 0, 3) && ACSM Input Plugin GUI Extension (0, 1, 0)
QPA platform: xcb
devicePixelRatio: 1.0
logicalDpi: 96.0 x 96.0
physicalDpi: 161.48344370860926 x 162.12765957446808
Failed to read metadata from font file: /home/linus/.local/share/fonts/material-design-icons.ttf No module named 'fontTools'
Failed to read metadata from font file: /home/linus/.local/share/fonts/weathericons.ttf No module named 'fontTools'
Failed to read metadata from font file: /home/linus/.local/share/fonts/octicons.ttf No module named 'fontTools'
Failed to read metadata from font file: /home/linus/.local/share/fonts/fontawesome.ttf No module named 'fontTools'
Failed to read metadata from font file: /home/linus/.local/share/fonts/file-icons.ttf No module named 'fontTools'
Failed to read metadata from font file: /home/linus/.local/share/fonts/all-the-icons.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationMono-Bold.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationMono-BoldItalic.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationMono-Italic.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationMono-Regular.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationSans-Bold.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationSans-BoldItalic.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationSans-Italic.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationSans-Regular.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationSerif-Bold.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationSerif-BoldItalic.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationSerif-Italic.ttf No module named 'fontTools'
Failed to read metadata from font file: /nix/store/idgj6mwi0jwv8r69r88ld73vz4kc5klh-calibre-7.16.0/share/calibre/fonts/liberation/LiberationSerif-Regular.ttf No module named 'fontTools'
[0.00] Starting up...
[0.00] Showing splash screen...
[1.04] splash screen shown
[1.04] Initializing db...
[1.07] db initialized
[1.07] Constructing main UI...
ACSM Input: GUI Plugin Genesis!
[1.73] GUI main window shown
[1.75] main UI initialized...
[1.75] Hiding splash screen
[1.75] splash screen hidden
[1.76] Started up in 1.76 seconds with 88 books
Traceback (most recent call last):
File "calibre_plugins.deacsm.init", line 547, in run_single
success, replyData = fulfill(path_to_ebook, deacsmprefs["notify_fulfillment"])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/linus/.config/calibre/plugins/ACSM Input.zip/libadobeFulfill.py", line 442, in fulfill
ret = operatorAuth(fulfillURL)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/linus/.config/calibre/plugins/ACSM Input.zip/libadobeFulfill.py", line 286, in operatorAuth
ret = doOperatorAuth(operatorURL)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/linus/.config/calibre/plugins/ACSM Input.zip/libadobeFulfill.py", line 238, in doOperatorAuth
replyData = sendRequestDocu(auth_req, authURL + "/Auth").decode("utf-8")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'decode'
ACSM Input v0.1.0: Trying to parse file Shea_The-Illuminatus_-Trilogy_9780307569646.acsm
ACSM Input v0.1.0: Try to fulfill ...
Post request caused HTTPError 403

These books all come from Libri GmbH and are sold by local bookshops. This is an interesting connection as fulfillment seems to work with Adobe's sample library (I do still get some error output but that seems unrelated).

sendPOSTHTTPRequest can return None while only printing the line

Post request caused HTTPError 403

in this line, right?

[frerich]

I'm getting the same error using Calibre 7.18 and DeACSM 0.0.16 on a book purchased from https://www.thalia.de/. I imported an .acsm file via Drag & Drop, this yields the debug log:

ApplicationPaletteChange event ignored
Using calibre Qt style: True
calibre Debug log
calibre 7.18  embedded-python: True
macOS-14.6.1-arm64-arm-64bit Darwin ('64bit', '')
('Darwin', '23.6.0', 'Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:04 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6020')
Python 3.11.5
OSX: ('14.6.1', ('', '', ''), 'arm64')
Interface language: None
EXE path: /Applications/calibre.app/Contents/MacOS/calibre-debug
Successfully initialized third party plugins: DeACSM (0, 0, 16) && DeDRM (10, 0, 3)
calibre 7.18  embedded-python: True
macOS-14.6.1-arm64-arm-64bit Darwin ('64bit', '')
('Darwin', '23.6.0', 'Darwin Kernel Version 23.6.0: Mon Jul 29 21:13:04 PDT 2024; root:xnu-10063.141.2~1/RELEASE_ARM64_T6020')
Python 3.11.5
OSX: ('14.6.1', ('', '', ''), 'arm64')
Interface language: None
EXE path: /Applications/calibre.app/Contents/MacOS/calibre-debug
Successfully initialized third party plugins: DeACSM (0, 0, 16) && DeDRM (10, 0, 3)
QPA platform: cocoa
devicePixelRatio: 2.0
logicalDpi: 72.0 x 72.0
physicalDpi: 81.50000122400722 x 80.50000120898872
[0.00] Starting up...
[0.01] Showing splash screen...
[1.17] splash screen shown
[1.17] Initializing db...
[1.18] db initialized
[1.18] Constructing main UI...
[1.42] GUI main window shown
[1.43] main UI initialized...
[1.43] Hiding splash screen
[1.49] splash screen hidden
[1.49] Started up in 1.49 seconds with 10 books
Running file type plugin DeACSM failed with traceback:
Traceback (most recent call last):
  File "calibre/customize/ui.py", line 199, in _run_filetype_plugins
  File "calibre_plugins.deacsm.__init__", line 416, in run
    success, replyData = fulfill(path_to_ebook, deacsmprefs["notify_fulfillment"])
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/frerich/Library/Preferences/calibre/plugins/DeACSM.zip/libadobeFulfill.py", line 442, in fulfill
    replyData = sendRequestDocu(fulfill_req_signed, fulfillURL).decode("utf-8")
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'decode'
DeACSM v0.0.16: Trying to parse file Towsey_Research-That-Scales_9781959029205.acsm
DeACSM v0.0.16: Try to fulfill ...
Post request caused HTTPError 403
Added Towsey Research-That-Scales 9781959029205 to db in: 0.0
Added 1 books in 0.6 seconds

I can reproduce this at will, let me know if you want me to try a patched version or anything.

[Leseratte10]

Looks like Thalia added Cloudflare in front of their servers and Cloudflare's WAF is probably detecting some kind of small difference in how ADE / legit adobe clients access this page and how my plugin does it. I can reproduce it with a free DRM'd book from Thalia.

[Leseratte10]

Issue should be fixed with the latest commit, please test again.

[SpringerJack]

I can confirm this fixes the issue, thank you!

[frerich]

Wow, amazingly fast turnaround! 🚀

Can confirm, the build at https://github.com/Leseratte10/acsm-calibre-plugin/actions/runs/10907951375#artifacts fixes the issue. Thanks! ❤️

[geronimoge]

Hello, I'm experiencing a similar error and am totaly unexoerienced with building software.
sorry for that.

So my question is: How can I use the build in calibre, which seems to solve the problem described here?

Currently I'm on Ubuntu Ubuntu 24.04.1 LTS, Calibre 7.19 with ACSM Input v0.1.0.

[Leseratte10]

No need to build the plugin yourself, you can just go to the link https://github.com/Leseratte10/acsm-calibre-plugin/actions/runs/10907951375#artifacts then scroll to the bottom and click the "calibre-plugins" link under Artifacts.

You'll get a ZIP file with two other ZIP files in it, the "calibre-plugin.zip" file can be imported into Calibre like any other plugin.

[geronimoge]

Thank you very much for the fast reply. That's fantastic!
So I can just download and install it without uninstalling the existing version of ACSM Input v.0.1.0 in Calibre, right?

[Leseratte10]

Yes. There's issues with updating / downgrading between 0.0.16 and the first beta versions of 0.1.0 (which is one of the reasons why 0.1.0 is still not released ...) but updating / downgrading in-between different 0.1.0 versions is fine.

[geronimoge]

Awesome, it worked:

1. Downloading the build here: https://github.com/Leseratte10/acsm-calibre-plugin/actions/runs/10907951375#artifacts
2. Extracting 'calibre-plugins.zip'
3. Opening Calibre, switching to the plugins >> load plugin from file >> go to path for extracted 'calibre-plugins.zip' and choose 'calibre-plugin.zip'
4. Wait for installation to end and click 'OK'
5. Exit Calibre
6. Restart Calibre
7. Choose ACSM file and wait for success

Thank you very much!

[jscholes]

@Leseratte10 After installing the version of the plug-in linked in this thread, I'm unfortunately getting the same error when trying to import a book purchased from Kobo. The sample books from the Adobe Library do work.

Here is the debug log output:

ApplicationPaletteChange event ignored
Using calibre Qt style: True
calibre Debug log
calibre 7.20  embedded-python: True
Windows-10-10.0.19045-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19045')
Python 3.11.5
Windows: ('10', '10.0.19045', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-debug.exe
Successfully initialized third party plugins: ACSM Input (0, 1, 0) && DeDRM (10, 0, 9) && ACSM Input Plugin GUI Extension (0, 1, 0)
calibre 7.20  embedded-python: True
Windows-10-10.0.19045-SP0 Windows ('64bit', 'WindowsPE')
('Windows', '10', '10.0.19045')
Python 3.11.5
Windows: ('10', '10.0.19045', 'SP0', 'Multiprocessor Free')
Interface language: None
EXE path: C:\Program Files\Calibre2\calibre-debug.exe
Successfully initialized third party plugins: ACSM Input (0, 1, 0) && DeDRM (10, 0, 9) && ACSM Input Plugin GUI Extension (0, 1, 0)
QPA platform: windows
devicePixelRatio: 1.25
logicalDpi: 96.0 x 96.0
physicalDpi: 113.41395348837209 x 113.70777202072539
[0.00] Starting up...
[0.00] Showing splash screen...
[1.16] splash screen shown
[1.16] Initializing db...
[1.16] db initialized
[1.16] Constructing main UI...
ACSM Input: GUI Plugin Genesis!
[1.39] GUI main window shown
[1.45] main UI initialized...
[1.45] Hiding splash screen
[1.45] splash screen hidden
[1.45] Started up in 1.45 seconds with 2 books
Traceback (most recent call last):
  File "calibre_plugins.deacsm.__init__", line 547, in run_single
  File "C:\Users\james\AppData\Roaming\calibre\plugins\ACSM Input.zip\libadobeFulfill.py", line 442, in fulfill
  File "C:\Users\james\AppData\Roaming\calibre\plugins\ACSM Input.zip\libadobeFulfill.py", line 286, in operatorAuth
  File "C:\Users\james\AppData\Roaming\calibre\plugins\ACSM Input.zip\libadobeFulfill.py", line 238, in doOperatorAuth
AttributeError: 'NoneType' object has no attribute 'decode'
stdout+stderr from file dialog helper: [b'', b'']
stdout+stderr from file dialog helper: [b'', b'']
piped data from file dialog helper: [b'[redacted]', b'C:\\Users\\james\\Downloads\\URLLink.acsm']
Added URLLink to db in: 0.0
ACSM Input v0.1.0: Trying to parse file URLLink.acsm
ACSM Input v0.1.0: Try to fulfill ...
Fulfill request:
<?xml version="1.0"?><adept:fulfill xmlns:adept="http://ns.adobe.com/adept"><adept:user>urn:uuid:[redacted]</adept:user><adept:device>urn:uuid:[redacted]</adept:device><adept:deviceType>standalone</adept:deviceType><fulfillmentToken xmlns="http://ns.adobe.com/adept" fulfillmentType="buy" auth="user">
  <distributor>urn:uuid:18fe3022-eb24-4fd1-8f6c-e63140c013ca</distributor>
  <operatorURL>https://acs4.kobo.com/fulfillment</operatorURL>
  <transaction>[redacted]</transaction>
  <purchase>2024-10-27T14:48:44-04:00</purchase>
  <expiration>2024-10-27T20:48:44-04:00</expiration>
  <resourceItemInfo>
    <resource>urn:uuid:[redacted]</resource>
    <resourceItem>0</resourceItem>
    <metadata>
      <dc:title xmlns:dc="http://purl.org/dc/elements/1.1/">[redacted]</dc:title>
      <dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">[redacted]</dc:creator>
      <dc:publisher xmlns:dc="http://purl.org/dc/elements/1.1/">[redacted]</dc:publisher>
      <dc:identifier xmlns:dc="http://purl.org/dc/elements/1.1/">urn:uuid:[redacted]</dc:identifier>
      <dc:format xmlns:dc="http://purl.org/dc/elements/1.1/">application/epub+zip</dc:format>
      <dc:language xmlns:dc="http://purl.org/dc/elements/1.1/">en-UK</dc:language>
    </metadata>
    <licenseToken>
      <resource>urn:uuid:[redacted]</resource>
      <permissions>
        <display/>
        <excerpt/>
        <print/>
        <play/>
      </permissions>
    </licenseToken>
  </resourceItemInfo>
  <hmac>[redacted]</hmac>
</fulfillmentToken>
<adept:targetDevice><adept:softwareVersion>9.3.58046</adept:softwareVersion><adept:clientOS>Windows Vista</adept:clientOS><adept:clientLocale>en</adept:clientLocale><adept:clientVersion>2.0.1.78765</adept:clientVersion><adept:deviceType>standalone</adept:deviceType><adept:productName>ADOBE Digitial Editions</adept:productName><adept:fingerprint>[redacted]</adept:fingerprint><adept:activationToken><adept:user>urn:uuid:[redacted]</adept:user><adept:device>urn:uuid:[redacted]</adept:device></adept:activationToken></adept:targetDevice></adept:fulfill>
Post request caused HTTPError 403

[Leseratte10]

Dang it. I wonder if there's some differences between how Thalia does that detection and how Kobo does it, or if they're actively trying to block this plugin ...

I'll take a look.