Image fields' URLs aren't being correctly generated (prepended with 6 file://<home directory path>)

[arturmatos]

I have a Django model with a ImageField, and I'm trying to use PDFTemplateView to generate a PDF that includes that image. However, the image URL isn't being properly generated in the PDF. If I put something like this in my template:

{{ user.get_profile.company.logo.url }}

This gets properly converted if I request it in HTML (with ?as=html):

/media/logos/abcfinancialservices.png

But if I'm generating the PDF I get this instead:

file:///Users/amatosfile:///Users/amatosfile:///Users/amatosfile:///Users/amatosfile:///Users/amatos/media/logos/abcfinancialservices.png

So basically I'm getting my home directory prepended 6 times in the URL.

I haven't had more time to look into it, but it looks like a bug. Or I'm doing anything wrong?

I'm not that familiar with Django, but assuming it's a bug I'm happy to submit a patch if I can get some guidance -I just need some brief explanation of what the problem might be.

[toudi]

hello.

i narrowed the issue down to the function called def make_absolute_paths(self, content): (utils.py : 127).

there are two possible solutions for this problem, both ot which require you to use full url (i.e. http://example.com/ ... )

the first is an easy one. just use full url in MEDIA_URL and STATIC_URL (both are required). so for example

STATIC_URL = 'http://example.com/static/'
MEDIA_URL = 'http://example.com/media/'

and then, use this in your template

<img src="{{ MEDIA_URL }}your-image" />

now, the downside of this is that it won't work if you are using django.contrib.staticfiles. as stated in django's documentation:

Additionally, when using staticfiles_urlpatterns your STATIC_URL setting can't be empty or a full URL, such as http://static.example.com/.

so the second one is to use the following hack:

154c154,155
<             content = content.replace(occur,
---
>             if not has_scheme.match(occur):
>                 content = content.replace(occur,

this works with contrib.staticfiles, but is a modification of this particular django app.

until a proper patch is issued, you can use monkeypatching or modify the utils.py locally on your virtualenv.

best regards

[george-silva]

Hello guys,

I've tried the monkey patching route, but it did not worked out.

For some reason, on windows, the file:/// syntax creates a PDF with the image container, but not image itself.

I've been experimenting with pure wkhtmltopdf to test the "correct" paths. It turns out that it does not need to append file:/// to the location of an image. I'm still not sure if it works for everything or not, but for now it's enough for me.

Here's what I've did, in utils.py.

def make_absolute_paths(content):
    """Convert all MEDIA files into a file://URL paths in order to
    correctly get it displayed in PDFs."""

    overrides = [
        {
            'root': settings.MEDIA_ROOT,
            'url': settings.MEDIA_URL,
        },
        {
            'root': settings.STATIC_ROOT,
            'url': settings.STATIC_URL,
        }
    ]
    has_scheme = re.compile(r'^[^:/]+://')

    for x in overrides:
        if has_scheme.match(x['url']):
            continue

        if not x['root'].endswith('/'):
            x['root'] += '/'

        occur_pattern = '''["|']({0}.*?)["|']'''
        occurences = re.findall(occur_pattern.format(x['url']), content)
        occurences = list(set(occurences))  # Remove dups
        for occur in occurences:
            if not has_scheme.match(occur):
                content = content.replace(occur,
                                          pathname2fileurl(x['root']) +
                                          occur[len(x['url']):])

                if sys.platform == "win32":
                    content = content.replace(r"file:///","")

    return content

[tachang]

Can someone explain the regex pattern:

    occur_pattern = '''["|']({0}.*?)["|']'''

I am a bit confused. It seems to be picking up more than it should.

Update: I figured the pattern out. The triple single quote beginning and ending is the start of a literal string in python.

The stuff in square brackets means double quote or single quote. So it is looking for anything inside a double or single quote. The {0} is a text replacement. In this case it is looking for settings.MEDIA_URL or settings.STATIC_URL.

[tachang]

My MEDIA_ROOT and MEDIA_URL were blank so it caused extra stuff to be picked up (because of the dynamic regex). I fixed it in my fork.

[metaljoe]

The project I've started working on doesn't use/need MEDIA_URL or MEDIA_ROOT set, so I've had a similar problem caused by that regex. Because they're still on the defaults (empty strings), the content had a lot of "file:///"s inserted everywhere. As a workaround, I've set MEDIA_URL for the moment - tachang's fix looks like it does the job.

[fonfon]

I ran into the MEDIA_URL problems as well.

Additionally, I'm generating images on-the-fly on the server side, so make_absolute_paths is not useful for me because it always puts in the file:// prefix.

My solution is to monkey-patch make_absolute_paths with a custom function. There I replace the path of my server-side generated images by the full URL. This works fine, but it's still monkey patching..

It would be cool to be able to customize the overrides in the settings -- to an extent that allows using both file: prefixes as well as http:// or whatever one would like. The current overrides could still be the default ones.

[johnraz]

@tachang do you mind submitting a PR for this ?

[johnraz]

@tachang another chance?

[amirkdv]

I just got bitten by the same problem. Debugging made clear that the real issue is the way the regular expression in make_absolute_paths is capturing matching patterns.

Example broken test:

# test/__init__.py

# test_22/settings.py
SECRET_KEY = 'x' # make django not crash

STATIC_ROOT = '/path/to/some/dir'
STATIC_URL = '/static/'

# test_22/test.py
from wkhtmltopdf.utils import make_absolute_paths

content = """
    <img src="/static/foo.png"/>
    <img src="/static/bar/static/foo.png"/>
"""
print(make_absolute_paths(content))

Result (depending on the order or matches returned by re either of the following):

$ DJANGO_SETTINGS_MODULE=test_22.settings python manage.py shell < test_22/test.py

    <img src="file:///path/to/some/dir/foo.png"/>
    <img src="/static/barfile:///path/to/some/dir/foo.png"/>

$ DJANGO_SETTINGS_MODULE=test_22.settings python manage.py shell < test_22/test.py

    <img src="file:///path/to/some/dir/foo.png"/>
    <img src="file:///path/to/some/dir/barfile:///path/to/some/dir/foo.png"/>

Note the consistently broken second URL.

What happens is this:

• The regex matches "/static/foo.png" but only captures /static/foo.png (does not capture quotes).
• Then when string replace is called all instances of the captured string are replaced which includes parts of the second URL.
• This can be fixed by capturing the quotes in the regular expression as well.

I have sent PR #167 that solves this by capturing the quotes and then putting them back in string replace.

Results:

$ DJANGO_SETTINGS_MODULE=test_22.settings python manage.py shell < test_22/test.py

    <img src="file:///path/to/some/dir/foo.png"/>
    <img src="file:///path/to/some/dir/bar/static/foo.png"/>