Wrong default in max content length if its not defined in site configuration

Describe the bug

If ckan site configuration does not define ckanext.datapusher_plus.max_content_length, it should use the default declared in

datapusher-plus/ckanext/datapusher_plus/config_declaration.yaml

Lines 12 to 16 in 474ce97

    
                 - key: ckanext.datapusher_plus.max_content_length 
        
                   editable: true 
        
                   default: 1256000000000 
        
                   description: | 
        
                     Maximum content length in bytes

Instead datapusher+ uses the value defined in

datapusher-plus/ckanext/datapusher_plus/config.py

Lines 86 to 88 in 474ce97

    
           MAX_CONTENT_LENGTH = tk.asint( 
        
               tk.config.get("ckanext.datapusher_plus.max_content_length", "5000000") 
        
           )

which results in an error shown with wrong value.

With default values, the code passes this check as conf.PRREVIEW_ROWS is 0.

datapusher-plus/ckanext/datapusher_plus/jobs/stages/download.py

Lines 277 to 286 in 474ce97

    
           # Check size before download 
        
           if cl: 
        
               try: 
        
                   if int(cl) > max_content_length and conf.PREVIEW_ROWS > 0: 
        
                       raise utils.JobError( 
        
                           f"Resource too large to download: {DataSize(int(cl)):.2MB} " 
        
                           f"> max ({DataSize(int(max_content_length)):.2MB})." 
        
                       ) 
        
               except ValueError: 
        
                   pass

But if fails this check as not 0 equals true.

datapusher-plus/ckanext/datapusher_plus/jobs/stages/download.py

Lines 423 to 426 in 474ce97

    
           if length > max_content_length and not conf.PREVIEW_ROWS: 
        
               raise utils.JobError( 
        
                   f"Resource too large to process: {length} > max ({max_content_length})." 
        
               )

To Reproduce
Steps to reproduce the behavior:

Do not define site configuration for datapusher+
Upload file larger than 5Mb and smaller than 1GB

Expected behavior
File should be handled by datapusher+ without errors.

Screenshots

Most likely this is some CKAN initialization order error, as conf.MAX_CONTENT_LENGTH contains the value 5000000 but if you run tk.config.get("ckanext.datapusher_plus.max_content_length") in the download.py, it contains the correct value.

Navigation Menu

Repository navigation

Wrong default in max content length if its not defined in site configuration #337

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	- key: ckanext.datapusher_plus.max_content_length
	editable: true
	default: 1256000000000
	description: \|
	Maximum content length in bytes

	MAX_CONTENT_LENGTH = tk.asint(
	tk.config.get("ckanext.datapusher_plus.max_content_length", "5000000")
	)

	# Check size before download
	if cl:
	try:
	if int(cl) > max_content_length and conf.PREVIEW_ROWS > 0:
	raise utils.JobError(
	f"Resource too large to download: {DataSize(int(cl)):.2MB} "
	f"> max ({DataSize(int(max_content_length)):.2MB})."
	)
	except ValueError:
	pass

	if length > max_content_length and not conf.PREVIEW_ROWS:
	raise utils.JobError(
	f"Resource too large to process: {length} > max ({max_content_length})."
	)

Wrong default in max content length if its not defined in site configuration #337

Description

Activity

Zharktas commented on Aug 10, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions