Skip to content

Scraper exits with 137 exit code #223

Open

Description

See e.g. https://farm.openzim.org/pipeline/1b6d8178-864c-4736-9b64-58d7df44e295, https://farm.openzim.org/pipeline/b6bdec4d-b64e-4cd0-ac6f-790e18f96474, https://farm.openzim.org/pipeline/8a3b159d-17e8-42f0-9154-a7bb8d6d53dc or https://farm.openzim.org/pipeline/fe7ac08c-43e0-497c-bb05-0bf53df2514f

[2024-08-05 00:34:14,552: INFO] starting zimfarm task-worker for 8a3b159d-17e8-42f0-9154-a7bb8d6d53dc.
[2024-08-05 00:34:14,552: INFO] configuration:
	username=michaelblob
	webapi_uris=['https://api.farm.openzim.org/v1']
	workdir=/data
	task_id=8a3b159d-17e8-42f0-9154-a7bb8d6d53dc
[2024-08-05 00:34:14,552: INFO] testing workdir at /data…
[2024-08-05 00:34:14,553: INFO] 	workdir is available and writable
[2024-08-05 00:34:14,553: INFO] testing private key at /etc/ssh/keys/zimfarm…
[2024-08-05 00:34:14,626: INFO] 	private key is available and readable (db0be1c33347c1b3b37a9565e619db62)
[2024-08-05 00:34:14,626: INFO] testing authentication with https://api.farm.openzim.org/v1…
[2024-08-05 00:34:15,265: INFO] 	authentication successful
[2024-08-05 00:34:15,266: INFO] testing docker API on /var/run/docker.sock…
[2024-08-05 00:34:15,303: INFO] 	docker API access successful
[2024-08-05 00:34:15,325: INFO] Host hardware resources:
	CPU : 24 (total) ;  18 (avail)
	RAM : 160 GiB (total) ;  144 GiB (avail)
	Disk: 300 GiB (configured) ; 574.04 GiB (avail) ; 68.29 GiB (reserved) ; 231.71 GiB (remain)
[2024-08-05 00:34:15,325: INFO] registering exit signals
[2024-08-05 00:34:15,325: INFO] Fetching task details for 8a3b159d-17e8-42f0-9154-a7bb8d6d53dc
[2024-08-05 00:34:15,631: INFO] Updating task-status=started
[2024-08-05 00:34:15,942: INFO] Setting-up workdir
[2024-08-05 00:34:15,949: INFO] Starting DNS cache
[2024-08-05 00:34:16,251: DEBUG] DNS Cache started using IPs: ['172.17.0.4']
[2024-08-05 00:34:16,251: INFO] Starting scraper. Expects files at: /nfs/zimfarm/data/8a3b159d-17e8-42f0-9154-a7bb8d6d53dc 
[2024-08-05 00:34:16,253: DEBUG] Pulling image ghcr.io/openzim/ted:3.0.3
[2024-08-05 00:34:16,691: INFO] Updating task-status=scraper_started
[2024-08-05 06:21:25,448: INFO] Updating task-status=scraper_completed. Exit code: 137
[2024-08-05 06:21:25,957: DEBUG] Dumping docker logs to file…
[2024-08-05 06:21:26,187: DEBUG] Starting log uploader container…
[2024-08-05 06:21:26,480: DEBUG] No artifacts configured for upload
[2024-08-05 06:21:35,605: INFO] Scraper log upload complete: 0
[2024-08-05 06:21:35,605: INFO] Stopping and removing log_uploader
[2024-08-05 06:21:35,620: INFO] Sending scraper log filename: 8a3b159d-17e8-42f0-9154-a7bb8d6d53dc_ted.log
[2024-08-05 06:21:35,930: INFO] Updating task-status=failed

I'm not sure what we could do in such situations, exit code 137 usually means RAM depletion, so it looks like under some circumstances make_zim_file function is consuming too much RAM.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions