pgloader MySQL to PostgreSQL Conversion Heap Issues

[zatowle]

I am attempting to migrate from MySQL to PostgreSQL via pgloader.

MySQL Version: 8.0.37 MySQL Community Server
PostgreSQL Version: 15.12
pgloader Version: pgloader version "3.6.7~devel"
compiled with SBCL 2.2.10-1.rhel9

Here is my pgloader command being run at the time of the error:

pgloader --verbose --debug --dynamic-space-size 18432 --with "prefetch rows = 500" mysql://:@:3306/<DB_NAME> pgsql://:@:5432/<DB_NAME>

I have been continuously seeing the heap error so I increased my RAM and CPU on the VM temporarily in hopes it would allow it to process if I specified a higher value for --dynamic-space-size. I have been trying all sorts of combinations for --dynamic-space-size and prefetch rows but it continues to fail. The MySQL database being converted is only 3GB in size, what can I do to get this to complete properly?

This is the most recent error which does not look to be related to heap but I am not sure why it failed, any help is apprecited!

2025-07-29T21:38:28.140234+01:00 DATA < #("98213" "2020-05-01 11:18:29.000000" "2025-05-07 09:21:40.866597"
"PRIORITY" "type/Decimal" NIL #1="t" NIL #1# "34" "5404" NIL "Priority"
"normal" NIL "2020-05-01 12:33:03.000000" NIL NIL
"{"global":{"distinct-count":2,"nil%":0.9989350372736954},"type":{"type/Number":{"min":225.0,"max":225.0,"avg":225.0,"sd":null,"q1":225.0,"q3":225.0}}}"
"4" "NUMBER" NIL NIL "34" "0" "type/Decimal" NIL NIL #2="f" #2# #2# NIL NIL
#2# "0")
2025-07-29T21:38:28.140234+01:00 DATA < #("98214" "2020-05-01 11:18:29.000000" "2025-05-07 09:21:41.054957"
"ITERATE_PARAM_INDEX" "type/Decimal" NIL #1="t" NIL #1# "10" "5404" NIL
"Iterate Param Index" "normal" NIL "2020-05-01 12:33:03.000000" NIL NIL
"{"global":{"distinct-count":2,"nil%":0.001064962726304579},"type":{"type/Number":{"min":-1.0,"max":-1.0,"avg":-1.0,"sd":0.0,"q1":-1.0,"q3":-1.0}}}"
"4" "NUMBER" NIL NIL "10" "0" "type/Decimal" NIL NIL #2="f" #2# #2# NIL NIL
#2# "0")
2025-07-29T21:38:28.140234+01:00 DATA < #("98215" "2020-05-01 11:18:29.000000" "2025-05-07 09:21:41.099189"
"PARENT_STEP_ID" "type/Decimal" NIL #1="t" NIL #1# "11" "5404" NIL
"Parent Step ID" "normal" NIL "2020-05-01 12:33:03.000000" NIL NIL
"{"global":{"distinct-count":513,"nil%":0.001064962726304579},"type":{"type/Number":{"min":-1.0,"max":1909979.0,"avg":949943.960554371,"sd":903830.5577293859,"q1":42596.51114246267,"q3":1885718.775465599}}}"
"4" "NUMBER" NIL NIL "11" "0" "type/Decimal" NIL NIL #2="f" #2# #2# NIL NIL
#2# "0")
2025-07-29T21:38:28.140234+01:00 INFO Stopping monitor

What I am doing here?

The value
NIL
is not of type
PGLOADER.CATALOG:COLUMN

[zatowle]

@dimitri Any idea on this one?

[biovolt]

pgloader version "3.6.7~devel"
compiled with SBCL 2.2.9.debian

I get this error. Running
WITH
data only,
on error stop,
batch rows = 1,
prefetch rows = 1,
truncate,
quote identifiers,
batch size = 8MB

Heap exhausted during allocation: 66912256 bytes available, 69131648 requested.
Immobile Object Counts
Gen layout fdefn symbol code Boxed Cons Raw Code SmMix Mixed LgRaw LgCode LgMix Waste% Alloc Trig Dirty GCs Mem-age
0 0 0 0 0 0 1 0 0 0 287 0 0 933 5.5 37803824 10737418 - 0 0.0000
1 0 0 0 0 1 3 14 0 0 7 8280 0 0 0.2 271683328 145479962 8295 1 0.4960
2 0 0 0 0 0 4 16 0 0 10 6101 0 0 0.3 200291584 55704810 6118 1 0.2245
3 46 0 0 66 30 21 25 1 4 38 3777 0 0 1.0 126418656 2000000 3803 0 0.4546
4 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0 0 2000000 0 0 0.0000
5 0 0 0 0 0 0 0 0 0 0 0 0 0 0.0 0 2000000 0 0 0.0000
6 2137 36032 30989 33784 775 841 694 19 78 67 100 0 290 2.2 91790720 2000000 13 0 0.0000
Tot 2183 36032 30989 33850 806 870 749 20 82 409 18258 0 1223 0.9 727988112 [67.8% of 1073741824 max]
GC control variables:
GC-INHIBIT = false
GC-PENDING = true
STOP-FOR-GC-PENDING = false
2025-08-20T10:40:43.744452Z ERROR A SB-KERNEL::HEAP-EXHAUSTED-ERROR condition without bindings for heap statistics. (If
you did not expect to see this message, please report it.