Reduce wheel size by omitting debug information

[marcelm]

I suggest to compile the Linux wheels with -g0 to reduce their size from 2.5 to ~0.5 MB.

The Linux wheels for zstandard are quite large at typically over 2 MB, especially compared to the Windows and macOS ones which have around 500 kB. The larger size seems to come from including debug information on Linux. To confirm this, I unpacked the .whl file manually, ran strip on the .so file and re-packed it. The resulting wheel has a size of just 400 kB.

I would like to suggest to pass CFLAGS=-g0 when creating the wheels, which leads to a similar size reduction. I have done this in multiple projects without detrimental effects.

See pycompression/python-isal#108.

[indygreg]

I respectfully disagree. I think the debugging information can be useful to debug crashes. 2.5 MB is not large by today's standards except in embedded environments. And embedded environments are likely so special anyway that pre-built binary wheels are often not used.

Someone would need to make the case that 2.5 MB is sufficiently large and harmful that it warrants losing out on debug symbols. As much as I am sympathetic to achieving peak performance and efficiency, I'm just not seeing the argument at this time. But I'm open minded in case I'm ignorant about real harms being incurred here.

[rhpvorderman]

I think the debugging information can be useful to debug crashes.

As the python-isal maintainer, I also maintain a C extension. In my perspective, the C debugging information is not very useful for users. When the C code crashes, usually the only information given is "Segmentation Fault". The debugging information is only useful when used in combination with a debugger such as gdb or catchsegv. That is beyond most (virtually all) users. In these cases it seems easier just to ask the user for data that reproduces the crash and do the debugging as a developer.

Also there is a difference here between the C-extension and the included zstandard distribution. My guess is that if you compile the zstd library with -g0 and the C-extension without such information that will already have saved 90% of the wheel size, whilst keeping the debugging information for your project.

Also it is good to know that -g0 only removes the debugging symbol table. NOT the entire symbol table. It will still be possible to ascertain in which function the error occured using a tool such as catchsegv. It will only be impossible to do a line by line walktrough with gdb, but again, that is beyond virtually all users. If they are able to do that, they are also able to compile it themselves with debugging information.

So that is why I decided to compile without debugging information. I never had any user report a bug to me that was made easier if only they had the debugging symbols. YMMV of course, (python-isal is not as popular).