The metadata file is too large

[lintingbin]

We have a table with thousands of columns. Every time a column is added, all the old columns will be retained in the schemas field of the metadata, and will not be deleted again. In this way, after hundreds of column addition operations, the metadata will have a size of several hundred MB.
Is there any way to clean up the schemas field. Does the metadata need to save all the historical schemas if it is just adding columns?

[RussellSpitzer]

I actually wrote something for cleaning up Partition Specs because of a slightly related issue.

#3462

But we could always extend this to schema's as well. Basically just a "cleanMetadata" function.

[lintingbin]

@RussellSpitzer This sounds like a good idea. However, the iceberg community staff did not express any of their opinions, so your code has not been merged into master.

[RussellSpitzer]

Yes as I noted, this was something we used internally and I didn't push for it to be merged here because I wasn't sure there was any utility. This also would not cover your specific issue but provides a template for what could be a solution. As for community interest, I'm sure we could get something like this in if it is not solely for fixing our internal issues.

[github-actions]

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

[github-actions]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'