What problem does this address?

If a theme adds CSS to customize captions, the options in the caption toolbar does not reflect this.
For example if the caption is already italic, choosing the italic option does nothing, and you also can not turn off the italic style.

In this core trac ticket for Twenty Seventeen, the reporter writes:

If you add a caption to an image in the Image Block, the caption is in italics on the front end but not in the editor. This means that if you select the caption text you have the option to make it italic (which has no change on the front end) but not the option to deselect italics.

https://core.trac.wordpress.org/ticket/56747

What is your proposed solution?

Add an option to the caption toolbar that clears the theme styling (overrides it).
Or, allow themes to use a filter to make any styling option in the toolbar enabled by default. Deselecting the option should then remove the style.