In the current WordPress text editor, the QuickTags buttons are properly labeled (the labels are also translatable):

In Gutenberg, they're not (yet?):

The current rendered markup is, for example:
<button class="editor-text-editor__bold">b</button>

All these buttons need aria-label attributes with translatable strings, as in the current text editor.
To evaluate: do they need also a type="button" attribute?

Aside: as mentioned in another issue, personally I'm not fully convinced QuickTags should stay but if they have to stay they should be accessible and not introduce accessibility regressions.