The Featured image component could be improved a bit for accessibility. There are probably a few possible improvements:

• the text "Click the image to edit or update"
• keep or change the current Set/Remove button
• the image alt text

Regarding the text "Click the image to edit or update", I'd totally be in favour of re-considering it, but please let me explain why it was added. The featured image, once set, gets wrapped in a link. However, there was just a title attribute to communicate the action associated to the link. Title attributes work just on hover and may be ignored by screen readers, depending on the screen reader and also on the user verbosity settings.

For this reason, in https://core.trac.wordpress.org/changeset/36000 the title attribute was replaced with some visible text. The text is associated to the link using aria-describedby so screen readers announce it automatically when the link is focused.

If the text "Click the image to edit or update" is going to stay, at the very least it should be associated to the link in the same way, using aria-describedby (and so there's the need of a unique ID). Worth noting it's possible to hide this text: screen readers should announce elements targeted with aria-describedby even when they're hidden.

However, I'm wondering if the whole control could be improved, also visually. The new media widgets just introduced a new pattern: the Image widget uses four controls: Add/Edit/Replace/Delete that make the available actions self-explanatory. Maybe worth exploring a new design to communicate (in an accessible way) the image is clickable and the associated action.

Regarding the alt text:
in core, the featured image in the admin UI currently uses the image alt text (if set), otherwise an empty alt text. In Gutenberg instead, the image uses an hardcoded alt text: alt="Featured image". As far as I see, the returned media object contains an alt_text property. It would be great to pair the current behaviour with the core one and even improve it a bit. The key here is that the authoring context is different from the front-end. While editing, users need to know what the image is, and that's communicated by the image alt attribute. I'd propose to consider something like:
In the admin UI:

• if there's an alt_text, use that
• otherwise, use as fallback something like "Featured image: no alt text provided"
  Instead, on the front-end the featured image should respect the author's intent and use the image alt text (if set) or an empty alt="".