Here's the page inspector that can be accessed when editing a page in the site editor:

It details which template the page uses, and provides access to edit that template.

What is missing is a way to switch templates.

We can potentially reuse the "Choose a pattern" modal that appears when creating pages / templates, and display the options therein.

Initially the modal would include any existing $custom page templates as options, and in the future might be enhanced to include a way to create a new template on-the-fly based on patterns.

cc @WordPress/gutenberg-design