Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support returning a preferred remote controller #9978

Merged
merged 13 commits into from
May 11, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .vscode/settings.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// Place your settings in this file to overwrite default and user settings.
{
"files.exclude": {
"out": true, // set this to true to hide the "out" folder with the compiled JS files
"out": false, // set this to true to hide the "out" folder with the compiled JS files
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is ok to have, i keep setting this.

"**/*.pyc": true,
".nyc_output": true,
"obj": true,
Expand Down
1,360 changes: 875 additions & 485 deletions TELEMETRY.md

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions package.nls.json
Original file line number Diff line number Diff line change
Expand Up @@ -288,8 +288,8 @@
"DataScience.jupyterSelectPasswordPrompt": "Enter your password",
"DataScience.removeRemoteJupyterServerEntryInQuickPick": "Remove",
"DataScience.jupyterNotebookFailure": "Jupyter notebook failed to launch. \r\n{0}",
"DataScience.remoteJupyterConnectionFailedWithServer": "Failed to connect to the remote Jupyter Server {0}. Verify the server is running and reachable.",
"DataScience.remoteJupyterConnectionFailedWithServerWithError":"Failed to connect to the remote Jupyter Server {0}. Verify the server is running and reachable. ({1}).",
"DataScience.remoteJupyterConnectionFailedWithServer": "Failed to connect to the remote Jupyter Server '{0}'. Verify the server is running and reachable.",
"DataScience.remoteJupyterConnectionFailedWithServerWithError":"Failed to connect to the remote Jupyter Server '{0}'. Verify the server is running and reachable. ({1}).",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was previously displaying the baseUrl, but in the case of somethign like Azure ML, we can display the displayName (which could contain spaces), hence quoting this string.

"DataScience.remoteJupyterConnectionFailedWithoutServerWithError": "Connection failure. Verify the server is running and reachable. ({0}).",
"DataScience.jupyterNotebookRemoteConnectFailedWeb": "Failed to connect to remote Jupyter server.\r\nCheck that the Jupyter Server URI can be reached from a browser.\r\n{0}. Click [here](https://aka.ms/vscjremoteweb) for more information.",
"DataScience.jupyterNotebookRemoteConnectSelfCertsFailed": "Failed to connect to remote Jupyter notebook.\r\nSpecified server is using self signed certs. Enable Allow Unauthorized Remote Connection setting to connect anyways\r\n{0}\r\n{1}",
Expand Down
3 changes: 2 additions & 1 deletion src/extension.node.ts
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,8 @@ export async function activate(context: IExtensionContext): Promise<IExtensionAp
registerPythonApi: noop,
registerRemoteServerProvider: noop,
showDataViewer: () => Promise.resolve(),
getKernelService: () => Promise.resolve(undefined)
getKernelService: () => Promise.resolve(undefined),
getSuggestedController: () => Promise.resolve(undefined)
};
}
}
Expand Down
3 changes: 2 additions & 1 deletion src/extension.web.ts
Original file line number Diff line number Diff line change
Expand Up @@ -117,7 +117,8 @@ export async function activate(context: IExtensionContext): Promise<IExtensionAp
registerPythonApi: noop,
registerRemoteServerProvider: noop,
showDataViewer: () => Promise.resolve(),
getKernelService: () => Promise.resolve(undefined)
getKernelService: () => Promise.resolve(undefined),
getSuggestedController: () => Promise.resolve(undefined)
};
}
}
Expand Down
2 changes: 1 addition & 1 deletion src/kernels/jupyter/commands/serverSelector.ts
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ export class JupyterServerSelectorCommand implements IDisposable {
// Find one that is the default for this remote
if (notebook) {
// Recompute the preferred controller
await this.controllerManager.computePreferredNotebookController(notebook);
await this.controllerManager.initializePreferredNotebookController(notebook);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was not only computing the preferred controller, it was also setting it using the VS Code api.
I've renamed this to be more specific.
The method computePreferredNotebookController is now used to return the preferred controller without changing it.


// That should have picked a preferred
const preferred = this.controllerManager.getPreferredNotebookController(notebook);
Expand Down
33 changes: 20 additions & 13 deletions src/kernels/jupyter/jupyterConnection.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,9 @@ import { inject, injectable } from 'inversify';
import { IExtensionSyncActivationService } from '../../platform/activation/types';
import { Identifiers } from '../../platform/common/constants';
import { IDisposableRegistry } from '../../platform/common/types';
import { RemoteJupyterServerUriProviderError } from '../../platform/errors/remoteJupyterServerUriProviderError';
import { IJupyterConnection } from '../types';
import { computeUriHash, createRemoteConnectionInfo } from './jupyterUtils';
import { computeServerId, createRemoteConnectionInfo } from './jupyterUtils';
import { ServerConnectionType } from './launcher/serverConnectionType';
import {
IJupyterServerUri,
Expand Down Expand Up @@ -65,7 +66,7 @@ export class JupyterConnection implements IExtensionSyncActivationService {
private async getUriFromServerId(serverId: string) {
// Since there's one server per session, don't use a resource to figure out these settings
const savedList = await this.serverUriStorage.getSavedUriList();
return savedList.find((item) => computeUriHash(item.uri) === serverId)?.uri;
return savedList.find((item) => computeServerId(item.uri) === serverId)?.uri;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Renamed computeUriHash to be computeServerId, the implementation should not define the method name, its a server id, hence the name should reflect that.

}
private async createConnectionInfoFromUri(uri: string) {
// Prepare our map of server URIs
Expand All @@ -92,18 +93,24 @@ export class JupyterConnection implements IExtensionSyncActivationService {
public async updateServerUri(uri: string): Promise<void> {
const idAndHandle = this.extractJupyterServerHandleAndId(uri);
if (idAndHandle) {
const serverUri = await this.jupyterPickerRegistration.getJupyterServerUri(
idAndHandle.id,
idAndHandle.handle
);
this.uriToJupyterServerUri.set(uri, serverUri);
// See if there's an expiration date
if (serverUri.expiration) {
const timeoutInMS = serverUri.expiration.getTime() - Date.now();
// Week seems long enough (in case the expiration is ridiculous)
if (timeoutInMS > 0 && timeoutInMS < 604800000) {
this.pendingTimeouts.push(setTimeout(() => this.updateServerUri(uri).ignoreErrors(), timeoutInMS));
try {
const serverUri = await this.jupyterPickerRegistration.getJupyterServerUri(
idAndHandle.id,
idAndHandle.handle
);
this.uriToJupyterServerUri.set(uri, serverUri);
// See if there's an expiration date
if (serverUri.expiration) {
const timeoutInMS = serverUri.expiration.getTime() - Date.now();
// Week seems long enough (in case the expiration is ridiculous)
if (timeoutInMS > 0 && timeoutInMS < 604800000) {
this.pendingTimeouts.push(
setTimeout(() => this.updateServerUri(uri).ignoreErrors(), timeoutInMS)
);
}
}
} catch (ex) {
throw new RemoteJupyterServerUriProviderError(idAndHandle.id, idAndHandle.handle, ex);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Throw a custom error so we can handle this, e.g. possible the jupyter server is down and user can now choose to ignore this connection completely.

}
}
}
Expand Down
41 changes: 35 additions & 6 deletions src/kernels/jupyter/jupyterUriProviderRegistration.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,12 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.
import { inject, injectable } from 'inversify';
import { inject, injectable, named } from 'inversify';
import { Memento } from 'vscode';

import { IExtensions } from '../../platform/common/types';
import { GLOBAL_MEMENTO, IExtensions, IMemento } from '../../platform/common/types';
import { swallowExceptions } from '../../platform/common/utils/decorators';
import * as localize from '../../platform/common/utils/localize';
import { noop } from '../../platform/common/utils/misc';
import { JupyterUriProviderWrapper } from './jupyterUriProviderWrapper';
import {
IJupyterServerUri,
Expand All @@ -12,12 +15,17 @@ import {
JupyterServerUriHandle
} from './types';

const REGISTRATION_ID_EXTENSION_OWNER_MEMENTO_KEY = 'REGISTRATION_ID_EXTENSION_OWNER_MEMENTO_KEY';
@injectable()
export class JupyterUriProviderRegistration implements IJupyterUriProviderRegistration {
private loadedOtherExtensionsPromise: Promise<void> | undefined;
private providers = new Map<string, Promise<IJupyterUriProvider>>();
private providerExtensionMapping = new Map<string, string>();

constructor(@inject(IExtensions) private readonly extensions: IExtensions) {}
constructor(
@inject(IExtensions) private readonly extensions: IExtensions,
@inject(IMemento) @named(GLOBAL_MEMENTO) private readonly globalMemento: Memento
) {}

public async getProviders(): Promise<ReadonlyArray<IJupyterUriProvider>> {
await this.checkOtherExtensions();
Expand All @@ -26,7 +34,7 @@ export class JupyterUriProviderRegistration implements IJupyterUriProviderRegist
return Promise.all([...this.providers.values()]);
}

public registerProvider(provider: IJupyterUriProvider) {
public async registerProvider(provider: IJupyterUriProvider) {
if (!this.providers.has(provider.id)) {
this.providers.set(provider.id, this.createProvider(provider));
} else {
Expand All @@ -53,14 +61,35 @@ export class JupyterUriProviderRegistration implements IJupyterUriProviderRegist
}

private async loadOtherExtensions(): Promise<void> {
const extensionIds = new Set<string>();
this.globalMemento
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the case of github extensions, they can register the lazy proxy controller when a notebook opens and they register the provider with us.
When we reload vscode, we try to get the provider information, however its possible github or other extensions haven't activated yet, for such extensions we don'e need an entry in package.json for pythonRemoteServerProvider, instead we can just keep track of the extensions that own a specific provider and load and activate those extensions.

.get<{ extensionId: string; providerId: string }[]>(REGISTRATION_ID_EXTENSION_OWNER_MEMENTO_KEY, [])
.forEach((item) => extensionIds.add(item.extensionId));

const list = this.extensions.all
.filter((e) => e.packageJSON?.contributes?.pythonRemoteServerProvider)
.map((e) => (e.isActive ? Promise.resolve() : e.activate()));
.filter((e) => e.packageJSON?.contributes?.pythonRemoteServerProvider || extensionIds.has(e.id))
.map((e) => (e.isActive ? Promise.resolve() : e.activate().then(noop, noop)));
await Promise.all(list);
}

private async createProvider(provider: IJupyterUriProvider): Promise<IJupyterUriProvider> {
const info = await this.extensions.determineExtensionFromCallStack();
this.updateRegistrationInfo(provider.id, info.extensionId).catch(noop);
return new JupyterUriProviderWrapper(provider, info.extensionId);
}
@swallowExceptions()
private async updateRegistrationInfo(providerId: string, extensionId: string): Promise<void> {
const registeredList = this.globalMemento.get<{ extensionId: string; providerId: string }[]>(
REGISTRATION_ID_EXTENSION_OWNER_MEMENTO_KEY,
[]
);
registeredList.forEach((item) => this.providerExtensionMapping.set(item.providerId, item.extensionId));
this.providerExtensionMapping.set(providerId, extensionId);

const newList: { extensionId: string; providerId: string }[] = [];
this.providerExtensionMapping.forEach((providerId, extensionId) => {
newList.push({ extensionId, providerId });
});
await this.globalMemento.update(REGISTRATION_ID_EXTENSION_OWNER_MEMENTO_KEY, newList);
}
}
8 changes: 7 additions & 1 deletion src/kernels/jupyter/jupyterUriProviderWrapper.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,9 @@ export class JupyterUriProviderWrapper implements IJupyterUriProvider {
return this.provider.id;
}
public getQuickPickEntryItems(): vscode.QuickPickItem[] {
if (!this.provider.getQuickPickEntryItems) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extensions such as github or others won't need to implement getQuickPickEntryItems (see type definition)

return [];
}
return this.provider.getQuickPickEntryItems().map((q) => {
return {
...q,
Expand All @@ -26,10 +29,13 @@ export class JupyterUriProviderWrapper implements IJupyterUriProvider {
};
});
}
public handleQuickPick(
public async handleQuickPick(
item: vscode.QuickPickItem,
back: boolean
): Promise<JupyterServerUriHandle | 'back' | undefined> {
if (!this.provider.handleQuickPick) {
return;
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
if ((item as any).original) {
// eslint-disable-next-line @typescript-eslint/no-explicit-any
Expand Down
13 changes: 10 additions & 3 deletions src/kernels/jupyter/jupyterUtils.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ import { ConfigurationTarget, Uri } from 'vscode';
import { IApplicationShell, IWorkspaceService } from '../../platform/common/application/types';
import { noop } from '../../platform/common/utils/misc';
import { IJupyterConnection } from '../types';
import { IJupyterServerUri } from './types';
import { IJupyterServerUri, JupyterServerUriHandle } from './types';
import { getJupyterConnectionDisplayName } from './launcher/helpers';
import { IConfigurationService, IWatchableJupyterSettings, Resource } from '../../platform/common/types';
import { getFilePath } from '../../platform/common/platform/fs-paths';
import { DataScience } from '../../platform/common/utils/localize';
import { sendTelemetryEvent } from '../../telemetry';
import { Telemetry } from '../../platform/common/constants';
import { Identifiers, Telemetry } from '../../platform/common/constants';

export function expandWorkingDir(
workingDir: string | undefined,
Expand Down Expand Up @@ -133,6 +133,13 @@ export function createRemoteConnectionInfo(
};
}

export function computeUriHash(uri: string) {
export function computeServerId(uri: string) {
return hashjs.sha256().update(uri).digest('hex');
}

export function generateUriFromRemoteProvider(id: string, result: JupyterServerUriHandle) {
// eslint-disable-next-line
return `${Identifiers.REMOTE_URI}?${Identifiers.REMOTE_URI_ID_PARAM}=${id}&${
Identifiers.REMOTE_URI_HANDLE_PARAM
}=${encodeURI(result)}`;
}
8 changes: 4 additions & 4 deletions src/kernels/jupyter/remoteKernelFinder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ import { IJupyterSessionManagerFactory, IJupyterSessionManager } from './types';
import { sendKernelSpecTelemetry } from '../raw/finder/helper';
import { traceError, traceInfoIfCI } from '../../platform/logging';
import { IPythonExtensionChecker } from '../../platform/api/types';
import { computeUriHash } from './jupyterUtils';
import { computeServerId } from './jupyterUtils';

// This class searches for a kernel that matches the given kernel name.
// First it searches on a global persistent state, then on the installed python interpreters,
Expand Down Expand Up @@ -68,9 +68,9 @@ export class RemoteKernelFinder implements IRemoteKernelFinder {
kind: 'startUsingRemoteKernelSpec',
interpreter: await this.getInterpreter(s, connInfo.baseUrl),
kernelSpec: s,
id: getKernelId(s, undefined, computeUriHash(connInfo.url)),
id: getKernelId(s, undefined, computeServerId(connInfo.url)),
baseUrl: connInfo.baseUrl,
serverId: computeUriHash(connInfo.url)
serverId: computeServerId(connInfo.url)
};
return kernel;
})
Expand Down Expand Up @@ -101,7 +101,7 @@ export class RemoteKernelFinder implements IRemoteKernelFinder {
},
baseUrl: connInfo.baseUrl,
id: s.kernel?.id || '',
serverId: computeUriHash(connInfo.url)
serverId: computeServerId(connInfo.url)
};
return kernel;
});
Expand Down
53 changes: 41 additions & 12 deletions src/kernels/jupyter/serverSelector.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,21 @@ import {
InputFlowAction
} from '../../platform/common/utils/multiStepInput';
import { captureTelemetry, sendTelemetryEvent } from '../../telemetry';
import { Telemetry, Identifiers } from '../../webviews/webview-side/common/constants';
import { Telemetry } from '../../webviews/webview-side/common/constants';
import {
IJupyterUriProvider,
IJupyterUriProviderRegistration,
IJupyterServerUriStorage,
JupyterServerUriHandle
} from './types';
import { IDataScienceErrorHandler } from '../../platform/errors/types';
import { handleExpiredCertsError, handleSelfCertsError, computeUriHash } from './jupyterUtils';
import { IConfigurationService, IsWebExtension } from '../../platform/common/types';
import {
handleExpiredCertsError,
handleSelfCertsError,
computeServerId,
generateUriFromRemoteProvider
} from './jupyterUtils';
import { JupyterConnection } from './jupyterConnection';
import { JupyterSelfCertsError } from '../../platform/errors/jupyterSelfCertsError';
import { RemoteJupyterServerConnectionError } from '../../platform/errors/remoteJupyterServerConnectionError';
Expand Down Expand Up @@ -79,6 +84,32 @@ export class JupyterServerSelector {
const multiStep = this.multiStepFactory.create<{}>();
return multiStep.run(this.startSelectingURI.bind(this, allowLocal), {});
}

@captureTelemetry(Telemetry.EnterJupyterURI)
@traceDecoratorError('Failed to enter Jupyter Uri')
public async enterJupyterURI(): Promise<string | undefined> {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrote this code for server picker for our own lazy controller, don't see any harm keeping this here.
Not used for now, as i didn't want to checkin the layzy proxy controller as that API could change.
Requries a bit more work to remove this hence left it, hope its ok (the plan is to make use of this in the future)

let initialValue = defaultUri;
try {
const text = await this.clipboard.readText().catch(() => '');
const parsedUri = Uri.parse(text.trim(), true);
// Only display http/https uris.
initialValue = text && parsedUri && parsedUri.scheme.toLowerCase().startsWith('http') ? text : defaultUri;
} catch {
// We can ignore errors.
}
// Ask the user to enter a URI to connect to.
const uri = await this.applicationShell.showInputBox({
title: DataScience.jupyterSelectURIPrompt(),
value: initialValue || defaultUri,
validateInput: this.validateSelectJupyterURI,
prompt: ''
});

if (uri) {
await this.setJupyterURIToRemote(uri, true);
return computeServerId(uri);
}
}
@captureTelemetry(Telemetry.SetJupyterURIToLocal)
public async setJupyterURIToLocal(): Promise<void> {
await this.serverUriStorage.setUri(Settings.JupyterServerLocalLaunch);
Expand Down Expand Up @@ -108,7 +139,7 @@ export class JupyterServerSelector {
sendTelemetryEvent(Telemetry.FetchError, undefined, { currentTask: 'connecting' });
}
await this.errorHandler.handleError(
new RemoteJupyterServerConnectionError(userURI, computeUriHash(userURI), err)
new RemoteJupyterServerConnectionError(userURI, computeServerId(userURI), err)
);
// Can't set the URI in this case.
return;
Expand Down Expand Up @@ -187,6 +218,9 @@ export class JupyterServerSelector {
_input: IMultiStepInput<{}>,
_state: {}
): Promise<InputStep<{}> | void> {
if (!provider.handleQuickPick) {
return;
}
const result = await provider.handleQuickPick(item, true);
if (result === 'back') {
throw InputFlowAction.back;
Expand All @@ -197,18 +231,10 @@ export class JupyterServerSelector {
}
private async handleProviderQuickPick(id: string, result: JupyterServerUriHandle | undefined) {
if (result) {
const uri = this.generateUriFromRemoteProvider(id, result);
const uri = generateUriFromRemoteProvider(id, result);
await this.setJupyterURIToRemote(uri);
}
}

private generateUriFromRemoteProvider(id: string, result: JupyterServerUriHandle) {
// eslint-disable-next-line
return `${Identifiers.REMOTE_URI}?${Identifiers.REMOTE_URI_ID_PARAM}=${id}&${
Identifiers.REMOTE_URI_HANDLE_PARAM
}=${encodeURI(result)}`;
}

private async selectRemoteURI(input: IMultiStepInput<{}>, _state: {}): Promise<InputStep<{}> | void> {
let initialValue = defaultUri;
try {
Expand Down Expand Up @@ -275,6 +301,9 @@ export class JupyterServerSelector {
const providers = await this.extraUriProviders.getProviders();
if (providers) {
providers.forEach((p) => {
if (!p.getQuickPickEntryItems) {
return;
}
const newProviderItems = p.getQuickPickEntryItems().map((i) => {
return { ...i, newChoice: false, provider: p };
});
Expand Down
Loading