|
| 1 | +# VSCode Deepnote Extension Architecture |
| 2 | + |
| 3 | +This extension adds support for Deepnote notebooks in Visual Studio Code. Deepnote is a collaborative data science notebook platform, and this extension allows users to open, edit, and manage Deepnote project files (`.deepnote` files) directly within VS Code. |
| 4 | + |
| 5 | +## Key Components |
| 6 | + |
| 7 | +### 1. Notebook Serializer (`deepnoteSerializer.ts`) |
| 8 | + |
| 9 | +The core component responsible for converting between Deepnote's YAML format and VS Code's notebook format. |
| 10 | + |
| 11 | +**Responsibilities:** |
| 12 | + |
| 13 | +- **Deserialization**: Converts Deepnote YAML files into VS Code NotebookData format |
| 14 | +- **Serialization**: Converts VS Code notebook changes back to Deepnote YAML format |
| 15 | +- **State Management**: Maintains original project data for accurate serialization |
| 16 | + |
| 17 | +**Key Methods:** |
| 18 | + |
| 19 | +- `deserializeNotebook()`: Parses YAML, converts blocks to cells |
| 20 | +- `serializeNotebook()`: Converts cells back to blocks, updates YAML |
| 21 | +- `findCurrentNotebookId()`: Determines which notebook to deserialize using manager state |
| 22 | + |
| 23 | +### 2. Data Converter (`deepnoteDataConverter.ts`) |
| 24 | + |
| 25 | +Handles the transformation between Deepnote blocks and VS Code notebook cells. |
| 26 | + |
| 27 | +**Responsibilities:** |
| 28 | + |
| 29 | +- Convert Deepnote blocks (code, markdown, SQL, etc.) to VS Code cells |
| 30 | +- Convert VS Code cells back to Deepnote blocks |
| 31 | +- Preserve block metadata and outputs during conversion |
| 32 | + |
| 33 | +**Supported Block Types:** |
| 34 | + |
| 35 | +- Code blocks (Python, R, JavaScript, etc.) |
| 36 | +- Markdown blocks |
| 37 | + |
| 38 | +### 3. Notebook Manager (`deepnoteNotebookManager.ts`) |
| 39 | + |
| 40 | +Manages the state of Deepnote projects and notebook selections. |
| 41 | + |
| 42 | +**Responsibilities:** |
| 43 | + |
| 44 | +- Store original project data for serialization |
| 45 | +- Track which notebook is selected for each project |
| 46 | +- Maintain project-to-notebook mapping using project IDs |
| 47 | + |
| 48 | +**Key Features:** |
| 49 | + |
| 50 | +- In-memory caching of project data |
| 51 | +- Project-ID based notebook selection tracking |
| 52 | +- Support for multiple notebooks per project |
| 53 | + |
| 54 | +**Key Methods:** |
| 55 | + |
| 56 | +- `getTheSelectedNotebookForAProject()`: Retrieves selected notebook ID for a project |
| 57 | +- `selectNotebookForProject()`: Associates a notebook ID with a project ID |
| 58 | +- `storeOriginalProject()`: Caches project data and sets current notebook |
| 59 | + |
| 60 | +### 4. Explorer View (`deepnoteExplorerView.ts`) |
| 61 | + |
| 62 | +Provides the sidebar UI for browsing and opening Deepnote notebooks. |
| 63 | + |
| 64 | +**Responsibilities:** |
| 65 | + |
| 66 | +- Create and manage the tree view in VS Code's sidebar |
| 67 | +- Handle user interactions (clicking on notebooks/files) |
| 68 | +- Register commands for notebook operations |
| 69 | + |
| 70 | +**Commands:** |
| 71 | + |
| 72 | +- `deepnote.refreshExplorer`: Refresh the file tree |
| 73 | +- `deepnote.openNotebook`: Open a specific notebook |
| 74 | +- `deepnote.openFile`: Open the raw .deepnote file |
| 75 | +- `deepnote.revealInExplorer`: Show active notebook info |
| 76 | + |
| 77 | +### 5. Tree Data Provider (`deepnoteTreeDataProvider.ts`) |
| 78 | + |
| 79 | +Implements VS Code's TreeDataProvider interface for the sidebar view. |
| 80 | + |
| 81 | +**Responsibilities:** |
| 82 | + |
| 83 | +- Scan workspace for `.deepnote` files |
| 84 | +- Parse project files to extract notebook information |
| 85 | +- Provide tree structure for the explorer view |
| 86 | +- Watch for file system changes |
| 87 | + |
| 88 | +**Features:** |
| 89 | + |
| 90 | +- Automatic workspace scanning |
| 91 | +- File system watching for real-time updates |
| 92 | +- Caching for performance optimization |
| 93 | + |
| 94 | +### 6. Activation Service (`deepnoteActivationService.ts`) |
| 95 | + |
| 96 | +Entry point for the Deepnote functionality within the extension. |
| 97 | + |
| 98 | +**Responsibilities:** |
| 99 | + |
| 100 | +- Register the notebook serializer with VS Code |
| 101 | +- Initialize the explorer view |
| 102 | +- Set up extension lifecycle |
| 103 | + |
| 104 | +## Data Flow |
| 105 | + |
| 106 | +### Opening a Notebook |
| 107 | + |
| 108 | +1. **User Action**: User clicks on a notebook in the sidebar |
| 109 | +2. **Explorer View**: Handles the click, stores notebook selection using project ID |
| 110 | +3. **Notebook Manager**: Associates the notebook ID with the project ID |
| 111 | +4. **VS Code**: Opens the document using the base file URI and calls `deserializeNotebook()` |
| 112 | +5. **Serializer**: |
| 113 | + - Uses `findCurrentNotebookId()` to determine which notebook to load |
| 114 | + - Reads the YAML file and finds the selected notebook |
| 115 | + - Converts blocks to cells using the Data Converter |
| 116 | +6. **Display**: VS Code displays the notebook in the editor |
| 117 | + |
| 118 | +### Saving Changes |
| 119 | + |
| 120 | +1. **User Action**: User makes changes and saves (Ctrl+S) |
| 121 | +2. **VS Code**: Calls the serializer's `serializeNotebook()` method |
| 122 | +3. **Serializer**: |
| 123 | + - Retrieves original project data from Notebook Manager |
| 124 | + - Converts cells back to blocks using the Data Converter |
| 125 | + - Updates the YAML structure |
| 126 | + - Writes back to file |
| 127 | +4. **File System**: Updates the `.deepnote` file |
| 128 | + |
| 129 | +## File Format |
| 130 | + |
| 131 | +### Deepnote YAML Structure |
| 132 | + |
| 133 | +```yaml |
| 134 | +version: 1.0 |
| 135 | +metadata: |
| 136 | + modifiedAt: '2024-01-01T00:00:00Z' |
| 137 | +project: |
| 138 | + id: 'project-uuid' |
| 139 | + name: 'Project Name' |
| 140 | + notebooks: |
| 141 | + - id: 'notebook-uuid' |
| 142 | + name: 'Notebook Name' |
| 143 | + blocks: |
| 144 | + - id: 'block-uuid' |
| 145 | + type: 'code' |
| 146 | + source: "print('Hello')" |
| 147 | + outputs: [] |
| 148 | +``` |
| 149 | +
|
| 150 | +### VS Code Notebook Format |
| 151 | +
|
| 152 | +```typescript |
| 153 | +interface NotebookData { |
| 154 | + cells: NotebookCellData[]; |
| 155 | + metadata: { |
| 156 | + deepnoteProjectId: string; |
| 157 | + deepnoteProjectName: string; |
| 158 | + deepnoteNotebookId: string; |
| 159 | + deepnoteNotebookName: string; |
| 160 | + deepnoteVersion: string; |
| 161 | + }; |
| 162 | +} |
| 163 | +``` |
| 164 | + |
| 165 | +## Multi-Notebook Support |
| 166 | + |
| 167 | +The extension supports opening multiple notebooks from the same `.deepnote` file: |
| 168 | + |
| 169 | +1. **Project-Based Selection**: The Notebook Manager tracks which notebook is selected for each project |
| 170 | +2. **State Management**: When opening a notebook, the manager stores the project-to-notebook mapping |
| 171 | +3. **Fallback Detection**: The serializer can detect the current notebook from VS Code's active document context |
| 172 | + |
| 173 | +## Technical Decisions |
| 174 | + |
| 175 | +### Why YAML? |
| 176 | + |
| 177 | +Deepnote uses YAML for its file format, which provides: |
| 178 | + |
| 179 | +- Human-readable structure |
| 180 | +- Support for complex nested data |
| 181 | +- Easy to read Git diffs |
| 182 | + |
| 183 | +### Why Project-ID Based Selection? |
| 184 | + |
| 185 | +- Simpler than URI-based tracking - uses straightforward project ID mapping |
| 186 | +- The VS Code NotebookSerializer interface doesn't provide URI context during deserialization |
| 187 | +- Allows for consistent notebook selection regardless of how the document is opened |
| 188 | +- Manager-based approach centralizes state management and reduces complexity |
0 commit comments