Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 4 additions & 54 deletions docs/host_services_setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@
This guide will walk through setting up Kolibri Studio for local development, where you'll run Studio's Python apps and all of Studio's services on your host machine, without the need for docker.

## Prerequisites
- [volta](https://docs.volta.sh/guide/getting-started)
- [pyenv](https://kolibri-dev.readthedocs.io/en/develop/howtos/installing_pyenv.html) and [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv#installation)
For detailed instructions on installing and configuring Volta, pyenv, and pyenv-virtualenv, please see the [Prerequisites](./local_dev_host.md#prerequisites) section in our Local Development with host guide.

## Install system dependencies and services
Studio requires some background services to be running:
Expand Down Expand Up @@ -36,7 +35,7 @@ brew link --force imagemagick@6

### Windows

Windows is no longer supported due to incompatibilities with some required packages.
**Note:** Windows is no longer supported due to incompatibilities with some required packages. If you are developing on Windows, it is recommended to use WSL (Windows Subsystem for Linux). Please follow the [WSL setup guide](./local_dev_wsl.md) for detailed instructions.

## Set up the database

Expand Down Expand Up @@ -72,32 +71,8 @@ Press <kbd>Ctrl</kbd>+<kbd>D</kbd> to exit the `psql` client. Finally
exit # leave the postgres account
```


## Build your python virtual environment
To determine what version of Python studio needs, you can check the `runtime.txt` file:
```bash
$ cat runtime.txt
# This is the required version of Python to run Studio currently.
# This is determined by the default Python 3 version that is installed
# inside Ubuntu Bionic, which is used to build images for Studio.
# We encode it here so that it can be picked up by Github's dependabot
# to manage automated package upgrades.
python-3.10.13
```
So to install python 3.10.13 through `pyenv` and set up a virtual environment:
```bash
pyenv install 3.10.13
pyenv virtualenv 3.10.13 studio-py3.10
pyenv activate studio-py3.10
```
Now you may install Studio's Python dependencies:
```bash
pip install -r requirements.txt -r requirements-dev.txt
```
To deactivate the virtual environment, when you're finished developing on Studio for the time being:
```bash
pyenv deactivate
```
For complete instructions on installing Python 3.10.13, creating and activating the virtual environment, and installing Studio’s Python dependencies, please refer to the [Build Your Python Virtual Environment](./local_dev_host.md#build-your-python-virtual-environment) section in our Local Development with host guide.

### A note about `psycopg2`
The packages `postgresql-12`, `postgresql-contrib`, and `postgresql-server-dev-all` are required to build `psycopg2` python driver.
Expand All @@ -112,32 +87,7 @@ export LDFLAGS="-L/opt/homebrew/opt/openssl/lib"
```

## Install frontend dependencies
The project requires `Node 16.X` as the runtime and `Yarn >= 1.22.22` as the package manager. We make use of [`Volta`](https://docs.volta.sh/guide/getting-started) to manage the same automatically. Please make sure you have volta installed and your shell configured to use volta. You can then install all the dependencies by running:
```bash
yarn install
```


## Run the services

Having installed all the necessary services, initialized your python virtual environment, and installed `yarn`, you're now ready to start the services. Generally speaking, you'll want to open a separate terminal/terminal-tab to run the services. The following will ensure all services are started, in addition to starting the celery workers service:
```bash
yarn run services
```

## Initializing Studio
With the services running, in a separate terminal/terminal-tab, we can now initialize the database for Studio development purposes. The command below will initialize the database, in addition to adding a user account for development:
```bash
yarn run devsetup
```

## Running the development server
With the services running, in a separate terminal/terminal-tab, and the database initialized, we can start the dev server:
```bash
yarn run devserver:hot # with Vue hot module reloading
# or
yarn run devserver # without hot module reloading
```
For guidance on installing Node 16.X, Yarn, and all required frontend dependencies, running the services, initializing Studio, and running the development server , please refer to the [Install Frontend Dependencies](./local_dev_host.md#install-frontend-dependencies) section in our Local Development with host guide.

Either of the above commands will take a few minutes to build the frontend. When it's done, you can sign in with the account created by the `yarn run devsetup` command:
- url: `http://localhost:8080/accounts/login/`
Expand Down
34 changes: 5 additions & 29 deletions docs/local_dev_docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,38 +2,14 @@

The following guide utilizes docker and docker-compose to run select services required for Studio to function. It's our recommended setup. However, if you would rather install these services on your host, please follow the [host-setup guide](./local_dev_host.md).

## Prerequisites
Please install these prerequisites, or alternatives for setting up your local development environment:
- [volta](https://docs.volta.sh/guide/getting-started) or a different node.js manager
- [pyenv](https://kolibri-dev.readthedocs.io/en/develop/howtos/installing_pyenv.html) and [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv#installation)
- [docker](https://docs.docker.com/install/) and [docker-compose](https://docs.docker.com/compose/install/)
**Note:** If you are developing on Windows, it is recommended to use WSL (Windows Subsystem for Linux). Please follow the [WSL setup guide](./local_dev_wsl.md) for detailed instructions.

## Prerequisites
For detailed instructions on installing and configuring Volta, pyenv, and pyenv-virtualenv, please see the [Prerequisites](./local_dev_host.md#prerequisites) section in our Local Development with host guide.

## Build your python virtual environment
To determine the preferred version of Python, you can check the `runtime.txt` file:
```bash
$ cat runtime.txt
# This is the required version of Python to run Studio currently.
# This is determined by the default Python 3 version that is installed
# inside Ubuntu Bionic, which is used to build images for Studio.
# We encode it here so that it can be picked up by Github's dependabot
# to manage automated package upgrades.
python-3.10.13
```
Use `pyenv` to install the version of Python listed in that file, and to also set up a virtual environment:
```bash
pyenv install 3.10.13
pyenv virtualenv 3.10.13 studio-py3.10
pyenv activate studio-py3.10
```
Now you may install Studio's Python dependencies:
```bash
pip install -r requirements.txt -r requirements-dev.txt
```
To deactivate the virtual environment, when you're finished developing on Studio for the time being:
```bash
pyenv deactivate
```
For complete instructions on installing Python 3.10.13, creating and activating the virtual environment, and installing Studio’s Python dependencies, please refer to the [Build Your Python Virtual Environment](./local_dev_host.md#build-your-python-virtual-environment) section in our Local Development with host guide.


### A note about dependencies on Apple Silicon M1+
If you run into an error with `pip install` related to the `grcpio` package, it is because it currently [does not support M1 with the version for `grcpio` Studio uses](https://github.com/grpc/grpc/issues/25082). In order to fix it, you will need to add the following environmental variables before running `pip install`:
Expand Down
3 changes: 2 additions & 1 deletion docs/local_dev_host.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

This guide will walk through setting up Kolibri Studio for local development, where you'll run Studio's Python apps and all of Studio's services on your host machine, without the need for docker.

**Note:** If you are developing on Windows, it is recommended to use WSL (Windows Subsystem for Linux). Please follow the [WSL setup guide](./local_dev_wsl.md) for detailed instructions.

## Prerequisites
- [volta](https://docs.volta.sh/guide/getting-started)
- [pyenv](https://kolibri-dev.readthedocs.io/en/develop/howtos/installing_pyenv.html) and [pyenv-virtualenv](https://github.com/pyenv/pyenv-virtualenv#installation)
Expand Down Expand Up @@ -72,7 +74,6 @@ Press <kbd>Ctrl</kbd>+<kbd>D</kbd> to exit the `psql` client. Finally
exit # leave the postgres account
```


## Build your python virtual environment
To determine what version of Python studio needs, you can check the `runtime.txt` file:
```bash
Expand Down
192 changes: 192 additions & 0 deletions docs/local_dev_wsl.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,192 @@
# Kolibri Studio Local Setup Guide Using WSL

This guide will walk you through setting up Kolibri Studio for local development using Windows Subsystem for Linux (WSL). We will cover everything from installing WSL and a Linux distribution to configuring your environment and running the necessary commands.

## Table of Contents

1. [Install WSL and Ubuntu](#install-wsl-and-ubuntu)
2. [Update and Upgrade Packages](#update-and-upgrade-packages)
3. [Install Prerequisites](#install-prerequisites)
4. [Install System Dependencies and Services](#install-system-dependencies-and-services)
5. [Set Up the Database](#set-up-the-database)
6. [Configure .bashrc](#configure-bashrc)
7. [Set Up Python Virtual Environment](#set-up-python-virtual-environment)
8. [Install Frontend Dependencies](#install-frontend-dependencies)
9. [Run the Services](#run-the-services)
10. [Initialize Studio](#initialize-studio)
11. [Run the Development Server](#run-the-development-server)
12. [Access Kolibri Studio](#access-kolibri-studio)

## Install WSL and Ubuntu

1. **Enable WSL**: Open PowerShell as Administrator and run the following command to enable WSL:

```sh
wsl --install
```

This command will install WSL and the default Linux distribution (Ubuntu). If you need to install a specific version of Ubuntu, you can follow the instructions on the [official WSL installation guide](https://docs.microsoft.com/en-us/windows/wsl/install).

2. **Set Up Your Linux Username and Password**: After the installation is complete, you will be prompted to set up your Linux username and password.

3. **Set WSL Version to 2**:

```sh
wsl --set-default-version 2
```

## Update and Upgrade Packages

Open your WSL terminal and update the package lists:

```sh
sudo apt update
sudo apt upgrade -y
```

Fork the repo and clone it by running the following command:

```sh
git clone https://github.com/$USERNAME/studio.git
```

Replace `$USERNAME` with your GitHub username.

## Install Prerequisites
For detailed instructions on installing and configuring Volta, pyenv, and pyenv-virtualenv, please see the [Prerequisites](./local_dev_host.md#prerequisites) section in our Local Development with host guide.

## Install System Dependencies and Services

Studio requires some background services to be running:

* Minio - a local S3 storage emulation
* PostgreSQL - a relational database
* Redis - a fast key/value store useful for caching

Install the necessary system dependencies and services:

```sh
sudo apt-get update
sudo apt-get install -y python-tk postgresql-server-dev-all postgresql-contrib postgresql-client postgresql-12 ffmpeg libmagickwand-dev redis-server wkhtmltopdf
```

### Install MinIO

Download and install MinIO:

```sh
wget https://dl.minio.io/server/minio/release/linux-amd64/minio -O bin/minio
sudo chmod +x ~/bin/minio
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just realized that a recent change we made likely needs a revision here. Not necessarily in-scope of this PR but if you have a chance to try this, I believe we need to add a command to create the bucket in the minio instance. Previously there was code that automatically created the bucket, but that was deleted because it was only for development and the docker image we use now performs it automatically.

```

## Set Up the Database

### Start PostgreSQL Service

Make sure PostgreSQL is running:

```sh
sudo service postgresql start
```

### Create Database and User

Start the client with:

```sh
sudo su postgres
psql
```

Run the following SQL commands:

```sql
CREATE USER learningequality WITH NOSUPERUSER INHERIT NOCREATEROLE CREATEDB LOGIN NOREPLICATION NOBYPASSRLS PASSWORD 'kolibri';
CREATE DATABASE "kolibri-studio" WITH TEMPLATE = template0 ENCODING = "UTF8" OWNER = "learningequality";
```

Exit the PostgreSQL client:

```sh
\q
exit
```

## Set Up Python Virtual Environment
For complete instructions on installing Python 3.10.13, creating and activating the virtual environment, and installing Studio’s Python dependencies, please refer to the [Build Your Python Virtual Environment](./local_dev_host.md#build-your-python-virtual-environment) section in our Local Development with host guide.

### Note about psycopg2

The packages `postgresql-12`, `postgresql-contrib`, and `postgresql-server-dev-all` are required to build the `psycopg2` Python driver.

## Install Frontend Dependencies

For guidance on installing Node 16.X, Yarn, and all required frontend dependencies, running the services, initializing Studio, and running the development server , please refer to the [Install Frontend Dependencies](./local_dev_host.md#install-frontend-dependencies) section in our Local Development with host guide.

## Access Kolibri Studio

Either of the above commands will take a few minutes to build the frontend. When it's done, you can sign in with the account created by the `yarn run devsetup` command:

* URL: <http://localhost:8080/accounts/login/>
* Username: `a@a.com`
* Password: `a`

## Contributing to the Codebase with Visual Studio Code and WSL

Once you have completed the setup of Kolibri Studio in your WSL environment using the terminal, you can use Visual Studio Code (VS Code) to open the project and start contributing to the codebase. Here’s how you can do it:

### Step 1: Install Visual Studio Code

1. **Download and Install VS Code**:
* Download VS Code from the [official website](https://code.visualstudio.com/).
* Install it on your Windows machine.

### Step 2: Install the Remote - WSL Extension

1. **Open VS Code**.
2. **Go to the Extensions View**:
* Click on the Extensions icon in the Activity Bar on the side of the window or press `Ctrl+Shift+X`.
3. **Search for "Remote - WSL"**:
* Install the extension by Microsoft.

### Step 3: Connect VS Code to WSL

1. **Open the Command Palette**:
* Press `Ctrl+Shift+P` to open the Command Palette.
2. **Select "Remote-WSL: New Window"**:
* This will open a new VS Code window connected to your WSL environment.

### Step 4: Open Your Project in VS Code

1. **In the WSL-connected VS Code Window**:
* Use the File Explorer to open your `~/studio` directory.
* You can do this by selecting `File > Open Folder` and navigating to `~/studio`.

### Running the Project in VS Code

Now that you have your project open in VS Code, you can run the same commands you used in the terminal to activate the environment and start the services:

1. **Open the Integrated Terminal in VS Code**:
* You can open the terminal by selecting `Terminal > New Terminal` from the menu or by pressing \`Ctrl+\`\`.
2. **Activate the Python Virtual Environment**:

```sh
pyenv activate studio-py3.10
```
3. **Run the Services**:

```sh
yarn run services
```
4. **Initialize the Studio**:

```sh
yarn run devsetup
```
5. **Start the Development Server**:

```sh
yarn run devserver:hot
```

By following these steps, you can set up a productive development environment in VS Code with WSL and start contributing to the Kolibri Studio codebase.
Loading