-
Notifications
You must be signed in to change notification settings - Fork 1
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Install Daft on existing Kubernetes cluster and submit jobs using daf…
…t-launcher (#44) * Instructions to install Ray and Daft on an existing Kubernetes cluster, and support BYOC k8s clusters in daft-launcher Added docs for kuberay + daft installation, fixed minor linter issue * Addressed PR comments, command groups, provisioned and byoc instead of aws and k8s * switched to DaftProvider enum * Addressed PR comments * Commented out tests --------- Co-authored-by: Jessie Young <jessie@eventualcomputing.com>
- Loading branch information
1 parent
8fd7a17
commit db3b824
Showing
14 changed files
with
1,508 additions
and
675 deletions.
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
# BYOC (Bring Your Own Cluster) Mode Setup for Daft | ||
|
||
This directory contains guides for setting up Ray and Daft on various Kubernetes environments for BYOC mode: | ||
|
||
- [Local Development](./local.md) - Setting up a local Kubernetes cluster for development | ||
- [Cloud Providers](./cloud.md) - Instructions for EKS, GKE, and AKS setups | ||
- [On-Premises](./on-prem.md) - Guide for on-premises Kubernetes deployments | ||
|
||
## Prerequisites | ||
|
||
Before using `daft-launcher` in BYOC mode with Kubernetes, you must: | ||
1. Have a running Kubernetes cluster (local, cloud-managed, or on-premise) | ||
2. Install and configure Ray on your Kubernetes cluster | ||
3. Install Daft on your cluster | ||
|
||
Please follow the appropriate guide above for your environment. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
# Cloud Provider Kubernetes Setup | ||
|
||
This guide covers using Ray and Daft with managed Kubernetes services from major cloud providers. | ||
|
||
## Prerequisites | ||
|
||
### General Requirements | ||
- `kubectl` installed and configured | ||
- `helm` installed | ||
- A running Kubernetes cluster in one of the following cloud providers: | ||
- Amazon Elastic Kubernetes Service (EKS) | ||
- Google Kubernetes Engine (GKE) | ||
- Azure Kubernetes Service (AKS) | ||
|
||
### Cloud-Specific Requirements | ||
|
||
#### For AWS EKS | ||
- AWS CLI installed and configured | ||
- Access to an existing EKS cluster | ||
- `kubectl` configured for your EKS cluster: | ||
```bash | ||
aws eks update-kubeconfig --name your-cluster-name --region your-region | ||
``` | ||
|
||
#### For Google GKE | ||
- Google Cloud SDK installed | ||
- Access to an existing GKE cluster | ||
- `kubectl` configured for your GKE cluster: | ||
```bash | ||
gcloud container clusters get-credentials your-cluster-name --zone your-zone | ||
``` | ||
|
||
#### For Azure AKS | ||
- Azure CLI installed | ||
- Access to an existing AKS cluster | ||
- `kubectl` configured for your AKS cluster: | ||
```bash | ||
az aks get-credentials --resource-group your-resource-group --name your-cluster-name | ||
``` | ||
|
||
## Installing Ray and Daft | ||
|
||
Once your cloud Kubernetes cluster is running and `kubectl` is configured, follow the [Ray Installation Guide](./ray-installation.md) to: | ||
1. Install KubeRay Operator | ||
2. Deploy Ray cluster | ||
3. Install Daft | ||
4. Set up port forwarding | ||
5. Submit test jobs | ||
|
||
> **Note**: For cloud providers, you'll typically use x86/AMD64 images unless you're specifically using ARM-based instances (like AWS Graviton). |
Oops, something went wrong.