Skip to content

Commit

Permalink
feat: Doc updates, Issue and PR templates are added (#1970)
Browse files Browse the repository at this point in the history
* Issue templates are added to the repo

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* removed Google CLA requirement

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* Updated ghcr.io registry references in the workflow

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* Added Pull request template

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* Updated Main README.md with Kubeflow header and new Slack channel link

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* Removed the License header and it will be replaced with Kubeflow guidelines

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* Revert "Removed the License header and it will be replaced with Kubeflow guidelines"

This reverts commit b892f5c.

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* Readme line revert for gcp docs

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* pre-commit run -a updates

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* fixed the helm lint issue by upgrading the Helm chart version

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* fixed docker image tag and updated chart docs (#1969)

Signed-off-by: Andrew Chubatiuk <andrew.chubatiuk@gmail.com>
Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

* rebase from master

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>

---------

Signed-off-by: Vara Bonthu <vara.bonthu@gmail.com>
Signed-off-by: Andrew Chubatiuk <andrew.chubatiuk@gmail.com>
Co-authored-by: Andrii Chubatiuk <andrew.chubatiuk@gmail.com>
  • Loading branch information
vara-bonthu and AndrewChubatiuk authored Apr 14, 2024
1 parent 39b79d7 commit 69e9841
Show file tree
Hide file tree
Showing 7 changed files with 163 additions and 41 deletions.
46 changes: 46 additions & 0 deletions .github/ISSUE_TEMPLATE/bug_report.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
---
name: Bug report
about: Create a report to help us improve
title: '[BUG] Brief description of the issue'
labels: bug
---

## Description
Please provide a clear and concise description of the issue you are encountering, and a reproduction of your configuration.

If your request is for a new feature, please use the `Feature request` template.

- [ ] ✋ I have searched the open/closed issues and my issue is not listed.

## Reproduction Code [Required]

<!-- REQUIRED -->

Steps to reproduce the behavior:


## Expected behavior

<!-- A clear and concise description of what you expected to happen -->

## Actual behavior

<!-- A clear and concise description of what actually happened -->

### Terminal Output Screenshot(s)

<!-- Optional but helpful -->


## Environment & Versions

- Spark Operator App version:
- Helm Chart Version:
- Kubernetes Version:
- Apache Spark version:

## Additional context

<!-- Add any other context about the problem here -->


32 changes: 32 additions & 0 deletions .github/ISSUE_TEMPLATE/feature_request.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
name: Feature request
about: Suggest an idea for this project
title: '[FEATURE] Brief description of the feature'
labels: enhancement
---

<!--- Please keep this note for the community --->

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

<!--- Thank you for keeping this note for the community --->

#### What is the outcome that you are trying to reach?

<!-- A clear and concise description of what the problem is. -->

#### Describe the solution you would like

<!-- A clear and concise description of what you want to happen. -->

#### Describe alternatives you have considered

<!-- A clear and concise description of any alternative solutions or features you've considered. -->

#### Additional context

<!-- Add any other context or screenshots about the feature request here. -->
20 changes: 20 additions & 0 deletions .github/ISSUE_TEMPLATE/question.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
---
name: Question
about: I have a Question
title: '[QUESTION] Brief description of the Question'
labels: question
---

- [ ] ✋ I have searched the open/closed issues and my issue is not listed.

#### Please describe your question here

<!-- Provide as much information as possible to explain your question -->

#### Provide a link to the example/module related to the question

<!-- Please provide the link to the example related to this question from this repo -->

#### Additional context

<!-- Add any other context or screenshots about the question here -->
38 changes: 38 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
### 🛑 Important:
Please open an issue to discuss significant work before you start. We appreciate your contributions and don't want your efforts to go to waste!

For guidelines on how to contribute, please review the [CONTRIBUTING.md](CONTRIBUTING.md) document.

## Purpose of this PR
Provide a clear and concise description of the changes. Explain the motivation behind these changes and link to relevant issues or discussions.

**Proposed changes:**
- <Change 1>
- <Change 2>
- <Change 3>

## Change Category
Indicate the type of change by marking the applicable boxes:

- [ ] Bugfix (non-breaking change which fixes an issue)
- [ ] Feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that could affect existing functionality)
- [ ] Documentation update

### Rationale

<!-- Provide reasoning for the changes if not already covered in the description above. -->


## Checklist
Before submitting your PR, please review the following:

- [ ] I have conducted a self-review of my own code.
- [ ] I have updated documentation accordingly.
- [ ] I have added tests that prove my changes are effective or that my feature works.
- [ ] Existing unit tests pass locally with my changes.

### Additional Notes

<!-- Include any additional notes or context that could be helpful for the reviewers here. -->

10 changes: 5 additions & 5 deletions .github/workflows/main.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ jobs:

- name: Build Spark-Operator Docker Image
run: |
docker build -t gcr.io/spark-operator/spark-operator:latest .
docker build -t ghcr.io/kubeflow/spark-operator:latest .
- name: Check changes in resources used in docker file
run: |
Expand All @@ -89,7 +89,7 @@ jobs:
if ! git diff --quiet origin/master -- $resource; then
## And the appVersion hasn't been updated
if ! git diff origin/master -- charts/spark-operator-chart/Chart.yaml | grep +appVersion; then
echo "resource used in gcr.io/spark-operator/spark-operator has changed in $resource, need to update the appVersion in charts/spark-operator-chart/Chart.yaml"
echo "resource used in ghcr.io/kubeflow/spark-operator has changed in $resource, need to update the appVersion in charts/spark-operator-chart/Chart.yaml"
git diff origin/master -- $resource;
echo "failing the build... " && false
fi
Expand Down Expand Up @@ -174,12 +174,12 @@ jobs:

- name: Build local spark-operator docker image for minikube testing
run: |
docker build -t gcr.io/kubeflow/spark-operator:local .
minikube image load gcr.io/kubeflow/spark-operator:local
docker build -t ghcr.io/kubeflow/spark-operator:local .
minikube image load ghcr.io/kubeflow/spark-operator:local
# The integration tests are currently broken see: https://github.com/kubeflow/spark-operator/issues/1416
# - name: Run chart-testing (integration test)
# run: make integation-test
# run: make integration-test

- name: Setup tmate session
if: failure()
Expand Down
12 changes: 0 additions & 12 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,18 +3,6 @@
We'd love to accept your patches and contributions to this project. There are
just a few small guidelines you need to follow.

## Contributor License Agreement

Contributions to this project must be accompanied by a Contributor License
Agreement. You (or your employer) retain the copyright to your contribution;
this simply gives us permission to use and redistribute your contributions as
part of the project. Head over to <https://cla.developers.google.com/> to see
your current agreements on file or to sign a new one.

You generally only need to submit a CLA once, so if you've already submitted one
(even if it was for a different project), you probably don't need to do it
again.

## Code reviews

All submissions, including submissions by project members, require review. We
Expand Down
46 changes: 22 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,24 @@
# Kubeflow Spark Operator
[![Go Report Card](https://goreportcard.com/badge/github.com/kubeflow/spark-operator)](https://goreportcard.com/report/github.com/kubeflow/spark-operator)

**This is not an officially supported Google product.**
## Overview
The Kubernetes Operator for Apache Spark aims to make specifying and running [Spark](https://github.com/apache/spark) applications as easy and idiomatic as running other workloads on Kubernetes. It uses
[Kubernetes custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
for specifying, running, and surfacing status of Spark applications. For a complete reference of the custom resource definitions, please refer to the [API Definition](docs/api-docs.md). For details on its design, please refer to the [design doc](docs/design.md). It requires Spark 2.3 and above that supports Kubernetes as a native scheduler backend.

## Community
The Kubernetes Operator for Apache Spark currently supports the following list of features:

* Join our [Slack](https://kubernetes.slack.com/messages/CALBDHMTL) channel on [Kubernetes on Slack](https://slack.k8s.io/).
* Check out [who is using the Kubernetes Operator for Apache Spark](docs/who-is-using.md).
* Supports Spark 2.3 and up.
* Enables declarative application specification and management of applications through custom resources.
* Automatically runs `spark-submit` on behalf of users for each `SparkApplication` eligible for submission.
* Provides native [cron](https://en.wikipedia.org/wiki/Cron) support for running scheduled applications.
* Supports customization of Spark pods beyond what Spark natively is able to do through the mutating admission webhook, e.g., mounting ConfigMaps and volumes, and setting pod affinity/anti-affinity.
* Supports automatic application re-submission for updated `SparkApplication` objects with updated specification.
* Supports automatic application restart with a configurable restart policy.
* Supports automatic retries of failed submissions with optional linear back-off.
* Supports mounting local Hadoop configuration as a Kubernetes ConfigMap automatically via `sparkctl`.
* Supports automatically staging local application dependencies to Google Cloud Storage (GCS) via `sparkctl`.
* Supports collecting and exporting application-level metrics and driver/executor metrics to Prometheus.

## Project Status

Expand Down Expand Up @@ -72,26 +85,11 @@ If you are running the Kubernetes Operator for Apache Spark on Google Kubernetes

For more information, check the [Design](docs/design.md), [API Specification](docs/api-docs.md) and detailed [User Guide](docs/user-guide.md).

## Overview

The Kubernetes Operator for Apache Spark aims to make specifying and running [Spark](https://github.com/apache/spark) applications as easy and idiomatic as running other workloads on Kubernetes. It uses
[Kubernetes custom resources](https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/)
for specifying, running, and surfacing status of Spark applications. For a complete reference of the custom resource definitions, please refer to the [API Definition](docs/api-docs.md). For details on its design, please refer to the [design doc](docs/design.md). It requires Spark 2.3 and above that supports Kubernetes as a native scheduler backend.

The Kubernetes Operator for Apache Spark currently supports the following list of features:

* Supports Spark 2.3 and up.
* Enables declarative application specification and management of applications through custom resources.
* Automatically runs `spark-submit` on behalf of users for each `SparkApplication` eligible for submission.
* Provides native [cron](https://en.wikipedia.org/wiki/Cron) support for running scheduled applications.
* Supports customization of Spark pods beyond what Spark natively is able to do through the mutating admission webhook, e.g., mounting ConfigMaps and volumes, and setting pod affinity/anti-affinity.
* Supports automatic application re-submission for updated `SparkApplication` objects with updated specification.
* Supports automatic application restart with a configurable restart policy.
* Supports automatic retries of failed submissions with optional linear back-off.
* Supports mounting local Hadoop configuration as a Kubernetes ConfigMap automatically via `sparkctl`.
* Supports automatically staging local application dependencies to Google Cloud Storage (GCS) via `sparkctl`.
* Supports collecting and exporting application-level metrics and driver/executor metrics to Prometheus.

## Contributing

Please check [CONTRIBUTING.md](CONTRIBUTING.md) and the [Developer Guide](docs/developer-guide.md) out.

## Community

* Join our [Kubeflow Slack Channel](https://kubeflow.slack.com/archives/C06627U3XU3)
* Check out [who is using the Kubernetes Operator for Apache Spark](docs/who-is-using.md).

0 comments on commit 69e9841

Please sign in to comment.