Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update doc and sample dag for Quicksight #23653

Merged
merged 1 commit into from
May 12, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -19,11 +19,12 @@
from datetime import datetime

from airflow import DAG
from airflow.models.baseoperator import chain
from airflow.providers.amazon.aws.operators.quicksight import QuickSightCreateIngestionOperator
from airflow.providers.amazon.aws.sensors.quicksight import QuickSightSensor

DATA_SET_ID = os.getenv("DATA_SET_ID", "DemoDataSet_Test")
INGESTION_NO_WAITING_ID = os.getenv("INGESTION_NO_WAITING_ID", "DemoDataSet_Ingestion_No_Waiting_Test")
DATA_SET_ID = os.getenv("DATA_SET_ID", "data-set-id")
INGESTION_ID = os.getenv("INGESTION_ID", "ingestion-id")

with DAG(
dag_id="example_quicksight",
Expand All @@ -36,20 +37,21 @@
# and does not wait for its completion
# [START howto_operator_quicksight_create_ingestion]
quicksight_create_ingestion_no_waiting = QuickSightCreateIngestionOperator(
task_id="quicksight_create_ingestion_no_waiting",
data_set_id=DATA_SET_ID,
ingestion_id=INGESTION_NO_WAITING_ID,
ingestion_id=INGESTION_ID,
wait_for_completion=False,
task_id="sample_quicksight_no_waiting_dag",
)
# [END howto_operator_quicksight_create_ingestion]

# The following task checks the status of the QuickSight SPICE ingestion
# job until it succeeds.
# [START howto_sensor_quicksight]
quicksight_job_status = QuickSightSensor(
task_id="quicksight_job_status",
data_set_id=DATA_SET_ID,
ingestion_id=INGESTION_NO_WAITING_ID,
task_id="check_quicksight_job_status",
ingestion_id=INGESTION_ID,
)
# [END howto_sensor_quicksight]
quicksight_create_ingestion_no_waiting >> quicksight_job_status

chain(quicksight_create_ingestion_no_waiting, quicksight_job_status)
36 changes: 20 additions & 16 deletions docs/apache-airflow-providers-amazon/operators/quicksight.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,43 +15,47 @@
specific language governing permissions and limitations
under the License.

=================
Amazon QuickSight
=================

Amazon QuickSight Operators
========================================
`Amazon QuickSight <https://aws.amazon.com/quicksight/>`__ is a fast business analytics service to build visualizations,
perform ad hoc analysis, and quickly get business insights from your data.
Amazon QuickSight seamlessly discovers AWS data sources, enables organizations to scale to hundreds of thousands of
users, and delivers fast and responsive query performance by using the Amazon QuickSight Super-fast,
Parallel, In-Memory, Calculation Engine (SPICE).

Prerequisite Tasks
------------------

.. include:: _partials/prerequisite_tasks.rst

Overview
--------

Airflow to Amazon QuickSight integration allows users to create and start the SPICE ingestion for dataset.

- :class:`~airflow.providers.amazon.aws.operators.quicksight.QuickSightCreateIngestionOperator`
- :class:`~airflow.providers.amazon.aws.sensor.quicksight.QuickSightSensor`
Operators
---------

.. _howto/operator:QuickSightCreateIngestionOperator:

Amazon QuickSight CreateIngestion Operator
"""""""""""""""""""""""""""""""""""""""""""
Amazon QuickSight create ingestion
==================================

The QuickSightCreateIngestionOperator Creates and starts a new SPICE ingestion for a dataset.
The operator also refreshes existing SPICE datasets
The ``QuickSightCreateIngestionOperator`` creates and starts a new SPICE ingestion for a dataset.
The operator also refreshes existing SPICE datasets.

.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_quicksight.py
:language: python
:dedent: 4
:start-after: [START howto_operator_quicksight_create_ingestion]
:end-before: [END howto_operator_quicksight_create_ingestion]

Sensors
-------

.. _howto/sensor:QuickSightSensor:

Amazon QuickSight Sensor
""""""""""""""""""""""""
Amazon QuickSight ingestion sensor
==================================

The QuickSightSensor wait for Amazon QuickSight CreateIngestion until it reaches a terminal state
The ``QuickSightSensor`` waits for an Amazon QuickSight create ingestion until it reaches a terminal state.

.. exampleinclude:: /../../airflow/providers/amazon/aws/example_dags/example_quicksight.py
:language: python
Expand Down
1 change: 1 addition & 0 deletions docs/spelling_wordlist.txt
Original file line number Diff line number Diff line change
Expand Up @@ -892,6 +892,7 @@ hiveconfs
hivemeta
hiveserver
hmsclient
hoc
homebrew
honoured
hookable
Expand Down