Skip to content

Conversation

@acmiyaguchi
Copy link
Contributor

@acmiyaguchi acmiyaguchi commented Jan 7, 2019

This script runs jobs registered with mozetl.cli using the databricks rest api.

Here are a couple of examples of running a fork of this repo to run jobs. The jobs will show up in the Clusters page. Clicking on Go To Job Run will contain the same id as the one in the logs.

python mozetl-databricks.py \
	--git-path https://github.com/acmiyaguchi/python_mozetl.git \
	--git-branch python3-part2 \
	--python 3 \
	--num-workers 5 \
	--token <TOKEN>  \
	churn \
		--start_date 20181201 \
		--bucket telemetry-test-bucket \
		--period 3 \
		--slack 1 \
		--sample
python mozetl-databricks.py \
	--git-path https://github.com/acmiyaguchi/python_mozetl.git \
	--git-branch python3-part2 \
        --python 3 \
        --num-workers 5 \
	--token <TOKEN> \
	search_clients_daily  \
		--submission_date 20181202 \
		--bucket telemetry-test-bucket \
		--prefix test_search

Copy link
Contributor

@akkomar akkomar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, that's useful!
I'd add one of these example calls somewhere in the repo (either readme or as a comment in the script).

@acmiyaguchi
Copy link
Contributor Author

I'll add the documentation in a follow-up PR, since I should add instructions for the mozetl-submit.sh script too.

@acmiyaguchi acmiyaguchi merged commit e284577 into mozilla:master Jan 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants