Skip to content

Commit

Permalink
Fixed using TODO in place of variable. Updated README with disclaimer…
Browse files Browse the repository at this point in the history
… of 1 billing account
  • Loading branch information
kardiff18 committed Jul 21, 2019
1 parent 9167697 commit e8640d1
Show file tree
Hide file tree
Showing 2 changed files with 30 additions and 22 deletions.
13 changes: 5 additions & 8 deletions tools/kunskap/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,16 +18,13 @@ The solution adjusts each project's cost by generating new line items in the out
<h2>Installation/Set-up</h2>
This project assumes that you already have project set up with billing data exported to BigQuery. Note the billing project id, dataset ids, and the table names, as you will need these later on when configuring the Cloud Function source code.

This project also assumes that you only have <b>one</b> billing account for your GCP organization.

<h3>Install/Configure the gcloud command line tool:</h3>

1. [Install and initialize the Cloud SDK](https://cloud.google.com/sdk/docs/how-to)

2. In a terminal window, enter the following to add the gcloud components for beta products
````
gcloud components install beta
````

3. Update all components:
2. Update all components:
````
gcloud components update
````
Expand Down Expand Up @@ -142,11 +139,11 @@ where [FUNCTION_NAME] is the name that you want to give the function and [TOPIC_
2. In the window, enter:

````
gcloud beta scheduler jobs create pubsub [JOB] --schedule [SCHEDULE] --topic [TOPIC_NAME] --message-body [MESSAGE_BODY]
gcloud scheduler jobs create pubsub [JOB] --schedule [SCHEDULE] --topic [TOPIC_NAME] --message-body [MESSAGE_BODY]
````
where [JOB] is a unique name for a job, [SCHEDULE] is the frequency for the job in UNIX cron, such as "0 */12 * * *" to run every 12hours, [TOPIC_NAME] is the name of the topic created in the step above when you deployed the Cloud Function, and [MESSAGE_BODY] is any string. An example command would be:
````
gcloud beta scheduler jobs create pubsub daily_job --schedule "0 */12 * * *" --topic cron-topic --message-body "bi-daily job"
gcloud scheduler jobs create pubsub daily_job --schedule "0 */12 * * *" --topic cron-topic --message-body "bi-daily job"
````

<h3>Run the job:</h3>
Expand Down
39 changes: 25 additions & 14 deletions tools/kunskap/cud_sud_attribution_query.sql
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,7 @@ CREATE TEMP FUNCTION
service.id AS service_id,
service.description AS service_description,
project.id AS project_id,
project.name AS project_name,
labels_1.value AS label_1_value,
labels_2.value AS label_2_value,
usage.unit AS unit,
Expand All @@ -83,7 +84,8 @@ CREATE TEMP FUNCTION
WHERE
service.description = "Compute Engine"
AND (
FALSE OR (LOWER(sku.description) LIKE "%instance%"
FALSE
OR (LOWER(sku.description) LIKE "%instance%"
OR LOWER(sku.description) LIKE "% intel %")
OR LOWER(sku.description) LIKE "%memory optimized core%"
OR LOWER(sku.description) LIKE "%memory optimized ram%"
Expand Down Expand Up @@ -195,6 +197,7 @@ CREATE TEMP FUNCTION
unit_type,
unit,
project_id,
project_name,
label_1_value,
label_2_value,
SUM(usage_amount) AS usage_amount,
Expand All @@ -220,7 +223,8 @@ CREATE TEMP FUNCTION
9,
10,
11,
14)
12,
15)
UNION ALL (
-- Second query pulls out CUD and SUD Credit usage and cost. This is done in a separate
-- SELECT and unioned because if we unnest the repeated field for credit types, we can
Expand All @@ -235,6 +239,7 @@ CREATE TEMP FUNCTION
unit_type,
unit,
project_id,
project_name,
label_1_value,
label_2_value,
SUM(usage_amount) AS usage_amount,
Expand All @@ -255,6 +260,7 @@ CREATE TEMP FUNCTION
unit_type,
unit,
project_id,
project_name,
label_1_value,
label_2_value,
unit_price,
Expand Down Expand Up @@ -300,7 +306,8 @@ CREATE TEMP FUNCTION
10,
11,
12,
15)
13,
16)
GROUP BY
1,
2,
Expand All @@ -313,7 +320,8 @@ CREATE TEMP FUNCTION
9,
10,
11,
14) ),
12,
15) ),
-- project_credit_breakout sums usage amount and cost
-- across the cost organization schema of interest: labels within projects
project_label_credit_breakout AS (
Expand All @@ -323,6 +331,7 @@ CREATE TEMP FUNCTION
service_description,
region,
project_id,
project_name,
label_1_value,
label_2_value,
cud_type,
Expand Down Expand Up @@ -364,7 +373,8 @@ CREATE TEMP FUNCTION
7,
8,
9,
10),
10,
11),
-- BA_credit_breakout sums usage amount and cost
-- across the entire Billing Account within each unique CUD scope <location, unit_type, cud_type>
-- so that we know what the total cost is that we need to attribute across each project/label
Expand Down Expand Up @@ -413,6 +423,7 @@ CREATE TEMP FUNCTION
p.unit_type,
p.cud_type,
p.project_id,
p.project_name,
p.label_1_value,
p.label_2_value,
BA_commitment_usage_amount,
Expand Down Expand Up @@ -527,7 +538,7 @@ CREATE TEMP FUNCTION
TIMESTAMP(usage_date) AS usage_start_time,
TIMESTAMP_ADD(TIMESTAMP(usage_date), INTERVAL ((3600*23)+3599) SECOND) AS usage_end_time,
STRUCT ( project_id AS id,
"" AS name,
project_name AS name,
ARRAY<STRUCT<key STRING,
value STRING>> [] AS labels,
"" AS ancestry_numbers) AS project,
Expand All @@ -544,9 +555,9 @@ CREATE TEMP FUNCTION
"USD" AS currency,
1.0 AS currency_conversion_rate,
STRUCT ( 0.0 AS amount,
"TODO" AS unit,
IF(LOWER(unit_type) LIKE "ram", "byte-seconds", "gibibyte hour") AS unit,
0.0 AS amount_in_pricing_units,
"TODO" AS pricing_unit ) AS usage,
IF(LOWER(unit_type) LIKE "ram", "seconds", "hour") AS pricing_unit ) AS usage,
ARRAY<STRUCT<name STRING,
amount FLOAT64>> [] AS credits,
STRUCT ( FORMAT_DATE("%Y%m", usage_date) AS month) AS invoice,
Expand All @@ -570,7 +581,7 @@ CREATE TEMP FUNCTION
TIMESTAMP(usage_date) AS usage_start_time,
TIMESTAMP_ADD(TIMESTAMP(usage_date), INTERVAL ((3600*23)+3599) SECOND) AS usage_end_time,
STRUCT ( project_id AS id,
"" AS name,
project_name AS name,
ARRAY<STRUCT<key STRING,
value STRING>> [] AS labels,
"" AS ancestry_numbers) AS project,
Expand All @@ -587,9 +598,9 @@ CREATE TEMP FUNCTION
"USD" AS currency,
1.0 AS currency_conversion_rate,
STRUCT ( 0.0 AS amount,
"TODO" AS unit,
IF(LOWER(unit_type) LIKE "ram", "byte-seconds", "gibibyte hour") AS unit,
0.0 AS amount_in_pricing_units,
"TODO" AS pricing_unit ) AS usage,
IF(LOWER(unit_type) LIKE "ram", "seconds", "hour") AS pricing_unit ) AS usage,
ARRAY<STRUCT<name STRING,
amount FLOAT64>> [(IF(LOWER(unit_type) LIKE "ram",
"Committed Usage Discount: RAM",
Expand All @@ -612,7 +623,7 @@ CREATE TEMP FUNCTION
TIMESTAMP(usage_date) AS usage_start_time,
TIMESTAMP_ADD(TIMESTAMP(usage_date), INTERVAL ((3600*23)+3599) SECOND) AS usage_end_time,
STRUCT ( project_id AS id,
"" AS name,
project_name AS name,
ARRAY<STRUCT<key STRING,
value STRING>> [] AS labels,
"" AS ancestry_numbers)
Expand All @@ -630,9 +641,9 @@ CREATE TEMP FUNCTION
"USD" AS currency,
1.0 AS currency_conversion_rate,
STRUCT ( 0.0 AS amount,
"TODO" AS unit,
IF(LOWER(unit_type) LIKE "ram", "byte-seconds", "gibibyte hour") AS unit,
0.0 AS amount_in_pricing_units,
"TODO" AS pricing_unit ) AS usage,
IF(LOWER(unit_type) LIKE "ram", "seconds", "hour") AS pricing_unit ) AS usage,
ARRAY<STRUCT<name STRING,
amount FLOAT64>> [("Sustained Usage Discount",
P_alloc_sud_credit_cost)] AS credits,
Expand Down

0 comments on commit e8640d1

Please sign in to comment.