Feature: dashboards without time series #1459

AdityaHegde · 2022-12-16T12:53:12Z

Works off of #1446

Frontend work for this PR wil be done in #1518

…-impl

…-impl-timeseries # Conflicts: # runtime/queries/column_api_benchmark_test.go # runtime/server/queries_timeseries.go

runtime/services/catalog/migrator/metricsviews/metrics_views.go

web-local/src/lib/components/workspace/explore/ExploreContainer.svelte

begelundmuller

Backend looks good (and so does frontend, but not sure I have all context)

* Use RefreshedOn in runtime query cache (#1559) * Remove timestamp required from CTA buttons * Dashboard without time dimension * Remove logs * Add lint fixes * Address review comments * Use single container component * Add grid to measures * Increase max size for measures * Metrics View: Value exclusion should keep nulls (#1535) * exclude with nulls * exclude with nulls * exclude with nulls for timeseries * Add explicit is null check instead of coalesce with UUID * fix failing test * Adress review + Resolve filter code duplication Co-authored-by: egor-ryashin <egor.ryashin@rilldata.com> Co-authored-by: Nishant Bangarwa <nishant.monu51@gmail.com> Co-authored-by: Benjamin Egelund-Müller <b@egelund-muller.com> * Remove redundant import * Fix imports * Remove time params * Connection pool for DuckDB (#1405) * connection pooling for duckdb * close channel * set pool size * fix test * fix stopping of pw * formatting * start mulitple jobs are getting unpaused * conn pool dequeue * fix test * better concurrency test of priority worker * comment * Merging with context affinity * Pass pool size through DSN * Formatting * Added a priority semaphore * Update duckdb driver to use priority semaphore * Maybe fix failing test case * Document priorityqueue.Semaphore * Bind time series to one connection * Use sqlx.Conn + fix hanging errors * Temporarily committing spending benchmark in examples * fix failing test * formatting * Commit js benchmark * Removing benchmarks * duckdb driver fix, tests with pool size > 1, separate meta connection * use conn from pool in migrate * use single conn in migrate * use built in connection pool * fix conn release race condition * fix linter errors * fmt * gofmt fix * gofmt fix * fix tests * upgrade duckdb driver * Meta and OLAP sems; ensuring safe release * Use WithConnection for temporary tables * formatting * Review Co-authored-by: Benjamin Egelund-Müller <b@egelund-muller.com> * Improve handling of time ranges (#1560) * Improve handling of time ranges (maybe) * Run prettier * Update tests * Update web-local/src/lib/temp/time-control-types.ts Co-authored-by: Hamilton Ulmer <hamilton.ulmer@gmail.com> * Make `rill init` Fail When Passed Arguments (Probably By Accident) (#1566) Make `rill init --example <example>` fail instead of quietly using the "default" example project when used instead of `--example=example`. The latter form is required due to a bug in cobra. See #1398 (comment) * Left align measures * Change copy, add tooltips * Remove CTA from metrics no timeseries icon * dyanamic number of columns upto 3 based on measures * Fix nightly builds * Remove old runtime release CI * more from r to f for integer formatting in summary numbers (#1572) * Show error on refresh failure (#1492) * Show error on refresh failure * Improving error display in sources * Fix lint * Fixing navigation blocked during profiling (#1561) * Release notes 0.18 (#1565) * release notes for 0.18 * Update 0.18.md Co-authored-by: Marissa Gorlick <marissagorlick@MacBook-Pro-2.local> * Handle long labels * Increase margin top to 20 * For measures with timeseries align center * Align measures on the left side * Align measures with timecharts * Align measures from the top * Increase gap between measures * use observer in measures container * changes the edit tooltip (#1575) * When a model query is blank, hide the table & inspector (#1568) * adds in a placeholder when a model is blank * ensures preview table is hidden when model is empty * makes a more dramatic yet more subtle style for empty models * Fix S3 Region not recognised (#1583) * changing aws.region to region * fixing ci * Add S3/GCS region to docs (#1581) * Add region to docs As proposed in #1577 * Update project-files.md * Update project-files.md * Check rows.Err() after all relevant queries (#1540) * check rows.Err() after all relevant DB queries * removed extra lines * fixed test cases * requested changes * adding err check for drivers.Result * revert deleted code * Use measure height and prevent overflow * Dispatch trigger for nightlies (#1586) * Testing trigger dynamics * Enable manual nightly trigger * Typo * Improve comments * Glob support for local_file, S3 and GCS (#1578) * initial commit * download plus ingestion support * adding config for limit * linting issues * linting issues * const name declaration * ut fix * config fix * config fix * batch mode fix * read parquet changes * ingestion by folder * review comments * unused code * relative local paths * local invalid path handling * review nit picks * review nit picks * Update error messages Co-authored-by: anshul khandelwal <kanshul@anshuls-MacBook-Pro.local> Co-authored-by: Benjamin Egelund-Müller <b@egelund-muller.com> * Adding gci for import grouping (#1587) * adding gci for import groupin * cleanup requested changes * Run DuckDB tests in CI (#1584) * fix olap test * run duckdb olap test always * Reflow measures on window resize * Fix spinner alignment * reflow on resize * Use max content for measures Co-authored-by: Benjamin Egelund-Müller <b@egelund-muller.com> Co-authored-by: Egor Riashin <egor-ryashin@users.noreply.github.com> Co-authored-by: egor-ryashin <egor.ryashin@rilldata.com> Co-authored-by: Nishant Bangarwa <nishant.monu51@gmail.com> Co-authored-by: Parag Jain <pjain1@apache.org> Co-authored-by: Hamilton Ulmer <hamilton.ulmer@gmail.com> Co-authored-by: Christian G. Warden <cwarden@xerus.org> Co-authored-by: Aditya Hegde <adityahegderocks@gmail.com> Co-authored-by: Marissa Gorlick <magorlick@gmail.com> Co-authored-by: Marissa Gorlick <marissagorlick@MacBook-Pro-2.local> Co-authored-by: rakeshsharma14317 <rakeshsharma14317@gmail.com> Co-authored-by: marcelteraflow <116802182+marcelteraflow@users.noreply.github.com> Co-authored-by: Anshul <12948312+k-anshul@users.noreply.github.com> Co-authored-by: anshul khandelwal <kanshul@anshuls-MacBook-Pro.local>

hamilton · 2023-01-13T05:44:25Z

We probably should have merged this a few weeks ago, but now that this branch has become a full feature branch, we'll have to resolve the Go-related merge conflicts once Aditya's back.

ericpgreen2 · 2023-01-13T19:42:04Z

web-local/package.json

@@ -29,8 +29,8 @@
    "@babel/preset-env": "^7.16.11",
    "@playwright/test": "^1.28.1",
    "@rollup/plugin-typescript": "^8.3.1",
-    "@sveltejs/adapter-static": "^1.0.0-next.41",
-    "@sveltejs/kit": "^1.0.0-next.463",


Was this a conscious decision? It might have some repercussions.

@AdityaHegde would be more aware of these changes.

This switches to using stable versions. We can do it in a separate PR.

web-local/src/lib/components/workspace/explore/time-controls/TimeControls.svelte

web-common/src/features/models/utils/model-is-empty/index.ts

web-local/src/lib/components/workspace/explore/utils.ts

web-local/src/lib/components/workspace/explore/filters/Filters.svelte

ericpgreen2 · 2023-01-13T20:12:41Z

A more general comment: I see a "dependent query" pattern in a handful of places.

Rather than ...

if (condition) {
    query = useQuery(...)
}

... we should use the built-in "dependent query" feature via the enabled parameter. See: https://tanstack.com/query/v4/docs/react/guides/dependent-queries

Here are the files where I see the pattern:

Filters.svelte
DimensionDisplay.svelte
Leaderboard.svelte
LeaderboardDispaly.svelte
MeasuresContainer.svelte
MetricsTimeSeriesCharts.svelte

djbarnwal · 2023-01-16T14:50:11Z

A more general comment: I see a "dependent query" pattern in a handful of places.

The totals, toplist, timeseries and other queries are using the above paradigm. I think it would be better to address those concerns in another isolated PR.

ericpgreen2 · 2023-01-17T06:10:51Z

A more general comment: I see a "dependent query" pattern in a handful of places.

The totals, toplist, timeseries and other queries are using the above paradigm. I think it would be better to address those concerns in another isolated PR.

Sure, let's just try to use the enabled option going forward though.

* queries-caching-impl * queries-caching-impl * caching for time grain query * caching: column numeric histogram * caching: column numeric histogram * caching: table cardinality fix * caching: rug histogram * caching: time range * caching: time range * caching: column cardinality * caching: rollup interval * caching: column with all nulls * caching: code style * caching: rug * caching: code style * caching: ts fix * caching: ts fix * caching: ts fix * caching: code style * caching: timeseries * caching: table columns * caching: metricsview totals * caching: metricsview toplist * caching: metricsview toplist * caching: metricsview timeseries * Making metrics dashboard less strict * adds rudimentary dashboard support for no time series * Removing model path filter (#1488) * Adding support for reading public buckets (#1491) * Refresh when local file changes (#1489) * Refresh when local file changes * Fixing test failure * spaces in timestamp column name (#1490) * spaces in timestamp column name * spaces in timestamp column name * spaces in timestamp column name Co-authored-by: egor-ryashin <egor.ryashin@rilldata.com> * npm install * Temporarily disable docker releases * Setup golangci-lint in rill-developer (#1417) * golangci-lint initial commit * added new line * adding .golangci.yml config * latest version and comment GH action on pull_request * go version update * go version update + prettier * for current branch * typo * run with config file * adding timeout * commenting disable all * default run * setup go changes * go setup changes * lint version latest * Adding golangci.yml file * with config arg * adding goreportcard-action * removed some linters from enable list * fix suggested by golangci linters * updaing GH action args with config and verbose * errors fix by linters * trigger ci check * trigger ci check * trigger ci check * trigger ci check * fixed errcheck issues * on push main branch and pull_request * adding golangci-lint-setup for test * golangci-lint error fix * more fixes * fixed statuscode const etc * fixes after merging with main * PR requested changes * Fixed PR changes and other errors * prettier on go-test.yml * Final changes * fixed PR requested changes * fixed some more changes after merge with main branch * reverted errcheck and gofumpt and fixed the changes * gofmt * updated CONTRIBUTING.md for golint setup * updated GH actions for golint * test action * fixed some more errors from linters * updated GH action and fixed typo * udpated CONTRIBUTING.md for golint instructions * changes in GH actions for golint * Release notes 0.17 (#1497) * release notes for 0.17 * fixing static image in docusaurus * Maybe fixes builds Co-authored-by: Marissa Gorlick <marissagorlick@MacBook-Pro-2.lan> Co-authored-by: Benjamin Egelund-Müller <b@egelund-muller.com> * Update 0.17.md * Fix suppress tooltip not being reactive (#1493) * [Dashboard] Fix jarring rearrange when include/exclude is toggled from filter pills (#1494) * Fixing misused older types * Fixing the jarring rearrange on toggling include/exclude * Update help links to new docs structure (#1499) Co-authored-by: Marissa Gorlick <marissagorlick@MacBook-Pro-2.lan> * Move design system to `web-common` (#1486) * Move files from `web-local` to `web-common` * Move files out of design system and into app * Move utils into `web-common` * Add typescript & eslint config files to `web-common` * Update imports (design system) * Delete blank file * Update imports (application) * Move duckdb data types to `web-common` * Fix import of chip component * Fix two missing imports * Edit path in the menu component's README * Add `svelte` as a `web-common` dependency * Remove old paths from typscript and vite configs * Fix imports in `/dev` routes * Add back imports to fix lint * Add `web-common` to tailwind config * Move to one top-level `eslint` file * Move `data-types` into `web-common/components` * Move `data-graphic` into `web-common/components` * Bugfix: show nav border * Add more data file extensions to `.prettierignore` * Fix prettier warning * Publish nightly docker image (#1500) * golint improvements (#1502) * skip proto files and only-new-issues is set true * testing with adding few errors * revert errors and commented dupl * adding dupl * dupl with thresold setting * removed dupl linter * removing only-new-issues * Improved dimension validations * Addressing PR comments * Enable dashboards with no time dimension (#1544) * Use RefreshedOn in runtime query cache (#1559) * Remove timestamp required from CTA buttons * Dashboard without time dimension * Remove logs * Add lint fixes * Address review comments * Use single container component * Add grid to measures * Increase max size for measures * Metrics View: Value exclusion should keep nulls (#1535) * exclude with nulls * exclude with nulls * exclude with nulls for timeseries * Add explicit is null check instead of coalesce with UUID * fix failing test * Adress review + Resolve filter code duplication Co-authored-by: egor-ryashin <egor.ryashin@rilldata.com> Co-authored-by: Nishant Bangarwa <nishant.monu51@gmail.com> Co-authored-by: Benjamin Egelund-Müller <b@egelund-muller.com> * Remove redundant import * Fix imports * Remove time params * Connection pool for DuckDB (#1405) * connection pooling for duckdb * close channel * set pool size * fix test * fix stopping of pw * formatting * start mulitple jobs are getting unpaused * conn pool dequeue * fix test * better concurrency test of priority worker * comment * Merging with context affinity * Pass pool size through DSN * Formatting * Added a priority semaphore * Update duckdb driver to use priority semaphore * Maybe fix failing test case * Document priorityqueue.Semaphore * Bind time series to one connection * Use sqlx.Conn + fix hanging errors * Temporarily committing spending benchmark in examples * fix failing test * formatting * Commit js benchmark * Removing benchmarks * duckdb driver fix, tests with pool size > 1, separate meta connection * use conn from pool in migrate * use single conn in migrate * use built in connection pool * fix conn release race condition * fix linter errors * fmt * gofmt fix * gofmt fix * fix tests * upgrade duckdb driver * Meta and OLAP sems; ensuring safe release * Use WithConnection for temporary tables * formatting * Review Co-authored-by: Benjamin Egelund-Müller <b@egelund-muller.com> * Improve handling of time ranges (#1560) * Improve handling of time ranges (maybe) * Run prettier * Update tests * Update web-local/src/lib/temp/time-control-types.ts Co-authored-by: Hamilton Ulmer <hamilton.ulmer@gmail.com> * Make `rill init` Fail When Passed Arguments (Probably By Accident) (#1566) Make `rill init --example <example>` fail instead of quietly using the "default" example project when used instead of `--example=example`. The latter form is required due to a bug in cobra. See #1398 (comment) * Left align measures * Change copy, add tooltips * Remove CTA from metrics no timeseries icon * dyanamic number of columns upto 3 based on measures * Fix nightly builds * Remove old runtime release CI * more from r to f for integer formatting in summary numbers (#1572) * Show error on refresh failure (#1492) * Show error on refresh failure * Improving error display in sources * Fix lint * Fixing navigation blocked during profiling (#1561) * Release notes 0.18 (#1565) * release notes for 0.18 * Update 0.18.md Co-authored-by: Marissa Gorlick <marissagorlick@MacBook-Pro-2.local> * Handle long labels * Increase margin top to 20 * For measures with timeseries align center * Align measures on the left side * Align measures with timecharts * Align measures from the top * Increase gap between measures * use observer in measures container * changes the edit tooltip (#1575) * When a model query is blank, hide the table & inspector (#1568) * adds in a placeholder when a model is blank * ensures preview table is hidden when model is empty * makes a more dramatic yet more subtle style for empty models * Fix S3 Region not recognised (#1583) * changing aws.region to region * fixing ci * Add S3/GCS region to docs (#1581) * Add region to docs As proposed in #1577 * Update project-files.md * Update project-files.md * Check rows.Err() after all relevant queries (#1540) * check rows.Err() after all relevant DB queries * removed extra lines * fixed test cases * requested changes * adding err check for drivers.Result * revert deleted code * Use measure height and prevent overflow * Dispatch trigger for nightlies (#1586) * Testing trigger dynamics * Enable manual nightly trigger * Typo * Improve comments * Glob support for local_file, S3 and GCS (#1578) * initial commit * download plus ingestion support * adding config for limit * linting issues * linting issues * const name declaration * ut fix * config fix * config fix * batch mode fix * read parquet changes * ingestion by folder * review comments * unused code * relative local paths * local invalid path handling * review nit picks * review nit picks * Update error messages Co-authored-by: anshul khandelwal <kanshul@anshuls-MacBook-Pro.local> Co-authored-by: Benjamin Egelund-Müller <b@egelund-muller.com> * Adding gci for import grouping (#1587) * adding gci for import groupin * cleanup requested changes * Run DuckDB tests in CI (#1584) * fix olap test * run duckdb olap test always * Reflow measures on window resize * Fix spinner alignment * reflow on resize * Use max content for measures Co-authored-by: Benjamin Egelund-Müller <b@egelund-muller.com> Co-authored-by: Egor Riashin <egor-ryashin@users.noreply.github.com> Co-authored-by: egor-ryashin <egor.ryashin@rilldata.com> Co-authored-by: Nishant Bangarwa <nishant.monu51@gmail.com> Co-authored-by: Parag Jain <pjain1@apache.org> Co-authored-by: Hamilton Ulmer <hamilton.ulmer@gmail.com> Co-authored-by: Christian G. Warden <cwarden@xerus.org> Co-authored-by: Aditya Hegde <adityahegderocks@gmail.com> Co-authored-by: Marissa Gorlick <magorlick@gmail.com> Co-authored-by: Marissa Gorlick <marissagorlick@MacBook-Pro-2.local> Co-authored-by: rakeshsharma14317 <rakeshsharma14317@gmail.com> Co-authored-by: marcelteraflow <116802182+marcelteraflow@users.noreply.github.com> Co-authored-by: Anshul <12948312+k-anshul@users.noreply.github.com> Co-authored-by: anshul khandelwal <kanshul@anshuls-MacBook-Pro.local> * Check for zero heights * address PR comments * Increase column gap and fix node bug Co-authored-by: egor-ryashin <egor.ryashin@rilldata.com> Co-authored-by: Hamilton Ulmer <hamilton.ulmer@gmail.com> Co-authored-by: Egor Riashin <egor-ryashin@users.noreply.github.com> Co-authored-by: Benjamin Egelund-Müller <b@egelund-muller.com> Co-authored-by: rakeshsharma14317 <rakeshsharma14317@gmail.com> Co-authored-by: Marissa Gorlick <magorlick@gmail.com> Co-authored-by: Marissa Gorlick <marissagorlick@MacBook-Pro-2.lan> Co-authored-by: Eric P Green <ericpgreen2@gmail.com> Co-authored-by: Himadri Singh <himadri.singh@gmail.com> Co-authored-by: Dhiraj Barnwal <sumankumaribarnwal@gmail.com> Co-authored-by: Nishant Bangarwa <nishant.monu51@gmail.com> Co-authored-by: Parag Jain <pjain1@apache.org> Co-authored-by: Christian G. Warden <cwarden@xerus.org> Co-authored-by: Marissa Gorlick <marissagorlick@MacBook-Pro-2.local> Co-authored-by: marcelteraflow <116802182+marcelteraflow@users.noreply.github.com> Co-authored-by: Anshul <12948312+k-anshul@users.noreply.github.com> Co-authored-by: anshul khandelwal <kanshul@anshuls-MacBook-Pro.local>

egor-ryashin and others added 30 commits December 6, 2022 18:31

queries-caching-impl

7f1700b

queries-caching-impl

c133a22

caching for time grain query

3126c35

caching: column numeric histogram

eda43c1

caching: column numeric histogram

c680489

Merge remote-tracking branch 'origin/main' into query-cache-interface…

235356f

…-impl

caching: table cardinality fix

7d095d3

caching: rug histogram

4d6a455

caching: time range

3895bf1

caching: time range

c9b94ac

caching: column cardinality

5d93116

caching: rollup interval

b4d6095

caching: column with all nulls

0670dec

caching: code style

d34946b

caching: rug

7822a6a

caching: code style

5f45640

caching: ts fix

8923aab

Merge remote-tracking branch 'origin/main' into query-cache-interface…

0d8f659

…-impl

caching: ts fix

5c94aa9

caching: ts fix

fd3d586

caching: code style

3750567

Merge remote-tracking branch 'origin/main' into query-cache-interface…

c69f7fd

…-impl

caching: timeseries

1f0f02f

Merge remote-tracking branch 'origin/main' into query-cache-interface…

954d9ad

…-impl-timeseries # Conflicts: # runtime/queries/column_api_benchmark_test.go # runtime/server/queries_timeseries.go

caching: table columns

7c33d50

caching: metricsview totals

3387238

caching: metricsview toplist

b8c3bad

caching: metricsview toplist

0e89a64

caching: metricsview timeseries

40c7c27

Making metrics dashboard less strict

7849bf7

Improved dimension validations

34ffbb8

begelundmuller requested changes Dec 23, 2022

View reviewed changes

runtime/services/catalog/migrator/metricsviews/metrics_views.go Outdated Show resolved Hide resolved

web-local/src/lib/components/workspace/explore/ExploreContainer.svelte Outdated Show resolved Hide resolved

Addressing PR comments

f0b952d

AdityaHegde requested a review from begelundmuller December 23, 2022 10:29

begelundmuller approved these changes Dec 23, 2022

View reviewed changes

Merge branch 'main' into less-strict-metrics-view

e5f9f01

hamilton assigned AdityaHegde Jan 4, 2023

hamilton requested review from ericpgreen2 and removed request for hamilton January 13, 2023 05:43

hamilton changed the title ~~Less strict metrics view~~ Feature: dashboards without time series Jan 13, 2023

ericpgreen2 reviewed Jan 13, 2023

View reviewed changes

web-local/src/lib/components/workspace/explore/time-controls/TimeControls.svelte Outdated Show resolved Hide resolved

ericpgreen2 reviewed Jan 13, 2023

View reviewed changes

web-common/src/features/models/utils/model-is-empty/index.ts Outdated Show resolved Hide resolved

ericpgreen2 reviewed Jan 13, 2023

View reviewed changes

web-local/src/lib/components/workspace/explore/utils.ts Outdated Show resolved Hide resolved

ericpgreen2 reviewed Jan 13, 2023

View reviewed changes

web-local/src/lib/components/workspace/explore/filters/Filters.svelte Outdated Show resolved Hide resolved

nishantmonu51 added the blocker A release blocker issue that should be resolved before a new release label Jan 16, 2023

djbarnwal added 3 commits January 16, 2023 22:08

Check for zero heights

025db73

address PR comments

3675f54

Increase column gap and fix node bug

d2e66a0

ericpgreen2 approved these changes Jan 17, 2023

View reviewed changes

Merge branch 'main' into less-strict-metrics-view

3c701ba

djbarnwal merged commit e7b0a06 into main Jan 17, 2023

djbarnwal deleted the less-strict-metrics-view branch January 17, 2023 07:09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature: dashboards without time series #1459

Feature: dashboards without time series #1459

AdityaHegde commented Dec 16, 2022 •

edited by hamilton

Loading

begelundmuller left a comment

hamilton commented Jan 13, 2023

ericpgreen2 Jan 13, 2023

djbarnwal Jan 16, 2023

AdityaHegde Jan 17, 2023

ericpgreen2 commented Jan 13, 2023

djbarnwal commented Jan 16, 2023

ericpgreen2 commented Jan 17, 2023

Feature: dashboards without time series #1459

Feature: dashboards without time series #1459

Conversation

AdityaHegde commented Dec 16, 2022 • edited by hamilton Loading

begelundmuller left a comment

Choose a reason for hiding this comment

hamilton commented Jan 13, 2023

ericpgreen2 Jan 13, 2023

Choose a reason for hiding this comment

djbarnwal Jan 16, 2023

Choose a reason for hiding this comment

AdityaHegde Jan 17, 2023

Choose a reason for hiding this comment

ericpgreen2 commented Jan 13, 2023

djbarnwal commented Jan 16, 2023

ericpgreen2 commented Jan 17, 2023

AdityaHegde commented Dec 16, 2022 •

edited by hamilton

Loading