Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for defining filters on measures #3624

Merged
merged 26 commits into from
Dec 18, 2023
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
2f56587
Initial commit
AdityaHegde Dec 5, 2023
102ac29
Using enum for expression and adding to metricsview_aggregation
AdityaHegde Dec 6, 2023
ccfeb0e
Merge branch 'main' into adityahegde/filters-on-measures
AdityaHegde Dec 7, 2023
d439c54
Comparison in measure filters
AdityaHegde Dec 7, 2023
655ab2e
Adding to metricsview toplist
AdityaHegde Dec 7, 2023
4808486
Updated proto spec
AdityaHegde Dec 11, 2023
74bddbd
Add generic expression syntax
AdityaHegde Dec 12, 2023
3bc151a
Final proto draft
AdityaHegde Dec 13, 2023
d0d5228
Integrate and test where filter
AdityaHegde Dec 13, 2023
0f9c159
Integrate where clause everywhere
AdityaHegde Dec 13, 2023
171a7f2
Merge branch 'main' into adityahegde/filters-on-measures
AdityaHegde Dec 13, 2023
9d9b016
Adding some tests for having clauses
AdityaHegde Dec 13, 2023
aa1b379
Add backwards compatibility for filter
AdityaHegde Dec 14, 2023
e9ff93b
fix lint
AdityaHegde Dec 14, 2023
a828601
Adding alias support in metricsview_comparison_toplist
AdityaHegde Dec 14, 2023
5a7526a
Apply sum() only when having is enabled
AdityaHegde Dec 14, 2023
38db0ff
Fix lint
AdityaHegde Dec 14, 2023
0ddcba5
PR comments
AdityaHegde Dec 15, 2023
f02b422
Move filter helpers to pkg/expressionpb
AdityaHegde Dec 15, 2023
a4086af
refactor filter builder to not use map
AdityaHegde Dec 15, 2023
1e49865
Backwards compatibility for older sort types
AdityaHegde Dec 15, 2023
0d02580
PR comments
AdityaHegde Dec 15, 2023
025a62d
Testing on druid and fixes
AdityaHegde Dec 18, 2023
b5754b3
Merge branch 'main' into adityahegde/filters-on-measures
AdityaHegde Dec 18, 2023
d1186d1
Using outer query
AdityaHegde Dec 18, 2023
1832ebf
Removing unused testdata
AdityaHegde Dec 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Apply sum() only when having is enabled
  • Loading branch information
AdityaHegde committed Dec 14, 2023
commit 5a7526a5b362b35b811bead31b5b739ee071ab32
31 changes: 20 additions & 11 deletions runtime/queries/metricsview_comparison_toplist.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,14 +6,14 @@
"fmt"
"io"
"strings"
// Load IANA time zone data

Check failure on line 9 in runtime/queries/metricsview_comparison_toplist.go

View workflow job for this annotation

GitHub Actions / lint

File is not `gci`-ed with --skip-generated -s standard -s default -s blank -s dot --custom-order (gci)
_ "time/tzdata"

runtimev1 "github.com/rilldata/rill/proto/gen/rill/runtime/v1"
"github.com/rilldata/rill/runtime"
"github.com/rilldata/rill/runtime/drivers"
"github.com/rilldata/rill/runtime/pkg/pbutil"
"google.golang.org/protobuf/types/known/structpb"

Check failure on line 16 in runtime/queries/metricsview_comparison_toplist.go

View workflow job for this annotation

GitHub Actions / lint

File is not `gci`-ed with --skip-generated -s standard -s default -s blank -s dot --custom-order (gci)
)

type MetricsViewComparison struct {
Expand All @@ -32,7 +32,7 @@
ResolvedMVSecurity *runtime.ResolvedMetricsViewSecurity `json:"security"`
Exact bool `json:"exact"`

// TODO: backwards compatibility
// backwards compatibility
Filter *runtimev1.MetricsViewFilter `json:"filter"`

Result *runtimev1.MetricsViewComparisonResponse `json:"-"`
Expand Down Expand Up @@ -165,6 +165,7 @@
return fmt.Errorf("error building query: %w", err)
}

fmt.Println(sql, args)
AdityaHegde marked this conversation as resolved.
Show resolved Hide resolved
rows, err := olap.Execute(ctx, &drivers.Statement{
Query: sql,
Args: args,
Expand Down Expand Up @@ -461,14 +462,23 @@
var columnsTuple string
var labelTuple string
if dialect != drivers.DialectDruid {
// TODO: only add `sum()` only if having is enabled
columnsTuple = fmt.Sprintf(
"sum(base.%[1]s) as %[1]s, sum(comparison.%[1]s) AS %[2]s, sum(base.%[1]s - comparison.%[1]s) AS %[3]s, sum((base.%[1]s - comparison.%[1]s)/comparison.%[1]s::DOUBLE) AS %[4]s",
safeName(m.Name),
safeName(m.Name+"__previous"),
safeName(m.Name+"__delta_abs"),
safeName(m.Name+"__delta_rel"),
)
if q.Having != nil {
columnsTuple = fmt.Sprintf(
"sum(base.%[1]s) as %[1]s, sum(comparison.%[1]s) AS %[2]s, sum(base.%[1]s - comparison.%[1]s) AS %[3]s, sum((base.%[1]s - comparison.%[1]s)/comparison.%[1]s::DOUBLE) AS %[4]s",
safeName(m.Name),
safeName(m.Name+"__previous"),
safeName(m.Name+"__delta_abs"),
safeName(m.Name+"__delta_rel"),
)
} else {
columnsTuple = fmt.Sprintf(
"base.%[1]s, comparison.%[1]s AS %[2]s, base.%[1]s - comparison.%[1]s AS %[3]s, (base.%[1]s - comparison.%[1]s)/comparison.%[1]s::DOUBLE AS %[4]s",
safeName(m.Name),
safeName(m.Name+"__previous"),
safeName(m.Name+"__delta_abs"),
safeName(m.Name+"__delta_rel"),
)
}
labelTuple = fmt.Sprintf(
"base.%[1]s AS %[5]s, comparison.%[1]s AS %[2]s, base.%[1]s - comparison.%[1]s AS %[3]s, (base.%[1]s - comparison.%[1]s)/comparison.%[1]s::DOUBLE AS %[4]s",
safeName(m.Name),
Expand Down Expand Up @@ -565,7 +575,7 @@
if err != nil {
return "", nil, err
}
havingClause = "HAVING " + havingClause
havingClause = "GROUP BY 1 HAVING " + havingClause
args = append(args, havingClauseArgs...)
}

Expand Down Expand Up @@ -700,7 +710,6 @@
) comparison
ON
base.%[2]s = comparison.%[2]s OR (base.%[2]s is null and comparison.%[2]s is null)
GROUP BY 1
%[16]s
ORDER BY
%[6]s
Expand Down
2 changes: 1 addition & 1 deletion runtime/queries/metricsview_rows.go
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ type MetricsViewRows struct {
MetricsView *runtimev1.MetricsViewSpec `json:"-"`
ResolvedMVSecurity *runtime.ResolvedMetricsViewSecurity `json:"security"`

// TODO: backwards compatibility
// backwards compatibility
Filter *runtimev1.MetricsViewFilter `json:"filter,omitempty"`

Result *runtimev1.MetricsViewRowsResponse `json:"-"`
Expand Down
2 changes: 1 addition & 1 deletion runtime/queries/metricsview_timeseries.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ type MetricsViewTimeSeries struct {
MetricsView *runtimev1.MetricsViewSpec `json:"-"`
ResolvedMVSecurity *runtime.ResolvedMetricsViewSecurity `json:"security"`

// TODO: backwards compatibility
// backwards compatibility
Filter *runtimev1.MetricsViewFilter `json:"filter,omitempty"`

Result *runtimev1.MetricsViewTimeSeriesResponse `json:"-"`
Expand Down
2 changes: 1 addition & 1 deletion runtime/queries/metricsview_toplist.go
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ type MetricsViewToplist struct {
MetricsView *runtimev1.MetricsViewSpec `json:"-"`
ResolvedMVSecurity *runtime.ResolvedMetricsViewSecurity `json:"security"`

// TODO: backwards compatibility
// backwards compatibility
Filter *runtimev1.MetricsViewFilter `json:"filter,omitempty"`

Result *runtimev1.MetricsViewToplistResponse `json:"-"`
Expand Down
2 changes: 1 addition & 1 deletion runtime/queries/metricsview_totals.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ type MetricsViewTotals struct {
MetricsView *runtimev1.MetricsViewSpec `json:"-"`
ResolvedMVSecurity *runtime.ResolvedMetricsViewSecurity `json:"security"`

// TODO: backwards compatibility
// backwards compatibility
Filter *runtimev1.MetricsViewFilter `json:"filter,omitempty"`

Result *runtimev1.MetricsViewTotalsResponse `json:"-"`
Expand Down
Loading