bug 1955428 - Use column replacement to place extended attribution and distribution info in client_info with the base info

[chutten]

Description

Per bug 1955428 (well, per the design doc beforehand), this takes the specially-named object metrics glean.attribution.ext and glean.distribution.ext and slides them over into client_info next to the rest of client_info.attribution.* and client_info.distribution.* (see also mozilla/glean#3091, where the client_info sections are added to the SDK and mozilla-services/mozilla-pipeline-schemas#841 where the sections and their fields are added to the schema)

Related Tickets & Documents

• bug 1955428 aka DENG-8191
• https://docs.google.com/document/d/1TIZhpBeZcJSEnIZJwj0Cj9saL2Tfs0X6oi_4gFU35eM/edit?tab=t.0
• Bug 1955428 - Implement attribution and distribution glean#3091
• bug 1955428 - Add client_info.attribution and client_info.distribution mozilla-services/mozilla-pipeline-schemas#841

Reviewer, please follow this checklist

[chutten]

sqlglot.errors.ParseError: Expecting ). Line 9, Col: 53. pointing to the . between client_info and attribution in

    metrics.object.glean_attribution_ext AS client_info.attribution.ext,

sure reinforces Ben's comment from mozilla-services/mozilla-pipeline-schemas#841 that REPLACE and nested fields don't play nicely.

... @BenWu ... should I leave this rewriting piece to you or someone else who knows bigquery-etl better than me?

[BenWu]

If you don't need this soon (in the next 1-2 weeks), I can see if I or someone else can take a look. It should be straightforward to do and similar to what's done for baseline pings:

bigquery-etl/sql_generators/stable_views/__init__.py

Lines 188 to 192 in c0c358b

	if schema.bq_table == "baseline_v1":
	replacements += [
	"mozfun.norm.glean_baseline_client_info"
	"(client_info, metrics) AS client_info"
	]

but recreating structs does have some performance cost so this could be expensive to do for all events pings for example so it might be worth some testing. Although we already are doing this for some other structs so maybe it wouldn't have a large additional impact

[chutten]

I'm willing to wait. I'll file a bug specifically for the column replacement, mirrored to a DENG ticket, and post it here. Thank you, @BenWu !

[chutten]

Filed https://bugzilla.mozilla.org/show_bug.cgi?id=1957476 mirrored to https://mozilla-hub.atlassian.net/browse/DENG-8208

I'll close this out.