Varnishtop 6 does not report RespUnset tags

[yched]

Expected Behavior

varnishtop should show RespUnset tags, just like any other VSL tag
that's the case in 5.2

for instance, when http_gzip_support=on and serving zipped beresp to clients with no Accept-Encoding:gzip,
varnishtop 5.2 does show

   218.15 RespHeader     Content-Encoding: gzip
   218.15 RespUnset      Content-Encoding: gzip

Current Behavior

varnishtop 6 (tested with 6.0.7) does not show any RespUnset tag

in the "serving zipped beresp to no-gzip reqs" case mentioned above,
varnishtop 6.0.7 only shows

   527.12 RespHeader     Content-Encoding: gzip

without the corresponding RespUnset (which is indeed listed by varnishlog when looking at individual request logs)

[nigoroll]

first look:
Because of the 0, 0,, Unset is flagged as "not used in either req or resp"

varnish-cache/include/tbl/vsl_tags_http.h

Line 86 in e89cd5f

SLTH(Unset, HTTP_HDR_UNSET, 0, 0, "unset header",

So SLT_F_UNUSED gets added here

varnish-cache/include/tbl/vsl_tags.h

Line 220 in e89cd5f

SLTM(Resp##tag, (resp ? 0 : SLT_F_UNUSED), "Client response " sdesc, \

which is skipped in varnishtop here

varnish-cache/bin/varnishtop/varnishtop.c

Line 134 in e89cd5f

if (VSL_tagflags[tag])

VSL_tagflags was added in 63807dd and I think at that point the check in varnishtop (and maybe elsewhere) should have been limited to binary and unsafe records?

[nigoroll]

Or should RespUnset not be flagged SLT_F_UNUSED to begin with?

[yched]

This is largely above my head, to be fair, but :
is RespUnset baing flagged SLT_F_UNUSED also related to the (sometimes puzzling IMO) behavior that
varnishncsa still displays header values even if they have in fact been unset ?

e.g :

set resp.http.X-My-Header = 'not-actually-sent';
unset resp.http.X-My-Header;

varnishncsa -f "%{X-My-Header}o" --> "not-actually-sent" ?

(sorry if this is something else entirely...)

[nigoroll]

FTR Unsetadded to vcl_tags_http.h a8e79f6
SLT_F_UNUSED introduced in 6725221

[nigoroll]

@yched my remarks are for coordination with fellow developers.
The varnishncsa issue would somehow be related, but a different bug. Please open a new ticket.

[yched]

Sure, I know you didn't expect me to answer that 😁
varnishncsa : ok, will do

[nigoroll]

after input from @mbgrydeland on IRC this one should be easy to fix:

(13:30:57) martin: slink: The SLT_F_UNUSED is just an artifact I think, in order to not list out and document (in the vsl2rst binary) some of the generated combinations that are impossible
(13:32:00) martin: comes as a result of how the tbl/vsl_tags_http.h include table is used
(14:41:53) slink: martin: ok cool, so in fact it should not be set for tags which we do emit, right?

[mbgrydeland]

(14:41:53) slink: martin: ok cool, so in fact it should not be set for tags which we do emit, right?

I'm not sure I understand the question.

As I unserstand it, the only defined effect of SLT_F_UNUSED is to skip it when creating the RST doc output in vsl2rst. The matrix of SLT_F_{Req,Resp,Obj,Bereq,Beresp}{Method,URL,Status,...} becomes quite large, and would bloat the vsl2rst output with a lot of duplicate and non-useful information. So this trick with the flag was introduced to limit that output.

VSL_tagflags was added in 63807dd and I think at that point the check in varnishtop (and maybe elsewhere) should have been limited to binary and unsafe records?

I think that is a correct analysis. Assuming that any tag flag is the same and any set should skip it is wrong.