Background
Raw logs are not especially useful on their own. It would be nice to either have just an index of tags for where significant conversations happened by topic, or, even better, generate summaries of feedback and decisions and interactions worth highlighting and preserving in some fashion. The suggestions below are one approach to accomplishing this this.
The idea
Tag the chat logs by subject at conversation granularity, so the reasoning that
never made it into documentation or issues becomes findable — design arguments
settled in chat, bug diagnoses nobody wrote up, practices the project follows
because someone explained them well once.
The logs are already public and complete. What's missing is any way into them
other than reading chronologically.
Scope
Selenium 4 shipped on 2021-10-13. Everything from that date onward in
selenium-tlc/ is the target: post-4 architecture, BiDi, Selenium Manager, the
Grid rework, and the decisions still shaping the project today. Earlier material
and the #selenium channel can follow once the approach is proven.
|
|
| Day files |
1,543 |
| Messages |
59,079 |
| Conversation segments |
4,271 (median 6 messages) |
| Messages already inside Slack threads |
41% |
Why it looks feasible
Segmentation is mostly free. build_slack.py renders thread replies indented
four spaces, so for 41% of messages the conversation boundaries are exact rather
than heuristic. A silence gap handles the remainder. The unit of tagging should be
the conversation, not the day file — a day holds several unrelated threads, and a
day-level tag points at a date rather than a discussion.
Small exchanges carry the weight. 53% of segments involve exactly two people,
and another 14% are one person writing at length. Any filter requiring several
participants would discard most of the corpus, including its clearest
explanations — one maintainer explaining something properly to one other person is
frequently the best statement of a rationale that exists anywhere.
Approach
Three passes over segments, escalating in capability as the set narrows:
- Triage. Score every segment for whether it contains a claim, rationale,
decision or disagreement, and propose free-form topic labels. Those proposals
get curated once into a fixed vocabulary — a closed set matters, because
independent per-segment labelling otherwise yields w3c-compliance, W3C spec
and webdriver-spec as three separate tags and the index stops being usable.
- Review. Over survivors only. Confirm or reject the triage call, assign
topics from the curated vocabulary, and record what was concluded together with
a verbatim supporting quote so every claim stays checkable against the log.
- Synthesis. Per topic, across the full range: what was decided and when,
what reversed, and what was argued without ever resolving. Where a topic never
reached a decision, saying so plainly is the finding — this is the pass most
likely to manufacture a decisions list that isn't there.
Two implementation constraints worth recording up front:
- Tags cannot live in the log files.
build_slack.py opens each day file with
"w", so anything written inline is destroyed the next time an export is
processed. Sidecar files only.
- Address segments by the timestamp of their first message, not by line numbers,
which shift whenever a day is regenerated from a corrected export.
A useful signal that falls out for free: a segment linking a GitHub issue or PR is
evidence the reasoning already got written down somewhere, which makes it a
reasonable thing to rank below the conversations that never did.
What should and shouldn't be published here
This repository is a record of what was said. Interpretation of that record is a
different kind of artifact, and the line between them is worth drawing
deliberately.
A topic index — which segments touch the Grid, with dates and permalinks — is a
factual pointer. It makes no claim about anyone, it's checkable at a glance, and
it belongs alongside the logs.
A conclusion — "X rejected Y because Z" — is a model-written characterisation of a
named contributor's position, and it will sometimes be subtly wrong. That's the
kind of sentence that gets quoted back at someone years later. Conclusions and
synthesis output should stay out of this repository, or at minimum sit behind
human review somewhere with its own editorial context. Requiring a supporting
quote alongside every conclusion is what makes that review tractable.
The intermediate working files — scores, draft conclusions, proposed topics — are
build artifacts and shouldn't be committed at all.
Background
Raw logs are not especially useful on their own. It would be nice to either have just an index of tags for where significant conversations happened by topic, or, even better, generate summaries of feedback and decisions and interactions worth highlighting and preserving in some fashion. The suggestions below are one approach to accomplishing this this.
The idea
Tag the chat logs by subject at conversation granularity, so the reasoning that
never made it into documentation or issues becomes findable — design arguments
settled in chat, bug diagnoses nobody wrote up, practices the project follows
because someone explained them well once.
The logs are already public and complete. What's missing is any way into them
other than reading chronologically.
Scope
Selenium 4 shipped on 2021-10-13. Everything from that date onward in
selenium-tlc/is the target: post-4 architecture, BiDi, Selenium Manager, theGrid rework, and the decisions still shaping the project today. Earlier material
and the
#seleniumchannel can follow once the approach is proven.Why it looks feasible
Segmentation is mostly free.
build_slack.pyrenders thread replies indentedfour spaces, so for 41% of messages the conversation boundaries are exact rather
than heuristic. A silence gap handles the remainder. The unit of tagging should be
the conversation, not the day file — a day holds several unrelated threads, and a
day-level tag points at a date rather than a discussion.
Small exchanges carry the weight. 53% of segments involve exactly two people,
and another 14% are one person writing at length. Any filter requiring several
participants would discard most of the corpus, including its clearest
explanations — one maintainer explaining something properly to one other person is
frequently the best statement of a rationale that exists anywhere.
Approach
Three passes over segments, escalating in capability as the set narrows:
decision or disagreement, and propose free-form topic labels. Those proposals
get curated once into a fixed vocabulary — a closed set matters, because
independent per-segment labelling otherwise yields
w3c-compliance,W3C specand
webdriver-specas three separate tags and the index stops being usable.topics from the curated vocabulary, and record what was concluded together with
a verbatim supporting quote so every claim stays checkable against the log.
what reversed, and what was argued without ever resolving. Where a topic never
reached a decision, saying so plainly is the finding — this is the pass most
likely to manufacture a decisions list that isn't there.
Two implementation constraints worth recording up front:
build_slack.pyopens each day file with"w", so anything written inline is destroyed the next time an export isprocessed. Sidecar files only.
which shift whenever a day is regenerated from a corrected export.
A useful signal that falls out for free: a segment linking a GitHub issue or PR is
evidence the reasoning already got written down somewhere, which makes it a
reasonable thing to rank below the conversations that never did.
What should and shouldn't be published here
This repository is a record of what was said. Interpretation of that record is a
different kind of artifact, and the line between them is worth drawing
deliberately.
A topic index — which segments touch the Grid, with dates and permalinks — is a
factual pointer. It makes no claim about anyone, it's checkable at a glance, and
it belongs alongside the logs.
A conclusion — "X rejected Y because Z" — is a model-written characterisation of a
named contributor's position, and it will sometimes be subtly wrong. That's the
kind of sentence that gets quoted back at someone years later. Conclusions and
synthesis output should stay out of this repository, or at minimum sit behind
human review somewhere with its own editorial context. Requiring a supporting
quote alongside every conclusion is what makes that review tractable.
The intermediate working files — scores, draft conclusions, proposed topics — are
build artifacts and shouldn't be committed at all.