readme: update information on why specs

[tynes]

Description

We need a clear explanation about why specs and how to think about them.
Opening this PR for feedback.

[clabby]

Maybe my own preference, but I never quite liked the heavy use of design docs, barring when there was more of a research problem to answer. Design docs feel great when an idea of the feature is still being shaped, but I do prefer specs that end up with a concrete "why" on top of the "how."

Creating a spec first, esp. with us leaning into having multiple implementations, allows for more people to think about the feature through the lens of how they'd implement it. IMO this makes it easier to think of the benefits & challenges to push important discussion forwards, rather than focusing too much on "it looks great on paper."

[axelKingsley]

To counter @clabby a bit -- I think the design-doc phase is important because it ensures the spec has at least one strong implementation. Our design docs tend to discuss the properties of a good solution, and the limitations/boundaries within the solution space, and without that backdrop it feels easy to come up with specs that don't fit the real world.

If you spell out in specification what a Node must do, but doesn't describe any implementation, it is possible there is no implementation possible for a given feature. And even in cases where the implementation is technically possible, the tradeoffs when adding it to an already complex system could make it infeasible in production. Exploring those tradeoffs and ensuring the spec is achievable is important, but is not part of the spec itself.

As a toy example, we could say "Nodes must validate that all transactions are purple when considering a Block as valid". Sounds great on paper, now you have purple blocks. But if the computation to check transaction color is expensive, maybe you can only process a fraction of the transactions you used to (let alone the complexity to fault-prove it). And maybe purple-ness isn't even the end goal. A design doc can spell out the need, the potential solutions, and the feasibility of the proposed solution before it shows up as a specification to be followed.

[ajsutton]

I think I land somewhere in the middle. There are times when design docs are useful but the majority of our work shouldn't need one and we don't want to spend all our time writing docs just to follow "the process", especially docs like design docs which quickly become out of date and are no longer useful (or even misleading). The spec is the living document that will be maintained over time. So I think it needs to include some "why" but it is a reasonably light explanation - eg "the sequencing window exists to ensure L1 censorship resistance is inherited" but not "censorship resistance is an important property to give users confidence in the ability to withdraw their funds". That kind of technical why helps people implementing the spec to understand the details by setting their frame of reference and future spec changes by documenting key properties we want to preserve even if they're not strictly part of the implementation.

I'd also say that our experience shows that neither design docs, nor specs are good at ensuring the design actually works and can be implemented. I can't think of the last time a design doc or spec survived the first encounter with real world implementation.

So when we talk about doing specs first, we can't afford for that to mean doing waterfall development where we do a high level design (design doc), low level design (spec) and then implement. The software industry has consistently shown that approach doesn't work. We need to be doing design, spec and implementation in parallel so we benefit from the feedback loop that actually building an implementation and testing it provides.

[mds1]

Include a light description of the problem that is being solved and the rationale but the majority of this should be recorded during the design doc phase

From past conversations, my understanding of design docs is that they are not intended to be deliverables themselves, and they should typically result in some other upstream change. For example, in ethereum-optimism/design-docs#21 we agreed on a new way to version and release L1 contracts, and we upstreamed it to the monorepo so it's more visible to contributors.

Therefore, I agree the problem and rationale should be recorded during the design doc phase, but I also think that info should still be upstreamed to the specs repo, to act as the permanent, single source of truth. It would provide a worse UX if readers had to jump to design docs—which are likely to become stale as specs are tweaked—to fully understand a spec

[mds1]

I think we should have more emphasis on the idea that specs should:

• Focus on describing the interfaces, behaviors, requirements, and safety conditions (e.g. when to error)
• Avoid mentioning implementation details and executable code snippets (e.g. interface code snippets are ok, implementation snippets are not).

This is an important distinction and leaking implementation details is an easy, expected mistake to make for people new to the spec-writing process. Quoting from a reply I left in #233 (comment):

To expand a bit so I don't think we've discussed this much, the rationale for "not leaking implementation details into the spec" is:

• Simplify spec maintenance: By specifying as little as possible to implement the requirements, we make it easier to keep the spec up to date by having less code to maintain. The code in this spec is not tested or audited, and we may end up tweaking it.
• Readability: The less code, the easier it is for a less-technical reader to understand the spec and ensure it's sound.
• Testing: The implementer of the spec has flexibility around the final result, as long as it meets the requirements. When implementing the spec, we write tests for all the requirements in here, but we wouldn't write a test to e.g. ensure we're using assembly since that's an implementation detail and not a requirement.

[ajsutton]

I think I land somewhere in the middle. There are times when design docs are useful but the majority of our work shouldn't need one and we don't want to spend all our time writing docs just to follow "the process", especially docs like design docs which quickly become out of date and are no longer useful (or even misleading). The spec is the living document that will be maintained over time. So I think it needs to include some "why" but it is a reasonably light explanation - eg "the sequencing window exists to ensure L1 censorship resistance is inherited" but not "censorship resistance is an important property to give users confidence in the ability to withdraw their funds". That kind of technical why helps people implementing the spec to understand the details by setting their frame of reference and future spec changes by documenting key properties we want to preserve even if they're not strictly part of the implementation.

I'd also say that our experience shows that neither design docs, nor specs are good at ensuring the design actually works and can be implemented. I can't think of the last time a design doc or spec survived the first encounter with real world implementation.

So when we talk about doing specs first, we can't afford for that to mean doing waterfall development where we do a high level design (design doc), low level design (spec) and then implement. The software industry has consistently shown that approach doesn't work. We need to be doing design, spec and implementation in parallel so we benefit from the feedback loop that actually building an implementation and testing it provides.

[ajsutton]

I don't think this is true and is actually an anti pattern. We shouldn't write the spec in isolation of implementation because we need the feedback cycle from actually implementing and testing it to know the spec actually works. It's extremely unusual for a good spec to be written without at least a proof of concept implementation being built alongside it. Even with the multi-client environment of L1, EIPs almost always have some kind of reference implementation or proof of concept that was used to work through the issues and get to an initial EIP.