Doc: add tests pages #10
Conversation
|
|
||
| ## Testing everything you should not | ||
|
|
||
| Automating a test is not mandatory, and there are cases there you should not automate them. |
There was a problem hiding this comment.
Automating a test is not mandatory, and there are cases that you should not automate.
There was a problem hiding this comment.
Thank you @CrochetFeve0251 for those additional pages.
The document about testing AJAX is good, it could use a complete example maybe?
About the other pages, I feel they are more general guidelines than technical standards per say. Therefore, maybe consider moving them to the folder ways_of_working? AC/communication/testing strategy per projects are more ideology guidelines than actual technical standards.
The testing/index.md might feel like very broad, and not carry much usable information. As a newcomer, I amn ot sure this would make me learn anything? see my comments there.
|
|
||
| Each line of code ever wrote will add up to the legacy that you will have to maintain. | ||
|
|
||
| This is the reason why we always want to write our test before changing any line of code inside the application. |
There was a problem hiding this comment.
This paragrpah is not very actionable on its own. Not everyone uses TDD and this is not something we enforce. So I am not sure about how this paragprah brings value?
There was a problem hiding this comment.
@MathieuLamiot TDD is not writing code before a test, but that is just a common practice, a good way to prevent mistakes and to prevent having tests being heavily coupled with implementation details.
I don't see any reason we should teach new comers to write tests in a way which is more error-prone.
There was a problem hiding this comment.
I agree with your comment. To clarify mine, I might have wrongly used the word TDD. I mostly agree with the guideline you suggest, but I don't think this text is teaching anything or providing value. Also, "writing tests first" is not an enforced practice in the team, so, at this stage, the combination of:
- Not actionable guideline ;
- Not commonly enforced practice in the team
makes the handbook a bit disconnected from what the team does, without an easy way to get closer to it.
I would recommend starting first with gathering commonly accepted and used practices in the team.
|
|
||
| All that will make the difference between automated and manual testing is if the time you are going to save from the test running will be higher from the time you invested. | ||
|
|
||
| That way, achieving high coverage is still an objective, but achieving 100% coverage is in fact a way to detect something wrong in the practices. |
There was a problem hiding this comment.
As part of a handbook, guidelines should be actionble. As a new comer, I am not sure I'd be able to know what to test/what not reading this. Maybe some examples would be beneficial here?
There was a problem hiding this comment.
@MathieuLamiot yes, I can definitely find examples for the first part.
For the second part, would it be too much pointing finger?
There was a problem hiding this comment.
For the second part, would it be too much pointing finger?
To avoid pointing fingers, find positive examples, not negative ones (ie. examples in which the guideline is correctly followed). You can probably find in the codebase or build examples of code that should not necessarily be covered by tests, that is actually not covered by test.
| To avoid fragile tests are using a testing strategy based mainly on integration tests and try to rely as little as possible on unit tests. | ||
|
|
||
| This makes our testing pyramid look more like a diamond as integration tests are more numerous than your unit tests. | ||
|
|
||
| The choice of integration tests over unit ones is related to multiple reasons. | ||
|
|
||
| First, they are black box tests or in other words, they do not rely on the implementation of the application unlike unit tests which are close to the implementation. | ||
|
|
||
| This allows tests to be anti-fragile and ease the amount of maintenance required by our tests. | ||
|
|
||
| Second, we value tests as a way to communicate by using integration tests primary instead of unit tests, we are putting business requirements at the center of our testing process allowing communication with the product team and QA. | ||
|
|
||
|  |
There was a problem hiding this comment.
@MathieuLamiot @CrochetFeve0251 Was this discussed anywhere? This has never been an official guideline for our team
There was a problem hiding this comment.
@Tabrisrp I think we talked about this during the dojo on tests, but we can discuss that at the retreat or here.
The main concern with unit tests is that they are too close to the implementation, which makes them fragile, and due to that takes a lot of time and effort in order for them to keep working when most of these tests value is today also handled inside integration tests.
That also slows the process of refactoring as changing anything in the implementation will lead you to end up with a ton of tests to fix and leads devs into wrong choices just to not have to handle the mess if they do a breaking change on tests.
Unlike unit tests, integration tests are black box which makes them anti-fragile, and due to that nature they end up not being that cost-intensive to maintain while still providing good coverage and also carrying the notion of done.
This is why we prioritize writing integration tests when possible instead of unit tests and keep unit when we are in a context not favorable for integration testing.
@MathieuLamiot Indeed they are more vision and guidelines my point on that is that the handbook should first provide a vision to its readers in the generic pages such as the test page and then translate them into actionable steps later on in more specific sections. That's why I put those pages here and not in a specific guideline part. |
|
Overall, I'd prefer we take a bottom-up approach where we first capture, in the handbook, actionable practices that we already partly use, to spread the knowledge. General vision will naturally come from them. I am afraid specifying a vision without applicable practices will just make the handbook shallow and not used. Let's discuss those points on Monday @CrochetFeve0251, and maybe have further feedbacks in the meantime. |
Description
Added a couple of pages about the testing strategy and how to test an ajax action.