Technical implementation documents and proposals at project level
Keep low-level technical implementation proposals and documents inside projects instead of in high-level RFCs.
Problem Description
The RFC process can sometimes contain lots of low-level technical implementation that distract us from agreeing on the approach and requirements.
For example:
When we create a new ComponentWidget, we need to check that is is not a duplicate of an existing ComponentWidget.
We also need to check that the process that created the ComponentWidget has created it based on the latest state of the system.
The integrity checks must be done at the database level to avoid race conditions in the duplication and state checks.
To do this we will create an index on columns widgetid and componentid,
when componentid is not null. To check that the ComponentWidget was
based on the latest state of the system, we will have the following
model...
[actual python implementation of a model here]
Including specific implementation details like that in the proposal stage can become out of date very quickly if we come across edge cases during the implementation. It can also delay acceptance of the approach because we are trying to validate the actual implementation, which should be validated during implementation with code reviews, tests, and UAT evidence.
Solution
We can separate low-level technical documentation from RFCs by using the C4 model. An RFC should be at levels 1, 2, and 3 (System Context, Containers, and Components), and technical documentation should be level 4 (Code).
With this in mind, the above proposal might be better stated as:
When we create a new ComponentWidget, we need to check that is is not a duplicate of an existing ComponentWidget.
We also need to check that the process that created the ComponentWidget has created it based on the latest data.
The integrity checks must be done at the database level to avoid race conditions in the duplication and state checks.
To do this we will create constraints in the database that stop duplicate _data being inserted, and keep a record of updates to the system state, so if two processes try to create a new ComponentWidget based on the same version of the system state, one of them would fail because the database would not allow two changes to be branched from the same initial state. This is to make sure the decisions are always based on the latest data in the system.
This describes the approach, and we can still review this to make sure it meets requirements, and flag up areas where implementation might be a problem. In the above example, we might flag up automated processes that are creating ComponentWidgets, and need to be able to deal with integrity errors raised when the database does not allow the duplicates. The approach of introducing a state version when the state is updated and if a database index can enforce this is what is important in this example. The actual columns used or the exact model structure used to keep track of system state is an implementation detail to be decided during implementation.
Separating the high-level approach proposal and the low-level technical implementation will allow us to start implementation sooner and discover any deficiencies in the approach much more easily. high-level RFCs can be amended if we need to revisit them during implementation.
Still, the implementation needs to be proposed and documented.
We should have a docs/ directory inside each project. Individual teams
can use that for documents that are specific to implementation and do
not need to go to a wider audience. For new projects, this should be
mandatory. Existing projects can start it along with the
code ownership files.
docs/ should contain markdown files that will be transformed into HTML
by the blueprints build process. When blueprints builds, it will copy
the docs/ and code ownership files and build it as part of the site,
linking to it automatically from the navigation menus.
Depending on the team's workflow, they can use docs/ for an internal
RFC process using Architectural Decision Records,
or can use it to include documentation in the same PR that has the code.
Alternatives
We could use confluence, but this has all the disadvantages we detailed when we came up with the RFC process.
In some cases, RFCs could reference working prototypes, although this should be reserved for demonstrations of the approach and not concrete implementation.
Caveats
N/A
Operation
N/A
Security Impact
N/A
Performance Impact
N/A
Developer Impact
Developers will need to submit implementation documentation with their code PRs.
Data Consumer Impact
N/A
Deployment
Deployment of the blueprints site will need to access projects too.
Dependencies
N/A
References
N/A