Configuration as Code
Reference Documents
| Reference | Document Location |
|---|---|
| CHA1 | Operations Dashboard |
| CHA2 | Channels Domain Design |
| PLAT1 | Helm |
Problem Description
When we deploy a service, we sometimes need to provide it with data that will ultimately impact its behaviour. This is referred to as configuration.
Configuration can be: * User driven configuration * Administrative configuration
Please note that client configuration, such as language preference is considered part of the functionality of a service and not configuration, and therefore out of the scope of this blueprint.
In the past, in order to reduce friction for users, we have exposed user interfaces to manage this configuration. This presents a few problems: * Access Management - if the configuration is designed to change the behaviour of the system, it is possible that not everyone should have permission to make this changes * Change Review - Even when changes are done by the right person, mistakes can be made and lack of peer review can be an issue * Disaster Recovery and reliability - If changes are persisted on data stores that are deleted and repopulated, the configuration can drift and might be hard to be replicated in case of a disaster recovery situation or if needs to be replicated in a staging environment * Fragmentation of user interfaces - With Ebury 2.0 and the breaking of our monoliths we should expect a growing number of production services. This can be confusing for administrators and operators to navigate a large set of fragmented user interfaces.
Background
Most user configurations have been embedded into Django UIs, while admin configuration has been in options files or admin panel. This has driven otherwise headless services to have a UI framework that brings larger maintenance requirements.
Also since Django provides a builtin admin interface which enables administration of many aspects of a service, Teams migrating services from Django applications are unclear if they need to recreate this functionality.
Solution
For both user and administrative configuration that requires a small amount of changes, configuration SHOULD be stored as code.
Developers SHOULD administer service configuration with file based config via a git repository, hosted on GitHub, where changes can be tracked, easily updated and version controlled (aka gitops).
It is recommended that it is stored using a pre-agreed JSON schema in a file.
User friendly solutions to create a PR with configuration changes can be provided to other teams such as business apps or operations.
What is configuration
Configuration is data that is provided to a service statically (not dynamically) and that it is used to determine the behaviour of the service. Configuration data is not served to service users in any form, incluing post-processing.
Configuration data examples * client or services whitelisting * Performance tunning parameters * Response limits * Cut-off times in which services will not be available
This is not configuration data: * client/customer preferences * Seed data such as currency codes or customer spreads
Please note that seed data, must be provided statically and under change control, but other methods to serve this data might be used but not discarding ConfigMaps as a viable option.
Headless services
Backend services should be by default headless, in case that user configuration that is often changing and where configuration as code is not practical, the services should offer a CRUD interface and implement a UI architecture inline with the Operation Dashboard (see CHA1)
The creation of a web based UI for the administration of config / reference data MUST be considered an option of last resort, requiring an exception agreed in advance (at planning level) by the VP of Engineering.
A clear disaster recovery plan for these services has to be specified in the blueprint.
Alternatives
Continue using administrative interfaces - creates issues with control access and security Configuration in Kafka topics - same configuration data could be fed to services via a kafka topic. Although this provides good disaster recovery it can increase complexity of the service.
Operation
Deployment and provisioning of configuration should be as a Kubernetes ConfigMap mounted as a volume file, rather than in memory as a variable.
Any changes to this configuration will require a pull request to be approved by a code owner on the repository.
Deployment would be included in the Helm chart of the service to be deployed (See PLAT1)
Security Impact
Providing configuration as code improves the security of the service by having a traceability and approval control to changes that can otherwise change the behaviour of the platform.