Treasury Migration Strategy

Proposal of a strategy to migrate TMS-related processes from BOS and FXSuite to new services.

Reference Documents

The reference document table contains links to relevant information for the RFC:

Reference Document Location
BPAQT01 Aggregated Quantum reporting
BPTDE01 Treasury Domain Design Overview
BPAEB01 Ebury 2.0
BPTTF01 Trade Finance
BPPMP01 Monitoring Platform

Problem Description

Currently, all relevant BOS transaction updates are reported by a periodic task that runs at a fixed frequency ( now every 5 minutes). The task is responsible for transforming BOS reportable data into treasury instruments that are sent to the TMS. However, the current reporting process has problems, like reporting fee updates(more details on BPAQT01) and the inability to report aggregate data over specific periods.

In addition to the reporting limitations issues, the current reporting process relies on FXSuite to send data to the TMS. However, FXSuite is considered deprecated, and TMS-related functions need to be moved to a new service.

This RFC proposes a solution to migrate and remove the FXSuite dependency from the reporting process and implement the necessary work to enable aggregated reporting capabilities described on BPAQT01

The scope of this RFC excludes the migration of the synchronization process of MT940 and MT942 FIN files(a type of message used on the SWIFT network) with TMS to the new architecture. The migration of the SWIFT files synchronization process with the TMS will be addressed in a subsequent RFC.

Background

The QusSendInstrument BOS's task is responsible for transforming BOS reportable data into financial instruments that are sent to the TMS. On the BOS side, every create/update/delete operation over a reportable BOS data has associated triggers that insert the element on the TMSBuffer table. The table stores a foreign key to the element and action and its status:

  • Actions: create/update/delete/drawdown/rollback/fixrate/feeupdate/cancelfee
  • Status: unprocessed, process started, processed

The task has a series of filters that selects the unprocessed elements over the TMSBuffer table and serializes them accordingly to their type. It's important to note that there isn't a one-to-one relation between an element in TMSBuffer and the number of serialized instruments generated. The unprocessed elements on the buffer are sent in the following order:

  • Counterparty
  • Cash Account
  • Bank Account Rule
  • Deal Set
  • Foreign Exchange Deal
  • Cash Flow

In addition, each instrument type has subtypes(like spots and fixed_forward for Foreign Exchange Deal related instruments). Each sub-type is associated with one of the "actions": created, updated, deleted).

The order that the elements are serialized and sent to TMS is essential. For example, for TMS to be able to read a new spot deal (Foreign Exchange Deal instrument), the deal set associated with it needs to be first created on the TMS side. If the deal set creation instrument is sent after the instrument that creates the spot on TMS, the spot data will be ignored ( related files will be moved to "errors") and need to be manually fixed.

Two types of reportable instruments don't follow the same logic: daily deal set spot book and end-of-day recon. Instead, the data for the two instruments is reported once per day.

After the task selects the instruments to be reported and serializes them, the resulting data will be sent to FXSuite via an HTTP post request(one request for each type of instrument). The data sent to FXSuite is in the format of a list of dictionaries.

On the FXSuite side, the data received from BOS is grouped, parsed, and mapped into QXT files(a type of DOS format text file that the TMS understands). The instruments are grouped and sent in different files to limit the maximum file size.

The following diagram demonstrates the current TMS reporting process:

Current TMS reporting process

Solution

The proposed solution will be strategic and involve two phases. The first consists of removing the FXSuite dependency, and the second one moves all the current QusSendInstrument behavior to a new service following BPAEB01 architecture.

The first phase will help remove many potential complications that arise with the migration of the instruments from BOS. For example, without removing FXSuite first, the process of migrating where the instruments are generated (BOS or the new services) would be a lot more complex, and it would require a complicated synchronization process between BOS, FXSuite, and the new service.

Phase 01: Removing FXSuite dependency

The proposed solution to remove the FXSuite dependency essentially involves using the QuantumGateway service to transmit BOS's data to the TMS. The QuantumGateway service is a generic service capable of sending data to the TMS(Quantum), and it is currently used in production to report Trade Finance related data [BPTTF01].

  • A new table, QUS_SERIALISED_INSTRUMENTS, will be created on BOS, storing the same data currently sent to FXSuite. CDC events from the table will be published on a Kafka topic (e.g., ebury.treasury.bos-serialised-instruments)
  • A new service will consume messages from the topic, parse the data, and publish it to the ebury.treasury.quantum.outgoing topic.
  • An instance of QuantumGateway service will consume the messages from ebury.treasury.quantum.outgoing and send qxt files to the TMS.

The following diagram gives an overview of the solution:

Phase 01 Diagram

To ensure that the QXT data generated by the Treasury service is the same as the current one, the solution permits running the new service concurrently with the FXSuite one.

  • An instance of QuantumGateway can be configured not to send any qxt files coming from the new service and instead save the files to a s3 bucket.
  • The files generated on the FXSuite side can be compared to the ones generated using the new service
  • After confirming that the files are equal, the HTTP request to send data to FXSuite can be disabled, and QuantumGateway can start to send qxt files to the TMS.

Audit Trail

A reference to the source of the instrument's data will be included as a meta field to the message to QuantumGateway. For example, when BOS generates a list of serialized treasury instruments and inserts the data to the table QUS_SERIALISED_INSTRUMENTS, the primary key of the instruments on BOS can be added in a __META_Source field when the new Treasury service process the message before sending it to QuantumGateway.

Phase 02: Migrating instruments from BOS

The second phase of the migration involves removing the responsibility of creating treasury instruments from the BOS side. The instruments will be created by the treasury service based on BOS CDC events. The following diagram demonstrates the proposed solution(detailed on BPTDE01):

Treasury Target Architecture

  1. Whenever there is any update in a reportable BOS data, CDC events will be published to their Kafka topic.
  2. Ebury services with relevant data required to generate an aggregated Treasury Instrument publish events each time the data changes.
  3. The Treasury Instrument Aggregation Service will aggregate the data from BOS CDC events and publish a Treasury instrument.
  4. The Treasury service will consume data from the Treasury Instrument Aggregation Service and map it to Treasury instruments(accordingly to business rules). The Treasury instruments will be published to the QuantumGateway input topic.
  5. QuantumGateway will consume Treasury Instrument events and send them to the TMS.

Considering the number of different instrument types and subtypes, the safest way to migrate instruments will be in groups. The suggested order of the migration is in the inverse order that they are sent from BOS. The reason is that doing this will ensure that an instrument that depends on prior instruments is sent only after the necessary ones are sent. The general process to migrate an instrument to the new service will be:

  1. Select an instrument type/subtype to migrate
  2. Ensure that the Treasury Instrument Aggregation Service consumes all necessary data to build the migrated instrument.
  3. The Treasury service will filter out the instrument being migrated from the BOS serialized instruments data and will start to send its own generated data for the migrated type.
  4. After the migration is confirmed to work, stop BOS from sending instruments for the migrated instrument type.

This phase will be considered completed when the QusSendInstrument task stops publishing the instrument's data.

Audit Trail

When a treasury instrument is migrated from BOS, the Treasury Aggregation service will be responsible for creating them. Similar to the audit trail solution for the first phase, the source will be included in __META key messages to QuantumGateway. In addition, an explanation for the audit trail for each instrument is included on BPTDE01.

Monitoring

The current monitors implemented for the QusSendInstrument task and TMS-related ones implemented on FXSuite will be replicated in the new services, using the tools proposed on BPPMP01.

Service Ownership

New Service Service Name Service Owner
Yes Treasury TRE Team
Yes Treasury-Aggregation-Service TRE Team
No Quantum-Gateway TRE team

Alternatives

The alternative for the proposed solution is to start migrating the instrument from BOS to the new service, keeping FXSuite dependency until all instruments are migrated. The problem with this approach is that it would require solving some issues like:

  1. Ensuring that BOS and new reporting services have the same configuration(when the new service starts to report a new instrument, BOS needs to stop reporting it)
  2. Ensuring that the new service will send no duplicates
  3. Ensuring that the data required for sending a migrated instrument to Quantum was already sent by BOS and consumed by
  4. Ensuring a simple rollback

The proposed solution solves the mentioned problems by moving the responsibility to control which instrument will be reported by BOS and which will be reported by the new service to the new service.

Operation

Considering that the new services will only consume data from internal services, they can be run on natonly cluster.

Security Impact

The proposed solution will require access to the same data BOS uses to serialize Treasury instruments. As it requires access to sensitive data, it requires the same access restrictions used on the BOS side.

Performance Impact

The performance impact on the BOS side should be negligible, as the solution only requires a maximum of six new inserts to a table.

Developer Impact

The proposed solution will change how financial instruments are reported to the TMS system, removing FXSuite dependency.