ISO Migration - Inbound MVP - Master RFC

Reference Documents

Reference	Document Location
PRD	ISO Migration - Inbound MVP
Diagrams	ISO Migration - Current Status
Message Definitions	ISO Migration - Message Definition
Target Architecture	Money Flows - Target Architecture
R-Transactions	R-Transactions use-cases for Incoming Funds

Glossary

• FXS: FX Suite
• FIN files: Files currently used by SWIFT to transport MT messages (text-based, proprietary encoded)
• IA files: New standard of files proposed by SWIFT to transport MX messages (XML-based, open standard), also known as InterAct files.
• Inbound: refers to the direction of traffic, meaning that this RFC only considers the files that Ebury receives from the SWIFT network

Problem Description

ISO 20022 is an ISO standard for data exchange between financial institutions that is being implemented in various financial market agents and clients that we use today. Today our database, our systems and our processes are based on SWIFT MT messages and we must be migrated to the new SWIFT ISO 20022 CBPR+ format, also known as MX format. Adhering to this new format is not a straightforward change. This document will present the proposed architecture to meet the business requirements, define the domains and analyse the opportunities for improvement identified for implementing this new incoming flow.

Background

Our communication with the SWIFT network is done through exchanging files, using SWIFTs own file server called “Autoclient”. Ebury access Autoclient servers through SFTP protocol. Each Autoclient server is located physically inside Ebury premises (not in a cloud environment), and we currently have 3 of them.

Every time Ebury wants to send a message to the SWIFT network, we need to place a message file in only one of the Autoclient servers.

For redundancy, there are multiple instances of the SWIFT Autoclient software running on different physical servers. Each SWIFT Autoclient retrieves the same files from SWIFT. The gateway is responsible for retrieving a file and deleting the replicas from all instances of the SWIFT Autoclient.

Our current stack is able to read and write SWIFT MT messages, which are transported in .FIN file format. The current stack used for the INCOMING flow is served by 1 legacy application, FX Suite (aka FXS).

This stack is used primarily to generate Incoming Funds (debits and credits) reconciliation in BOS, and in order for that communication to happen, there are 2 extra steps needed: Transformation and Categorization.

Transformation is a step that converts the incoming funds messages from the SWIFT MT format to a standardised JSON format. This is necessary because the MT standard is semantically poor, and each bank can decide to add specific details on specific fields, making the act of parsing these MT messages challenging. For illustration purposes, at the time of this writing Transformer has at least 11 different APIs to parse MT messages from banks to return a single JSON format. The formats even have variations within a single bank (Barclays generic vs. Barclays UK vs. Barclays Italy, for example).

Categorization is a best-effort guess done partially by FXS and partially by another service called Swift Funds Categorizer, that tries to classify an incoming fund as either “Broker Funds”, “Client Funds”, “Company Funds”, “Client Payment Funds” or “Return Funds”. This is done through a machine learning algorithm inside the service, which was last updated 2 years ago (in 2021), and by a set of overriding IF statements inside FXS code after the ML categorisation.

The communication between both FXS and Transformer and FXS and Categorizer is done through HTTP API calls.

The last step in this pipeline is where FX Suite generates a BOS-specific Json message, which is called Bank Account Entry. BOS expects these messages in a SQS queue.

The final picture of the current stack, focusing on Incoming Funds, is the one below:

The SWIFT ISO 20022 migration project, in the context of Incoming Funds, proposes 2 changes:

1) The messages will be changed from MT (text-based, semantically poor) to MX (XML-based, ISO 20022 standard, semantically rich); 2) The transport mechanism of these messages will be IA files (InterAct files), and not FIN files anymore;

InterAct files do not transport MX messages directly. The ISO messages are embedded in an envelope (called DataPDU) that contains extra information about the source and destination financial institutions of the message, such as the BIC code, which is needed to apply certain business rules such as the funds categorisation.

This DataPDU envelope is SWIFT-specific, and unrelated to the ISO 20022 format.

The migration from the MT format to the ISO 20022 format will take place over the next few years, and we will need to support both MT and MX formats in our production environment while the market moves from one format to the other. How long this is going to take is not subject to discussion of this RFC, and the SWIFT migration timelines are detailed in the PRD.

This first iteration of the Inbound ISO migration project focuses on receiving Incoming Funds, but next in line will be the expansion to handle more complex scenarios such as handling GPI Tracker requests and handling Query Management, such as Cancellation Request (camt.056). These new capabilities are not handled by our current stack using FX Suite, and the exact process of how these will be handled is still in the discovery phase.

For that reason, this RFC will be focusing only on the message types required for Incoming Funds reconciliation..

Important note: With the implementation of the Incoming Funds flow with the new stack and adoption of the ISO standard by the market, we will be one step closer to decommission FX Suite, Transformer and Categorizer. The missing part will be to handle Outgoing Payments for the SWIFT scheme, using this new stack, and at the moment that this document has been written, we’re working in the discovery phase for that. One of the goals of this project is to decommission FX Suite.

Solution

For Incoming Funds reconciliation we will be using 3 ISO messages for 2 specific purposes:

• Intraday Reconciliation in BOS: camt.054 and camt.052
• Next Day Reconciliation in Treasury: camt.053

Below we’re illustrating the relationship between an InterAct file and an ISO message, in the context of Incoming Funds.

The same diagram can be repeated using camt.052 and camt.053 examples, with minor differences that are not relevant to this discussion.

The SWIFT network works with InterAct files the same way it works with FIN files: We will receive the same file in 3 different Autoclient servers, and our service is responsible for synchronising all 3 servers after consuming the files from one of them:

The data necessary for next-day reconciliation using camt.053 will be available to the consumers through Kafka after the InterAct files are acquired and extracted into DataPDUs by the Gateway:

The next logical step is to add a layer to communicate with BOS, translating the DataPDUs from the InterAct files into Bank Account Entry that BOS understands, like in the design below, using the Ambassador pattern and keeping the same interface that BOS uses to consume Bank Account Entries, which means SQS:

It’s important to note that each InterAct file can contain any number of DataPDUs. Each DataPDU has a limit of 999.999 bytes (fitting into a kafka message). Specifics about message retention and size are to be discussed in the Gateway RFC.

At this stage we’re handling a few different steps: - Receiving the file from SWIFT; - Removing the replica files from the other 2 Autoclient servers; - Unpacking each InterAct file into DataPDUs; - Transforming DataPDUs that contains camt.054 and camt.052 messages into Bank Account Entries for BOS;

Each time we have a new Bank Account Entry generated to BOS, what we actually have is a new Debit or a new Credit notification. And these notifications are Domain Events, that will be used by other services like Outgoing Payment Service, in the scenario of a Payment Return.

We also have business specific rules that must be implemented somewhere regarding filtering or blacklisting specific bank accounts, such as accounts related to SEPA payments or FPS payments. Today these rules exist in FX Suite, and we will need to replicate them in the new stack:

This 3-tiered approach gives us a good foundation of Domains, isolating both SWIFT on the left and legacy platforms on the right, and allowing us to generate domain events in the middle. In a more compact way, this will be the final design:

Adding boundaries and defining what’s the expected language and context:

The usage of a Gateway is consistent with the Gateway pattern.
The usage of the Adapter is consistent with the Event Standard.
The usage of a Domain Connector is consistent with the Ambassador Services pattern.

A separate RFC will be created for each component in this topology. The discussion in this document is a high-level view.

The communication between the services will be done through Kafka. BOS is a special case, since it's legacy and already contains a SQS interface to import Bank Account Entries, so we'll re-use this interface instead of creating a new one.

This topology considers a separation of domains and establishes boundaries of conceptual integrity between the layers. The layers are separated into:

• Payment Network Format - This layer is responsible for receiving the file and forwarding information according to pre-established contracts. The domain that this layer interfaces with is related to the steps required to connect to the autoclient servers, access the files, download the files, save the files and send the information extracted from the files. The application with this responsibility is called Swift-Gateway. More information can be found in the RFC for this component.

• Ebury Internal Domain - After receiving the files and extracting the useful information, it's necessary to adapt the raw data received into a common language. ISO 20022 proposes an ubiquitous language for payments and using this advantage, the model defined by ISO Technical Committee TC68 Financial Services will be used as the basis for the canonical representation of debits and credits for Ebury's domain but simplifying some ISO structures where it will make handling them easier in business logic in domain services.The domain that this layer interfaces with is related to the steps required to adapt, map, categorise and filter incoming messages for the company's internal context. The applications that will implement the business requirements are the following:
• Adapter: This application will work as a transformer that receives DataPDUs, maps the messages and then publishes them for interested parties.
• Financial-codecs: This module stores the schemas and models that we will transit between the Gateway and the Adapter. Even though ISO20022 aims to address this problem, it is expected that the network-specific payloads may have variations depending on the specific payment provider (e.g. Citi UK requires an optional field to be present, or one field to contain information in one format while another bank in a different way). Also note that "external codes" (enums, such as country, currency codes, purpose of payment, etc.) are defined in separate documents, and can change without a change in the message schema version. For these reasons the base codec classes will implement the mandatory elements of the message standard, and their subclasses implemented in service code can add to or modify the base class implementation. The additional unit tests for the specific behaviour is service code responsibility.

• Ebury Legacy Domain - After processing messages, it’s necessary to categorise and transform the information to a format that legacy consumers understand. Filtering is also needed, as some bank accounts are prohibited of creating Bank Account Entries in BOS. In the scope of this phase of the project, we’ll be creating bank account entries in BOS and the information needs to be sent via SNS/SQS. BOS consumes the messages from SQS only. This is an anti-corruption layer that centralises responsibility for connecting and accessing services with legacy and/or external domains.

Caveats

The ISO 20022 implementation period will happen over the period of a few years, and Ebury works with 17 different banks that will need to implement this standard. During the period where a Bank will be implementing the new standard, Ebury will need to receive Incoming Funds messages in both legacy and new platforms, meaning we will have the same Debit and Credit coming from 2 sources, a MT message and a MX message.

We expect this coexistence phase to last for a long time.

That means we need to provide 2 functionalities that will be detailed later, as we’re not sure if we will put this functionality in the Financial Adapter or Domain Connector yet: - Intercepting Bank Account Entry messages coming from FX Suite, and decide if the message that will flow to BOS will be from the new or the old system; - Allowing the users to quickly switch, for each bank account, which will be the source of the debit or credit.

We’re calling this strategy “proxy filtering”, and it’s final design is still under development as there are open questions regarding how can we match a message coming from both systems, to make sure we’re not risking duplication of entries or missing any entries when communicating to BOS, and will be detailed further in the project, but the initial design looks like this:

Another important caveat is the lack of a proper volume of messages to test in the staging environment. It is expected that each bank that migrates to the new standard will have their own specific oddities when sending the camt.053 and camt.054 messages to us, and as much as we can test the whole solution in staging, we will need to evaluate the solution with real-world data. For this reason, we’re predicting that the Domain Connector will need this filtering step, allowing us to implement the solution for a bank and cross-checking if we’re able to match the quality and quantity of the messages from the legacy MT stack against the new MX stack.

We’ll need tooling to automate as much as possible this comparison. The specific tooling is subject to the Domain Connector RFC, and will be discussed in detail in a later stage of the project.

We also expect to rollout bank implementations gradually, working first with a single bank and adjusting the solution until it works correctly. Once the whole solution works for one bank, we can tackle a second bank and make all necessary changes for that bank. This cycle will repeat until we have all 17 banks being processed on the new solution.

Idempotency concerns

Each time we notify about a new file, a new DataPDU or a new DomainEvent, the event producer will also generate an unique ID for that event.

The initial approach is to use an “at-least-once” strategy for kafka messages. Missing messages would create problems, but receiving them twice is “handleable”.

That means that Event duplication is something that “may” happen in exceptional cases, so we’ll need to guarantee that: 1) The Unique Key generation in the producers is consistent, and will always generate the same key for the same event; 2) The consumer is responsible for handling idempotency;

Decommissioning the Legacy stack

At this point, it will be easier to decommission the whole FX Suite legacy stack, instead of trying to disable only parts of the code. Since all the Bank Account Entry messages coming from FXS will be filtered out by the new stack, we will have enough confidence to disable it entirely, working through phases, like described below:

Addendum: R-transactions in the context of incoming funds

Disclaimer: This is a new section added to the already approved RFC, needs to be reviewed.

R-Transactions in the context of Incoming Funds are split between the 2 use-cases below:

• Return of fund;
• Recall of fund;

R-Transaction Use Case: Return of fund

Return of fund is an action initiated by Ebury, and it means that Ebury is returning a Credit that has been received by an Ebury bank account, which means that:

1) The money is already present in a bank account that belongs to Ebury; 2) Ebury was notified about this credit through an ISO 20022 camt.05x message; 3) Ebury does not want that credit anymore for any business reason and wants to return the money; 4) Ebury has an agreement with the destination bank about the protocol to return the money;

The 10 thousand feet view for the return process is this:

Breaking it down to include the services inside Ebury, plus Operations Team participation, we get a more detailed view:

1) In order to return a fund via SWIFT, Ebury needs to add a reason for the return - this will be initiated in BOS in the Unnalocated Funds queue - this is the same behavior that exists today for returning SEPA funds; 2) BOS will call an API inside the Incoming Connector calling for the return of a specific fund; 3) This API is synchronous, idempotent and will use optimistic locking, meaning that it can be called for the same fund multiple times without generating multiple return calls to SWIFT; 4) The API will generate a Return Fund message in a kafka topic to the Adapter; 5) The Adapter will construct a pacs.004 message, enrich it with any missing data, and encapsulate it into a DataPDU for delivery; 6) The DataPDU will be relayed to the Gateway through a kafka topic; 7) The Gateway delivers the message to SWIFT; 8) After some time, the bank returns the fund and generates a Debit in Ebury's account; 9) The Debit is received and reconciled against the previous credit by the Operations team;

Alternate flows/errors will propagate information back to the Incoming Connector, informing the users through email about the error(s), and allowing them to investigate or correct anything wrong with the fund.

After that, they're able to sending another Return fund command.

R-Transaction Use Case: Recall of funds

Recall of fund is an action initiated by a 3rd party, and it means that an external entity is requesting back funds that have been received by an Ebury bank account, which means that:

1) The money is already present in a bank account that belongs to Ebury; 2) Ebury was notified about this credit through an ISO 20022 camt.05x message; 3) The money might already have been spent by the client to whom the fund was allocated; 4) Ebury has an option to accept or reject the recall request; 5) Ebury has an agreement with the destination bank about the protocol to return the money;

The 10 thousand foot view of the recall process is this:

The catch is the volume. The amount of recalls that Ebury receives is very small, about 1 recall each 2 months (6 per year). That being said, the Product team decided to not invest in the end to end automation for this process. The only automated part of the recall will be a notification (through e-mail) to the Operations team, so they can handle the recall manually:

1) If the Operations team decides to NOT accept the Recall, the Reporting section above won't happen; 2) The Operations team will generate camt.029 messages manually as a response to the original camt.056 message, or will handle recall response outside the SWIFT messaging system - how they will do it is outside the scope of this RFC;

Overview

The flows included in the scope of this project MVP are defined in the following table.

Flow	Action	Message		Name
Incoming Funds	Receive	camt.052.001.08	Cash Management (camt)	Bank to Customer Account Report
Incoming Funds	Receive	camt.053.001.08	Cash Management (camt)	Bank To Customer Statement
Incoming Funds	Receive	camt.054.001.08	Cash Management (camt)	Bank To Customer Debit Credit Notification
R-Transactions	Receive	camt.056.001.08	Cash Management (camt)	FI To FI Payment Cancellation Request
R-Transactions	Receive	camt.055.001.08	Cash Management (camt)	Customer Payment Cancellation Request
R-Transactions	Send	camt.029.001.09	Cash Management (camt)	Resolution Of Investigation
R-Transactions	Send	pacs.004.001.09	Payments Clearing and Settlement (pacs)	Payment Return

Service Ownership

New Service	Service Name	Owner
Yes	SWIFT Gateway	COP Team
Yes	Adapter	COP Team
Yes	Domain Connector	COP Team
No	Outgoing Payment Service	PAY Team
TBD	TRE Intermediary Service	TRE Team
No	BOS	TEG Team

Operation

N/A as this is the master RFC. Specifics will be discussed in the specific RFC.

Security Impact

Since we’re ingesting new files from a 3rd party (SWIFT), the security team will be involved in the Swift Gateway RFC in order to discuss non-functional requirements to protect Ebury regarding the ingestion of these new files.

Performance Impact

In the context of Incoming Funds, the most similar workload that we have today is the one from FX Suite that handles MT940 and MT942 messages.

At the risk of over-simplification, the workflow of a MT940 is similar to a camt.053, and the workflow of MT942 is similar to camt.054 and camt.052.

That means that for each new entry in one of Ebury accounts we expect to receive a file that contains a camt.054 (or camt.052, depending on the bank), and for each account Ebury has we expect to receive one camt.053 daily.

In the past 24 hours (from October 05th, 2023) we had 1.900+ MT 940 messages and 11.300+ MT 942 messages, by checking Kibana logs. This is a typical volume if we check the last 30 days of logs.

There’s also an useful dashboard showing the current flow of MT942 messages. Focus on the "Received MT942" panel, as it shows how many messages we received and had to analyse.

We expect about the same amount of files for camt.054, camt.053 and camt.052 as described above daily, as the focus is similar which translates roughly to: - 11.000+ daily messages combining camt.054 and camt.052; - 1.500+ daily messages of camt.053;

We also expect an increase in size, as the legacy MT message format is more compact than the ISO 20022 XML format. Without actual production data there’s no way to have a real value, but we can estimate from the samples below: - 1 MT942 file with a single entry has 500 bytes; - 1 InterAct file, containing 1 camt.054 message with a single entry has 13.000 bytes; - The camt.052 message is similar to camt.054 by ISO standard;

Taking only these 2 files into consideration we have these estimations:

Scenario	Rationale	Final Size
MT942	11.300 messages * 500 Bytes	5.5 MB/day
Camt.054 + Camt.052	11.300 messages * 13.000 Bytes	143.8 MB/day

That’s a 26x increase in size.

There’s also a seasonal component on these messages, as historically at the end of the year the volume of Incoming Funds messages increases a lot. Unfortunately we don’t have hard statistics regarding the volume, as we’re storing this information only in Kibana, but this is common knowledge inside Money Flows and Operations teams, as they need to deal with larger volumes of fund reconciliations.

Data Contracts

The InterAct files are received in this format:

Data Sources

SWIFT network.

Deployment

Services deployed in Kubernetes MUST support concurrency, as there’s expected overlap between the current pod and the previous pod in a deployment.

This is described in the Developers Handbook and it’s a reasonable non-functional requirement.

Dependencies

No specific dependency defined.

Based on RFC Template Version 1.1