SCT Outgoing SEPA Core

Reference Documents

Reference	Document Location
SEPA Core Incoming	Blueprint for SEPA Core Incoming
Outgoing Payment Service	Blueprint for Outgoing Payment Service
Payments Hub API	Payments Hub API documentation
SCT Incoming RFC	SCT Incoming RFC
Product Requirements Document (PRD)	Product Requirements Document

Problem Description

In order to be able to send outgoing SEPA Credit Transfer (SCT) payments via Santander, we need to make changes to existing SEPA Core service, extending its existing SCT Incoming functionality. This RFC introduces the changes required for achieving the outgoing payment flow, both for messages originating from Ebury services destined for Santander, and for messages originating from Santander, destined for Ebury services.

Background

Santander acts as a sponsoring bank for Sepa Credit Transfer for Ebury for incoming payments. We now need to implement changes in SEPA Core for processing outgoing payments as well, which will cover the second phase of the SEPA Payments Integration with Santander.

Solution

Architecture overview

SCT Outgoing payment flow example

The following diagram illustrates an example flow for an outgoing payment starting from Outgoing Payment Service (P1...P4, P stands for Payment) and a payment status returned from Santander (R1...R7, R stands for Response).

This does not cover all cases but can illustrate the generic flow and the role of the various components in the diagram.

Payment: - P1: An outgoing payment message is produced to the respective Kafka topic by Outgoing Payment Service. - P2: The message is consumed by SEPA Core. Message validation is successful and the message is converted to ISO20022 format (pacs.008) as expected by Santander. - P3: The pacs.008 is backed up to S3 bucket. - P4: SEPA Core sends the payment message via Santander's API.

Payment status (response): - R1: When payment is processed by Santander and the rest of the banking system (EBA, beneficiary bank etc.), Santander posts a message to respective webhook exposed by Kong Gateway. - R2: Benthos instance sinks the message from the webhook to a Kafka topic. - R3: SEPA Core consumes from the Kafka topic and receives the links to download the pacs.002 message. - R4: SEPA Core makes a request to Santander API using the link received in the message, and receives a response with the contents of the pacs.002 (payment status). - R5: The pacs.002 is backed up to S3. - R6: SEPA Core parses the pacs.002, converts it into a message format expected by Outgoing Payment Service and produces it to a Kafka topic. - R7: Outgoing Payment Service consumes the message from the Kafka topic.

Communicating with other Ebury services

SEPA Core will communicate with other Ebury services by producing messages to Kafka topics, at the moment to be consumed only by Outgoing Payments Service, an orchestrator service that will coordinate various Ebury services (such as BOS, PSSDS, the-BAVS, Email Service) to achieve the SCT Outgoing flow logic. Outgoing Payments Service will produce messages to Kafka topics that SEPA Core will consume, perform validations and convert them to the ISO20022 message format, back them up to S3 and eventually send them to Santander via Payments Hub API. SEPA Core will then receive response messages from Santander (such as payment status, payment returns or payment recall accepted/rejected), back up the messages to S3 and produce Kafka messages, consumed by Outgoing Payment Service, in a format that will be determined during implementation.

Receiving messages from Santander

Contrary to the SCT Incoming Payments flow, Santander will not send SCT Outgoing messages to a SQS queue. Instead, it will post to Ebury webhooks, which are publicly exposed by Kong Gateway that handles authentication, incoming traffic routing and load balancing. A Benthos instance will then receive the request and pipe it to a Kafka sink, which will then be consumed by SEPA Core.

Alternatively, because Benthos is yet to be used in Production, SEPA Core can consume the webhooks Santander will post messages to, directly from Kong Gateway.

Since messages sent by Santander to our webhooks will not include the message contents, but only a link to where these contents can be retrieved from, SEPA Core will construct a request to Santander's API to retrieve them. An example request would be something like: https://{root URL}/payments/pacs008/v08/{PaymentsHubId}, with the root URL part being stored on our side and the rest being taken from messages received by Santander. The message contents in the response will be in JSON format and once retrieved, they will be backed up to the S3 bucket (already used by SCT Incoming). Finally, these messages will be converted to a format expected by Outgoing Payment Service before being produced to the respective queues consumed by it (or any other services interested in those events).

Sending messages to Santander

For sending messages to Santander, SEPA Core will convert them into the format expected by Santander's API (construct ISO20022 messages) and call the Payments Hub API, as it currently does in SCT Incoming.

In general, depending on whether we have sent payment before or after cutoff, we expect to receive response same day or next business day. If we do not receive response to payment we have sent in a couple of days, we are not considering it as failed. AML (Anti-Money Laundering) checks could be one of the reasons for holding the payment. In such a case, we can contact Santander to confirm if the payment is still pending or if it indeed failed. We can also query API at any point to get the status of the payment.

Consumers

An 'outgoing flow consumer' will be introduced in SEPA Core that will consume Kafka events that Outgoing Payment Service produces to.

An 'incoming flow consumer' will be introduced that will receive messages originating from Santander. Santander will post messages to our webhooks, exposed by Kong Gateway, and Benthos will sink those messages to Kafka topics consumed by the 'incoming flow consumer'.

Outgoing flow consumer will consume messages for: - outgoing payments and convert them to pacs.008. - payment recalls and convert them to camt.056.

Incoming flow consumer will consume messages for: - payment status as pacs.002. - payment returns as pacs.004. - payment recall positive result as pacs.002/pacs.004. - payment recall rejection as camt.029.

Messages to and from Santander

Common to all messages received from Outgoing Payment Service, destined to Santander, SEPA Core will validate the message, and if invalid, will immediately produce a payment status message to respective Kafka topic. If valid, it will convert it to the respective ISO20022 format expected by Santander, back it up to S3 and send it to Santander via Payments Hub API.

Common to all messages sent received from Santander, the contents of the messages will not be received directly, instead, the messages sent to our webhooks exposed by Kong Gateway and sinked to Kafka topics by Benthos, will contain a link which SEPA Core will use to download the respective message contents from Santander's API. The downloaded contents (ISO20022 formatted messages) will be backed up to S3 before further parsing and processing.

Caveats

We will receive the responses from Santander to webhooks exposed by Kong Gateway and with the use of Benthos instances we will sink them to Kafka topics consumed by SEPA Core. This is the first time that this is used in production.

Since the communication with Santander is decoupled, it does not allow any responses from SEPA Core to Santander, similarly to the current incoming flow. So in case we receive payment messages that we cannot parse for any reason or if they are missing mandatory fields, or if we cannot retrieve the message contents from Santander's API, we can only raise alerts that will then require communication with Santander.

As we progress with the project, we can agree, in coordination with Santander, how to handle these cases and who our point of contact will be.

Operation

The SEPA Core service is an internal service running in Kubernetes, accessible only by other Ebury services, in this case, Outgoing Payment Service. It has external access to send requests and receive responses for those requests, in this example to and from Santander. Any other incoming traffic directly is not allowed.

Payments are sent during working days so support for this service will be 24x5 (24hours during the 5 working days in a week unless they are a Santander banking holiday). Therefore, any runbooks for alerts that would require Support or SRE teams intervention would be valid during those times.

The team responsible for SEPA Core is PAY team (Slack: #payments-team) and as first person of contact is PAY team member, Karim Bechara (karim.bechara@ebury.com).

Deployment

Because payment messages cannot be supported partially, SCT Outgoing flow will not be active in production until all payment flows are implemented and tested in Staging. As parts of the flow will be implemented and deployed to production, we will have logs and alerts in place to ensure things are flowing and behaving as expected, but the SCT payment scheme will not be enabled in production so Outgoing Payment Service will not forward such payments to SEPA Core.

The Staging environment will be utilised to test everything and once the whole flow is completed and fully tested there, only then can it go live. After going live, Penny tests will take place in production to test different payment flows.

Security Impact

The service will be deployed in a private network (Kubernetes cluster). Internal resources will communicate through Kafka topics using SSL and SASL authentication (username/password).

External resources: Santander API will be called using HTTPS and following Payments Hub authentication guide, using JWT Bearer tokens to obtain access token required to access the API. Our webhooks will accept HTTPS requests from Santander, and by using Kong Gateway, the security measures already in place will be followed.

Messages sent and received from Santander will be stored for backup purposes in an encrypted bucket in S3.

Santander's IP addresses will be whitelisted in Kong Gateway to allow requests to the respective webhooks.

Performance Impact

According to the Product Requirements Document, for 6 months (1st of June 2022 to 28th of November 2022) Ebury released 67,307 payments. The assumption is that 75% of those payments would be eligible to be released via SCT scheme, which would mean around 50,000 payments. This was based on the amount of payments we currently send to the SEPA Zone but of course it also depends on the beneficiary's reachability. Therefore, according to the above, SEPA Core is expected to process around 400 payments per day.

Developer Impact

Not applicable.

Data Contracts

Messages received from Outgoing Payment Service will be the same ones that it receives from BOS. SEPA Core will use the ISO20022 format to convert these messages into pacs/camt messages before sending them to Santander. The https://developer.paymentshub.io/en/products/paymentshub_iso20022_v1?tab=documentation&documentationId=message-fields-collection will be used as a reference on how to construct these messages.

Metrics will be maintained for various statistics relating to the payment flows and displayed in a grafana dashboard for readability. Example metrics: - Number of payments received from Outgoing Payment Service. - Number of payments that failed validation. - Number of payments successfully sent to Santander. - Number of payments that failed to be sent to Santander.

Data Sources

SEPA Core data sources: - Outgoing Payment Service (via Kafka topics) - Santander (via webhooks)

Dependencies

These changes heavily depend on the implementation of Outgoing Payment Service (RFC). They also depend on Santander implementing their side of things (sending SCT outgoing payment related messages to our webhooks).