SEPA incoming files via ABN AMRO

Prepare our system for receiving SEPA files from ABN AMRO instead of Arkea bank.

Problem Description

We are currently receiving/sending SEPA movements via Arkea bank (with Sentenial), but in a few months, we will receive these files via ABN AMRO bank.

ABN AMRO bank only sends files via FileAct and Sentenial hasn't got that functionality, so we will be like a proxy between ABN and Sentenial. Here you could see the ABN/Sentenial workflow.

Sentenial is a third party which is giving us some functionality for managing payments/account that we don't have implemented in our system so will keep continue using them.

We are currently sending/receiving files with FileAct format (via AutoClient) for Swift payments, and we will receive ABN files also via Autoclient. The main functionality that we should implement is build a proxy for interchange ABN files files between AutoClient (FileAct) and Sentenial SFTP.

To get more information about the AutoClient process, you could review this page.

AutoClient is an SFTP used to send and receive messages and files over SWIFTNet. We use more than one AutoClient and our current process in FXS reading from the reception folder, it is only reading from one AutoClient instance. As we are receiving the same message in all our AutoClient instances, a process in our side called "Garbage Collector" is removing the files that were previously processed for avoiding duplicate files.

To get more information about our SEPA workflows, you could review this guide.

Background

In the past, we have built a new service for managing AutoClient files called "AutoClient Balancer" (ACBalancer) and we will use it for processing ABN files via AutoClient. If you want to know more about how are we using ACBalancer, please review this document.

We've already ask product about if we will receive the same files in all our AutoClients or only in one. They think so (because this is how AutoClient works), so we will share our proposal including a lock system for avoiding uploading to Sentenial and AutoClient duplicated files. In case that this will change, we could remove the lock system part.

Solution - Incoming Process

Our proposal wants to build a new service to be a proxy between AutoClient and the Sentenial SFTP.

For reading/uploading SEPA files in AutoClient, we will use ACBalancer. It will be the service which will only manage the credentials for AutoClient, so in terms of security, this is a mandatory requirement.

Here we want to describe the process for reading and uploading files to AutoClient.

Here you could check the incoming process proposal.

ACBalancer was build to made the connections between AutoClient and Ebury services so the new process should be very simple. This will take files from a new folder for SEPA files in AutoClient, it will upload these files to S3 (as backup), and will publish a message (S3 url, not the file) in an SNS (and SQS).

Instead of create a new process for managing duplicate files as we are doing with the "Garbage Collector", we want to read files from all our AutoClient instances and include a lock system (using the one we have at Ebury with Redis) to avoid consuming the same message twice. We propose to manage this (as a first step) on the service which will upload files to Sentenial SFTP to avoid to include complexity to ACBalancer process. On a second phase, ACBalancer will only send to Proxy Service the message once.

The process for uploading files to Sentenial SFTP is a new one completely different to the SEPA workflow for incoming/outgoing Sentenial files, so we don't want to include this new functionality in our SEPA Service. We suggest to create a new service to be the proxy between ACBalancer and the Sentenial SFTP. We could call it "Proxy Service" during this phase. This process will be processing messages from the SQS, including a "processing" lock, and once the file is properly uploaded to Sentenial, the "processing" lock will be released and a new one on "complete" status will be generated. This will manage the concurrency problems previously commented. We want to create it based on the lock system implemented on Swift Service.

Following the same way of working, we will also create a new process in our Proxy Service for reading files from Sentenial SFTP and send it to a new SNS (and SQS). Then, ACBalancer will upload these files to AutoClient. You could check that on L3 proposal outgoing files tab.

Incoming Project Phases

In this section, we would like to analyze how we will adding value in each project phase.

Phase 1

We will create a new process in ACBalancer reading files from only one AutoClient, uploading files to S3 and sending the message to the deadletter. Create a new service (Proxy Service) for uploading file to Sentenial SFTP. This service will have a process reading files from the SQS and uploading them to Sentenial SFTP (without retries or managing errors).

Phase 2 Include error management and retry system in Proxy Service.

Phase 3 Include check concurrency in Proxy Service. We will include a lock system in this new service for avoiding upload the same file twice to Sentenial SFTP (lock system).

Phase 4 ACBalancer will read files from all our AutoClient instances and sending all these files to the SNS. We will start "cleaning" files from other AutoClients at this point (our Garbage Collector process). At this point, before active, we should execute an script for cleaning files from other AutoClients. I.E, if we will implement the lock system like the one we have in our Swift Service, we will include a lock in Redis with "complete" status for each file during a month. If after a month we will try to process the same file, we won't find it on Redis so we could process it again.

Phase 5 ACBalancer will be reading files from all our AutoClient instances but only sending one message to the SNS.

Phase 6 Improve Sentenial connections for keeping connections open. With this change we will try to decrease the number of Sentenial connections.

Summary We could active the project in phase one but if something fails uploading the file to Sentenial SFTP (or any error happens), we should manage each error manually. Furthermore, if AutoClient is down, we could not process SEPA files and is a huge effort detect in other AutoClient instances which files were processing and were not. We suggest to finish phases 1,2,3 and 4 to enabled it in production.

Alternatives

  • Create a new service (Proxy Service) for processing/sending files from/to ACBalancer and processing/sending files from/to Sentenial SFTP (FINAL SOLUTION)

    • Con: Plus of time for creating the new service and CI process (1 week)
    • Con: We need to set up an active health check system with Prometheus so more connections to Sentenial SFTP
    • Pro: We keep logic separated (abstract code), we don't need to change SEPAService. This logic is not affecting the current SEPA workflow, is a different one
    • Pro: SEPA-Service has django and another additional stack that we won't need to use

  • By infrastructure with Autoclient in AWS

    • Con: Devops do it
    • Con: There are too many questions about how to do this (no time estimation)
    • Con: We never did this before
    • Pro: Flexible and you avoid a lot of code

  • ACBalancer managing read from AutoClient and uploading files to Sentenial SFTP

    • Pro: The faster solution
    • Con: We break keep it ACBalancer simple doing only Autoclient things
    • Con: Too many erros to handle

  • Swift service for processing/sending files from/to ACBalancer and processing/sending files from/to Sentenial SFTP

    • Con: Going around
    • Con: ABN files received are not Swift files so is not related to Swift Workflow
    • Con: Include lock system for incoming files (we are not doing this for incoming error files)
    • Pro: Including lock system for incoming files in Swift Service will be reuse for Swift incoming process
    • Pro: All FileAct files go to swift service

  • SEPA service processing/sending files from/to ACBalancer and processing/sending files from/to Sentenial SFTP

    • Con: More process to SEPA Service and we want to include a refactor in the future.
    • Pro: Everything is already done, you just need to add new processes.
    • Pro: Sentenial is everything in the same place.

Caveats

  • We are having lots of connection problems with Sentenial SFTP so we don't want to create a process automatically reading files and uploading them to Sentenial in only one step because we could lose file information.

Operation

This is a new process so we don't need to include a switch in our system. For active this workflow we just need to run the new processes created.

Security Impact

To avoid payment fraud, we should follow SWIFT SCP and to do that, ACBalancer should be the only service with AutoClient credentials.

Performance Impact

ACBalancer is prepared to read files from different AutoClients (not only one).
In case of a massive files reception, we could run more than one process in Proxy Service for uploading files to Sentenial SFTP.

We have connection problems with Sentenial SFTP so we should be concern about how are we uploading files to Sentenial and the number of connection that we are generating.

Technology

* Base Code: Python latest version 3.8.1
* Communication system: AWS - SQS/SNS
* Locking system: Redis
* Deployment: ECS - NAT-only cluster

Deployment

We will need DevOps help to deploy the new service in production.