SEPA outgoing files via ABN AMRO

Prepare our system for sending SEPA files from ABN AMRO instead of Arkea bank.

Problem Description

Today, Arkea acts as sponsoring bank for SEPA SCT for Ebury’s BICs under Ebury Partners UK and Ebury Partners Belgium.

As of September 2020, ABM Amro will act as sponsoring bank for SEPA SCT for Ebury’s BICs under Ebury Partners UK and Ebury Partners Belgium. Such as Arkea, ABN Amro is a direct participant in STEP2 (EBA) and can sponsor our BICs as an indirect participant to the SCT scheme.

We moved our sponsorship from Arkea to ABM Amro in keeping the same implementation we´ve got with Sentenial for managing all r-transactions thanks to the Origix Platform.

ABN AMRO bank only sends/receive files via FileAct and Sentenial hasn't got that functionality, so we will be like a proxy between ABN and Sentenial. Here you could see the ABN/Sentenial workflow.

We are currently sending files with FIN format (via AutoClient) for Swift payments, and we will send FileAct files also via Autoclient. The main functionality this RFC is going to cover is to create a new process for sending Sentenial files via AutoClient (FileAct) to ABN.

If you want to know more about ABN/Sentenial and the incoming flow, please review this RFC.

Background

In the past, we have built a new service for managing AutoClient files called "AutoClient Balancer" (ACBalancer) and we will use it for sending Sentenial files via AutoClient. If you want to know more about how are we using ACBalancer, please review this document.

For sending FileAct messages via AutoClient, the back-office application must prepare the following files: * The payload file that you want to send to your counterparty, embedded in a FileAct file transmission. A FileAct payload file is any file not ending with a reserved extension for emission files. The reserved extensions are: .par, .fa, .fin, .ia, .lau. * A companion file, which contains the SWIFTNet FileAct routing information for the associated payload file and specifies how the file must be sent (.fa).

When AutoClient detects a .fa companion file, AutoClient processes the companion file, retrieves the referenced payload file and sends the files to the server.

Sending FileAct files through AutoClient

  1. A data file and its companion file, (.par file, or .fa file in XMLv2 format), prepared on our back-office application, are ready to be sent.
  2. The back-office application places both files in the emission directory, in the following order:
    • data file
    • companion file
  3. AutoClient scans the emission directory, finds both a data file and a .fa file, and starts processing the files.
  4. AutoClient checks whether the files meet the basic requirements.
  5. On successful validation, AutoClient uploads the files to the Alliance Lite2 server.
  6. During upload, the files remain in the emission directory.
  7. On successful completion of the file upload:
    • AutoClient moves the data file, the .par file, and the LAU file (if LAU enabled) to the archive directory.
    • The data file is submitted to FileAct according to the routing information contained in the companion .par file.

We will receive from Sentenial a payload file (.I) and our system should generate the companion file for routing the message.

The structure of the companion file is a JAVA properties file, an XML, or an XMLv2 file.

We will interchange messages between Sentenial and ABN using XMLv2 format (like we are currently doing on the incoming flow).

XMLv2: The .fa extension must be used to identify the XMLv2 companion file. The XMLv2 companion file contains the reference to the location of the payload file. The payload file must be stored under the emission directory or one of its subdirectories. The structure of the .fa companion file is SAA XMLv2 revision 3 preceded by the SAA preamble. The preamblestructure is as follows:

  • Prefix: 1 byte (0x1f)
  • Length: 6 bytes, total length of the signature + DataPDU
  • Signature: 24 bytes (must contain NULL characters if LAU is not used)
  • DataPDU: XML structure

An XMLv2 file contains 31 additional bytes in front of the DataPDU. These 31 bytes (prefix, length and signature) are also present in the XMLv2 file produced by Alliance Lite2.

DataPDU elements to include in our companion file to route a message:

  • Revision: The revision to a version of an XML schema. Absence of the Revision element indicates that the document has revision 0 ("zero").
  • Header: Contains all information relevant to the processing of Alliance Access.
    • Message:
      • SenderReference: Reference provided by the application for reconciliation of a DataPDU carrying a message with the corresponding report DataPDUs. Length is 70 characters.
      • MessageIdentifier: The identification of the message. For an MX message, the format is: ...
      • Format: The message format:
        • for an MT message: MT
        • for an MX message: MX
        • for an FpML message: FpML
        • for a File message: File
        • for any message in an XML format not included in a deployment package: AnyXML
      • SubFormat: The message sub-format. Possible values are: Input (default value) or Output. Both values can be used for the "From" as well as the "To" direction.
      • Sender: The address of the message sender.
      • Receiver: The address of the message receiver.
      • NetworkInfo: Network-related information managed by Alliance Access.
      • SecurityInfo: Security-related information managed by Alliance Access.
      • FileLogicalName: Logical name of the file (1 to 254 characters).
  • Body: Contains the message "text". Present in a DataPDU carrying either a message, a delivery notification, or a report that includes the original message. For an MX message: this element contains the Application Header (if required(2)) and the Business Document as defined in the Solutions documentation. No encoding is required since these structures contain XML data.

An example of a companion file that we should generate:

        001497ZiAxVT0mYBqiuT81pX9q0g==<?xml version="1.0" encoding="UTF-8"?>
        <DataPDU xmlns="urn:swift:saa:xsd:saa.2.0">
           <Revision>2.0.3</Revision>
           <Header>
              <Message>
                 <SenderReference>OEBURGB2LXXX008$2006200000001</SenderReference>
                 <MessageIdentifier>pacs.008.001.02</MessageIdentifier>
                 <Format>File</Format>
                 <SubFormat>Output</SubFormat>
                 <Sender>
                    <DN>ou=rra-et,o=abnanl2a,o=swift</DN>
                    <FullName>
                       <X1>EBURGB2LXXX</X1>
                       <X2>rra-et</X2>
                    </FullName>
                 </Sender>
                 <Receiver>
                    <DN>o=eburgb2l,o=swift</DN>
                    <FullName>
                       <X1>ABNANL2AXXX</X1>
                    </FullName>
                 </Receiver>
                 <NetworkInfo>
                    <Priority>Normal</Priority>
                    <IsPossibleDuplicate>false</IsPossibleDuplicate>
                    <Service>swift.generic.fa!p</Service>
                    <Network>SWIFTNet</Network>
                    <SessionNr>0000</SessionNr>
                    <SeqNr>0000</SeqNr>
                 </NetworkInfo>
                 <SecurityInfo>
                    <SWIFTNetSecurityInfo>
                       <FileDigestAlgorithm>SHA-256</FileDigestAlgorithm>
                       <FileDigestValue>iNb2ZNSFEottD/3phyrn8cEYOozPBh40+FiYNS48igk=</FileDigestValue>
                    </SWIFTNetSecurityInfo>
                 </SecurityInfo>
                 <FileLogicalName>S202SCTEBURGB20200608144742007.I</FileLogicalName>
              </Message>
           </Header>
           <Body>S202SCTEBURGB20200608144742007.I</Body>
        </DataPDU>

Solution - Outgoing process

Our proposal wants to build a new process to receive files from Sentenial and send them to ABN via FileAct format (AutoClient).

For uploading SEPA files in AutoClient, we will use ACBalancer. It will be the service which will only manage the credentials for AutoClient, so in terms of security, this is a mandatory requirement.

Here we want to describe the flow for uploading files to AutoClient.

A new process in FA-Sentenial-Proxy will be reading files from Sentenial SFTP OUT folder, uploading them to S3 and publishing a message with S3 URL to an SNS (and SQS). This SNS/SQS will be the connection between our FA-Sentenial-Proxy and ACBalancer.

ACBalancer will have a new process for reading files from the SQS and then upload them to AutoClient. This new process should create the companion file related to the payload one received from Sentenial, uploading them to AutoClient and also avoiding upload duplicate files to AutoClient.

ACBalancer will contains a lock system for avoiding upload duplicated files to AutoClient. This lock system will be implemented following the same logic like in the incoming flow. ACBalancer will include the pending lock, will do the actions needed and then it will release the pending lock, creating complete one. This behaviour allows us to implement a retry system in case of an error uploading a file.

We've decided to include the companion file generation in ACBalancer because a companion file is related to AutoClient and FileAct messages. A companion file doesn't exist if we don't use AutoClient and we should also include the signature of the file (Only ACBalancer knows this) on this new file. For these reasons, we generate the companion in ACBalancer.

For generating the companion file, we have 2 possible options:

  • Transfomer: Send a payload file to Transformer and it returns the companion one. Transformer is a product designed to provide message transformation, enrichment and XML/non-XML interoperability based on a rich data dictionary. We use this tool for files generation related to our payment executions. As we have pacs, camt and pain official libraries, it's a good idea to use Transfomer for generating companion files, processing the information received from Sentenial (pacs, camt and pain messages).

  • Python: A function for generating a companion file, given the payload one.

Here you could see an image of the outgoing flow using Transformer:

SEPA ABN_OUTGOING_FLOW

Alternatives

  • Generate companion file in ACBalancer (FINAL SOLUTION)

    • Con: ACBalancer doing extra things, not only take files and upload/read them to/from Autoclient
    • Pro: A companion file is a routing file that AutoClient needs for interchanging FileAct messages so in our opinion, the generation of this file should be done by ACBalancer because it's the only Service that knows how AutoClient works. Fa-Sentenial-Proxy should only interchange messages between SEPA and AutoClient. We want to include all processes entirely related to AutoClient in ACBalancer. Furthermore, for the incoming flow, we are also checking the lau and the digest information in ACBalancer so for the outgoing flow we should do in the same Service

  • Generate companion file in Fa-Sentenial-Proxy Service

    • Con: Fa-Sentenial-Proxy should be a proxy between Sentenial and ABN and with this solution we are including AutoClient logic in a Service which is not directly connected to it.
    • Pro: ACBalancer only reading files, signing and uploading them to AutoClient

Caveats

  • We are having lots of connection problems with Sentenial SFTP so we don't want to create a process automatically reading files and uploading them to AutoClient in only one step because we could lose file information.

Operation

This is a new process so we don't need to include a switch in our system. For enabling this workflow we just need to run the new processes created.

Security Impact

To avoid payment fraud, we should follow SWIFT SCP and to do that, ACBalancer should be the only service with AutoClient credentials.

Performance Impact

ACBalancer is prepared to send files to different AutoClients. In case of a massive files reception, we could run more than one process in ACBalancer for uploading files to AutoClient and as we will implement the lock system we won't have problems about upload duplicated files.

We have connection problems with Sentenial SFTP so we should be concern about how are we reading files from Sentenial and the number of connection that we are generating. We will have only one process reading files from Sentenial SFTP.

Technology

* Fa-Sentenial: Python latest version 3.8
* ACBalancer: Python 3.7
* Communication system: AWS - SQS/SNS
* Locking system: Redis
* Deployment: ECS - NAT-only cluster
* Active Monitoring system: Prometheus
* File generation: Transformer 3.7

Deployment

We will need DevOps help to deploy the new processes in production.