2-factor authentication id pass-through for regulatory reporting

Currently, we are extracting regulatory information about Strong Customer Authentication in payments with 2-factor authentication (2FA) using fuzzy queries, because there are missing links between Verify, EBO and BOS.

Problem Description

Twice a year Ebury has to extract regulatory information about Strong Customer Authentication (SCA) in payments with 2-factor authentication (2FA) in the previous 6-month period.

This report is extracted from BOS and Verify, and reconciling the payments using fuzzy queries. This is done, because currently we're not persisting information about authorization linked with the payment.

This process takes a lot of time to work, due to fuzzy queries being required to ran and manually adjusted and verified to match the ids. Also, since it's mostly manual, it's error-prone, which is specially undesired in a regulatory report.

There is no easy way to link the 2FA with the actual payment using the fuzzy queries. Imagine a user trying to associate 2 different spreadsheets with thousands of records on each one, where there's no direct link between them.

Background

When the client instructs a payment or a group of payments from EBO, the application calls BOS API to create the actual payment. Depending on the workflow, different endpoints on the BOS API are called. The endpoints will be detailed later in this RFC.

Some payment workflows in EBO requires 2FA. Which ones are out of scope for this document, but they fall into 3 different categories:

  • Single payment
  • Many payments
  • Multi payments

When the payments are created in EBO, EBO calls Verify to provide the 2FA authorization for the payment. Verify then responds with OK/NOK and, in case of OK, also provides an authorization-id for that specific payment.

The key part is the authorization-id field is not stored anywhere along with the payment created in BOS.

When a payment workflow requires 2FA, the high-level sequence can be described as:

Happy Path

  • When the payment information is only for one payment (payment entity), Verify returns an authorization-id for this payment. (1 to 1 cardinality)
  • When the payment information is for more than one payment associated with a trade (many payments), Verify returns an authorization-id for all these payments together. (1 to many cardinality)
  • When the payment information is in a multi payment object (multipayment entity), Verify returns an authorization-id for all the payments involved in the multi payment. (1 to many cardinality)

From the EBO point of view, this process is more complex, since EBO has to handle exceptions, timeouts and other cases.

These details are not relevant for this RFC, but you can check them by reading this document created by Miguel Moreno.

Each payment in BOS is represented by the Django model BeneficiaryPayment, pointing to the settlements_beneficiarypayment table.

The simplest possible solution for this problem is just to add one more field to this model but this table has problems with its size and restriction on adding new fields. There's even a proposal to break this table into smaller and more specialized ones.

The simplest possible solution is not feasible in this case.

Solution

The proposed solution requires passing through the authorization-id aggregated along with the current payment info to BOS, so this information can be returned to the user and also reused in future queries when necessary.

The proposed sequence is similar to the current one, with the only difference being the pass-through and persistence of the authorization-id field:

Happy Path

For more details on this sequence, please check slide 12 from this presentation from Miguel Moreno. It contains details on the workflow, including 3 possible responses from Verify after sending a payment for verification: Yes, No and Authorization Required. This RFC is covering the 3rd case, Authorization Required, as the first 2 cases do not trigger 2FA requests.

BOS Requirements

  • Must provide a safe place to store and retrieve the authorization-id
  • Must not add a new field on the settlements_beneficiarypayment table.
  • Must be retro-compatible in the API layer, so the new field must be optional.
  • Must produce the minimal possible impact on performance affecting the BeneficiaryPayment model.

BOS Models and DB

From the Models perspective, there will be a 0..1 relationship between a BeneficiaryPayment and an authorization-id, meaning the field is optional and must have at most 1 authorization for that specific payment.

Because of this "at most 1" constraint, the proposed model requires 2 tables and a clever primary key. Solution2

Alternative: We could simplify this model by using a de-normalized table, but we would lose the ability to enforce the "at most one" constraint at the database level. This "at most 1" constraint would have to be enforced by the application itself during the INSERT phases. Solution2

This solution will allow the reporting to be created by directly matching authorization-id on both sides, instead of using fuzzy queries.

BOS API

It'll be required to modify the APIs described on the section below to accept the new optional authorization-id field.

Adding the field as optional will provide the necessary retro-compatibility needed.

EBO Changes

  • Must pass the authorization-id field to BOS API when necessary.

The BOSApiClient class must be modified to include the optional argument authorization-id in the methods that wrap calls to BOS API.

Before sending the authorization-id to BOS, EBO is responsible for verification of its validity. This should be done by sending the authorization-id with corresponding payment data to the /authorization/check endpoint of the verify service.

Data Information

The authorization-id field is created on Verify, on verify.serializers.CheckViewResponseSerializer.get_authorization_id which uses models from tokens.models.action_token.ActionToken.

    request_id = models.CharField(max_length=64, unique=True, help_text='identify the request from client side')

The field is a Char(64), and contains stringyfied representation of UUIDs. So it's safe to assume this same type/size for storage on BOS.

Examples:

a98fb026-7491-4a2a-8bba-09c7fbebb969
6c45d268-8957-408e-bcf4-e4303f287169
1d9faaf9-b20e-4c44-be58-c59fea544028

Impacted systems, workflows and service endpoints

BOS will be impacted, as it has to be prepared to store and serve the new field on different workflows/endpoints.

EBO will be impacted, as different workflows will have to be adapted, tested and deployed.

Impacted EBO workflows and their correspondent BOS endpoints:

EBO Workflow Endpoint on BOS
Send payment /api/v1.0/client/payment/independent/create/
History of Trades (Add payment) /api/v1.0/client/payments/add/
Add payment Fixed Forward /api/v1.0/client/trade/confirm_complete/
Add payment Spot deal /api/v1.0/client/trade/confirm_complete/
Add payment Drawdown /api/v1.0/client/trade/confirm_complete_drawdown/
Multi payments /api/v1.0/client/multipayment/confirm_complete/

Alternatives

Storing the matching information on EBO

  • PRO: Since EBO has both Verify authorization-id and the PaymentInfo from BOS, EBO could be used as the repository for this matching information.
  • CON: Against EBO architecture. EBO has to be kept as a thin client and store as minimal information as necessary.
  • CON: EBO does not have stored information in the database about payments.

Caveats

In this RFC we're not discussing how possible corrections might be done in the data later, if any change is needed.

What do we do with historical data ?

No study has been done on back filling ids to previous payments in order to load historical data. While it's possible, it would require work on at least these steps: - defining a common layout to import the data into the system; - actually importing the data into the system; - reconciling the imported data against the golden source; - providing a way to correct mistakes;

Security Impact

No security impact is expected by this development.

Performance Impact

On BOS, for INSERTs on new payments that require 2FA, a minor impact is expected due to the overhead of INSERTing on 2 extra tables. The actual impact should be measured by the programmer at the development stage, as this table and model are a key part of the system. Testing concurrent load is a good idea.

We have to take a special attention to the Multipayment case, as we're going to create several Payment2FA objects in the database when the Multipayment requires 2FA.

On BOS, for QUERIES of the BeneficiaryPayment model, no impact is expected since the new field will be added as a lazy-loading and will touch the DB only when absolutely necessary and not for the daily activities.

Deployment

The deployment must be done in order, with BOS first. The proposed deployment strategy is:

  1. Deploy BOS with the database and model changes
  2. Add a switch on BOS to activate / deactivate the API changes, so we can disable it if we have performance issues in production, specially with multipayments
  3. Deploy BOS with the API changes
  4. Observe the changes in production for some time (a couple of days will suffice) to confirm retro-compatibility
  5. Deploy EBO changes passing the authorization-id to BOS
  6. Observe EBO changes in production for some time and confirm the new data saved in BOS
  7. Remove the switch added on step 2.

Dependencies

There are no external dependency for this implementation.

Developer Impact

Since both EBO and BOS systems will be impacted, it'll be necessary to allocate developers from both teams to implement the changes and run the appropriate tests and quality control.

Credits and External Documents

Tools used for this RFC