FenX API Integration
As part of the Ebury growth and the difficulties to handle regulations in each jurisdiction, we are moving our current onboarding and compliance processes from Salesforce into FenX. FenX is a complete Client Lifecycle Management tool (CLM) provided by Fenergo that streamlines processes like risk assessment, Anti-Money Laundering or ID&V.
Salesforce needs to communicate with FenX seamlessly as it is the main source of information that will feed the FenX system. This document covers how we are integrating Salesforce with FenX.
Problem Description
Salesforce will continue being the CRM and the land for Sales people to introduce prospects and dealers to store direct communication and actions with clients.
When the prospects are in a suitable stage to be onboarded, the data and onboarding process has to be handed over to FenX. This introduces a number of challenges. The ones that are covered in this document are mainly two:
Data synchronization: There are a number of fields that are shared between systems. Some of them are mastered in FenX but still have to be fresh in Salesforce and vice versa. This requires the integration to establish a communication between systems but also to prevent one to change data that is mastered in the other.
Transition to the new system: we can’t migrate hundreds of back office people to FenX on day one and expect everything is going to work fine. Not only in terms of data, but also on processes, this is going to be a massive change. The transition needs to be progressive and it needs to be a proof of the company is operational. This means that some people and processes will be available to work in FenX following a pilot program while others will remain in Salesforce.
These two challenges require us to describe a detailed process and tools to keep all consistent. Some of the actions will last till FenX is 100% live and running.
Background
Since December 2014 we have been using Salesforce not only as a CRM tool for Front Office but also as a CLM tool for our Back Office teams.
During these years, we have built a big number of flows, services and features to support their daily operations, such as:
-
FX flow (onboarding process): The onboarding team works on an account and performs all the necessary checks to ensure the prospect can become a client.
-
Re-assessment: In the same way we are checking the viability of a prospect to become a client, we need to perform almost the same checks periodically to ensure it is still safe to operate with an existing client.
-
ODD: It is similar to the reassessment flow, but it is run on demand for certain changes in the companies (changes on the shareholding, directors, etc.), the back office team has to review such changes.
-
Risk assessment: Previously using the Risk Macro and now using the CRR service, we are scoring prospects/customers with a Risk Rating. The number of checks can vary depending on the risks, or even further approvals could be necessary.
-
Sanctions screening: during the onboarding and re-assessment processes, we need to screen companies and individuals to ensure that they do not have sanctions or negative news. Such indicators might block them from becoming clients. In the past we were using Lexis Nexis and now we are using Comply Advantage.
-
ID&V: Apart from checking for potential sanctions, we need to ensure customers and related parties are who they say they are, so we are performing some identity verification checks for this. We are mainly using Jumio, but in certain countries we are using more specific tools (not automatic into SF).
However, as far as Ebury is growing into more and more jurisdictions, the complexity of these checks and processes, their particularities and differences are also growing, and we are aiming to delegate these flows and any other compliance processes into a specialised company, Fenergo, and their most recent technology for this, FenX.
Solution
Design overview
The end goal is to effectively remove our compliance back office flows from Salesforce (others like credit or lending will remain there) and have them all in FenX as well as the data that is strictly related with their business needs. But as the process will be a progressive transition, we’ll act accordingly moving partially a combination of features, data and users.
This requires us to progressively adapt our features and thus the migration to the new CLM platform. Back office usual operations like Screening, ODD, etc will be just turned on/off based on whether a client has been onboarded in FenX or not. This includes background routines or nightly jobs performing bulk updates/checks. The new logic for compliance flows would look like this:
A detailed description of how services will be switched on/off has been written here.
In the case of the "initiate onboarding" flow it will have a specific behaviour depending on whether the user initiating the process is in the pilot scope or not. This flow is the starting point where we’ll delegate the work to Fenergo from now on, but we need to keep the out of the scope clients working as they are right now. Initially we’ll rely on the Fen-X Salesforce plugin to initiate the onboarding but it’ll need to be customized to work as we want.
The account differentiation will occur as depicted in the following diagram.
This also affects edits on existing client records through the Salesforce standard UI. Thus, Salesforce will be accepting or blocking edits depending on where the onboarding was done, but will be responsible for keeping an eventual consistency between platforms.
So, Salesforce will update Fen-X records or listen to updates from Fen-X to update Salesforce DB accordingly.
All this translates into a number of requirements to be made by Ebury:
- Move a skinny shape (AKA shell records) of all data in scope into FenX*.
- Enable the synchronization of Salesforce <> FenX in both directions.
- Adopt and integrate with the FenX Salesforce plugin.
- Enable some users to work in FenX for new onboards.
- Switch down features selectively for those users.
- Disable edits on fields mastered in FenX for clients onboarded in the new platform, but sync any edit from FenX to Salesforce.
- Allow edits on all the fields for clients onboarded in Salesforce, but update FenX skinny records .
* #1 is out of the scope of this document and assumed as already in place. It will be done by using the ETL API provided by FenX. It’ll require a relatively small amount of dev work.
To accomplish all the above we’ll need to build a set of tools:
- An API integration from Salesforce to FenX to send updates.
- A framework in Salesforce to customize and interact with the FenX plugin.
- A mechanism to prevent changes in new onboards mastered in FenX.
- A service in Salesforce to listen to updates/new onboards from FenX.
- A service in Salesforce to orchestrate the communication between platforms.
- A framework to easily switch features on/off in Salesforce.
Design details
Let’s see each of these 6 work items individually and how they interact with each other.
1. API integration
Will create an API connector containing a set of methods with the actual calls to their respective rest endpoints. The connector will have the Fen-X API knowledge dealing with authentication, versioning, data transformation and the actual rest endpoint calls.
This layer will cover our toolset #1 meant to perform the actual queries/updates from Salesforce to Fenx.
2. Fen-X plugin framework
Fenergo provides a package that makes it possible for sales to interact with Fen-X. There is a widget to initiate the onboarding process and another to see and interact with the status of a case in Salesforce. The widgets are designed to live in the Account page, but our Sales need to work on the Deal instead where they have all their controls.
This requires us to limit the visibility of the widgets to just the accounts/users in scope (see the switch design document for more details), but also being able to place it in the Deal page (not supported natively).
The visibility piece is easy to resolve with the native Salesforce features. We will set up the visibility to be available only for the users in scope in the Lightning Page Builder.
On the other hand, we’ll need a bit more work in order to make these components to be displayed in the Deal. We’ll create a wrapper UI based in a framework for dependency injection in Salesforce. We have proved it works in this POC’s PR and also got successful results with a few small tweaks for Fen-X in the sandbox env we have created for the project.
3. Switch Engine
We have mentioned the switch design document already, and it has its own document for a reason. It is probably the most difficult part of the integration from the functional standpoint, because it is not an all or none behavior. We need to apply certain rules based on clients onboarded in Fen-X, the user performing the operation (it can be Fen-X itself) and the set of fields that are relevant to the integration.
To accomplish this, we need the aforementioned switch in place. From a high level perspective, we can simplify its responsibilities on:
-
Allowing or forbidding a requested change in Salesforce when applicable.
-
Asking for a request on Fen-X if the data needs to be reflected there (see the Fen-X Service section below).
We’ll see in #4 that Fen-X will notify us for updates and we have to distinguish this scenario. The switch will take this into account to allow these changes to happen in Salesforce but fail in any other use case.
4. Salesforce Webhook
Fen-X can be configured to send events on every change on the platform to webhooks. These events can be as generic as changes to legal entities to even fine grained like approval/rejection tasks on an in progress journey.
Among other things, the plugin will come with a webhook implementation (ETA TBC by Fenergo). It will listen to updates from FenX and keep clients (Salesforce records) in sync based on the mapping set up. However, it couldn't cope with Ebury's (or every other companies) specific architecture, so it focuses on maintaining only the account records updated (client representation in Salesforce). For other event types like new associations, we'll implement a solution on top of it.
To be more precise, the plugin provides an OOTB webhook implementation to listen to FenX events. And when a message is received, the plugin decrypts the messages and produces Salesforce Platform Events in the same order they have been originally emitted by FenX (TBC by Fenergo, otherwise we'll check the event data and will sort ourselves). We’ll use this OOTB plugin implementation to handle any relevant event not already processed by them and perform any necessary action on our side. In essence, we'll "subscribe" to these events and will hook the appropriate logic to create/update the relevant data. To ensure this doesn't conflict with the switch, we’ll set up the platform events to be run by a brand new integration user with specific permission sets.
The module subscribed to the events will delegate the operation to the Fen-X Service.
It is important to note that the webhooks feature is still being built by Fenergo. We have very little information about the final behaviour and structure of the messages. This section will be extended as soon as we have more information.
5. Fen-X Service
This is the glue of the integration. The FenXService will be an abstraction layer between the connector and the Fenergo Salesforce plugin API. It will form the interface the Salesforce domain knows and will encapsulate the logic required to orchestrate the connector/plugin calls. It will be invoked by other layers like the switch engine asking for account updates as well.
Some examples of the logic the Fen-X Service will contain are:
-
Update Entity From Fen-X: receives the ID, fetches the data from Fen-X and updates the Salesforce records accordingly.
-
Create association: receives ids of the Salesforce records and the type of association and makes the necessary number of calls to the FenXConnector. Handles errors and ensures transactionality.
-
Update client: gets the data that needs to be updated in Fen-X and invokes the relevant number of methods in the connector to do it.
The whole architecture would look like follows:
6. Feature toggle
This is actually a small part of the switch (seen in #3). For all onboarding flows, the switch engine will provide a set of methods to tell us if an account can be processed in Salesforce. Say for example we are going to screen an account, then when clicking in the "screen account" it will first invoke the switch engine with this account Id and will return if it is valid to proceed.
The engine will provide a standardised method to throw an exception when the operation is not allowed so that all the features work seemingly. It will be optional though, as some operations might be background processes that just need to skip the processing.
The services in the scope will be stored in environment variables (custom metadata types) so that we can easily toggle them on/off. Obviously this requires us to add that switch engine in all the onboarding services during the transition to finally delete all the code (not only the switch) once we are 100% operational in Fen-X.
Data Model
There is an obvious need of a mapping between FenX and Salesforce data. That modeling exercise is currently being carried out by the Product Team. It turns out to be a hard exercise to do, so we don't expect the definitive design to come soon. It is not an issue from the technical standpoint though as we are making it configurable via environment variables.
For now, we are not making changes at the Salesforce model. We expect it to remain as it is even some time after the pilot period has been completed. After that, we might consider to delete some fields from Salesforce as being no longer relevant for the system. This is however a detail not specified just yet as it depends on the aforementioned Product Team's mapping design.
Data lifecycle
As seen in the first few diagrams, the clients will be initially created in Salesforce and then transferred to FenX when it is ready for the Compliance Teams. At that point in time, the records in Salesforce will be flagged so that some fields are prevented from being changed by a Salesforce user. This is the responsibility of the Switch Engine explained in more detail here. Such set of fields, now mastered by FenX, might be synced back to Salesforce through the webhooks and well ensure the flow stops there.
On the other hand, while the client is being onboarded, new relationships might be added to it (Authorised Contacts, Directors or Shareholders). All these records will be created back in Salesforce when the completion of the onboarding journey event is received in Salesforce. When a record is an actual client in FenX, adding new relationships in Salesforce will be also blocked.
The exception will be the shell records that will always be mastered and updated in Salesforce until they are fully migrated into FenX.
Alternatives
The integration with the FenX API might be placed in its own service in our AWS infrastructure. However, it would not only complicate the design but also be an overhead due to the nature of the Salesforce platform as a separate cloud. The cost of integrating with an ad-hoc service vs Fen-X will be the same plus the service itself.
Caveats
Design overview
This is a quite big project, so it is subject to changing certain bits. The design has been challenged several times and although refined, it didn’t change substantially. The final decision on the data migration strategy might have a small impact on the integration design.
There are lots of decisions to be made by product, but we are in a level of uncertainty where we can make our own ones with respect to technical design.
API integration
Fenergo has built Fen-X following an API first approach and we’ll be using the same API than the Fen-X UI will. Making simple operations turns a bit more verbose than what we would wish, but it has been made in favor of flexibility. It seems they have made it with the UI in mind all the time making the operations run by the API to have a "visual effect" for end users when sometime is not desired. E.g. Modifying an Entity will require a Journey to be created in Fen-X. This might potentially flood the Entity history for onboarding, but there is nothing today to avoid it. The shell records concept should mitigate the number of updates required during the pilot period.
Operation
The above tools must exist before we promote the pilot program as they are key to ensure data consistency. Then, the first data migration can be done to our Fen-X sandbox.
We are in the analysis and design phase where either Fenergo and Ebury are defining the processes, so this work can be done in parallel and before the Fen-X final implementation is in place.
The pilot will consist of a number of users that are still TBD, but much probably will start by small jurisdictions. Then we’ll progressively be moving other small chunks to the new CLM.
Security Impact
We are expecting a positive impact on security as the permissions on Fen-X are going to be redesigned from scratch. Salesforce might sporadically be used as reference, but we are pursuing a well-defined set of permissions we can understand and control for our back office teams that adheres to Ebury's security standards.
Performance Impact
Sales users might experience a small performance impact during the pilot phase as an extra click will be added to send an account to onboarding. We will consider removing it later on. Also, the new flow, although will start at the same point it is starting now, it’ll behave differently as we rely on Fen-X now. But that’s expected and desired as part of the project and an eventual improvement.
On the other hand, performance is one of the reasons why we are moving to Fen-X. Perhaps not a platform performance, but users performance. It is true that the communication from Fen-X to Salesforce will be based on events (async), but it is expected to be nearly real time communication.
Salesforce won’t need to initiate clients in bulk. It will be user actions that will fire the process individually. We have no limitation on the number of daily callouts and haven’t identified any possible performance issues. (TBC response times < 200ms).
Developer Impact
There are a number of fields that will be moved to Fenergo in the long term future as they no longer make sense to Salesforce. We’ll keep the ones that are relevant to BOS to keep the integration unaltered though.
This transition will allow us to understand our users better and clean up the Salesforce code base. This will positively affect the efficiency of the developers creating new features, the security (better control) and the efficiency of creating new environments that is expected to speed up significantly.
Data Consumer Impact
The only affected consumer is the Data Team as they collect all the different onboarding cycle data from Salesforce. They use that to generate several different reports and statistics but also to run ODD processes directly in Salesforce.
This has been notified to them and will have to adapt their reports and in the medium term also the ODD process.
The rest of consumers won’t be affected by the migration to Fen-X.
Deployment
The transition will be phased. We’ll first do an initial data migration and turn the data sync on. The migration plan can be seen here.
Then a small group of sales people will be activated to start sending data to Fen-X. A small amount of back office people will be needed to start working in the new CLM.
We’ll repeat this process until the migration is complete to then start deprecating Salesforce services.
For this we don’t envisage any need from DevOps or other teams to collaborate.
Dependencies
Most of the work described in this document can be started straight away. However, there are some product decisions to be made. Also, we depend on Fenergo to complete the implementation of Fen-X and the Salesforce plugin which is in Beta version right now.