Fees Ledger
Ebury transactions cost a small amount of fee for the customer. Service charge on behalf of the company.
This charge (Fee) used to be applied on a per-trade bases. However recently the base for Fees has chaged from trade to a payment level.
The new processes have not yet been extended with Quantum reporting, which is now to be added within the scope of this task.
Problem Description
Quantum reports can't just be simply generated from Ebury transactional data automatically. Specific reporting workflows have to be implemented, suited for each target, so that it could be introduced in Quantum reports.
These workflows mainly consist of a strict data collection and serialization process, combined with a small degree of additional logic, that is regularly part of the requirements.
The base structure of current reporting workflows are the following.
- Extracting required information per transactional object
- Serializing extracted information to JSON
- JSON is transformed into QXT
Transactional objects that are targets for Quantum reporting are inserted into specific queues at creation time, and/or on each change. Generally speaking, the queue is regularly processed (multiple times per hour) according to the steps above. (Only a few workflows differ from this schedule, accumulating data per week/month/etc.)
We have to create the custom workflows for the new Payment Fees and Investment Fees transactional objects.
After their initial creation, a list of transactions may apply to Fees (funding, return, write-off, cancel, etc.). All related transactions generate Ledger Entries, visualized on the BOS UI Fee Ledger view. Now, within the scope of the current project, we need to report on all those Ledger Entries to Quantum.
Example: Fees ledger display with a set of Fees in different statuses.
We have the liberty to choose whether we prefer to report those changes on a per transaction bases, or rather as summarized values per day.
Two existing Quantum Instruments are to be used for QXT Record generation: PaymentFee and InvoicedFee.
Reporting goes along "the usual" Fee reporting rules. We need to report both on credit and debit in separate records (regardless whether individual or cumulative reporting).
Background
The new fees have been gradually introduced in 2021. Now it's only the Quantum reporting missing to, to reach full functionality.
The diagram below is showing the BOS Fee Account display for a Client's Investment Fees.
Solution
We recommend a tactical solution for this task, given pressing timelines, and major gaps in the Ebury 2.0 infrastructure and services, that would be essential foundations for this project to be able to go 'strategic'.
On the other hand, in BOS all related workflows and displays are ready available.
Going tactical, all we need to do is to
- extend existing financial workflows with passing on new objects for Quantum reporing at creation or modification (BOS)
- extend data collection and serialization workflows according to the new requirements and characteristics (BOS/TMS)
- re-use already existing QXT generation (FxSuite)
All the above is pretty straightforward.
Thus we are off-loading the tactical solution from implementing any business logic, and we only use the obsolate frame as "rails", to deliver reports via well-established channels.
Though summarized end-of-day reporting would be significantly more efficient than reporting per ledger, sadly it wouldn't quite fit the existing frame (data structures, workflows, QXT format, etc.). Furthermore, reporting calculated values over data aggregations requires particularly careful considerations in terms of audit trail.
All together it's non-trivial, requires careful planning... And most likely would rather be an inapproriate approach "squeezed" or "hacked" into existing legacy environment, where the proposed tactical solution is to be implemented. (See also Alternatives)
Thus, in the Phase 1, tactical solution of the task, we'd rather stick to the per-ledger reporting approach.
Alternatives
Strategic Soluton
Ideally we should not be adding new workflows to an obsolate system...
Yet, given the delays impacting pioneer Ebury 2.0 components, we would not be able to meet our deadlines if moving away from BOS. Implementing missing parts of the system would heavily increase the volume of the project, and put timely deliveries at risk.
Fees reporting workflows should be moved out of BOS similar to BrokerDeal E20 plans (see BrokerDeal Ebury Company Entity Branch Reporting, Phase 2 onwards). Yet at the time when this blueprint is being written, none of related new services would be available, but targets of ongoing/future work.
Of course, introducing a new service may not be particularly elaborate. However combined with deep and considerate definition of all data flows impacted by the current task, would increase the overhead to extents, that can not fit within the scope and delivery dates of this task.
Summarized reports
Instead of per-ledger reporting, it would be a lot more efficient to send daily credit/debit summaries to Quantum.
This feature is a good candidate for the Phase 2 stage of the project, carried out as an E20 solution. Namely, there should be a separate service responsible for generating summarized data, in a way that it's
- async
- traceable (audit trail!)
A need for summarized reporting is not unique to this project, and there is ongoing effort relating to this subject. Thus we hope for already existing patterns available to follow by the time this issue could be addressed.
Caveats
Tactical solution, preferably developed as portable as possible. Also, documentaion has an elevated importance, allowing to move the workflow out from the current frame hopefully in the future...
Operation
The new workflow will be implemented within the already existing reporting workflows, executed once a day.
Security Impact
No security impact is to be considered, there is no higher sensitivity on data in question, than data that's being equally processed in related workflows.
Performance Impact
Sadly current Quantum reporting workflows aren't most efficient. Within that frame this new process could be considered reasonably lightweight, with its Postgres data aggregation, bulk queries and once-a-day execution.
Developer Impact
The workflows must be accompaigned with comprehensive docs, allowing for fellow developers to get familiar with them, if needed. Undocumented Quantum reporting workflows easily get "mysterious", requiring non-negligable time spent on reverse engineering when modifications may need to be introduced on later requests.
Data Consumer Impact
The task is operating over existing data, without changing any of it. The only data-related "change" within these workflows is to mark processed entries in the TMSBuffer, as 'processed'.
Deployment
The new workflows will be introduced gradually, potentially with additional protection of feature flags, allowing for smooth rollback in case.
Dependencies
All dependent workflows (Payment Fees, Invoice Fees) are now in Production.
References
More detail on upcoming work will be reflected on the Fees Ledger TRE team epic.