How to manipulate amounts in EBO
We need to reach an agreement about how we are going to manipulate amounts when making calculations without rounding issues on EBO.
Problem Description
The floating point in JavaScript, Python and most other programming languages are based on the IEEE 754 standard, which means the numbers are represented as number times a power of two, so rational numbers like 1/10 or 1/3 cannot be represented exactly as a power of two.
Some examples are:
// JavaScript:
0.1 + 0.2 = 0.30000000000000004
0.8 - 0.1 = 0.7000000000000001
5.11 * 100 = 511.00000000000006
1.4 / 10 = 0.13999999999999999
# Python:
0.1 + 0.2 = 0.30000000000000004
0.8 - 0.1 = 0.7000000000000001
5.11 * 100 = 511.00000000000006
1.4 / 10 = 0.13999999999999999
As we can see, we get the same results for both languages.
In order to avoid these common floating point issues, we should work with some library which takes care about decimal precision. The proposed solution is use currency.js in JavaScript and decimal in python.
Using the libraries mentioned, if we repeat the same calculations, we get the next:
// JavaScript (using currency.js):
currency(0.1).add(0.2).value = 0.3
currency(0.8).subtract(0.1).value = 0.7
currency(5.11).multiply(100).value = 511
currency(1.4).divide(10).value = 0.14
# Python (using decimal):
from decimal import Decimal
Decimal('0.1') + Decimal('0.2') = 0.3
Decimal('0.8') - Decimal('0.1') = 0.7
Decimal('5.11') * Decimal('100') = 511.00
Decimal('1.4') / Decimal('10') = 0.14
But even choosing this solution proposed, we still could have issues working with decimals because is where we loose their precision, ie:
// JavaScript (using currency.js):
let amount = null;
amount = currency(1).divide(3).value = 0.33
currency(amount).multiply(3).value = 0.99
amount = currency(5).divide(3).value = 1.67
currency(amount).multiply(3).value = 5.01
# Python (using decimal):
from decimal import Decimal
Decimal('1') / Decimal('3') * Decimal('3') = 0.9999999999999999999999999999
Decimal('5') / Decimal('3') * Decimal('3') = 5.000000000000000000000000001
In EBO users can create a trade and when creating the trade they may specify the sell amount and currency and not the buy amount. This means that when users want to add payments to that trade, they are asked for the payment amount in the sell currency, but for BOS to create the payments it needs the buy currency / amount of the payments. This means we need to convert the sell currency to buy currency before sending the request to BOS and doing these calculations could lead to rounding issues as we saw in the previous examples.
Background
Let's take the following as an example. A client wants to pay three benes 1000 EUR each but in USD and they do not know, or mind, the exact USD amount. They will enter the trade details specifying a sell amount of 3000 EUR. If they then want to add payments to this trade, When they come to enter the payment amounts, they will need to specify the payment amount in EUR. If each payment was then selected to be EUR 1000 then ideally the payments would be created using this value but BOS cannot create the payments using the EUR 1000 amount and must create the payments in USD. This means that EBO must convert the payment amounts into USD and then send that amount to BOS which will then create the payments in USD. This can lead to rounding errors.
Some more examples are:
1) Case 0.01 less (Rate: USDEUR 0.3)
Trade: 3000 EUR -> 10000 (3000/0.3) USD
1000 EUR -> 1/3 -> 3333.33
1000 EUR -> 1/3 -> 3333.33
1000 EUR -> 1/3 -> 3333.33
Total of payments: 3333.33 * 3 = 9999.99
2) Case 0.01 more (Rate: EURGBP 1.666666)
Trade: 3000 EUR -> 5000 (3000 * 1.666666) GBP
1000 EUR -> 1/3 -> 1666.67
1000 EUR -> 1/3 -> 1666.67
1000 EUR -> 1/3 -> 1666.67
Total of payments: 1666.67 * 3 = 5000.01
The feature about having sell operation (a user fills the sell amount input field and is asked to add payments for the sell currency) is not new, it is something we already have in the EBO legacy, and we want to keep it for "Convert and Pay" given there are a significant number of users using it (around 25% of trades).
In the EBO legacy is being dealt in his own backend, working with floats and multiplying or dividing for the rate depending on the rate symbol, which causes some rounding issues as historically we have had many bugs for the misalignment between what was calculated on EBO and what was finally calculated on BOS.
Solution
The solution proposed for the previous problem, is adding or removing the difference to the last payment. Working from the previous examples shown:
1) Case 0.01 less (Rate: USDEUR 0.3)
Trade: 3000 EUR -> 10000 (3000/0.3) USD
1000 EUR -> 1/3 -> 3333.33
1000 EUR -> 1/3 -> 3333.33
1000 EUR -> 1/3 -> 3333.33
Total of payments: 3333.33 * 3 = 9999.99
10000 - 9999.99 = 0.01 + 3333.33 (last payment) = 3333.34
Total of payments: 3333.33 * 2 + 3333.34 = 10000
2) Case 0.01 more (Rate: EURGBP 1.666666)
Trade: 3000 EUR -> 5000 (3000 * 1.666666) GBP
1000 EUR -> 1/3 -> 1666.67
1000 EUR -> 1/3 -> 1666.67
1000 EUR -> 1/3 -> 1666.67
Total of payments: 1666.67 * 3 = 5000.01
5000.01 - 5000 = - 0.01 + 1666.67 (last payment) = 1666.66
Total of payments: 1666.67 * 2 + 1666.66 = 5000
So, if we see that the sum of the payments are equal to the sell amount of the trade, it means the user wants the trade fully allocated, so if making calculations the sum of the payments is smaller / greater than the total amount, we could adjust the error, adding or removing it to the last payment.
We also need to have the same behaviour of the other parts of the system, because JavaScript uses "ROUND_HALF_UP" rounding type, but Python3 by default is "ROUND_HALF_EVEN". In order to be consistent, we should be rounded using "ROUND_HALF_UP", and we can set it globally in Python. About round and quantize methods for rounding issues, we should use quantize because is more accurate for monetary applications.
Said that, to create a trade with payments based on a sell operation, we need to convert the payments amounts from the sell currency to buy currency, making calculations with decimals through a library to manage the loss of precision and adjust the error, but where should be that conversion code making calculations?
Alternatives
- In the EBO frontend making calculations using currency.js and show how the payments are distributed.
- In the EBO backend, similarly how is already implemented but making calculations with decimal library.
- Allow from BOS receive sell amounts for payments and calculate the final payment amounts, where EBO would not need to be concerned about it.
- Create a kind of payments system service.
Caveats
N/A
Operation
N/A
Security Impact
N/A
Performance Impact
To keep the feature, it should not exist any potential performance impact on the system, given we are going to make some calculations, and the maximum number of payments we can create on C&P are 10.
Developer Impact
Depends on what decision is taken, we will need to adapt EBO to BOS or BOS to EBO or integrate a new service.
Data Consumer Impact
N/A
Deployment
Depends on what decision is taken, it will need to deploy EBO or BOS or a new service.
Dependencies
At first, there are no dependencies to implement this RFC.