I am trying to retrieve a list of futures trades that happened on a clients’ USD-M account for the whole year of 2021. I know for certain that this account has well over 50k transactions for the whole year of 2021.
I am using this endpoint: ‘/fapi/v1/userTrades’. However I have encountered what might possibly be a bug in this API endpoint. This is the process of how initially I had designed the program to get the data:
- Make a call to this endpoint sending the ‘startTime’ parameter to be 1609459200000 (01-01-2021) and the ‘limit’ parameter to be 1000.
- If I get 1000 trades in the response I will take the Id of the last trade received and send another request, only this time I remove the ‘startTime’ parameter and instead use the ‘fromId’ parameter.
- If I get another 1000 trades I repeat the above and take the Id of the last trade retrieved and put it in the ‘fromId’ parameter.
- I stop doing this only when the amount of trades retrieved is smaller than 1000.
- I do all the above steps for all symbols that are supported for USD-M futures trades.
The problem I am facing is that when I make the first call with the ‘startTime’ parameter I only get 602 trades that range from the (01-01-2021) till (04-01-2021) (dd-MM-yyyy). If I move the ‘startTime’ parameter to be 1609718400000 (04-01-2021) I get 555 trades ranging from 04-01-2021 till 10-01-2021. So this proves my point that there is more data than the initial 602 trades received.
I then took the first trade Id I initially retrieved from the 602 trade batch and said let me put it in the ‘fromId’ from the start and not use the ‘startTime’ parameter anywhere. I did this and too my surprise I got 1000 trades, however the majority of them occurred in 2020. I find this to be a bug since the date when the trade placed in the ‘fromId’ parameter happened was 01-01-2021.
Can someone please answer why not all trades are sent back on my first call to this endpoint and why I get data that is before the Id sent in the ‘fromId’ parameter?
- Make a call to this endpoint sending the ‘startTime’ parameter to be 1609459200000 (01-01-2021) and the ‘limit’ parameter to be 1000.
The Account Trade List expects both startTime and endTime parameters to be sent. If one is omitted, the most recent 7 days is returned instead.
- If
startTime and endTime are both not sent, then the last 7 days’ data will be returned.
https://binance-docs.github.io/apidocs/futures/en/#position-information-v2-user_data
Your approach is still achievable with a change to the initial step. Iterate by week until the first trade is identified, then proceed using the fromId parameter.
You may further optimize the above suggestion by using the Aggregated Trades endpoint to directly obtain the market’s first tradeId for your desired timestamp. If the account contains insurance fund or ADLs trades, reduce the timestamp by 1 and perform a timestamp filter on the initial step mentioned above.
This optimization will remove the redunant week-by-week check for the initial tradeId using the startTime parameter, and can be used as the initial tradeId across all accounts per market.
The Aggregated Trades endpoint is for market data. I am working on private client data so unfortunately this cannot be used in my case.
The Aggregated Trades endpoint is for market data. I am working on private client data so unfortunately this cannot be used in my case.
The aggregate endpoint is to be used only to obtain the initial tradeId for the market, which is the same for all accounts as the tradeId is not unique by account, but rather to the market. Ex. BTCUSDT 2021 initial tradeId is 82782038
- The docs state that if BOTH startTime and endTime are not sent the most recent 7 days is returned. I sent the startTime, so either the docs need to be updated or your answer is not 100% correct. Even if what you said is true, the docs go on to say that if BOTH startTime and endTime are not sent, the recent 7 days history is returned. When I sent just the startTime I got back data from 2021, to be precise from 01-01-2021 till 04-01-2021, which is not even 7 days.
The documentation is correct, it was a misinterpretation from my end, however this does not impact the suggested approach.
- I still tried your approach and sent both the startTime and endTime with values being 1609459200000 and 1609934400000 respectively (NOT EVEN 7 DAYS BETWEEN THESE VALUES) I still get the initial 602 trades that were returned when the original problem popped up, where the trades in this response range from 01-01-2021 till 04-01-2021. I know for ccertain that this is incorrect as there are trades on the 5th and 6th.
Could you please provide the full URL used to query the endpoint (omit sensitive data) and a sample trade which should have been in included in the response but was missing?
- If I use the suggested approach to find the first Id and use that from then onwards, I still get in the same situation where data that happened in 2020 is returned, which is BEFORE the Id I found, is retrieved. This is most certainly a bug from the API endpoints’ side.
Can you confirm what the initial tradeId identified is and for which market?
The documentation is correct, it was a misinterpretation from my end, however this does not impact the suggested approach.
This impacts the approach because even when I gave it a startTime and endTime where the range between them is 7 days, I still didn’t get all the data.
Could you please provide the full URL used to query the endpoint (omit sensitive data) and a sample trade which should have been in included in the response but was missing?
I first use this call:
"/fapi/v1/userTrades?startTime=1609459200000&endTime=1609934400000&limit=1000&recvWindow=60000×tamp=1643713805000"
and this give me 602 trades where the time for the received trades ranges from 01-01-2021 and 04-01-2021. I know there were trades for the rest of the time range I gave it. If I take the last Id from those 602 trades so that use the ‘fromId’ parameter with this call:
"/fapi/v1/userTrades?fromId=215262298&limit=1000&recvWindow=60000×tamp=1643714146000"
I start getting the rest of the data, however the endpoint gives me trades that happened back in August 2020 up until February 2021, including the original 602 trades I got from the first response.
Can you confirm what the initial tradeId identified is and for which market?
The first Id happened in this account for 2021 was 1609513565362.
@ryanfalzon You seem to be missing the symbol parameter in both calls. Could you please specify which symbol you are querying?
The first Id happened in this account for 2021 was 1609513565362.
I assume this is the timestamp, not the tradeId. Could you provide the initial tradeId and the symbol? (tradeId is unique across symbol however the same tradeId is present across symbols)
Hi @tantialex I noticed that you were right and I wasn’t sending in the currency (I guess this is another bug as the query did not fail 
). However, I have now implemented the currency property and I am looping through all futures currencies and I still end up with those 602 trades after the looping is complete. Might it be that when we try and request data that goes beyond a year the API endpoint glitches out? because if I move the timestamp from 01-01-2021 to 02-02-2021 everything works out just fine.
I don’t know if its possible or not but perhaps we can schedule a call so that we can show you the actual problem live, because I fear there might be miscommunication when explaining a dev issue over text 
I noticed that you were right and I wasn’t sending in the currency (I guess this is another bug as the query did not fail ).
Thanks for pointing that out. Will escalate it to the team responsible.
Might it be that when we try and request data that goes beyond a year the API endpoint glitches out? because if I move the timestamp from 01-01-2021 to 02-02-2021 everything works out just fine.
No this should not be the case and this is the first time it is being reported. Could you please provide the full URL used and a sample of a missing trade? (omit sensitive data)
I don’t know if its possible or not but perhaps we can schedule a call so that we can show you the actual problem live, because I fear there might be miscommunication when explaining a dev issue over text
Not possible, sorry.
The below request gives me 83 trades
"/fapi/v1/userTrades?symbol=ETHUSDT&startTime=1609459200000&limit=1000&recvWindow=60000×tamp=1643721917000"
The below request give me 1000 trades
"/fapi/v1/userTrades?symbol=ETHUSDT&startTime=1612224000000&limit=1000&recvWindow=60000×tamp=1643717831000"
This is one particular trade missing from the first query:
{
"symbol":"ETHUSDT",
"id":308386368,
"orderId":8389765492096782624,
"side":"BUY",
"price":"1527.59",
"qty":"0.024",
"realizedPnl":"-17.75566232",
"marginAsset":"USDT",
"quoteQty":"36.66216",
"commission":"0.00026321",
"commissionAsset":"BNB",
"time":1612296633954,
"positionSide":"BOTH",
"buyer":true,
"maker":false
}
- The time between
startTime and endTime cannot be longer than 7 days.
There is a 7-day window when querying the Account Trade List endpoint with either the startTime or endTime parameters. If either of the parameters is omitted, the server assumes the missing parameter to be the opposite bound of the 7-day window.
The sample will not be part of the first query because it falls outside of the 7-day window from the startTime.
1 Like