missing trades in the futures trade stream

Websocket
1

When I subscribe to the aggregated futures trade stream, I noticed some trades are missing.

The messages come with an aggregated block id and also with a first and last trade ids for that block.

The aggregated block ids are sequential, so this suggests that I am receiving every block of data from Binance, but Binance has dropped some trades in between.

For example:

08/30/2021 13:45:51 FTMUSDT - :warning: missing trades, went from 89496196 to 89496198
08/30/2021 13:45:51 FTMUSDT - :warning: missing trades, went from 89496205 to 89496207
08/30/2021 13:46:19 ADAUSDT - :warning: missing trades, went from 415484061 to 415484070

but this data come from aggregated trades with sequential numbers (I print an error otherwise), so no data is dropped in the transmission.

what is happening here?

you can check the test code here: https://gitlab.com/ThomasD3/binancemonitor
it also highlights another issue with lagging for which I’ll open another question

2

Hi. Thanks for the feedback.
Could you provide some raw data for better investigation? It would be helpful if the record can show timestamp, symbol, and aggregate trade id ‘a’ before and after the missing row.

FYI, you can download the missing data from here ( https://data.binance.vision/?prefix=data/futures/um/daily/aggTrades/ ) when the file is available.

3

Hi,

Here is an example output with the ‘a’ field:

missing trades, went from 285709618:471533389 to 285709619:471533391 , trade time: 09/01/2021 10:26:19.265

In this case, we have:
agg trade id (a) = 285709618, last trade id = 471533389
then agg trade id = 285709619, first trade id = 471533391

So we can see that the agg trade messages are sequential, no data is missing from the stream. But one trade is missing (471533390) in this case.

The number of missing trades varies but it’s usually around 2-3, in average.

I have updated the code on gitlab, so you can run it and see the patterns.

This situation happens a lot, it’s not a rare isolated issue.

4

Sorry for not describing what I want to see clearly.

Could you provide some raw data for better investigation? It would be helpful if the record can show timestamp, symbol, and aggregate trade id ‘a’ before and after the missing row.

I hope you could provide the records you already collect “before” and “after” the missing id (instead of which ids are missing ) so that it’s easier to see the pattern. Besides, the timestamp I meant was UTC time, the date time you showed above must be a local time. This will be helpful to check if those ids are missing from the server side or the client side (your local). Thanks.

5

Alright, checked the data from our end. The trade ids indicated in aggregate trade stream are not the whole. For instance, IF and ADL are not included.