How to download historical ticks for last 2-3 days from Binance server?

API Futures API
1

My goal is to have a least last 2-3 days of ticks till current moment without gap.
At this moment I am interested in cryptocurrency futures.

I downloaded one day of agg trades from link
< Binance Data Collection >
But I am not able to download missing two day ago from current day using REST endpoint aggTrade.

For example today 2022-01-25 but I am able to download only 2022-01-23 or earlier.
So to fill a gap from 2022-01-24 T00:00:00 till 2022-01-25 Tcurrenttime I need to download missing tick data using the GET /fapi/v1/aggTrades endpoint.

I make requests from 2022-01-24 T00:00:00 for 1000 trades in forward circle. But Binance server allows to download only 9 hours of missing trades and then I receive a message:

aggTrades request error: {“code”:-1003,“msg”:“Too many requests; current limit of IP(182.32.229.148) is 2400 requests per minute. Please use the websocket for live updates to avoid polling the API.”}

But I make less then 2400 requests per minute. I make delay in 300 msec between my requests(1000 trades per aggTrades endpoint)
Maybe there is limit of total requests?

Is there some way to get last 2-3 day of ticks without gap?

2

Could you specify which endpoint you are referring to when you mention “aggTicks”?

4

But I make less then 2400 requests per minute. I make delay in 300 msec between my requests(1000 ticks per aggTrades endpoint)
Maybe there is limit of total requests?

A weight is attributed to each endpoint. This value represents how much of the request bandwidth is consumed per request. The 2400 “requests per minute” limit factors in weight.

Therefore, invoking /fapi/v1/aggTrades, which has a weight of 20, consumes 20/2400 requests per minute. Multiply this by an interval of 300ms and you will hit the rate limit in 40s. To avoid this scenario, you should increase the interval to > 500ms.

If you intend to keep up to date with aggTrades, I suggest using the websocket to avoid hitting rate limits.

3

GET /fapi/v1/aggTrades allows me to download only up-to-date 8-9 hours for BTCUSDT.

Initially I need last 2-3 days of agg trades till current moment but in the future I would fill gaps less then 3 days. All downloaded trades will be saved to file on my computer. 9 hours limit would be enough later.

5

Thanks for reply.
Thus, 20/2400 requests per min = 0.0083req / 1min.
It means 1min / (20/2400req) = 120min / 1req.
It equals to 2 hour per one request. My calculations correct?
if yes then it is not acceptable.

API doc say: Each route has a weight which determines for the number of requests each endpoint counts for

Please explain how to use ‘weight’ in order to calculate number of requests allowed per min.

How to download historical data with WebSocket? I read it is used for real time trade flow only.
For example my program did not receive trades tonight and how to fill 8-10 hour gap at the morning with WebSocket?

6

Thus, 20/2400 requests per min = 0.0083req / 1min.
It means 1min / (20/2400req) = 120min / 1req.
It equals to 2 hour per one request. My calculations correct?
if yes then it is not acceptable.

Sorry for the confusion, I did not intend to suggest that the endpoint consumes 20/2400 requests per minute.

The endpoint consumes 20 of the bandwidth. Therefore a bandwidth of 2400 will be consumed in 2400/20 = 120 requests to the endpoint.

Please explain how to use ‘weight’ in order to calculate number of requests allowed per min.

Rate limit per minute / weight = Requests per minute

How to download historical data with WebSocket? I read it is used for real time trade flow only.

This is correct. The websocket returns real-time data, not historicals.

For example my program did not receive trades tonight and how to fill 8-10 hour gap at the morning with WebSocket?

The gap will need to be recovered using the REST API.

7

So, aggTrade endpoint bandwidth limit is calculated this way:
(2400 req / 20 weight) / 1 min = 120 req / 1 min
It means I have to make delay between reqs => 60000 ms (1 min) / 120 req = 500 ms.
I will try to make delay more or equal to 500 ms.
Thanks now it is clear.

But I already used req delay = 500ms but got the same error “code”:-1003.
I will try 550 ms.

8

The response header X-MBX-USED-WEIGHT-1M can be used to keep track of the weight used.

9

I will try to check this header. Maybe server rises weight for my IP because I requests aggTrade too often. Or weight is fixed for each endpoint?

As I understand bandwidth of different endpoints is calculated with this formula:
(2400 req / endpoint_weight) / 1 min
where 2400 req is the same for each endpoint but endpoint_weight may be different

10

I will try to check this header. Maybe server rises weight for my IP because I requests aggTrade too often. Or weight is fixed for each endpoint?

Weight is fixed.

where 2400 req is the same for each endpoint but endpoint_weight may be different

IP weight rate limit is shared across all endpoints, therefore all endpoints contribute to the same bandwidth.