Im trying to pull more than 1000 candles with multiple api calls

Struggling to grab more than 1000 candles from Binance’s API without missing a beat? You’re not alone. Here’s a swift rundown:

The provided code does a decent job fetching candlestick data from Binance, but it consistently misses the last three candles. Why? The culprit lies in how it calculates the start time for each iteration. By simply adding one second to the timestamp of the last fetched candle, it overlooks potential gaps between candles.

To remedy this, adjust the code to set the start time for the next iteration just after the timestamp of the last candle fetched.

s_time = int(new_klines[-1][0]) + 1

Here’s the full code for your reference:

def _get_kline(self, cc, interval, limit, start_time='none', end_time='none'):
    """ Get historical kline by {interval}, {limit}, {time_end}, and {time_start} => Returns a json. """
    params = {
        'symbol': cc,
        'interval': intervals[interval],
        'limit': limit
    }
    if start_time != 'none':
        params['startTime'] = start_time
    if end_time != 'none':
        params['endTime'] = end_time

    return self._get(binance_endpoints['klines'], params)


def get_historical_kline(self, cc: str, interval: str, dataframe_limit: int, start_time=None, end_time=None, get_ts=False):
    klines = []
    curr_time = round_down_timestamp(get_curr_ts(), 15)
    s_time = reduce_from_timestamp(curr_time, 15, dataframe_limit)
    while True:
        # Fetch Klines
        new_klines = self._get_kline(cc, interval, 1000, s_time)
        # Append fetched Klines to the existing list
        klines += new_klines
        # If less than 1000 Klines were retrieved, we have reached the end of available data
        if len(new_klines) < 1000:
            break
        # Update start time for the next iteration
        s_time = int(new_klines[-1][0]) + 1

    return klines_to_dataframe(klines, get_ts)