New candlestick/kline generation behavior

Can you explain me this situation.
In websocket API I have stream btcusdt@kline_1m and sometimes when is that important for me i am checking candle status. When Candle status is True I am trying download last few candles using rest api klines function.
And situation looks like when i have ready signal from websocket API this latest candle (actual which one is under building time) one time exist and other time not. It have random begining in time.

This solution is not acceptable for me, because does not make sens to use REST API /klines? It generate only error in saved candles. NOW IS USLESS

For me it looks like binance dont generate empty candle after time cross full minute. They are waiting for first trade in new minute.

Befor last update in April 2020 that worked normal as should. They generate empty candle.

Does binance can fix this?

But if you dont get the kline from current minute, then you now 0 volume and last price of the last candle is open price for the new one, which is also highest and lowest price…

Empty klines are generated in the spot exchange after the minute boundary passes if there are no trades but it does take a few seconds for it to happen. An empty kline for the new minute will not be made as soon as it its xx:xx:xx.000, it can take up to 10 seconds.

1 Like

But before the upgrade it worked well. Immediately, I have an empty candle. For me it is a bit confusing.
I am asking for advice.
How can I download the correct number of candles? if binance sometimes gives me an empty candle in a klines? sometimes not.

Regards

There’s been no change to this behavior in any recent release.

In my opinion Binance have to generate empty candle when time cross xx:xx:xx.000, and they have to do it immediately and independently from information from websockets or rests APIs.

  1. Websocet should show that candle is ready and toghether with this info en unfinished candle should exist.
  2. rest API should show unfinished candle when time cross xx:xx:xx.000

Of course when i am using stream API i know that candle should be read and I can download data from date - to date. That is clear.
But… If you are using a only REST API for example when you download last 200 candles you can not be sure what is in newest candle. Question with no answer now is?

Was newest (last in the list) candle finished and program should save it, or is not finished and program should remove it as unfinished!?

If that info is not clear for API users, that make your API useless, because can be situation that program do not remove unfinished candle and next will download from unfinished candle!

Binance cannot say that situation like before will never happen. Because in long period of time this can be happened. One time program can catch situation with existing unfinished candle and other time can catch situation when unfinished candle is not prepare for API.

I do not know how to explain better this situation.

Candlesticks are uniquely identified by their openTime. When reading candlestick data from the restApi, the latest candle is the is newest and considered “open”. Once the next kline is generated, the previous one is closed and will not be modified.
To use a concrete example, if you access: https://api.binance.com/api/v3/klines?symbol=BNBUSDT&interval=1m&limit=2
The first candlestick is closed; the second candlestick is open.

So in your situation, if you want “save all closed candles” then you should consider a candlestick closed when the next candlestick is created.

1 Like

That is good that I discuss with someone who understand API.

I will answer your question later. I have to prepare an answer.

I would like to say that this bug is first one which I would like to discuss on forum. And really I would like to be convinced that the bug is in my scripts.

Ninj0r do you represent Binance side?
How can i attache a file, print-screen or show you a piece of any code?
Or should I describe what should you write and that will be enough?

You can copy and paste your code in here and wrap in it 3 back ticks (`) to mark it as code:

{
  "exampleType": "JSON",
  "formatted": true
}

You can also upload files/screenshots by clicking the “upload” button, but putting your code in here as text is better.

So again, there is no bug. The behavior is:

So in your situation, query the klines using startTime=“openTimeOfTheKlineYouAreAt”. If the size of the returned array is 1, that kline is still open. If it’s > 1, then it’s closed. Every kline returned is closed except for the final kline (assuming you haven’t hit the limit for the number of records to be returned which defaults at 500).

Here is an important piece of code which show that i do nothing special :

print('Time from file int: {}'.format(startTime))
print('url: {}'.format(url))
# for example https://api.binance.com/api/v3/klines?symbol=BTCUSDT&interval=1m&limit=1000&startTime=1589431380000

candles = requests.get(url).text

data = json.loads(candles)

# print(data)
print('Amount of return candles: {}. Test {}'.format(len(data), 'Pass' if len(data) == 3 else 'Fail'))
if len(data) != 3:
     print('Here is missing unfinished candle')
print('Candles: ')
for candle in data:
     print(candle)

As we can compare to https://binance-docs.github.io/apidocs/spot/en/#kline-candlestick-data link looks normal.

Now you have to carefully look on print screen. and you will see what happened there.

That is the problem. I do not have unfinished candle (sometimes).

I made short test with https://api.binance.com/api/v1/ but result seems to be the same - missing candle

Where is the unfinished candle?

It’s being made.
The candle can be found here: https://api.binance.com/api/v3/klines?symbol=BTCUSDT&interval=1m&limit=3&startTime=1589431500000

The system is an asynchronous system so the next candle will not be made instantly every time. If you get less than 3 klines in your request, I recommend waiting for a few seconds and then trying again.

To solve this problem you first have to know when a candle is going to close, to find out we can check the k.x value of the data returned by the kline websocket, when the k.x value is true it means that the candle is about to close and not that has already closed, this value is set to true at 999 milliseconds of the current candle, that is, the candle has not yet closed but in the next call to onmessage event it must already be closed, so the following code may cause that when requesting the last 3 candles only return 2 in some occasions as you can see in the image

 socket.onmessage = e => {
    data = JSON.parse(e.data)
    if (data.k.x)  klines(symbol, interval, limit, startTime, endTime, data => console.log(data))
 }

To fix this problem we have to cause a delay so that the request is executed in the next onmessage event that is executed after k.x is set to true which we can do in two ways either by using a global flag or by using a timer as follows

 candleClosed = false`
 socket.onmessage = e => {
    data = JSON.parse(e.data)
    if(candleClosed) {
            candleClosed = false
            klines(symbol, interval, limit, startTime, endTime, data => console.log(data))
    }
    if (data.k.x) candleClosed = true

Or

 socket.onmessage = e => {
    data = JSON.parse(e.data)
    if (data.k.x) setTimeout(klines(symbol, interval, limit, startTime, endTime, data => console.log(data)), 2000)
 }

Which solves the problem as can be seen in the image

Finally, if the only thing you want is to obtain the last three candles, you only need to set the limit of the query to 3 and you avoid having to calculate the startTime (because I suppose you are calculating it or at least saving it in an array), from so the request should be the following

https://api.binance.com/api/v3/klines?symbol=XXXX&interval=1m&limit=3

3 Likes

This solution worked best for me, I was pulling REST results with painful timers and getting “dupe results”, but in the end I adopted the websocket strategy with the REST functions explained in Saratoga’s reply. Everything works as intended.

1 Like

Thank you so much for taking the time to post this although I did not employ the websocket solution it illustrated how the Kline candlestick data works.

For me, the key is to recognize that the last candle only pops up once given sufficient time for all asynchronous operations from binance to complete

I am using python making requests for example to get 1h Kline candles for past 7 days using start and end time Paramus Using the binance rest API

I simply wait for the top of the hour to trigger and then sleep for 2 more seconds and by then the Klines are always ready with the full set. I should expect 168 candles since that’s how many hours are between 7 days.

If I don’t wait sleep the extra 2 seconds then sometimes I return 167 candles.

also the actual last close candle is at index -2 and the previous closed candle is at index -3 because the candle at index -1 hasn’t actually closed yet

Hello.
Thank you for answer. It is helpfully to see how other make it.

For me the problem is that we have to call get_candles minimum twice and that reduce ours limits. Is`nt it?