Why my Websocket connections not getting trades after some time?

Hi

I’m accessing Trade Streams (@trade) programatically, one websocket connection per symbol.

I’m getting trades but after one to three minutes times trades stop coming. I can still send websocket PONG messages without failures.

I send a websocket PONG control message immediately after I receive a websocket PING control message from Binance for the symbol. I assign 0xA as PONG opcode and send back the PING message’s payload and it’s length in PONG message.

Binance Websocket Market Streams documentation does not specify what I’m supposed to send as PONG message’s payload.

Please help. Appreciate.

Regards
Ling

See the changelog entry below:

2020-04-23

WEB SOCKET STREAM

WebSocket connections have a limit of 5 incoming messages per second. A message is considered:
A PING frame
A PONG frame
A JSON control message (e.g. subscribe, unsubscribe)
A connection that goes beyond the limit will be disconnected; IPs that are repeatedly disconnected 
may be banned.
A single connection can listen to a maximum of 1024 streams.

You need to implement a rate limiter to ensure that you limit websocket incoming messages to 5 per second. I suspect that’s whats causing some of your sockets to disconnect.

Have you tried connecting to a combined stream instead? You would only have to worry about one socket connection.

1 Like

Thank you Mathrio, I have the same feeling, Ling’s client is disconnected due to over the message limit, hope this change log can be helpful for him to debug. :smiling_face_with_three_hearts:

please can you show us your ping and pong frame ?

How often do you send pong messages? There is a time based limit of 5 per seconds to how many socket payloads you can send.

Hi Mathrio, thanks for the reply.

Yes, I had a burst send of PONG messages to every symbol on every minute since today (test runs only). But after seen your reply, I immediately stopped that. But I can still connect and get trades for multiple symbols.

Now the issue is when I start 8 symbols, for one or two symbols do not receive PING at all within first 180 seconds and they stop, Another two symbols, one or two PING come and they too stop in another two or three minutes time.

I still like to know what’s the payload expected by Binance for PONG messages?

  • Ling

Are you subscribing to a combined stream or 8 single streams?

It’s possible that you are sending too many pings at once and Binance disconnects you because you broke their limit.

Have you programmed a heartbeat function to keep the steams alive from your end?

There is no expected payload for pongs, you can simply send a noop function.

  1. I’m subscribing for 8 single streams, therefore, 8 separate websockets connections.

  2. I do not send PING to Binance. I only send PONG. But there was a critical bug in our application that we were sending two PONG messages to one Binance PING. One PONG from our websockets library and one from our app. Now that is fixed.

  3. Now our app is much improved. When our app run 8 separate websockets connections to Binance, now we get trades for those 8 asset pairs (symbols) for more than 15 mins (until we stop). But when we add one more asset pair (9th websocket), in about 3 mins one of those connections lose trade arrivals. It’s not the last connection added.

So the question is, is there a limit on number of websockets connections can make to Binance from an IP address?

– Ling

Thanks Mathrio and Dino

Our issue is still not fixed yet, we are now investigating our app to see is it due to application bug. This might take one or two days.

We have not built a rate limiter yet. We only send PONGs to Binance PINGs that we receive. That too is automatic by our framework and we have no control on that. We do not receive more than 5 Binance PINGs a second yet. It seems our websocket connections do not get disconnected, we do not receive Websocket Close control messages to sockets having this issue.

The changelog entry is for a single websocket connection. Ours is multiple websocket connections to Binance. I hope there is no limit on the number websocket connections that we can run in parallel to Binance. I have mentioned this early, but I did not receive a definitive answer yet. Hope there is no limit or else we have to restructure our application. We prefer multiple websocket connections for the reason if one or more connections are down, other connections are not affected.

– Ling

Thanks Ling.

First of all, we need this websocket message limit for server side protection. It’s enough for most clients that listen streams by combined them together.

I can see the difficult in your side. Many websocket client libraries handle the ping pong heartbeat automatically. In most clients/applications, this is a good implementation, library user don’t need to worry about sending pong message back, everything is done smoothly.
But when we talking about building a trading bot, everything changes. In a “normal” client, it’s not common to see client has a lot of streams listening to ws server. But in crypto trading, it’s what every traders doing. If the library is sending pong response automatically, with multiple ws connections, WS server can easily see the client sending message over the limit.

So in this case, there are some suggestions:

  • combine streams
  • disable auto pong heartbeat
  • split connections into different clients.

Hope this will be helpful.

Hi Dino

We are trying to reorganise our application so that 1) Connect once to Binance 2) Send SUBSCRIBE to trade streams subsequently.

We noted the Websocket documentation does not specify the path requirement for connect only. We tried endpoint wss://stream.binance.com:9443 with path “/”, “/ws/” and " ", but all fail to complete the websocket connection to Binance.

  1. What’s required to specify for the path to connect only?
    It clean for us if the connection and subscriptions to streams are separate.

  2. In case, currently, if the connection and subscriptions to streams are not separate, can the Binance implement a path requirement to connect only. Binance may enforce a timeout (eg. 10 mins) in case if the new connection is not subscribed within a certain time period to disconnect.

– Ling

This is already implemented, you are allowed to connect to the websocket server only:

wss://stream.binance.com:9443/ws or
wss://stream.binance.com:9443/stream

The difference of them is wss://stream.binance.com:9443/stream will return stream name in the response messages.

over the connection, send message to server for subscription or unsubscription, e.g.

{"method": "SUBSCRIBE","params":["btcusdt@miniTicker"],"id": 1}
{"method": "SUBSCRIBE","params":["btcusdt@kline_1h"],"id": 2}
{"method": "UNSUBSCRIBE","params":["btcusdt@miniTicker"],"id": 3}

The message has to be json string, invalid json format message will cause disconnection from server.

Hi Dino

I can safely say switch to single websocket to Binance solved our issue.

We ran more than 2hrs continuously with 20 live trade subscriptions and worked without an issue.

Thanks again to Dino and thanks for the valuable input of Mathrio too.

Best regards
– Ling

that’s great, happy to be helpful :slight_smile:

@Dino, i’m facing the same problem.
i have 200 symbols each one have one socket and after a while it stops working and gives me the following error message
“sent 1011 (unexpected error) keepalive ping timeout; no close frame received”

this is my websocket code

can you please help me?