managing websocket buffer

Regarding the last article : increasing delays on @bookTicker streams

Initial Prob: When i subscribe almost all(60~70) futures BookTicker streams through websocket, i endup getting latency (currentTime-transactionTime) of more than 500000ms within few hours.

I’ve tried several approaches suggested there.

  1. Changing t2.micro to t3.micro to get faster network speed,
  2. subscribing to AllbookTicker stream( !bookTicker) instead of combination of individual streams through ‘/’
  3. spliting tickers to several streams so that single stream has only 20-30 tickers each.
  4. regularly close ws and reconnect.

all of methods worked for only short period of time and got back to the same situation again.
I think the reason would be the piling up of the websocket incoming data. eventHandling speed can’t follow up the inflow so the requests are lining up which makes the delay.

So here’s the problem.

  1. How can i know how many requests are piled up?(i.e. ws property that’s indicating the length of buffered request yet to be processed)
  2. Is there any way that i can clear the buffer so that i can give up the piled data and start processing the new one?
  3. Or any other methods that this wouldn’t happen at all?.. I think many people are using AWS EC2 t3a.micro instance and binance wouldn’t make AllbookTicker stream if that popular intance couldn’t handle this. Maybe there’s a proper way to use the websocket ?..

Using nodejs/ws module/Ubuntu EC2 AWS Tokyo t3a.micro/Elastic IP bound.

More streams means more data coming from the connection. Handling network data requires CPU, memory, etc.
If you need the book ticker of all the symbols, it can be a challenge on that client settings.

Spin up a powerful machine or reduce the stream numbers.

@lolsm Did you find a solution to this? I have the same problem with the order book depth updates. It happens only with Binance.