When I’m trying to send very long subscribe command, the server does not respond, until the regular Ping comes from it.
It seems like the WS server reads the incoming web data in chunks of 4096 bytes and expects that every message that comes to it fits into this size. However, the documentation says nothing about the limits of how many subscriptions can I use in their live “SUBSCRIBE” method.
So, I investigate this a little and found out that the if the number of subscriptions in “SUBSCRIBE” between 200 and 300, the server suddenly switches from returning the result of “SUBSCRIBE” and the new subscribed data to just silence. After some more investigation, I could calculate the number of bytes in the message that goes on the wire while “SUBSCRIBE”-ing. So, if the number of bytes in the message is more than 4088 (plus 8 header bytes, the total is 4096), the server discontinues to respond.
I attach the script that shows that when trying to subscribe to:
- 216 aggregated trades, the “SUBSCRIBE” command is 4088 bytes long, so we receiving response;
- 217 aggregated trades, the “SUBSCRIBE” command is 4106 bytes long, so we failed to receive any response in several seconds (5 by default) window;
- 256 regular trades, the “SUBSCRIBE” command is 4076 bytes long, so we receiving response;
- 257 regular trades, the “SUBSCRIBE” command is 4091 bytes long, so we failed to receive any response in several seconds (5 by default) window;
To run the scenarios above, use the following commands:
cargo +nightly -Zscript binance-rust-client.rs -n216 --agg-trade
cargo +nightly -Zscript binance-rust-client.rs -n217 --agg-trade
cargo +nightly -Zscript binance-rust-client.rs -n256 --trade
cargo +nightly -Zscript binance-rust-client.rs -n257 --trade
Even more investigations also show, that the workaround exists. To every additional chunk of data we could send the PING message
in order to force the server (somehow) to read one more chunk. So the number of additional PING-s is floor(MSG_SIZE / 4096). E.g. when the size of the message is 10_000, we have to send floor(10_000 / 4096) = 2 PING-s to ensure the server will read our message fully. The other case when the server reads one more chunk is its own (server’s) PING (or, more accurately, our PONG to that PING).
So, my final questions are:
- do I have to use this workaround or maybe some simpler means exist?
- how can I force to fix or maybe just document this behavior of the Binance server?