How to avoid losing data across User Data stream disconnect/reconnects

Websocket
, ,
1

We use “order update” messages from User Data streams to create blotters for the purpose of trade surveillance and risk management. (Here, “order update” messages are "e":"executionReport" for spot and "e":"ORDER_TRADE_UPDATE" for futures). We do not place orders; we are a third party that only monitors trading activity on behalf of our clients.

The documentation states that we should expect a server-side disconnection at least once every 24 hours, and we do see this commonly. Upon disconnection, our software will reconnect within one second. My question has to do with how to ensure data integrity across these disconnects. There seems to be no way to detect what messages might have been missed during the reconnection lag.

There are helpful suggestions on this forum to manage these disconnections by creating a redundant websocket connection and synchronizing the messages based on the “update ID” field. This sounds reasonable, but unless I’m mistaken there is no field in the order update messages that serves as an “update ID”, nor any kind of message sequence number. Did I overlook something in these messages? (Reference: Binance API Documentation).

Is there a field in the messages that we could reliably use as a message sequence number? If not, then how exactly are we supposed to maintain data integrity across the forced disconnections? What about the case where an infrastructure problem (for example) caused an extended disconnection? It would be nice if the websocket API offered true message sequence numbers that allowed for easy gap detection and also had a “replay” ability for back filling gaps – virtually all FIX feeds have these capabilities. Given that the websocket feeds don’t have these capabilities, and there doesn’t seem to be a field in the messages that serves as a sequence number, then what is the correct method to ensure that no data is lost during a forced disconnect/reconnect cycle?

Thanks for any insights.

4

I am in a similar situation, it seems amazing to me that there is no way to do this. What workaround did you end up going with?

EDIT: Actually, it seems like you can use the U or u fields, no?

2

You can make use of the newClientOrderId when placing an order to keep track of ORDER_TRADE UPDATE messages. If you did not receive an update after placing an order, you should use the restful endpoint to query the order status (Binance API Documentation).

3

Thanks for the reply. We do not place orders – we are a third party that only monitors trading activity on behalf of our clients. We don’t know a priori when orders will be placed. In order to make use of your suggestion, we would have to levy a requirement on our clients to (possibly) change their trading software to always use newClientOrderId, and we would also have to set up stand-aside real-time channels with our clients to obtain newClientOrderIds so we would know to expect to see them in the websocket feed. Unfortunately, this is very likely impossible. We try very hard to not interfere with our clients’ existing workflows.