Kline stream returns mismatched tinterval data

dziekonskik · September 9, 2023, 5:22am

Hi, is this a bug i request 5m time frame on ETHBTC and i get updates for 5m time frame as well as 1h time frame

{
[0]  |   openTime: 1694236500000,
[0]  |   closeTime: 1694236799999,
[0]  |   open: 0.06317,
[0]  |   close: 0.06316,
[0]  |   high: 0.06317,
[0]  |   low: 0.06316,
[0]  |   baseVolume: 9.7678,
[0]  |   tradesCount: 20,
[0]  |   isClosed: false
[0]  | }
[0]  | {
[0]  |   openTime: 1694235600000,
[0]  |   closeTime: 1694239199999,
[0]  |   open: 0.06314,
[0]  |   close: 0.06316,
[0]  |   high: 0.06317,
[0]  |   low: 0.06313,
[0]  |   baseVolume: 24.3725,
[0]  |   tradesCount: 83,
[0]  |   isClosed: false
[0]  | }
[0]  | {
[0]  |   openTime: 1694235600000,
[0]  |   closeTime: 1694239199999,
[0]  |   open: 0.06314,
[0]  |   close: 0.06316,
[0]  |   high: 0.06317,
[0]  |   low: 0.06313,
[0]  |   baseVolume: 25.0995,
[0]  |   tradesCount: 84,
[0]  |   isClosed: false
[0]  | }
[0]  | {
[0]  |   openTime: 1694236500000,
[0]  |   closeTime: 1694236799999,
[0]  |   open: 0.06317,
[0]  |   close: 0.06316,
[0]  |   high: 0.06317,
[0]  |   low: 0.06316,
[0]  |   baseVolume: 10.4948,
[0]  |   tradesCount: 21,
[0]  |   isClosed: false
[0]  | }
[0]  | {
[0]  |   openTime: 1694236500000,
[0]  |   closeTime: 1694236799999,
[0]  |   open: 0.06317,
[0]  |   close: 0.06316,
[0]  |   high: 0.06317,
[0]  |   low: 0.06316,
[0]  |   baseVolume: 10.4948,
[0]  |   tradesCount: 21,
[0]  |   isClosed: false
[0]  | }

it is easy to spot by the volume and number of trades, should I be filtering this data before sending to the frontend or is it done with some purpose? Or is it a bug? Thanks

jonte · September 11, 2023, 2:23am

Hi, thanks for alerting us - it’s a strange occurrence. How often are you getting these unexpected 1h klines? Can you please show me your code implementation?

dziekonskik · September 13, 2023, 6:43am

Hi jonte, thanks for getting back to me, I did not have the time for further development since then so this one a one time i saw it. This is the route handler

import { observable } from "@trpc/server/observable";
import { EventEmitter } from "events";

import { BinanceSocket } from "../../../clients/binance/WebSocketClient";
import { publicProcedure } from "../../../routers/trpc";
import { throwError } from "../../../utils/binance/errorHandlers";
import {
  CandlestickInput,
  CandlestickStreamSchema,
} from "../../../validators/binance/anaysis/candlesticksValidator";

import type { Candlestick } from "../../../types/analysis";

const ee = new EventEmitter();

export const candlestickStream = publicProcedure
  .input(CandlestickInput)
  .subscription(async ({ input }) => {
    await new BinanceSocket(
      `ws/${input.symbol.toLowerCase()}@kline_${input.interval}`,
      ee
    );

    return observable<Candlestick>((emit) => {
      const update = (data: Buffer) => {
        try {
          const jsonData = JSON.parse(data.toString());
          const { k } = CandlestickStreamSchema.parse(jsonData);

          const newCandlestick: Candlestick = {
            openTime: k.t,
            closeTime: k.T,
            open: Number(k.o),
            close: Number(k.c),
            high: Number(k.h),
            low: Number(k.l),
            baseVolume: Number(k.v),
            tradesCount: k.n,
            isClosed: k.x,
          };

          emit.next(newCandlestick);
        } catch (error) {
          if (error instanceof Error) {
            emit.error(error);
            throwError(
              error,
              `ws/${input.symbol.toLowerCase()}@kline_${input.interval}`
            );
          }
        }
      };

      ee.on("update", update);
      return () => {
        ee.off("update", update);
      };
    });
  });

and this is BinanceSocket class just in case

import { EventEmitter } from "events";
import WebSocket from "ws";

import { throwError } from "../../utils/binance/errorHandlers";

export class BinanceSocket {
  #ws: WebSocket;
  #ee: EventEmitter;
  readonly #endpoint: string;

  constructor(endpoint: string, ee: EventEmitter) {
    this.#ws = new WebSocket(`${process.env.BINANCE_WS_URL}/${endpoint}`);
    this.#ee = ee;

    this.#endpoint = endpoint;
    this.#ws.on("open", this.onOpen);
    this.#ws.on("message", this.onMessage);
    this.#ws.on("close", this.onClose);
    this.#ws.on("error", this.onError);
  }

  private onOpen = () => {
    console.log("WebSocket connected");
  };

  private onMessage = (data: WebSocket.Data) => {
    this.#ee.emit("update", data);
  };

  private onClose = () => {
    console.log("WebSocket disconnected");
  };

  private onError = (error: Error) => {
    throwError(error, this.#endpoint);
  };
}

please let me know if you need any more details or if you would be able to advise on how to deal with this thanks!

jonte · September 13, 2023, 9:30am

Thanks for providing the code. I see that the Kline Interval is determined based on the input it receives. The Klines returned from your Websocket subscription should therefore always be of interval input.interval.

However, you can implement something similar to the my following example in order to automatically re-subscribe with the correct Kline Interval if it ever receives a Kline with an invalid or unexpected interval:

import { observable } from "@trpc/server/observable";
import { EventEmitter } from "events";
import { BinanceSocket } from "../../../clients/binance/WebSocketClient";
import { publicProcedure } from "../../../routers/trpc";
import { throwError } from "../../../utils/binance/errorHandlers";
import {
  CandlestickInput,
  CandlestickStreamSchema,
} from "../../../validators/binance/anaysis/candlesticksValidator";

import type { Candlestick } from "../../../types/analysis";

const ee = new EventEmitter();

// Mapping of valid intervals and their durations in milliseconds
const validIntervals = {
  '1m': 60 * 1000,
  '3m': 3 * 60 * 1000,
  '5m': 5 * 60 * 1000,
  // Add other valid Kline intervals here...
}; 

export const candlestickStream = publicProcedure
  .input(CandlestickInput)
  .subscription(async ({ input }) => {
    const expectedInterval = input.interval; // Track the expected interval

    const subscribeWebSocket = () => {
      new BinanceSocket(
        `ws/${input.symbol.toLowerCase()}@kline_${expectedInterval}`,
        ee
      );
    };

    // Initial WebSocket subscription
    subscribeWebSocket();

    return observable<Candlestick>((emit) => {
      const update = (data: Buffer) => {
        try {
          const jsonData = JSON.parse(data.toString());
          const { k } = CandlestickStreamSchema.parse(jsonData);

          // Calculate the interval based on openTime and closeTime
          const intervalDuration = k.T - k.t;

          // Find the matching interval
          let detectedInterval = null;
          for (const interval in validIntervals) {
            if (intervalDuration === validIntervals[interval]) {
              detectedInterval = interval;
              break;
            }
          }

          if (!detectedInterval) {
            // Interval mismatch, resubscribe
            console.log(
              `Received Kline with unrecognized interval. Resubscribing...`
            );
            subscribeWebSocket();
            return;
          }

          if (detectedInterval !== expectedInterval) {
            // Interval mismatch, resubscribe
            console.log(
              `Received Kline with interval ${detectedInterval}, expected ${expectedInterval}. Resubscribing...`
            );
            subscribeWebSocket();
            return;
          }

          // The rest of your code to process the valid Kline event
          // ...
         // ....

If you do end up implementing some check like this, it would be great if you could log it and provide it here as it should not be happening.

Cheers,

dziekonskik · September 13, 2023, 2:59pm

Hi, thank you for the directions, I will implement them, till now I’ve been sending string literal like so

const KlineTimeInterval = z.enum(["5m", "15m", "30m", "1h", "4h", "1d", "1w"]);

export const CandlestickInput = z.object({
  symbol: z.string(),
  interval: KlineTimeInterval,t
});

I thought that this is what the api expects, in the docs there is no mention that it should be numerical reprsentation, Binance API Documentation. If i will have errors after implementing your code I will post them. Thanks!

jonte · September 14, 2023, 1:59am

Yes you’re correct, the valid intervals are strings like "5m", "15m", "30m", "1h", "4h", "1d", "1w".

What the code I posted does is detect the interval of each received kline by subtracting the openTime from the closeTime and then converting that to the matching valid interval string. Eg. 60000 == "1m", 300000 == "5m", etc. It compares that with the expectedInterval from your input/the stream you’re subscribed to <symbol>@kline_<expectedInterval>. If it doesn’t match, it will then re-subscribe to the stream using the expectedInterval.

Again, it shouldn’t ever happen but to log/save as much context as possible within the if (detectedInterval !== expectedInterval) { statement and post the logs if it ever does happen to occur.

dziekonskik · September 15, 2023, 5:57am

Hi jonte i have implemented the code as you asked, this is the current version

import { EventEmitter } from "events";

import { BinanceSocket } from "../../../clients/binance/WebSocketClient";
import { publicProcedure } from "../../../routers/trpc";
import { throwError } from "../../../utils/binance/errorHandlers";
import {
  CandlestickInput,
  CandlestickStreamSchema,
  type KlineInterval,
} from "../../../validators/binance/anaysis/candlesticksValidator";

import type { Candlestick } from "../../../types/analysis";

const ee = new EventEmitter();

// Mapping of valid intervals and their durations in milliseconds
const validIntervals: Record<KlineInterval, number> = {
  "5m": 5 * 60 * 1000 - 1,
  "15m": 15 * 60 * 1000,
  "30m": 30 * 60 * 1000,
  "1h": 60 * 60 * 1000,
  "4h": 4 * 60 * 60 * 1000,
  "1d": 24 * 60 * 60 * 1000,
  "1w": 7 * 24 * 60 * 60 * 1000,
};

export const candlestickStream = publicProcedure
  .input(CandlestickInput)
  .subscription(async ({ input }) => {
    const expectedInterval = input.interval;
    const subscribeWebSocket = () => {
      new BinanceSocket(
        `ws/${input.symbol.toLowerCase()}@kline_${expectedInterval}`,
        ee
      );
    };
    subscribeWebSocket();

    return observable<Candlestick>((emit) => {
      const update = (data: Buffer) => {
        try {
          const jsonData = JSON.parse(data.toString());
          const { k } = CandlestickStreamSchema.parse(jsonData);

          // Calculate the interval based on openTime and closeTime
          const intervalDuration = k.T - k.t;

          // Find the matching interval
          let detectedInterval = null;
          for (const interval in validIntervals) {
            if (
              intervalDuration === validIntervals[interval as KlineInterval]
            ) {
              detectedInterval = interval;
              break;
            }
          }

          if (!detectedInterval) {
            // Interval mismatch, resubscribe
            console.log(
              `Received Kline with unrecognized interval. Resubscribing...`
            );
            console.log({
              expectedInterval,
              intervalDuration,
              detectedInterval,
            });
            subscribeWebSocket();
            return;
          }

          if (detectedInterval !== expectedInterval) {
            // Interval mismatch, resubscribe
            console.log(
              `Received Kline with interval ${detectedInterval}, expected ${expectedInterval}. Resubscribing...`
            );
            console.log({
              expectedInterval,
              intervalDuration,
              detectedInterval,
            });
            subscribeWebSocket();
            return;
          }

          const newCandlestick: Candlestick = {
            openTime: k.t,
            closeTime: k.T,
            open: Number(k.o),
            close: Number(k.c),
            high: Number(k.h),
            low: Number(k.l),
            baseVolume: Number(k.v),
            tradesCount: k.n,
            isClosed: k.x,
          };
          console.log(
            `emitting new candlestick of interval ${expectedInterval} time: ${new Date().toISOString()}`
          );
          emit.next(newCandlestick);
        } catch (error) {
          if (error instanceof Error) {
            emit.error(error);
            throwError(
              error,
              `ws/${input.symbol.toLowerCase()}@kline_${input.interval}`
            );
          }
        }
      };

      ee.on("update", update);
      return () => {
        ee.off("update", update);
      };
    });
  });

as you can see i have subtracted one on the interval i am testing because i was getting those console logs:

 Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '5m',
[0]  |   intervalDuration: 299999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '5m',
[0]  |   intervalDuration: 299999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '5m',
[0]  |   intervalDuration: 299999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '5m',
[0]  |   intervalDuration: 299999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '5m',
[0]  |   intervalDuration: 299999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '5m',
[0]  |   intervalDuration: 299999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '5m',
[0]  |   intervalDuration: 299999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '5m',
[0]  |   intervalDuration: 299999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '5m',
[0]  |   intervalDuration: 299999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '5m',
[0]  |   intervalDuration: 299999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | WebSocket connected
[0]  | file:///Users/seven/Dev/trading_spot/packages/api/src/utils/binance/errorHandlers.ts:58
[0]  |     throw new TRPCError({
[0]  |     ^
[0]  | 
[0]  | TRPCError: An unexpected error occurred on ws/ethbtc@kline_5m : AggregateError
[0]  |     at throwError (file:///Users/seven/Dev/trading_spot/packages/api/src/utils/binance/errorHandlers.ts:58:11)
[0]  |     at BinanceSocket.onError (file:///Users/seven/Dev/trading_spot/packages/api/src/clients/binance/WebSocketClient.ts:31:13)
[0]  |     at WebSocket.emit (node:events:514:28)
[0]  |     at emitErrorAndClose (/Users/seven/Dev/trading_spot/node_modules/ws/lib/websocket.js:1016:13)
[0]  |     at ClientRequest.<anonymous> (/Users/seven/Dev/trading_spot/node_modules/ws/lib/websocket.js:864:5)
[0]  |     at ClientRequest.emit (node:events:514:28)
[0]  |     at TLSSocket.socketErrorListener (node:_http_client:495:9)
[0]  |     at TLSSocket.emit (node:events:514:28)
[0]  |     at emitErrorNT (node:internal/streams/destroy:151:8)
[0]  |     at emitErrorCloseNT (node:internal/streams/destroy:116:3) {
[0]  |   code: 'INTERNAL_SERVER_ERROR',
[0]  |   [cause]: undefined
[0]  | }
[0]  | 
[0]  | Node.js v20.5.0
[0]  | [nodemon] app crashed - waiting for file changes before starting...
[0]  | [nodemon] restarting due to changes...
[0]  | [nodemon] starting `node --experimental-specifier-resolution=node --loader ts-node/esm src/app.ts`
[0]  | (node:41724) ExperimentalWarning: Custom ESM Loaders is an experimental feature and might change at any time
[0]  | (Use `node --trace-warnings ...` to show where the warning was created)

advise me please, i don’t have much exp with websockets, if i resubscribe without cleaning up previous connection those would stack up? I was getting more and more console logs and eventually the server crashed. After subtracting 1 from the equation it seems not to mismatch the candles, hovewer those rarely come in 2 sec interval as you can see on the console logs below:

 emitting new candlestick of interval 5m time: 2023-09-15T05:04:35.869Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:05:00.082Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:05:02.546Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:05:13.594Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:05:18.560Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:05:43.623Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:06:44.129Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:07:26.885Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:07:29.551Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:07:40.072Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:07:43.827Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:07:46.449Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:07:55.345Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:08:00.836Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:08:10.356Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:09:17.105Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:09:21.554Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:09:43.257Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:09:55.104Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:10:00.080Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:10:02.560Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:10:07.088Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:10:10.666Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:10:13.549Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:10:25.127Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:10:31.250Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:10:52.926Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:10:57.529Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:11:15.895Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:11:27.469Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:11:30.196Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:11:32.565Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:11:35.562Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:11:38.564Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:11:55.627Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:12:11.499Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:12:13.567Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:12:16.599Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:12:21.019Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:12:47.497Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:12:51.161Z
[0]  | emitting new candlestick of interval 5m time: 2023-09-15T05:12:53.566Z

I will keep monitoring this endpoint and leave the console logs, plase tell me if this 1ms gap is it expected, and should data be comming in regular 2s interval or this randomness is expected? Below i paste logs from all intervals i use to document if there is a mismatch

15min:

 WebSocket connected
[0]  | WebSocket connected
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '15m',
[0]  |   intervalDuration: 899999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | WebSocket connected
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '15m',
[0]  |   intervalDuration: 899999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '15m',
[0]  |   intervalDuration: 899999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '15m',
[0]  |   intervalDuration: 899999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '15m',
[0]  |   intervalDuration: 899999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '15m',
[0]  |   intervalDuration: 899999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '15m',
[0]  |   intervalDuration: 899999,
[0]  |   detectedInterval: null
[0]  | }

WARNING HERE! MISMATCH DETECTED on 30min

trading_spot_backend
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '30m',
[0]  |   intervalDuration: 899999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '30m',
[0]  |   intervalDuration: 1799999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '30m',
[0]  |   intervalDuration: 1799999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '30m',
[0]  |   intervalDuration: 899999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '30m',
[0]  |   intervalDuration: 1799999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '30m',
[0]  |   intervalDuration: 1799999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '30m',
[0]  |   intervalDuration: 1799999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '30m',
[0]  |   intervalDuration: 899999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '30m',
[0]  |   intervalDuration: 1799999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '30m',
[0]  |   intervalDuration: 1799999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected

1h:

| WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1h',
[0]  |   intervalDuration: 3599999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1h',
[0]  |   intervalDuration: 3599999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1h',
[0]  |   intervalDuration: 3599999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1h',
[0]  |   intervalDuration: 3599999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1h',
[0]  |   intervalDuration: 3599999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1h',
[0]  |   intervalDuration: 3599999,
[0]  |   detectedInterval: null
[0]  | }

4h:

 {
[0]  |   expectedInterval: '4h',
[0]  |   intervalDuration: 14399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '4h',
[0]  |   intervalDuration: 14399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '4h',
[0]  |   intervalDuration: 14399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '4h',
[0]  |   intervalDuration: 14399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '4h',
[0]  |   intervalDuration: 14399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '4h',
[0]  |   intervalDuration: 14399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '4h',
[0]  |   intervalDuration: 14399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '4h',
[0]  |   intervalDuration: 14399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '4h',
[0]  |   intervalDuration: 14399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected

1d:

WebSocket connected
[0]  | WebSocket connected
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1d',
[0]  |   intervalDuration: 86399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1d',
[0]  |   intervalDuration: 86399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1d',
[0]  |   intervalDuration: 86399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1d',
[0]  |   intervalDuration: 86399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1d',
[0]  |   intervalDuration: 86399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1d',
[0]  |   intervalDuration: 86399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | WebSocket connected
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1d',
[0]  |   intervalDuration: 86399999,
[0]  |   detectedInterval: null
[0]  | }
[0]  | Received Kline with unrecognized interval. Resubscribing...
[0]  | {
[0]  |   expectedInterval: '1d',
[0]  |   intervalDuration: 86399999,
[0]  |   detectedInterval: null
[0]  | }

I know that this is not enough data tell for sure but last time i had this mismatch i changed my interval from 1h to 5m and i had mismatch between those 2 intervals, now i had mismatch between 15 and 30 min after changing between those 2 intervals so there seems to be a pattern here, if Binance collects data about the intervals that users are using this part of code may be to blame. And again plaese tell me what should i be expecting about this irregularity in the time that candle data comes (rarely this is 2s interval), and if this 99999 ending is something that should be alerting me that something is wrong with the data i receive.
Hope this helps, if i can help any further please let me know.

jonte · September 19, 2023, 2:10am

Hi mate,

Thanks for providing that information. Firstly, I see that all the intervalDuration values are 1ms less than expected which is fine. Just add 1ms to the result/subtract 1ms from the detectedInterval so it doesn’t trigger the unrecognized interval console log.

Also it’s important to always close the previous websocket connection before subscribing to another one, otherwise the old connections will remain open which will cause issues as you’ve seen firsthand. The documentation covers the action of unsubscribing from a stream in the Live Subscribing/Unsubscribing to streams section.
Here’s an example snippet for how you can unsubscribe from a stream:

// Function which handles unsubscribing/disconnecting from Websocket Streams
function unsubscribeStream(streamName) {
  ws.send(JSON.stringify({ method: 'UNSUBSCRIBE', params: [streamName], id: 1 }));
}
// Eg. unsubscribe from the BTCUSDT@trade stream:
setTimeout(() => unsubscribeStream('BTCUSDT@trade'), 5000);

Once you implement the unsubscribe/disconnect functionality, I’m sure that you will no longer get the unrecognized interval logs when subscribing to a new stream with a different interval than the previous connection. The reason you’re currently getting those is because the old stream is still connected in the background so you’re essentially receiving 2 (or more) sets of Klines with differing intervals.

dziekonskik · September 19, 2023, 7:39pm

hi jonte

i know i had it on my radar will figure it out

thanks again!