Http2: Increase HEADER_TABLE_SIZE to 65536

API Futures API
1

Hello,

I’m using the futures api over Http2, and I’m trying to increase the dynamic table index from 4096 to 65536.

In the documentation it says you are allowed to use a mix of query string and response body. Which improves response times with http2, because the path gets indexed. e.g

/fapi/v1/countdownCancelAll?symbol=BTCUSDT

The default dynamic table size of 4096 is far too small to hold even 20 different query strings, so they keep getting evicted.

I tried to send the Binance server a SettingsFrame with the header table size set to 65536 but I end up with a GoAway { COMPRESSION_ERROR } after 4096 bytes. It seems that increasing this value is not supported? I confirmed this by using nghttp, command and output shown below.
Any chance this could be increased?

nghttp -nv -c 65536 https://fapi.binance.com/fapi/v1/time

    [  0.031] Connected
The negotiated protocol: h2
[  0.049] send SETTINGS frame <length=18, flags=0x00, stream_id=0>
          (niv=3)
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):100]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65535]
          [SETTINGS_HEADER_TABLE_SIZE(0x01):65536]
[  0.050] send PRIORITY frame <length=5, flags=0x00, stream_id=3>
          (dep_stream_id=0, weight=201, exclusive=0)
[  0.050] send PRIORITY frame <length=5, flags=0x00, stream_id=5>
          (dep_stream_id=0, weight=101, exclusive=0)
[  0.050] send PRIORITY frame <length=5, flags=0x00, stream_id=7>
          (dep_stream_id=0, weight=1, exclusive=0)
[  0.050] send PRIORITY frame <length=5, flags=0x00, stream_id=9>
          (dep_stream_id=7, weight=1, exclusive=0)
[  0.050] send PRIORITY frame <length=5, flags=0x00, stream_id=11>
          (dep_stream_id=3, weight=1, exclusive=0)
[  0.050] send HEADERS frame <length=50, flags=0x25, stream_id=13>
          ; END_STREAM | END_HEADERS | PRIORITY
          (padlen=0, dep_stream_id=11, weight=16, exclusive=0)
          ; Open new stream
          :method: GET
          :path: /fapi/v1/time
          :scheme: https
          :authority: fapi.binance.com
          accept: */*
          accept-encoding: gzip, deflate
          user-agent: nghttp2/1.41.0
[  0.063] recv SETTINGS frame <length=18, flags=0x00, stream_id=0>
          (niv=3)
          [SETTINGS_MAX_CONCURRENT_STREAMS(0x03):128]
          [SETTINGS_INITIAL_WINDOW_SIZE(0x04):65536]
          [SETTINGS_MAX_FRAME_SIZE(0x05):16777215]
[  0.063] recv WINDOW_UPDATE frame <length=4, flags=0x00, stream_id=0>
          (window_size_increment=2147418112)
[  0.063] recv SETTINGS frame <length=0, flags=0x01, stream_id=0>
          ; ACK
          (niv=0)
[  0.063] send SETTINGS frame <length=0, flags=0x01, stream_id=0>
          ; ACK
          (niv=0)
[  0.295] recv (stream_id=13) :status: 200
[  0.295] recv (stream_id=13) content-type: application/json
[  0.295] recv (stream_id=13) content-length: 28
[  0.295] recv (stream_id=13) date: Wed, 23 Sep 2020 12:11:01 GMT
[  0.295] recv (stream_id=13) server: Tengine
[  0.295] recv (stream_id=13) x-mbx-used-weight-1m: 1
[  0.295] recv (stream_id=13) x-response-time: 0ms
[  0.295] recv (stream_id=13) access-control-allow-origin: *
[  0.295] recv (stream_id=13) access-control-allow-methods: GET, POST, PUT, DELETE, OPTIONS
[  0.295] recv (stream_id=13) access-control-allow-credentials: true
[  0.295] recv (stream_id=13) access-control-allow-headers: *,clienttype,Content-Type,lang,x-ui-request-trace
[  0.295] recv (stream_id=13) content-security-policy: default-src 'self'
[  0.295] recv (stream_id=13) cache-control: no-cache, no-store, must-revalidate
[  0.295] recv (stream_id=13) expires: 0
[  0.295] recv (stream_id=13) pragma: no-cache
[  0.295] recv (stream_id=13) strict-transport-security: max-age=31536000; includeSubdomains
[  0.295] recv (stream_id=13) x-content-type-options: nosniff
[  0.295] recv (stream_id=13) x-frame-options: SAMEORIGIN
[  0.295] recv (stream_id=13) x-xss-protection: 1; mode=block
[  0.295] recv (stream_id=13) x-cache: Miss from cloudfront
[  0.295] recv (stream_id=13) via: 1.1 70b3d55a139dd5f3bf9f6b0ac188bac6.cloudfront.net (CloudFront)
[  0.295] recv (stream_id=13) x-amz-cf-pop: LHR62-C1
[  0.295] recv (stream_id=13) x-amz-cf-id: vvRmFqWwuOMxbSDwe5dTGrRyg1tftRWF9DY-JA6p0DY1z5kn97os3w==
[  0.295] recv HEADERS frame <length=618, flags=0x04, stream_id=13>
          ; END_HEADERS
          (padlen=0)
          ; First response header
[  0.295] recv DATA frame <length=28, flags=0x00, stream_id=13>
[  0.295] recv DATA frame <length=0, flags=0x01, stream_id=13>
          ; END_STREAM
[  0.295] send GOAWAY frame <length=8, flags=0x00, stream_id=0>
          (last_stream_id=0, error_code=NO_ERROR(0x00), opaque_data(0)=[])
2

Sorry I didn’t see this error COMPRESSION_ERROR from goaway frame.
In the log provided, the goaway has no error and seems a normal disconnection symbol.

3

There is COMPRESSION_ERROR in the output above because it didn’t use more than 4096 bytes. However thats very easy to do when making multiple API calls.

In the “send SETTINGS frame” section you can see I send [SETTINGS_HEADER_TABLE_SIZE(0x01):65536] which is asking for the table size to be increased.

In the “recv SETTINGS frame”, no table size header is included which means it’s the default of 4096.

4096 is really small, 65kb is the standard on most websites now as it greatly increases performance. Is there any chance this could be increased?

4

thank you for your feedback, we will review this.

1 Like