HTTP/3 client library for OpenResty

Background

HTTP/3 is the third major version of the Hypertext Transfer Protocol used to exchange information on the World Wide Web, complementing the widely-deployed HTTP/1.1 and HTTP/2.

Both HTTP/1.1 and HTTP/2 use TCP as their transport. HTTP/3 uses QUIC, a transport layer network protocol which uses user space congestion control over the User Datagram Protocol (UDP). The switch to QUIC aims to fix a major problem of HTTP/2 called “head-of-line blocking”: because the parallel nature of HTTP/2’s multiplexing is not visible to TCP’s loss recovery mechanisms, a lost or reordered packet causes all active transactions to experience a stall regardless of whether that transaction was impacted by the lost packet. Because QUIC provides native multiplexing, lost packets only impact the streams where data has been lost.

How to send HTTP/2 and HTTP/3 request from OpenResty/Nginx?

In fact, it’s difficult to support HTTP/2 and HTTP/3, because cosocket has below limitations:

• tcpsock:sslhandshake doesn’t support the ALPN.
• cosocket instance (connection) is bound to a specific HTTP downstream request, i.e. connection could not be shared globally, which disables HTTP2/HTTP3 multiplexing and concurrency then.

req is a golang library: Simple Go HTTP client with Black Magic.

Highlights:

• unique interface for all HTTP versions
• auto HTTP version selection
• easy download and upload

Why not encapsulate it so that we could reuse it in openresty?

lua-resty-ffi provides an efficient and generic API to do hybrid programming in openresty with mainstream languages (Go, Python, Java, Rust, Nodejs).

lua-resty-ffi-req = lua-resty-ffi + req

Check this repo for source code:

https://github.com/kingluo/lua-resty-ffi-req

Demo

simple get

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

local client, err = req:new_client()
local ok, res = client:request{
    url = "https://httpbin.org/anything?foo=bar",
    body = "hello",
    args = {
        foo1 = "foo1",
        foo2 = 2,
        foo3 = false,
        foo4 = 2.2,
    },
}
assert(ok)
ngx.say(inspect(res))
ngx.say(inspect(cjson.decode(res.body)))

Output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41

-- response
{
  body = '...',
  headers = {
    ["Access-Control-Allow-Credentials"] = { "true" },
    ["Access-Control-Allow-Origin"] = { "*" },
    ["Content-Length"] = { "609" },
    ["Content-Type"] = { "application/json" },
    Date = { "Wed, 05 Apr 2023 09:45:57 GMT" },
    Server = { "gunicorn/19.9.0" }
  },
  proto_major = 2,
  proto_minor = 0,
  status = 200,
  tls_version = 771
}
-- body
{
  args = {
    foo = "bar",
    foo1 = "foo1",
    foo2 = "2",
    foo3 = "false",
    foo4 = "2.2"
  },
  data = "hello",
  files = {},
  form = {},
  headers = {
    ["Accept-Encoding"] = "gzip",
    ["Content-Length"] = "5",
    ["Content-Type"] = "text/plain; charset=utf-8",
    Host = "httpbin.org",
    ["User-Agent"] = "req/v3 (https://github.com/imroc/req)",
    ["X-Amzn-Trace-Id"] = "Root=1-642d4355-2c0076244bb7d61c385efbb4"
  },
  json = <userdata 1>,
  method = "GET",
  origin = "34.193.132.77",
  url = "https://httpbin.org/anything?foo=bar&foo1=foo1&foo2=2&foo3=false&foo4=2.2"
}

request body writer and response body reader

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

local client, err = req:new_client()
local ok, res = client:request{
    method = req.HTTP_POST,
    url = "https://httpbin.org/anything",
    body = coroutine.wrap(function()
        coroutine.yield("hello")
    end),
    body_reader = true,
}
assert(ok)
ngx.say(inspect(res))

local body = {}
for chunk in res.body_reader do
    table.insert(body, chunk)
end
ngx.say(inspect(cjson.decode(table.concat(body, ""))))

HTTP3

Install Nginx HTTP3 version:

https://www.nginx.com/blog/binary-packages-for-preview-nginx-quic-http3-implementation/

# nginx.conf
http {
    # for better compatibility it's recommended
    # to use the same port for quic and https
    listen 8443 http3 reuseport;
    listen 8443 http2 ssl reuseport;
    ssl_certificate     /opt/certs/ca.crt;
    ssl_certificate_key /opt/certs/ca.key;
    ssl_protocols       TLSv1.3;

    location / {
        if ($arg_http3 = true) {
            add_header Alt-Svc 'h3=":8443"; ma=86400';
        }
        #proxy_pass http://httpbin_backend;
        #proxy_http_version 1.1;
        #proxy_set_header Connection "";
        return 200 ok;
    }
}

Test Alt-Svc:

1
2
3
4
5

local ok, res = client:request({url="https://foo.bar:8443/ip?http3=true"})
ngx.say(inspect(res))
ngx.sleep(0.1)
local ok, res = client:request({url="https://foo.bar:8443/ip?http3=true"})
ngx.say(inspect(res))

Output:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

-- first response
-- uses HTTP2
{
  body = "ok",
  headers = {
    ["Alt-Svc"] = { 'h3=":8443"; ma=86400' },
    ["Content-Length"] = { "2" },
    ["Content-Type"] = { "application/octet-stream" },
    Date = { "Wed, 05 Apr 2023 09:51:35 GMT" },
    Server = { "nginx/1.23.4" }
  },
  proto_major = 2,
  proto_minor = 0,
  status = 200,
  tls_version = 772
}
-- second response
-- switches to HTTP3
{
  body = "ok",
  headers = {
    ["Alt-Svc"] = { 'h3=":8443"; ma=86400' },
    ["Content-Length"] = { "2" },
    ["Content-Type"] = { "application/octet-stream" },
    Date = { "Wed, 05 Apr 2023 09:51:35 GMT" },
    Server = { "nginx/1.23.4" }
  },
  proto_major = 3,
  proto_minor = 0,
  status = 200,
  tls_version = 772
}

Auto HTTP version selection

req uses TLS ALPN and Alt-Svc header to do auto HTTP version selection.

HTTP1 to H2C (HTTP2 Clear Text)

This method is not supported yet.

HTTP2 ALPN

HTTP3 Alt-Svc

Redirect to the same site:

Redirect to a different site:

Benchmark

Due to golang http client implementation, lua-resty-ffi-req has only 70% of the performance of lua-resty-http, and it consumes 30% more cpu than lua-resty-http.

The performance is not that good yet, but there is room for improvement.

It is still worth sacrificing some performance for the ecosystem:

• No need to reinvent the wheel, especially when the wheel is big
• Cover maximum functionalities
• No need to bugfix yourself, because there is many contributors for popular open source project

FYI, here is the flamegraph of lua-resty-ffi-req (open in new tab page to browse the flamegraph svg):

Conclusion

With lua-resty-ffi, you could use your favorite mainstream programming language, e.g. Go, Java, Python, Rust, or Node.js, to do development in Openresty/Nginx, so that you could enjoy their rich ecosystem directly.

Welcome to discuss and like my github repo:

https://github.com/kingluo/lua-resty-ffi