Json vs Flatbuffers vs Protobuf in Lua

xmen

flatBuffers is an efficient cross platform serialization library for C++, C#, C, Go, Java, Kotlin, JavaScript, Lobster, Lua, TypeScript, PHP, Python, Rust and Swift. It was originally created at Google for game development and other performance-critical applications.

In lua/luajit, is flatbuffers really faster than json?

I make a simple benchmark. Let’s see what’s the result.

Source code

test.fbs

namespace Test;

table TextEntry {
    name:string;
    value:string;
}

table Req {
    conf:[TextEntry];
    key:string;
}

runjson.lua

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32


local cjson = require"cjson"

local t = {
    key = "hello",
    conf = {
        {name="foo", value="foo"},
        {name="bar", value="bar"},
        {name="fz", value="fz"},
        {name="baz", value="baz"},
    }
}

local _M = {}

function _M.marshal_bench()
    local cnt = 0
    for i = 1, 9000000 do
        cnt = cnt + #cjson.encode(t)
    end
    print(cnt)
end

function _M.unmarshal_bench()
    local data = cjson.encode(t)
    local cnt = 0
    for i = 1, 9000000 do
        cnt = cnt + #cjson.decode(data).key
    end
    print(cnt)
end

return _M

runfb.lua

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79


local flatbuffers = require("flatbuffers")
local test_req = require("Test.Req")
local text_entry = require("Test.TextEntry")
local tblnew = require"table.new"

local builder = flatbuffers.Builder(0)

local t = {
    key = "hello",
    conf = {
        {name="foo", value="foo"},
        {name="bar", value="bar"},
        {name="fz", value="fz"},
        {name="baz", value="baz"},
    }
}

local _M ={}

local function fb_marshal()
    builder:Clear()

    local key = builder:CreateString(t.key)
    local len = #t.conf
    local textEntries = tblnew(len, 0)
    for i = 1, len do
        local name = builder:CreateString(t.conf[i].name)
        local value = builder:CreateString(t.conf[i].value)
        text_entry.Start(builder)
        text_entry.AddName(builder, name)
        text_entry.AddValue(builder, value)
        local c = text_entry.End(builder)
        textEntries[i] = c
    end
    test_req.StartConfVector(builder, len)
    for i = len, 1, -1 do
        builder:PrependUOffsetTRelative(textEntries[i])
    end
    local conf_vec = builder:EndVector(len)

    test_req.Start(builder)
    test_req.AddKey(builder, key)
    test_req.AddConf(builder, conf_vec)
    local req = test_req.End(builder)
    builder:Finish(req)

    return builder:Output()
end

local function fb_unmarshal(data)
    local buf = flatbuffers.binaryArray.New(data)
    local req = test_req.GetRootAsReq(buf, 0)
    local len = req:ConfLength()
    local res = {key = req:Key(), conf = tblnew(len, 0)}
    for i = 1, len do
        local entry = req:Conf(i)
        res.conf[i] = {name = entry:Name(), value = entry:Value()}
    end
    return res
end

function _M.marshal_bench()
    local cnt = 0
    for i = 1, 9000000 do
        cnt = cnt + #fb_marshal()
    end
    print(cnt)
end

function _M.unmarshal_bench()
    local data = fb_marshal()
    local cnt = 0
    for i = 1, 9000000 do
        cnt = cnt + #fb_unmarshal(data).key
    end
    print(cnt)
end

return _M

runpb.lua

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44


local pb = require "pb"
local protoc = require "protoc"

local t = {
    key = "hello",
    conf = {
        {name="foo", value="foo"},
        {name="bar", value="bar"},
        {name="fz", value="fz"},
        {name="baz", value="baz"},
    }
}

assert(protoc:load [[
message Conf {
    required string name  = 1;
    required string value = 2;
}
message Test {
    required string key   = 1;
    repeated Conf   conf  = 2;
}
]])

local _M = {}

function _M.marshal_bench()
    local cnt = 0
    for i = 1, 9000000 do
        cnt = cnt + #pb.encode("Test", t)
    end
    print(cnt)
end

function _M.unmarshal_bench()
    local data = pb.encode("Test", t)
    local cnt = 0
    for i = 1, 9000000 do
        cnt = cnt + #pb.decode("Test", data).key
    end
    print(cnt)
end

return _M

Benchmark

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56


mkdir /tmp/fb_bench
cd $_

git clone https://github.com/google/flatbuffers flatbuffers.repo
cp -a flatbuffers.repo/lua/flatbuffers* .

apt install -y flatbuffers-compiler
flatc --lua test.fbs

luarocks install lua-cjson lua-protobuf

# serialization

time luajit -e "require('runfb').marshal_bench()"
1584000000

real    2m18.694s
user    2m14.777s
sys     0m0.446s

time luajit -e "require('runjson').marshal_bench()"
1242000000

real    0m13.712s
user    0m13.438s
sys     0m0.030s

time luajit -e "require('runpb').marshal_bench()"
477000000

real    0m29.054s
user    0m28.377s
sys     0m0.060s

# deserialization

time luajit -e "require('runfb').unmarshal_bench()"
45000000

real    1m59.111s
user    1m55.956s
sys     0m0.392s

time luajit -e "require('runjson').unmarshal_bench()"
45000000

real    0m28.408s
user    0m27.686s
sys     0m0.092s

time luajit -e "require('runpb').unmarshal_bench()"
45000000

real    0m28.245s
user    0m27.531s
sys     0m0.089s

Conclusion

benchmark

You could see that json is faster than flatbuffers a lot, even the serialization size is better a bit.

And the protobuf is half of speed in serialization compared to JSON, but deserialization is almost the same.

Of course, it’s for lua only, and my test schema only contains string types. You need to make a similar benchmark in your scenario.

I do not have time to investigate the reason, but I guess it’s due to the lua implementation of flatbuffers: it handle each field of struct via lua code, including table iteration, but cjson handles everything in C land, as well as lua-protobuf.

JSON is my favourite format.

Besides performance, it has below advantages:

no schema
no code generation
human readable

If you’re worrying about the size of JSON for huge data, you could use lz4 to do compression, which has low CPU overhead.

What’s your opinion? Welcome to discuss!

Json vs Flatbuffers vs Protobuf in Lua

Source code

Benchmark

Conclusion

See Also: