Implementing a JSON interface in Lua

Posted by Twisol on Mon 01 Mar 2010 06:24 AM — 5 posts, 17,335 views.

Twisol USA Mon 01 Mar 2010 06:24 AM #0

This post was copied from another thread; see there for other details.

http://www.gammon.com.au/forum/?id=10043&page=19

The json.encode function is complete; json.decode is next. I'm really happy with how this is turning out! Here's a brief explanation of what I've done.

json.encode takes Lua data and returns a "compiled" userdata-wrapped json_object. This object has :to_json() and :to_lua() methods. :to_json() will return a valid JSON string, whereas :to_lua() will return a decoded Lua representation. The compiled json objects can also be used as part of later json.encode calls.

Three helpful other items in the json table are json.array, json.object, and json.null. json.array and json.object take a table and return a compiled JSON userdata, treating the table as an array/object and ignoring any invalid keys. json.null is a pre-generated JSON userdata representing null, equivalent to json.encode(nil).

Examples:

chunk = json.encode{1, 2, nil, 4, json.null}
print(chunk:to_json())

--[[
Output:
  [1, 2, null, 4, null]
--]]

chunk = json.encode{foo=100, bar=chunk, baz=101}
print(chunk:to_json())

--[[
Output:
  { "baz": 101, "bar": [ 1, 2, null, 4, null ], "foo": 100 }
--]]

print(json.encode{}:to_json())
print(json.encode(json.array{}):to_json())
print(json.encode(json.object{}):to_json())

--[[
Output:
  ERROR: Ambiguous, could be array or object.
  [ ]
  { }
--]]

* :to_lua() isn't implemented yet, I'll be adding it along with json.decode.

EDIT: It's hard to see the difference between {} and () in the code font, so if you're confused by that, look closely.

Twisol USA Tue 02 Mar 2010 01:21 AM #1

Decoding is in, I'm just cleaning up now. I want to make sure allocated json objects are properly freed in the case of a Lua stack overflow, for example. But it's mostly done, and I'm really liking it if I do say so myself. The current changes are up on my Git repository.

A cool side effect of making the json_object the "active ingredient", so to speak, is that you can do things like this:

some_json = "[1, 2, 3]"
chunk = json.encode{foo = 42, bar = json.decode(some_json)}
print(chunk:to_json())
--[[
Output:
  { "bar": [1, 2, 3], "foo": 42 }
--]]

The .encode and :to_lua methods are eachother's inverse, as are the .decode and :to_json methods. Translating between Lua and JSON is necessarily a two-step operation, but it also provides a lot of flexibility.

EDIT: Sorry for all the typos, I wrote this from my iPhone.

Twisol USA Tue 02 Mar 2010 05:05 AM #2

Cleaned up and working fine! I've implemented everything I wanted to, except maybe json-c's incremental parsing, but unless it's needed this should be enough. The changes have been pushed to GitHub.

By the way, the C API for json-c is really quite easy to use. You shouldn't have any problem supporting a JSON-based protocol.

EDIT: It should go without saying that circular references are a bad thing, and MUSHclient will crash if you give it something like "t = {}; t[1] = t; json.encode(t)". I could write in a circular-reference checker which would cause an error if it detects any, if you really want.

Twisol USA Tue 02 Mar 2010 06:46 AM #3

Hey, this is slick. To check out exactly what I've changed while writing the JSON interface, go here:

http://github.com/Twisol/mushclient/compare/cbf2852aad145438fb41d2764c76feeba55f3a92...master

It's part of a new bunch of features the GitHub people added recently.

Twisol USA Tue 02 Mar 2010 07:46 PM #4

Uploaded a fix to arrays, I accidentally broke encoding to JSON arrays when I added the Lua-stack-overflow insurance.

Also, check this out. What follows is a series of tests comparing the JSON interface's performance to that of loadstring().

loadstring()()

local time = GetInfo(232)
for i=1, 1000 do loadstring("return {1, 2, 3}")() end
print(GetInfo(232) - time)
collectgarbage()

0.022395094914827

0.023329152172664

0.022670060017845

0.022370999664417

0.0077074041619198

0.023637222053367

0.022699393361108

0.022840542587801

0.022376237771823

0.022947818782995

Average: 0.021297392548877

json.decode():to_lua()

/local time = GetInfo(232)
for i=1, 1000 do json.decode"[1, 2, 3]":to_lua() end
print(GetInfo(232) - time)
collectgarbage()

0.020603247074177

0.018901075411122

0.019014218283701

0.018860637312173

0.018902332551079

0.018940186535474

0.019190567516489

0.019461342162685

0.018927894459921

0.018970777018694

Average: 0.019177227832551

Total average difference: 0.0021201647163255

So, perhaps surprisingly, the JSON interface is on average 2ms faster. And then there's the oddball result in the loadstring set, 0.007, which is so out of place you might not even count it...