Simple JSON Encode/Decode in Pure Lua
More Lua code

I've coded up some simple JSON encode/decode routines in pure Lua and thought I'd share them in case anyone else would find them useful. I use them in Adobe Lightroom, but they're pure Lua 5, so can be used anywhere Lua is.

Creative Commons License
JSON Encode/Decode in Pure LUA by Jeffrey Friedl is licensed under a Creative Commons Attribution 3.0 Unported License.
Download JSON.lua
Version 20170927.26 (version 26: September 27, 2017)

Full docs and changelog are in the code itself, but basic use is:

JSON = (loadfile "JSON.lua")() -- one-time load of the routines


local lua_value = JSON:decode(raw_json_text) -- decode example


local raw_json_text    = JSON:encode(lua_table_or_value)        -- encode example
local pretty_json_text = JSON:encode_pretty(lua_table_or_value) -- "pretty printed" version

Enjoy.

(You might also be interested in this comparison of Lua JSON packages.)


The 30 most-recent comments (out of 39; see all), most recent last...

Hi Jeffrey,

I’m new to lua (since yesterday) and I was playing around with your nice module.
I’m runnin into a few issues / unclarities though and hope to get them clarified.

The pretty_encode does not generate the same output as the encode when it comes to ‘arrays’.
Consider the following:

local Test = {1, 2, 3, [5]=5} -- would be a table with Test[4] = nil
-- encode output: [1, 2, 3, null, 5]
-- pretty_encode output: [1, 2, 3]

Secondly, in the comments you put:

-- We need to inspect all the keys... if there are any strings, we'll convert to a JSON
-- object. If there are only numbers, it's a JSON array.

Which would indicate that

Test = { [1] = "a", [2] = "b", ["3"] = "c" }

results in a JSON object (ther is ‘any strings’), instead i get a ‘mixed keys’ error.

Hope you can help me out,

Mattie (Netherlands)

Thanks for the heads up on these… I’ve pushed a new version that addresses both issues. —Jeffrey

— comment by Mattie on October 31st, 2013 at 5:25am JST (4 years ago) comment permalink

Hey there from Spain!

I just wanted to thank you for releasing this, it literally saved my life, as I am using Lua in a closed environment for a project, and the functions they provided me to work with JSON were pretty lame.

And I must also say thanks for your pure lua sha1 implementation! ; )

Regards.

— comment by Pablo PHG on November 28th, 2013 at 10:32pm JST (3 years, 11 months ago) comment permalink

Hi Jeffrey

Thanks for your JSON module. I stumbled across the following issue (using version 20131118.9):
JSON:decode(“[1,null,3]”) silently skips the “null” value and result in a table with two values only: {1, 3}.

This seems to be because of the problem described here: http://lua-users.org/wiki/StoringNilsInTables

The following one-line change would allow an application to define a nilPlaceholder in your module:

— JSON.lua.old 2013-11-18 13:07:36.000000000 +0100
+++ JSON.lua 2013-12-20 14:47:28.636625937 +0100
@@ -506,7 +506,7 @@
return false, start + 5

elseif text:find(‘^null’, start) then
– return nil, start + 4
+ return JSON.nilPlaceholder, start + 4

else
self:onDecodeError(“can’t parse JSON”, text, start, etc)

For example:

local NIL = {}
JSON.nilPlaceholder = NIL

The calling application could then compare the received values with NIL. If nilPlaceholder is not defined, it is simply “nil”.

Thanks and keep up the good work!

Oliver (Switzerland)

Yikes, I just (April 2014) noticed this comment sitting in moderation. Sorry. It turns out that I did notice a comment of someone else reporting the bug many months later, and have just (April 2014) pushed a fix. The fix doesn’t require any placeholder thing… it just works. Sorry for the delay(!) —Jeffrey

— comment by Oliver Hitz on December 20th, 2013 at 10:51pm JST (3 years, 10 months ago) comment permalink

Hello Jeffrey,

Thank you very much for making your code available.
When trying to decode a malformed JSON:

{"sleepCycle": 15,a"enabled": 1}

your code gets to line 550:

      -- should never get here... JSON parse errors should have been caught earlier
      assert(false, value)

and fails. Although it says it should not get there, it’s there.
Anyway I think the right code should be:

      -- should never get here... JSON parse errors should have been caught earlier
      -- assert(false, value)
	   if self.assert then
		  self.assert(false, message)
	   else
		  assert(false, message)
	   end

To be able to catch the exception. Do you agree?
Best regards.

Good catch… the “should never get here” comment is true only if the user’s JSON.assert actually breaks out of the pcall, as it does when I use it in Adobe Lightroom, so it’s not an appropriate comment for the general package, so I’ve updated it, and the code as you suggest (with s/message/value/). Thanks for the heads up. —Jeffrey

— comment by blue on January 16th, 2014 at 6:24am JST (3 years, 9 months ago) comment permalink

Hi Jeffrey,

When decoding [“1″,null,null,null,null,null,”marathon”], it returns an array of two elements ({1,marathon}).
It may be a normal behavior, but it would be helpful if it returns an array of 7 elements with nil values (something like {1,nil,nil,nil,nil,nil,marathon}).

Thank you for your work, it saved me a lot of time, here in Canada 🙂

Good catch, thanks. I’ve pushed a fix. —Jeffrey

— comment by haddock on April 18th, 2014 at 1:53am JST (3 years, 6 months ago) comment permalink

Nice job: thank you!

— comment by Rick77 on May 19th, 2015 at 6:40pm JST (2 years, 5 months ago) comment permalink

HI Jeffrey,
When decoding a 900K json file, json lib will cost 14M+ memory. I review json.lua, and find use many stringcat .. operation at grok_string function.
so I change VALUE = VALUE .. c to tmp_stream:insert(c). diff code is here: https://gist.github.com/lvzixun/80e5b900b82059ebf5d7/revisions. using table.concat would be faster and reduce string memory allocation.

you can review code : https://gist.github.com/lvzixun/80e5b900b82059ebf5d7 😉

Unfortunately, whatever this “insert” function is, it’s not available in the version of Lua that I use. —Jeffrey

— comment by zixun on June 12th, 2015 at 12:55pm JST (2 years, 4 months ago) comment permalink

Forked this repo for LuaRocks. Do you have GitHub account for this?
https://github.com/jiyinyiyong/json-lua

No, sorry, no GitHub. —Jeffrey

— comment by jiyinyiyong on June 26th, 2015 at 11:02pm JST (2 years, 4 months ago) comment permalink

Hi Jeffrey,
thanks for the code. This is exactly what I was looking for. Works like a charme – in a Lightroom plugin!
So far, only simple JSON responses are decoded (did this with a regexp before), but it will probably become more important when using more sophisticated WebAPIs of the Synology PhotoStation.

Greetings from Berlin,

Martin

— comment by Martin on September 19th, 2015 at 7:35am JST (2 years ago) comment permalink

Hi Jeffrey,
Thanks a lot for this code ! Excellent work ! We are using this serilizer together with Zabbix JSON-RPC API in a large distributed monitoring network. This code provided us with an excellent jumpstart at first when we made a prototype of the gateway from Zabbix to internal ESB. After a while we have decided not to touch anything as it works perfectly and performance is sufficient for this project.

Thanks a lot again,
Greetings from Kazakhstan,
Pavel.

— comment by Paul on October 7th, 2015 at 7:18pm JST (2 years ago) comment permalink

Hello

I am new to Lightroom plugin development: I can’t find the correct syntax to load your JSON.lua file. I have put the JSON.lua file in the plugin folder. I have tried some code found on this page:
JSON = (loadfile “JSON.lua”)()
JSON = assert(loadfile “JSON.lua”)()
JSON = loadfile(LrPathUtils.child(_PLUGIN.path, “JSON.lua”))
without success.
Could you please give the correct syntax to use (Lightroom 6 Plugin SDK ).
Thanks a lot
Thierry

In Lightroom you can use JSON = require "JSON.lua", but note that Lightroom won’t recognize any file added to a plugin folder after it starts, so after having added the JSON.lua file, you have to restart Lightroom before anything will work. —Jeffrey

— comment by Thierry on October 26th, 2015 at 10:21pm JST (2 years ago) comment permalink

During encode(), numbers larger than 1e14 were being changed to scientific notation.

Here is some lua code to show how tostring() handles large numbers
> x=100000000000001
> print(tonumber(x));
1e+14
> print(string.format(“%.0f”,x))
100000000000001

I changes the encoding to use the “%.0f” format -> works great

Here is the GIT DIFF
@@ -837,7 +837,10 @@ function encode_value(self, value, parents, etc, options, indent)

return “-1e+9999”
else
– return tostring(value)
+ return string.format(“%.0f”, value);
end

Ah, good catch, but unfortunately it’s not quite that simple. Try with your x value set to 10000000000054321, 10000000000054322, and 10000000000054321+1 and you’ll see (if your system is the same as mine) that at some point the string.format() version is wrong, due, I’m sure, to precision being lost in encoding big numbers in too-small a number of bits. I’d rather see that loss of precision represented, instead of a wrong number presented precisely.

There’s a range of decimals for which string.format() is indeed better, as your example shows; if I can determine for sure such cases, I’ll incorporate a change for them. I just tried the obvious way and it was lacking, but I’ll dig further.

Of course, we still need to make sure that a number like 3.14159 is output correctly.—Jeffrey

— comment by Russell Sullivan on April 22nd, 2016 at 9:29am JST (1 year, 6 months ago) comment permalink

I am not able to do loadfile with LUA called from HAProxy config

JSON = (loadfile “JSON.lua”)()

The program exists for loadfile call.
The caller lua and JSON.lua are placed in same folder.

Is there specific configuration needed for HAProxy?

— comment by Deepak Agarwal on May 17th, 2016 at 2:19am JST (1 year, 5 months ago) comment permalink

Any thoughts or ways around having decode properly handling nil values from a table?

For instance:
{
“somevalue”: null
}

will create a empty output since lua doesn’t support nil values as a key in a table element.

I’ve locally (and horribly) solved this by patching if nil to return a function that returns nil — but looking at the table I have to account for that too — then on the encode I just say if it’s a function then it is nil — good enough for me, but not for a general stance.
local function grok_object(self, text, start, etc)
[…]
VALUE[key] = new_val or function() return nil end

function encode_value(self, value, parents, etc, options, indent)
if type(value) == ‘function’ then value = nil end
if value == nil then
return ‘null’

Other than this small hiccup, I really love how the rest works… Efficient and stable!

As of version 20160526.15 I’ve added a “null” field to the encoding-options table. If your Lua table has a string value with the same value as the “null” option, it gets encoded into JSON as a raw null. Using null="\0" might be appropriate, for example. —Jeffrey

— comment by Adam B on May 25th, 2016 at 5:50pm JST (1 year, 5 months ago) comment permalink

Thanks a lot for this! I’m using to create and load save files for a game I’m working on, and it works great. One small suggestion would be to ignore functions while parsing, rather than throwing an error. This is only my opinion, of course.

Anyway, I am quite happy with this library. It was the first result when I searched for “lua json parser.”

As an aside, what do you use this library for in Lightroom?

Since you asked where I’m from: Portland, Oregon.

I’m not sure what you mean by “ignore functions while parsing”; JSON has no concept of “functions”. I use this in my Lightroom plugins to handle responses from a variety of photo-hosting-site’s APIs. —Jeffrey

— comment by MarkSill on May 29th, 2016 at 3:58pm JST (1 year, 5 months ago) comment permalink

I have found a bug when decoding an empty object, it is interpreted as an empty array, at least that is what shows on encode().
look at the field: ‘attributes’, it is an object in the string and then an array on decode()

lua
> JSON = (loadfile “./JSON.lua”)();– one-time load routines
> x='{“id”:”UserInfo_BILL”,”actions”:[],”attributes”:{}}’
> y=JSON:decode(x)
> print(JSON:encode(y));
{“actions”:[],”attributes”:[],”id”:”UserInfo_BILL”}

As mentioned in the docs, it’s handled correctly if you “JSON.strictTypes = true“. It’s not the default because supporting it requires adding metatables to the data, which can break some systems into which Lua is embedded. —Jeffrey

— comment by Russell Sullivan on June 4th, 2016 at 5:42am JST (1 year, 4 months ago) comment permalink

Thank you for making this module!

I think I have spotted an unintended global. On line 516, you have:
> local isNumber = {
> __index = isNumber,
> — <snip>
> }
However, when __index = isNumber on line 517 is parsed, nothing has been assigned to the local isNumber variable on line 516 yet, so at that point isNumber is treated as an uninitialised global, and __index is assigned a value of nil. (At least, this is how it works in Lua 5.1.)

I haven’t followed all the paths through the code to see if this causes any problems, but I guess that this isn’t what was intended.

Holy cow, you’re right. What a nubie mistake on my part. Good catch, thanks. Fixed. —Jeffrey

— comment by Jack Taylor on September 16th, 2016 at 3:22pm JST (1 year, 1 month ago) comment permalink

Hi!
This is an awesome module, thanks a lot!

Wanted to know, sadly in the project I’m testing, ‘the order’ of some keys in the json file needs to be preserved, my question is, is it possible when reading a json to keep the original order in which they were read?
decoding/encoding the same file changes the order of key-value pairs.

It’s not currently possible, sorry. Lua tables have no order for non-numeric keys, so to preserve the order you’d have to use metatables somehow, which just opens up a can of worms for handling that metadata (e.g. what to do when new keys are added in Lua). —Jeffrey

— comment by Alejandro on September 29th, 2016 at 7:44am JST (1 year ago) comment permalink

Hello again, and thank you for the updates to the module dealing with trailing garbage. I’ve spotted another nil global bug in the new code, so I thought I would let you know.

On line 1065, you have this variable declaration:

> local error_message

This declaration is in the scope of the if block starting at line 1060.

> if next_i ~= #text + 1 then

The problem is that on line 1070 you try and use error_message:

> return value, error_message

But at that point we have left the scope of the if block, so error_message is a nil global.

If you could update the code to fix this, it would be much appreciated.

Oops, sorry, I fixed this earlier in the month but goofed up pushing out the fixed version here. It (version 20161109.21) is here now. —Jeffrey

— comment by Jack Taylor on November 24th, 2016 at 12:29pm JST (10 months, 24 days ago) comment permalink

Hey, do you have the old versions 1-20 somewhere online?
It would be cool to have a look at them (push them to git for proper version history/changes etc).

Cheers

You can append the version string to the download link to get old ones, e.g. http://regex.info/code/JSON.lua-20140920.13 —Jeffrey

— comment by Gregor on March 17th, 2017 at 7:32pm JST (7 months, 1 day ago) comment permalink

Wow Jeffrey,

Thank you a million times for this library.

Jon

— comment by Jon on April 4th, 2017 at 7:30am JST (6 months, 15 days ago) comment permalink

local stock= ‘{“open”:332.44,”close”:329.87,”high”:332.5,”low”:328.15,”volume”:11038600.0,”actual_close”:331.29}’
local s = JSON:decode(stock)
return s[“high”]

This gives me a result of 332
or if I add JSON.decodeNumbersAsObjects = true I get: (empty list or set)

What do I need to get the right answer, 332.5?

Are you sure you’re not using it in some kind of integer context once you get the value back from whatever function this is? I changed your ‘return’ line to a ‘print’ line, and I got the proper 332.5. —Jeffrey

— comment by Greg on April 5th, 2017 at 12:57am JST (6 months, 14 days ago) comment permalink

You’re right, it turns out Redis turns floats into integers. Not sure about the behavior of JSON.decodeNumbersAsObjects = true.

Thanks for the library, I’m new to lua so still asking stupid questions.

— comment by Greg on April 13th, 2017 at 9:07pm JST (6 months, 5 days ago) comment permalink

hi, Jeffrey

I add a ‘array_newline’ option, which show array

[11,22,[33,44]]

as

[
| 11,
| 22,
| [
| | 33,
| | 44
| ]
]

https://github.com/yurenchen000/json-lua/commit/e870c6e702deacd3b9c5c2b1135893791fc51e16

thank you~

Seems like a good idea… I’ve added it, thanks! —Jeffrey

— comment by yurenchen on April 16th, 2017 at 4:59am JST (6 months, 3 days ago) comment permalink

Hi Jeffrey,
Thanks for your module! I am running into a problem and I cannot figure out how to solve it. I would appreciate some help, please.

I have a json object being returned from a server, which seems to be properly decoded into a LUA table as I can reference it’s properties. One of the properties contains another json like string and I am trying to convert it to a table but keep getting “JSON.decode must be called in method format.” Here is the complete JSON string:

{“Success”:true,”GalleryId”:20094,”postURL”:”https://upload.blah.net/upload/processUpload/?websiteId=5140\u0026eventId=6500″,”FileUploadId”:”b5ymbczi”,”UMIs”:”{\”IMG_3271.JPG\”:\”109678\”,\”IMG_3272.JPG\”:\”109679\”,\”IMG_3273.JPG\”:\”109680\”,\”IMG_3274.JPG\”:\”109681\”}”}

I can isolate the last property called response.UMIs and print it out as the following string.
2017-06-05 18:59:19 +0000, TRACE UMIs as string: {“IMG_3271.JPG”:”109678″,”IMG_3272.JPG”:”109679″,”IMG_3273.JPG”:”109680″,”IMG_3274.JPG”:”109681″}

UMIS = {}
When I try UMIs = JSON.decode(response.UMIs) is when I get the error.
The JSON string is verified by a JSON linter so I am not sure what I am doing wrong.

Thanks!

Use “JSON:decode” rather than “JSON.decode” and it should work. —Jeffrey

— comment by Julian Dormon on June 6th, 2017 at 4:09am JST (4 months, 13 days ago) comment permalink

Thanks man, this is exactly what I needed. I was just messing around with modding a game that I am playing at the moment. I was looking to export some data as JSON, so that I persist state between gaming sessions. This did the trick perfectly. You’ve saved me a bunch of time, and I really appreciate that you make this free to use.

Thank you!

— comment by Francois on July 2nd, 2017 at 2:30pm JST (3 months, 16 days ago) comment permalink

I’d like to request support for boolean keys, right now it says “can’t encode table with a key of type boolean”. A simple fix is to have


if type(key) == ‘boolean’ then
table.insert(string_keys, tostring(key))
elseif type(key) == ‘string’ then

Makes sense. Done. —Jeffrey

— comment by Anonymous on August 19th, 2017 at 9:12am JST (2 months ago) comment permalink

Using VERSION = ‘20170416.23’

At line 1325:

local encode_value – – must predeclare because it calls itself
function encode_value

Actually, you don’t need to predeclare a recursive function in Lua. The interpreter includes the function in its own _ENV before calling the function. You only need to predeclare pairs of recursive functions, like this:

local a
local function b()
  a()
end
function a()
  b()
end

I know about that property of Lua and use it elsewhere in the file, so I suppose there was something esoteric happening at one point during development that made me feel the need to do it explicitly. The Lua docs are pretty clear that the two expressions are identical, so I’m at a loss to guess what it might have been, but in any case, you’re right and I’ve tidied it up. Thanks. —Jeffrey

— comment by Chronos Phaenon Eosphoros on August 21st, 2017 at 11:26pm JST (1 month, 28 days ago) comment permalink

Using VERSION = ‘20170416.23’

I’m using you package for debugging code, so I need to encode table which includes functions. For my purposes, I just added this to encode_value():

elseif type(value) == ‘function’ then
  return tostring(value)

However, as a suggestion, I think it would be nice if you either:

1- included some kind of types whitelist as an option for encoodin; or
2- made JSON:onEncodeError() interactive, as is JSON:onDecodeError(); or
3- both of the above.

Anyways, I’m quite happy with this package. Very nice implementation, quick, efficient and very useful.

Good idea, thanks. Check out unsupportedTypeEncoder() in the new version I just posted. —Jeffrey

— comment by Chronos Phaenon Eosphoros on August 21st, 2017 at 11:34pm JST (1 month, 28 days ago) comment permalink

Hi Jeffrey,

Great package! We’ve been using it for a while now and it works pretty well. This seems to have been mentioned before but as of Version 20170823.25 (version 25: August 23, 2017) we haven’t been able to pass a null option for *decoding* JSON strings. This is useful to us because we need to retain keys in the lua table (since there is no distinction between keys that don’t exist and keys assigned as nil values). Having null configurable when encoding a JSON string is not sufficient because we decode, modify, and then subsequently encode it again. The patch we made was as follows:


1086: - return nil, start + 4
1086: + return options.null, start + 4

Is there any chance that this patch could be surfaced for the next release?

Cheers,
Max

Oops, yeah, that was mentioned four years ago… I completely missed it, sorry. Thanks for the bump. I’ve just pushed the update. —Jeffrey

— comment by Max Sindwani on September 27th, 2017 at 6:28am JST (3 weeks, 1 day ago) comment permalink
Leave a comment...


All comments are invisible to others until Jeffrey approves them.

Please mention what part of the world you're writing from, if you don't mind. It's always interesting to see where people are visiting from.


You can use basic HTML; be sure to close tags properly.

Subscribe without commenting