API | Overloading serialization | Development
ldump is a flexible serializer, able to serialize any data, starting with circular references, tables as keys, functions with upvalues, metatables and ending with coroutines, threads and userdata (by defining how they should be serialized). It outputs valid Lua code that recreates the original object, doing the deserialization through load(data)(). It aims for functionality and flexibility instead of speed and size, allowing full serialization of complex data, such as video game saves. The output is large, but can be drastically reduced with modern compression algorithms.
Inspired by Ser. Supports Lua 5.1, 5.2, 5.3, 5.4 and LuaJIT. Tested for edge cases, such as joined upvalues and _ENV redefinition. Fully annotated in compatibility with LuaLS.
WARNING: ldump‘s deserialization function is Lua’s builtin load, which can load malicious code. Treat serialized data as arbitrary Lua code (which it is), .safe_load is coming soon.
| Type | Support |
|---|---|
| nil, boolean, number, string | full |
| function | full |
| userdata | user-defined |
| thread | user-defined |
| table | full |
| metatables* | full |
local ldump = require("ldump") local upvalue = 42 local world = { name = "New world", get_answer = function() return upvalue end, } local serialized_data = ldump(world) -- serialize to a string local loaded_world = load(serialized_data)() -- deserialize the string assert.are_equal(world.name, loaded_world.name) assert.are_equal(world.get_answer(), loaded_world.get_answer())
See as a test at /tests/test_use_case.lua:7
local ldump = require("ldump") -- basic tables local game_state = { player = {name = "Player"}, boss = {name = "Boss"}, } -- circular references & tables as keys game_state.deleted_entities = { [game_state.boss] = true, } -- functions even with upvalues local upvalue = 42 game_state.get_answer = function() return upvalue end -- fundamentally non-serializable types if overriden local create_coroutine = function() return coroutine.wrap(function() coroutine.yield(1337) coroutine.yield(420) end) end -- override serialization game_state.coroutine = create_coroutine() ldump.serializer.handlers[game_state.coroutine] = create_coroutine local serialized_data = ldump(game_
12 Comments
bflesch
Nice – I wonder why something like this is not built-in to the language itself. Especially debugging tables is painful to say the least :)
ithkuil
"lump" would have been a nice name
sebstefan
So it also dumps functions and is able to import them back?
Does the function still need to be in memory to be loaded again ("does it just dump the pointer") or can I save it to disk, shut off the interpreter, boot it again and it imports it fine (in which case it somehow dumps them as code…?)?
Even in the linked test case on the readme you don't show the output/expectation of the serialization
lifthrasiir
Maybe I'm too pedantic but allowing anything to be "deserialized", which equals to "evaluated" here, is not secure. I think it only has to accept a very limited subset of Lua anyway, so you may switch to a non-Lua format which is made easy to parse. That way the library has a total control over what is being evaluated.
brunocroh
I will try it on my next love2d project, thank you!
JourneyJourney
I'm afraid I spent too much time with LUA lately and fell in love with its simplicity. Kinda hard to go back to OOP after that.
gvx
Cool to see you were inspired by Ser!
synergy20
newbie question,when is this useful in practice
jhatemyjob
Try running this in a repl and tell me what you get, OP:
098527222722
use computer divices
koeng
I've been looking for something similar! Here is what I'd like to do:
I have a long-running script. At several steps, the execution of the script has to pause for a long time for operations to be done in-real-life (biological experiments, so think wait time being like 2 days between running), before getting some data and continuing to run. From what I can see in this, I'd add yielding coroutines at data pause points, right? How would you handle that?
appleorchard46
Very cool! I was just needing something like this for my Defold game, this looks way better than my hacky solution.
Semi-unrelated – you say you're using tables as keys in your project. I didn't know you could do that! What are you using it for?