Enhancement request: (owned) key/string value deduplication
Created by: ewaldc
- Problem: large JSON strings that contain many keys with the same name (repeatative structures) consume a lot of memory as each key name is stored multiple times.
- Enhancement suggestion: deduplicate owned keys (and if that works eventually owned variant string values) during deserialization. Preliminary code on v6 and running code on v5 suggests it can save up to 25% of consumed RAM.
- Target platform: embedded systems with limited RAM memory e.g. ESP8266
I have tried to implement this myself here using two different approaches (see source code) but both seem to have the same erroneous effect: when you serialize the JSON document after deserialization, the tail elements of the JSON string seem to have gone missing (even though the serialized JSON string is semantically correct).
So far I have not figured out why. Although the code only returns a pointer to a previously allocated string in the _string pool (left side of the buffer, if one can be found) it somehow changes the slot/collection structure :-(.
On my modified 5.x version which implements a relocatable slot structure allowing for pre-parsed JSON strings to be stored in flash, it works OK. In that code the strings are stored in Flash, while the structure is copied into RAM.
Thanks in advance. Ewald