Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • A ArduinoJson
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 24
    • Issues 24
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Benoît Blanchon
  • ArduinoJson
  • Issues
  • #1303
Closed
Open
Issue created Jun 26, 2020 by Administrator@rootContributor

Enhancement request: (owned) key/string value deduplication

Created by: ewaldc

  • Problem: large JSON strings that contain many keys with the same name (repeatative structures) consume a lot of memory as each key name is stored multiple times.
  • Enhancement suggestion: deduplicate owned keys (and if that works eventually owned variant string values) during deserialization. Preliminary code on v6 and running code on v5 suggests it can save up to 25% of consumed RAM.
  • Target platform: embedded systems with limited RAM memory e.g. ESP8266

I have tried to implement this myself here using two different approaches (see source code) but both seem to have the same erroneous effect: when you serialize the JSON document after deserialization, the tail elements of the JSON string seem to have gone missing (even though the serialized JSON string is semantically correct).

So far I have not figured out why. Although the code only returns a pointer to a previously allocated string in the _string pool (left side of the buffer, if one can be found) it somehow changes the slot/collection structure :-(.

On my modified 5.x version which implements a relocatable slot structure allowing for pre-parsed JSON strings to be stored in flash, it works OK. In that code the strings are stored in Flash, while the structure is copied into RAM.

Thanks in advance. Ewald

Assignee
Assign to
Time tracking