Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • C csvkit
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 61
    • Issues 61
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 4
    • Merge requests 4
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • wireservice
  • csvkit
  • Merge requests
  • !357

Adding support for streaming json documents

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Administrator requested to merge github/fork/jennifersmith/355_streaming_json_documents into master Nov 04, 2014
  • Overview 1
  • Commits 3
  • Pipelines 0
  • Changes 2

Created by: jennifersmith

This resolves issue #355 (closed) by adding a further option to csvjson that allows you to specify that you want to output a stream of newline delimited json object rather than an array of all objects.

To see this in action, consult the tests or:

csvjson --stream examples/dummy3.csv

Result:

{"a": "1", "b": "2", "c": "3"}
{"a": "1", "b": "4", "c": "5"}

Original:

csvjson examples/dummy3.csv

Result:

[{"a": "1", "b": "2", "c": "3"}, {"a": "1", "b": "4", "c": "5"}]

Note: This will not work (and in fact error) if you to use the stream option alongside lat/lon/key options because all of these output a top level json object anyway.

Implementation notes:

I tried to keep the changes minimal, but I did extract dump_json to allow me to diversify where it was called from.

I am not too familiar with python. Tried to keep to conventions, but please feel free to met me know if you would like any improvements.

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: github/fork/jennifersmith/355_streaming_json_documents