Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • C csvkit
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 61
    • Issues 61
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 4
    • Merge requests 4
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • wireservice
  • csvkit
  • Issues
  • #596
Closed
Open
Issue created Apr 06, 2016 by Administrator@rootContributor

Unicode tests fail with Python 2

Created by: qauge

I have trouble getting tests right from the repository freshly cloned from master.

Some "unicode tests" won't pass with Python 2 (UnicodeDecodeError). Those are test_ndjson for csvjson, test_unicode for csvlook, test_sort_date for csvsort, and test_sort_string_reverse for csvsort.

Sample output:

ERROR: test_ndjson (tests.test_utilities.test_csvjson.TestCSVJSON)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/quentin/work/csvkit/tests/test_utilities/test_csvjson.py", line 108, in test_ndjson
    '{"text": "Unicode! Σ", "float": null, "datetime": null, "boolean": null, "time": null, "date": null, "integer": null}',
  File "/home/quentin/work/csvkit/tests/utils.py", line 56, in assertLines
    lines = self.get_output_as_list(args)
  File "/home/quentin/work/csvkit/tests/utils.py", line 44, in get_output_as_list
    return self.get_output(args).split('\n')
  File "/home/quentin/work/csvkit/tests/utils.py", line 37, in get_output
    utility.main()
  File "/home/quentin/work/csvkit/csvkit/utilities/csvjson.py", line 168, in main
    table = agate.Table.from_csv(self.input_file, sniff_limit=self.args.sniff_limit, column_types=self.get_column_types(), **self.reader_kwargs)
  File "/home/quentin/work/csvkit/.tox/pypy/site-packages/agate/table/__init__.py", line 362, in from_csv
    contents = contents.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 295: ordinal not in range(128)

Is it something on my side only?

Assignee
Assign to
Time tracking