Skip to content

Instantly share code, notes, and snippets.

@lelandbatey
Created July 22, 2022 19:21
Show Gist options
  • Save lelandbatey/26fb0be6b7891acdd000e1c01089ed31 to your computer and use it in GitHub Desktop.
Save lelandbatey/26fb0be6b7891acdd000e1c01089ed31 to your computer and use it in GitHub Desktop.
Remove all newlines from all cells of all rows and all columns of a CSV
#!/usr/bin/env python3
#
# Copyright (c) 2022 Leland Batey. All rights reserved.
#
# This work is licensed under the terms of the MIT license.
# For a copy, see <https://opensource.org/licenses/MIT>.
'''
Reads a CSV on stdin and writes a CSV on stdout. The output CSV should be
semantically identical to the input CSV, except that each cell will have
newline characters ('\n') replaced with an escaped newline ('\\n'). The
original use-case of this tool is to improve how CSVs displayed with `xsv
table` look in the terminal; see the following discussion on the XSV issue
tracker:
https://github.com/BurntSushi/xsv/issues/275
Example CSV:
headerOne,header2
foo,"baz
bar"
Hex dump of example CSV:
$ xxd ./example.csv
00000000: 6865 6164 6572 4f6e 652c 6865 6164 6572 headerOne,header
00000010: 320a 666f 6f2c 2262 617a 0a62 6172 220a 2.foo,"baz.bar".
Hex dump of example CSV *after* processing to remove inter-cell newlines:
$ cat ./example.csv | remove_csv_newlines.py | xxd
00000000: 6865 6164 6572 4f6e 652c 6865 6164 6572 headerOne,header
00000010: 320a 666f 6f2c 6261 7a5c 6e62 6172 0a 2.foo,baz\nbar.
'''
import csv
import sys
rdr = csv.reader(sys.stdin)
wtr = csv.writer(sys.stdout, quoting=csv.QUOTE_MINIMAL, lineterminator='\n')
for row in rdr:
newrow = list()
for col in row:
col = col.replace('\n', '\\n')
newrow.append(col)
wtr.writerow(newrow)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment