Does anyone know how to get sed to print the position of the illegal byte sequence? Or does anyone know what the illegal byte sequence is?
$ uname -a
Darwin Adams-iMac 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64
I got part of the way to answering the above just by using tr.
I have a .csv file that is a credit card statement and I am trying to import it into Gnucash. I am based in Switzerland so I have to deal with words like Zürich. Suspecting Gnucash does not like " " in numeric fields, I decide to simply replace all
; ;
with
;;
Here goes:
$ head -3 Auswertungen.csv | tail -1 | sed -e 's/; ;/;;/g'
sed: RE error: illegal byte sequence
I used od to shed some light: Note the 374 halfway down this od -c output
$ head -3 Auswertungen.csv | tail -1 | od -c
0000000 1 6 8 7 9 6 1 9 7 1 2 2 ; 5
0000020 4 6 8 8 7 X X X X X X 2 6
0000040 6 0 ; M Y N A M E I S X ; 1
0000060 4 . 0 2 . 2 0 1 9 ; 9 5 5 2 -
0000100 M i t a r b e i t e r r e s t
0000120 Z 374 r i c h
0000140 C H E ; R e s t a u r a n t s ,
0000160 B a r s ; 6 . 2 0 ; C H F ;
0000200 ; C H F ; 6 . 2 0 ; ; 1 5 . 0
0000220 2 . 2 0 1 9 \n
0000227
Then I thought I might try to persuade tr to substitute 374 for whatever the correct byte code is. So first I tried something simple, which didn't work, but had the side effect of showing me where the troublesome byte was:
$ head -3 Auswertungen.csv | tail -1 | tr . . ; echo
tr: Illegal byte sequence
1687 9619 7122;5468 87XX XXXX 2660;MY NAME ISX;14.02.2019;9552 - Mitarbeiterrest Z
You can see tr bails at the 374 character.
Using perl seems to avoid this problem
$ head -3 Auswertungen.csv | tail -1 | perl -pne 's/; ;/;;/g'
1687 9619 7122;5468 87XX XXXX 2660;ADAM NEALIS;14.02.2019;9552 - Mitarbeiterrest Z?rich CHE;Restaurants, Bars;6.20;CHF;;CHF;6.20;;15.02.2019