On , I learnt ...

Python’s csv.writer function defaults to Windows line endings

I was confused as to why awk wasn’t matching the final element in a CSV stream from csv.writer. Example:

$ python -c "
> import csv, sys
> w = csv.writer(sys.stdout, delimiter='\t')
> w.writerow(['foo', 'bar'])
> " | awk '$2 == "bar" { print $1 }'

This should print foo as the second element in the stream is bar. But it doesn’t.

This is because csv.writer defaults to using Windows-style line endings \r\n which prevents awk from matching the last field on each line.

The fix is to use Posix-style line endings \n:

$ python -c "
> import csv, sys
> w = csv.writer(sys.stdout, delimiter='\t', lineterminator='\n')
> w.writerow(['foo', 'bar'])
> " | awk '$2 == "bar" { print $1 }'
foo