cleanup.sh

With the text files edited, it’s time to clean up a little bit. That’s what cleanup.sh is for.

As usual, we start with the shebang line.

#!/usr/bin/env zsh

We remove all of the no longer necessary backups. The -f makes sure there are no complaints in case no backups exist.

# remove temporary files 
rm -f *.bak
rm -f *~

This section isn’t really necessary, but I added in to make working with one-off awk commands a little easier (tabs as delimiters are much easier to deal with than commas).

# tsv file for easier command line text processing
filepath=`realpath *.csv`
dirname=`dirname $filepath`
basename=`basename $filepath '.csv'`
mlr --c2t cut -o -f bill_type,bill_number,sponsor,action_date,committee $filepath > "$dirname/$basename.tsv"
echo "Created $dirname/$basename.tsv"

Miller really shines here. awk could do the same thing, but with Miller I can use the names of the columns. In awk, I’d have to remember which numeric field variable, e.g., $1 corresponds to which column.

Here, we add the names of all the text files in the directory to the clipboard for pasting into the spreadsheet.

ls *.txt | pbcopy
echo "Filenames copied to clipboard."

Now it’s time to get rid of the backup directory created by prep.sh if it exists.

PWD=$(pwd)
if [[ -d "${PWD}.bak" ]]
then
	rm -rf "${PWD}.bak"
	echo "Backup removed."
fi

Finally, we move the directory up to its final location on my hard drive.

mv $PWD $(dirname $(dirname $PWD))
echo "${PWD} moved."
cd ..

Up next

Now it’s time for a little data munging with the script update-tsv-data.sh.