Wednesday, October 28, 2015

Extracting and Organizing with Bash

Long week? Yeah, me too. I have my heavy metal Linux band in the motel room and no customers to attend to at the moment…let’s do some Bash scripting! Remember the “thumbnailing” script I did a few weeks ago? This is the script I use before doing thumbnails. It’s actually a bit more trippy and uses another script which I’ll cover in the next installment.
I begin by getting a temporary file setup:
  • #!/bin/bash
  • set -e
  • DateFile="/tmp/image-dates.$$"
  • [ -z "$DateFile" ] && echo "no datefile, bye." && exit 1
  • [ -f "$DateFile" ] && rm -f $DateFile
  • touch "$DateFile"
We have a guard: did we screw up our file name? That’s almost irrationally cautious, so you might feel justified in deleting that guard. The $$ symbol is a quick way to get your process ID (PID). You can use this for many things, but it is reasonably unique with low-frequency usage (that is, a few times a day).
You can only have 65535 process IDs, and there are 86400 seconds a day, so if you approach one process a second, you’ll get repeats. If we find an identical date file, let’s delete it anyhow and create a new one. (POP QUIZ: tell me a one-liner that does this.)
Turn that metal up! We’re about to “headbang” through a crazy lead solo of string manipulation:
  • ls *.jpg *.dng *.bmp *.tiff *.jpeg \
  • *.JPG *.DNG *.BMP *.TIFF *.JPEG 2>/dev/null \
  • | while read F
  • do
  • G=$( echo -n "$F" | perl -pe 'y/[A-Z]/[a-z]/' )
  • cmd="mv -v $F $G"
  • $cmd
  • done
Yeah! Let’s crush some upper-case file names! Why the 2>/dev/null though? Have you ever done an “ls * .txt” command just to get everything in your directory listed and followed by a warning: .txt: file not found ? Yeah. That error message, or those TIFF or JPEG files you probably won’t have will all give you that message. It’s a safe message to discard into /dev/null.
We’re taking each file name F, and piping it into perl and feeding it into the Sarlacc jaws of the y/// translator. This y/// operator is the perl equivalent to the ‘tr’ command: it translates one character pattern to another character pattern. We’re forcing all upper-case alphabetic characters into lower case characters. Why? Type in caps much? (The bassist does, the bastard.) Extra credit: give me the shell command that replaces that perl command.
Now G is the transformed name, and we just move the file from the original F value to the new G value. (POP QUIZ: Why, oh why would I assemble a string of this command? Could this be evidence of an erased echo statement?) Extra Credit: make this command safer for filenames with spaces in them.
Drum solo! We’re going to rattle out some dates with another wee utility called
  • ls *.{jpg,dng,bmp,tiff,jpeg,gif,png} 2>/dev/null \
  • | while read F
  • do
  • echo -n "$F " >> $DateFile
  • ~/bin/ $F >> $DateFile
  • done
Remember that the >> operator appends to files and echo -n prints stuff without a newline. Pay attention to spaces. We’re immediately going to use this output in a loop:
  • cat $DateFile | sort | uniq \
  • | while read D
  • do
  • echo "D $D"
  • imgfile=`echo "$D" | awk '{print $1}'`
  • imgdir=`echo "$D" | awk '{print $NF}'`
  • if [ ! -d "$imgdir" ]
  • then
  • mkdir -v $imgdir
  • fi
  • mv -v $imgfile $imgdir
  • done
What, another external program…awk? Isn’t that the sound your drummer makes after the fifth shot of Jagermeister? Hell yeah. Awk loves to toss out column values, so $1 is anything in the first column. What did we put in the first column of DateFile? Oh yeah…the drummer, no, the image file name. The last column is always $NF. Pop Quiz: Why not $2 in the second column? Warning: Don’t ask where why the drummer is called NF, he’s shy. Extra Credit #2: what if the file name has spaces in it?
Now we need to clean the floor of our motel room: /tmp. Let’s wipe up $DateFile.
  • rm -f $DateFile
I’m going to drag the drummer off to the tour bus, but I’m going to leave you with one more puzzler: If I delete $DateFile at the end of this party, why do I bother deleting it at the start of the party as well? Think about it…

No comments:

Post a Comment