Adding a zero to file names

Question

I've recently discovered the power of sed; regular expressions seem like they'll take a lifetime to master. This tiny script uses both to add a zero before the single digit in file names.

Can this be done more easily without sed? Is the regex sound?

#!/bin/bash

# > addzero.sh <
# Adds a zero before the single digit in file names separated by an underscore.
# Designed to help keep files in sequence in directory listings. Otherwise, you get listings like
# "_0.txt, _1.txt, _10.txt, _11.txt, _12.txt, ..., _2.txt, _20.txt, _21.txt, _22.txt, ..."

for filename in $(ls -1 ./*); do
  if [[ $filename =~ _[0-9]\. ]]; then
    new_filename=$(echo $filename | sed -n 's/_\([0-9]\)\./_0\1\./p')
    mv $filename $new_filename
  fi
done

Answer 1

What's wrong

for filename in $(ls -1 ./*); do

Ouch! As a general rule (with very few exceptions), do not use ls in scripts. What you wrote is almost equivalent to for filename in ./*; do, except that if there are any non-printable characters, whitespace characters, or \[?* in the file names, they will be mangled if you use ls. You don't need the ./ (except to ensure that the file name doesn't start with a -) but it doesn't hurt.

[[ $filename =~ _[0-9]\. ]]

could be written in a slightly simpler and more portable way: [[ $filename = *_[0-9].* ]]. And since you can use a simple glob pattern, you might as well not have iterated over all file names: for filename in *_[0-9].*; do. But there's a better way to express your script, to take advantage of BASH_REMATCH; see below.

echo $filename

Always put double quotes around variable and command substitutions. Exception: when you understand why you need to leave the double quotes off and why it's safe to do so. When the shell sees a variable substitution ($foo or ${foo}) or a command substitution (`foo` or $(foo)) outside double quotes, the result of the substitution undergoes word splitting and globbing (filename generation). That was one of the problems with $(ls -1 ./*) earlier. This should be echo "$filename".

In fact, it would be better printf "%s" "$filename", because echo itself performs expansions. In bash, unless you've set non-default options to enable backslash expansion, the only problem is that a few arguments beginning with a - look like option, and in this specific case the filename will begin with ./.

There is an edge case where your sed call won't work: if you have a file name that ends with a newline character. This doesn't happen in practice unless someone has made a mistake (like a rogue script or a bad copy-paste) or is deliberately trying to trick your script — so watch out in security contexts.

Incidentally, this is one of the few cases where it's safe to leave out the double quotes around a command substitution: in a variable assignment, there is an implicit pair of double quotes around the right-hand side, so new_filename=$(…) is equivalent to new_filename="$(…)". Note that this does not extend to export VARIABLE="$(value)", where the double quotes are necessary.

We now turn to your question about the use of sed. It is not necessary here; you can perform this substitution in bash. Bash has a pattern replacement parameter substitution feature, but it's limited to a constant replacement text. Bash also has a way to extract substrings from regexp matches with =~, through the BASH_REMATCH variable. After the match, ${BASH_REMATCH[0]} contains the portion of the string matched by the regexp, ${BASH_REMATCH[1]} contains the portion matched by the first parenthesized group and so on.

mv $filename $new_filename

Again, double quotes.

---

A working script

One possibility is to extract the text to replace from the regexp match, then perform a string replacement on it. Since I'm not using ./*, the file name may begin with a -, so I take care to use -- on the call to mv to ensure that the file name isn't seen as an option.

for filename in *; do
  if [[ $filename =~ _[0-9]\. ]]; then
    from=${BASH_REMATCH[0]}
    mv -- "$filename" "${filename//$from/${from/_/_0}}"
  fi
done

Another possibility is to match the whole file name as a regexp and splice a 0 into the bits. Note that this will behave differently in a corner case: if there are several occurrences of _[0-9]\., the code above replaces the first occurrence, while this replaces the last occurrence.

for filename in *; do
  if [[ $filename =~ ^(.*_)([0-9]\..*)$ ]]; then
    mv -- "$filename" "${BASH_REMATCH[1]}0${BASH_REMATCH[2]}"
  fi
done

You can also use other constructs and avoid regexps altogether, but it's more complicated. An advantage is that the script then works in all shells, not just bash.

for filename in *_[0-9].*; do
  digit=${filename%.${filename##*_[0-9].}}; digit=${digit##*.}
  mv -- "$filename" "${filename%%_[0-9].*}_0${digit}.${filename#*_[0-9].*}"
done

There are other methods to perform this kind of file renaming. If you're on Debian or derivative (including Ubuntu) or have the rename Perl script that's floating around (note that this is not the standard Linux rename utility):

rename 's/_([0-9]\.)/_0$1/' *_[0-9].*

Or with mmv:

mmv -x '*_[0-9].*' '#1_0#2.#3'

Or in zsh, after autoload zmv:

zmv '(*_)([0-9].*)' '${1}0$2'

Further reading

The rename and quoting tags on Unix Stack Exchange.

Answer 2

I wouldn't use regular expressions at all here. I'd use printf to pad the digits, and bash glob-patterns to extract the digits.

shopt -s extglob
for filename in *; do
  tmp=${filename%.txt}   # remove the ".txt" extension
  digits=${tmp##*_}      # remove everything up to the final underscore
  case $digits in
    +([0-9])) 
      # 'digits' contains only digits
      newname=${filename%_*}_$(printf "%02d" $digits).txt
      mv "$filename" "$newname"
      ;;
  esac
done