Bash script to replace substrings in a file with strings

Question

I am an editor of the Funtoo Linux wiki and was looking to create a shell script that would help to ease the updating of pages on the wiki that use old tags by replacing certain deprecated tags with their newer counterparts. For example, I am interested in replacing every occurrence of <tt> with {{c|. I wrote a bash script that accomplishes the task required, but seems to be somewhat bloated and 'hackish' and I am looking for ways that I can consolidate what I have written.

#!/bin/bash

# An application used to replace all instances of <tt>, <code>, and <console> in a wiki page
# with their corresponding updated tags.

FILE_TO_CONVERT="toConvert"
NEW_FILE_LOCATION="converted"

LINE_COUNT=1
CODE_COUNT=0
TT_COUNT=0
CONSOLE_COUNT=0
END_COUNT=0

replaceCharacters() {
    while read line; do
            if [[ $line =~ \<code\> ]]; then
                    let "CODE_COUNT++"

                    if [[ $line =~ \<\/code\> ]]; then
                            let "END_COUNT++"
                            replace_string=${line//<code>/"{{c|"}
                            replace_string=${replace_string//<\/code>/"}}"}
                            echo ${replace_string} >> ${NEW_FILE_LOCATION}
                            echo "[Line ${LINE_COUNT}]: Found <code> and </code>."
                    else
                            replace_string=${line//<code>/"{{c|"}
                            echo ${replace_string} >> ${NEW_FILE_LOCATION}
                            echo "[Line ${LINE_COUNT}]: Found <code>"
                    fi
            elif [[ $line =~ \<tt\> ]]; then
                    let "TT_COUNT++"

                    if [[ $line =~ \<\/tt\> ]]; then
                            let "END_COUNT++"
                            replace_string=${line//<tt>/"{{c|"}
                            replace_string=${replace_string//<\/tt>/"}}"}
                            echo ${replace_string} >> ${NEW_FILE_LOCATION}
                            echo "[Line ${LINE_COUNT}]: Found <tt> and </tt>"
                    else
                            replace_string=${line//<tt>/"{{c|"}
                            echo ${replace_string} >> ${NEW_FILE_LOCATION}
                            echo "Line ${LINE_COUNT}]: Found <tt>"
                    fi
            elif [[ $line =~ \<console\> ]]; then
                    let "CONSOLE_COUNT++"
                    replace_string=${line//<console>/"{{console|body="}
                    echo ${replace_string} >> ${NEW_FILE_LOCATION}
                    echo "[Line ${LINE_COUNT}]: Found <console>"
            elif [[ $line =~ \<\/[ct][ot].*\> ]]; then
                    replace_string=${line//<\/tt>/"}}"}
                    replace_string=${replace_string//<\/code>/"}}"}
                    replace_string=${replace_string//<\/console>/"}}"}
                    echo ${replace_string} >> ${NEW_FILE_LOCATION}
                    echo "[Line ${LINE_COUNT}]: Found an end (</tt>,</console> or </code>)"
                    let "END_COUNT++"
            else
                    echo ${line} >> ${NEW_FILE_LOCATION}
            fi
            let "LINE_COUNT++"
    done < "$FILE_TO_CONVERT"
}

displayReplaceInfo() {
    echo -e "\n================="
    for count in CODE_COUNT TT_COUNT CONSOLE_COUNT END_COUNT; do
            echo ${count}: $((count))
    done
}

# Run the functions
replaceCharacters
displayReplaceInfo

In particular, the replaceCharacters() function seems quite redundant. How can I simplify this code?

Here is an example of what a page I might be parsing through looks like:

=== Installing the Stage 3 tarball ===

Now that filesystems are created and your hardware and system clock are     set, the next step is downloading the initial Stage 3 tarball. The Stage 3 is a pre-compiled system used as a starting point to install Funtoo Linux. Notice: if you're using virtual machines (like Vbox) generic stage3 images are preferred rather than cpu-optimized ones.

To download the correct build of Funtoo Linux for your system, head over to the [[Subarches]] page. Subarches are builds of Funtoo Linux that are designed to run on a particular type of CPU, to offer the best possible performance. They also take advantage of the instruction sets available for each CPU.

The [[Subarches]] page lists all CPU-optimized versions of Funtoo Linux. Find the one that is appropriate for the type of CPU that your system has, and then click on its name in the first column (such as <code>corei7</code>, for example.) You will then go to a page dedicated to that subarch, and the available stage3's available for download will be listed.

For most subarches, you will have several stage3's available to choose from. This next section will help you understand which one to pick.

==== Which Build? ====

'''If you're not sure, pick <code>funtoo-current</code>.'''

Funtoo Linux has various different 'builds':

{{TableStart}}
<tr><th class="info">Build</th><th class="info">Description</th></tr>
<tr><td><code>funtoo-current</code></td><td>The most commonly-selected build of Funtoo Linux. Receives rapid updates and preferred by desktop users.</td></tr>
<tr><td><code>funtoo-stable</code></td><td>Emphasizes less-frequent package updates and trusted, reliable versions of packages over the latest versions.</td></tr>
{{TableEnd}}

==== Which Variant? ====

'''If you're not sure, pick <code>standard</code>.'''

Our "regular" stage3's are listed with a variant of <code>standard</code>. The following variant builds are available:

{{TableStart}}
<tr><th class="info">Variant</th><th class="info">Description</th></tr>
<tr><td><code>standard</code></td><td>The "standard" version of Funtoo Linux</td></tr>
<tr><td><code>pure64</code></td><td>A 64-bit build that drops multilib (32-bit compatibility) support. Can be ideal for server systems.</td></tr>
<tr><td><code>hardened</code></td><td>Includes PIE/SSP toolchain for enhanced security. PIE does require the use of PaX in the kernel, while SSP works with any kernel, and provides enhanced security in user-space to avoid stack-based exploits. For expert users.</td></tr>
{{TableEnd}}

==== Download the Stage3 ====

Once you have found the stage3 that you would like to download, use <code>wget</code> to download the Stage 3 tarball you have chosen to use as the basis for your new Funtoo Linux system. It should be saved to the <code>/mnt/funtoo</code> directory as follows:

<console># ##i##cd /mnt/funtoo
# ##i##wget http://build.funtoo.org/funtoo-current/x86-64bit/generic_64/stage3-latest.tar.xz
</console>

Answer 1

The way you replace characters is extremely tedious, hard to follow, and error prone. A lot easier way is using sed. Here's an example to get you started, replacing two patterns:

sed -e 's/<tt>/{{c|/g' -e 's?</tt>?}}?g' input.txt > output.txt

This won't print such details like "Line ${LINE_COUNT}]: Found <tt>" as your original, but hopefully that's not too important to you. If on the other hand those diagnostic lines are important to you, then you can either write another script to generate those details, or rewrite this script in another language that's better suited for line-by-line processing and transformation, like Python or Perl.

Answer 2

Your logic looks like it would break if a line had <tt>x=1</tt> <code>, it would hit the code block first and skip the tts.

You don't need as much branching structure, you could go for:

replaced=${line//<code>/"{{c|"}
if [ $replaced!=$line ]; then
    echo "[Line $LINE_COUNT]: found <code>"
    line=${replaced}
fi

replaced=${line//<tt>/"{{c|"}
if [ $replaced!=$line ]; then
    echo "[Line $LINE_COUNT]: found <tt>"
    line=${replaced}
fi
...
echo ${replace_string} >> ${NEW_FILE_LOCATION}

It would end up slightly shorter, clearer and able to handle mixed things on one line.

With Python, you could pair up the patterns and replacements, and have a clear loop:

pairs = [('<tt>',       '{{c|'),
         ('</tt>',      '}}'),
         ('<code>',     '{{c|'),
         ('</code>',    '}}'),
         ('<console>',  '{{console|body='),
         ('</console>', '}}')]

counts = {}
# ..open wiki_file and output file..

for line_num, line in enumerate(wiki_file, 1):
    for old_text, new_text in pairs:
        if old_text in line:
            counts[old_text] = counts.get(old_text, 0) + 1
            line = line.replace(old_text, new_text)
            print "[Line {}]: found {}".format(line_num, old_text)

    # print line to new file

print counts

This isn't to say "you should use Python"*, it's to offer a different structure that's probably possible in any scripty/interpreted language (but I can't write Bash like that off the top of my head).

_{*you should, though}