Edit: The goal of this Question was only seeking code review and guidance in learning or tips for improvement. I apologize if that was not clear or aligned properly with the goals of this forum.
Script Goal: audit file existence and differences between two red hat Enterprise Linux Servers
- Place information in sql server tables to use for analysis
- Maintain ability to view the diff in a web page
- Process Steps
- Set array with Directorys to check
- Set array of file types to look for
- Build a list of existing files for each Environment
- Script will solve the following
- If the files exist in both servers are they the identical? (md5)*
- If no what is different (Diff)?
- Does the file contain hard coded values (grep)
- View Diff in web page (currently reading text file using into site php)
Code
DIR=(/Dir/Durp/DurpaDurp/Scriptdir1 /Dir/Durp/DurpaDurp/Scriptdir2 /Dir/Durp/DurpaDurp/Scriptdir3 ) f_Type=("*.sh" "*.txt" "*.log") f_Type2=("*.img" "*.rpt" ) for((i=0; i<${#DIR[@]}; i++)) do echo "CHECKING: ${DIR[$i]}" cd "${DIR[$i]}" for((x=0; x<${#f_Type[@]}; x++)) do echo "FOR FILE TYPE: ${f_Type[$x]}" find $PWD -type f -name "${f_Type[$x]}" | sed 's/^/ENV1|/'| column -t >> "$filelistENV1" done if [[ ${DIR[$i]} == "/Dir/Durp/DurpaDurp/Script3" ]]; then for((y=0; y<${#f_Type[@]}; y++)) do echo "FOR FILE TYPE: ${f_Type2[$y]}" find $PWD -type f -name "${f_Type2[$y]}" | sed 's/^/ENV1|/'| column -t >> "$filelistENV1" done fi done
After these files are output to the text files the script will delete the existing data from a staging table in SQL Server 2008 R2 and insert the new data.
for((i=0; i<${#ENV[@]}; i++)) do sqlcmd -S $_DB_CONN -d $_DB -Q "DELETE FROM ['$_DB']..['$_TABLE'] WHERE ENV = '${ENV[$i]}'"; done bcp $_DB_CONN.."$_TABLE" in "$filelistENV1" -f "$_SCRIPTDIR/STG.fmt" -e $_ERRDIR/ERROR_STG$(date -d "today" +"%Y%m%d%H%M").txt -S $_DB_CONN -d "$_DB"
the format file creates 2 columns
AbsoluteFilePath | ENV
gets a list of files from the database to compare
`
sqlcmd -S $_DB_CONN -d $_DB -s "|" -h-1 -m -1 -W -i $_SCRIPTDIR/SQL/EXPORT_COMPARE.sql -o $_INPUT/comp_list.txt set NOCOUNT ON;
compare the md5sum of the files.
for i in $(cat "$_INPUT/comp_list.txt") do export filename=$(basename "$i") export path=$(dirname "$i") env1_md5sum=$(md5sum "$i") env1="${env1_md5sum%% *}" export tmpdir=("$_TMPDIR$path") if ssh "$_CONN" stat $path'$filename' \> /dev/null 2\>\&1 then env2_md5sum=$(ssh $_CONN "cd $path; find -name '$filename' -exec md5sum {} \;") env2_md5="${env2_md5sum%% *}" if [[ $env1_md5 == $env2_md5 ]]; then echo $filename $path >> "$matchingMD5" else echo "md5 does not match, getting copy of file" echo "$i" >> "$no_matchMD5" mkdir -p $tmpdir scp $_CONN:$i $tmpdir fi fi done
run a diff on files that do not match
for x in $(cat "$no_matchMD5") do comp_filename=$(basename "$x") env2file=(/"$ScriptsDir"/tmp"$x") DIFF=$(diff --ignore-all-space --ignore-blank-lines --brief "$x" "$env2file" &>/dev/null