3
\$\begingroup\$

I have the following files:

  • STUDENT_D19001.T103412_HU.txt
  • STUDENT_D19001.T103412_KU.txt
  • STUDENT_D19002.T113030_HU.txt
  • STUDENT_D19002.T115055_KU.txt
  • STUDENT_D19003.T115055_KU.txt
  • STUDENT_D19004.T135040_HU.txt

I want to process the common/pair files separately in this order (HU1 and then KU1; HU2 and then KU2) and the uncommon/unpaired files separately.

These two files (STUDENT_D19001.T103412_HU.txt and STUDENT_D19001.T103412_KU.txt) are common/pair files because they have same day of the year 001 in their name. However, these two files (STUDENT_D19003.T115055_KU.txt and STUDENT_D19004.T135040_HU.txt) are uncommon/unpaired because the do not have the same day of year with other files.

The output of the script is illustrated below for further clarification:

*** Uncommon/Unpaired files: ***
Processing './STUDENT_D19003.T115055_KU.txt'
Processing './STUDENT_D19004.T135040_HU.txt'

*** Common/Pair files: ***
1 of : 4 './STUDENT_D19001.T103412_HU.txt'
2 of : 4 './STUDENT_D19001.T103412_KU.txt'
3 of : 4 './STUDENT_D19002.T113030_HU.txt'
4 of : 4 './STUDENT_D19002.T115055_KU.txt'

Here is the script for review:

#!/bin/bash

SOURCE_DIR="."

#
# Create an array of all the HU files
#
HU_files=()
while IFS='' read -r -d '' file
do
    HU_files+=("$file")
done < <(find "$SOURCE_DIR" -maxdepth 1 -type f -iname "*HU*" -printf '%p\0' | sort -zn)

#
# Calculate HU total files
#
total_HU="${#HU_files[@]}"

#
# Create an array of all the BFV files
#
KU_files=()
while IFS='' read -r -d '' file
do
    KU_files+=("$file")
done < <(find "$SOURCE_DIR" -maxdepth 1 -type f -iname "*KU*" -printf '%p\0' | sort -zn)

#
# Calculate KU total files
#
total_KU="${#KU_files[@]}"


# Find common/pair HU and KU files considering partial part name 'STUDENT_D19001'
COMMON_files=()
i=0
k=0

while [ $i -lt $total_HU ] 
do
    j=0 
    while [ $j -lt $total_KU ] 
    do
        if [[ "${HU_files[$i]/.T*/}" == "${KU_files[$j]/.T*/}" ]]
        then
            COMMON_files[$((k++))]="${HU_files[$i]}"
            COMMON_files[$((k++))]="${KU_files[$j]}"
            break
        else 
            ((j++))
        fi
    done
    ((i++))
done

# Process uncommon/unpaired HU and KU files
echo "*** Uncommon/Unpair files: ***"
while IFS='' read -r -d '' file
do
    if [[ ! "${COMMON_files[@]}" =~ "${file}" ]]
    then
    echo "Processing '$file'"
    fi
done < <(find "$SOURCE_DIR" -maxdepth 1 -type f \( -iname "*HU*.txt" -o -iname "*KU*.txt" \) -printf '%p\0' | sort -zn)

#
# Process the common/pair HU and KU files
#
total_COMMOM="${#COMMON_files[@]}"
counter=0
echo "*** Common/Pair files: ***"
for file in "${COMMON_files[@]}"
do  
    echo "$((++counter)) of $total_COMMOM: '$file'"
done

Your inputs are appreciated :-)

\$\endgroup\$
5
  • \$\begingroup\$ Imagine that there would be 2 HU files (not HU and KU) with same D19001, like STUDENT_D19001.T103412_HU.txt and STUDENT_D19001.T103515_HU.txt. How would that effect the expected result and how it should look in such case? \$\endgroup\$ – RomanPerekhrest Dec 13 '19 at 17:58
  • \$\begingroup\$ @RomanPerekhrest Such a case will not occur because there are two files HU and KU with day of the year part of their name. However, sometimes it happens that there is only either HU or KU which are considered unpaired. \$\endgroup\$ – Sherzad Dec 13 '19 at 18:02
  • \$\begingroup\$ Is some real processing implied for "uncommon" files or it's just for printing? \$\endgroup\$ – RomanPerekhrest Dec 13 '19 at 20:58
  • \$\begingroup\$ @RomanPerekhrest of course there is. Here the print is only for debugging purpose. \$\endgroup\$ – Sherzad Dec 13 '19 at 21:13
  • \$\begingroup\$ @RomanPerekhrest they could be moved to another directory for further investigation for example ... \$\endgroup\$ – Sherzad Dec 13 '19 at 21:21
1
\$\begingroup\$

Per shell checkers feedback, I suggest the following changes:

Fix#1 To prevent globing and word splitting it is recommended to use double quote. Hence, update this while [ $i -lt $total_HU ] to while [ $i -lt "$total_HU" ], and this while [ $j -lt $total_KU ] to while [ $j -lt "$total_KU" ]. I am aware we are dealing with number rather than string but no harm to double quote variables.

Fix#2 Per SC2199 and SC2076 this line of code if [[ ! "${COMMON_files[@]}" =~ "${file}" ]] is problematic, therefore, I changed it to if [[ ! "${COMMON_files[*]}" =~ ${file} ]].

In this context I am sure that I want to match literally so I could also use a trick like this if [[ ! "${COMMON_files[*]}" =~ "${file}"* ]].

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.