Listing my StackEchange communities user id

Question

Extending this answer solution, I've wrote the following bash script to get the html of my StackExchange accounts page and parse it to a YAML file. The objective was list for each community I participate its url and my user id.

get-stackexchange-info.sh

#!/bash/bin

NETWORK_PROFILE_URL=$1 # https://stackexchange.com/users/3581081/artu-hnrq
read -r ID USER <<< \
    $( cut -d'/' -f5,6 <<< $NETWORK_PROFILE_URL \
        | sed 's/\// /g' \
    )

OUTPUT_FILE=stackexchange.yml

echo \
"user: $USER
stackexchange:
  url: http://stackexchange.com
  id: $ID" \
> $OUTPUT_FILE

parse(){
    local HTTPS="https:\/\/"
    local COMMUNITY="[^\.]*"
    local DOMAIN="[^\/]*"
    local PROFILE_ID="\d*"

    local MODEL="\1:\n  url: $HTTPS\1\2\n  id: \3"

    sed -E "s/($COMMUNITY)($DOMAIN)\/($PROFILE_ID)/$MODEL/"
}

curl "https://stackexchange.com/users/$ID/$USER?tab=accounts" \
    | grep "account-container" -A1 \
    | grep "http" \
    | cut -d"/" -f3,5 \ # stackoverflow.com/3581081
    | parse \
    >> $OUTPUT_FILE

I would like to receive some opinions about the parse process (usage of grep, cut and sed commands) and about control flow management could be improved to the script

output example:

user: artu-hnrq
stackexchange:
  url: https://stackexchange.com
  id: 3581081
askubuntu:
  url: https://askubuntu.com
  id: 689894
stackoverflow:
  url: https://stackoverflow.com
  id: 2989289
codereview:
  url: https://codereview.stackexchange.com
  id: 213097

Thanks a lot!

Answer 1

I see 2 errors and have some more advice.

The Shebang is wrong.

#!/bash/bin
should be
#!/bin/bash

Do not put comment after the \ escape for a new line

| cut -d"/" -f3,5 \ # stackoverflow.com/3581081
# should be
| cut -d"/" -f3,5 \

I would move all functions (in your case only parse()) to the top of the file.

Use lowercase variables (uppercase is reserved for system variables like PATH).

NETWORK_PROFILE_URL=
# change to
network_profile_url=

Combine cut and sed

read -r ID USER <<< \
    $( cut -d'/' -f5,6 <<< $NETWORK_PROFILE_URL \
        | sed 's/\// /g' \
    )
# can be replaced by
read -r ID USER < <(awk -F "/" '{print $(NF-1) " " $NF }' <<< $NETWORK_PROFILE_URL)
# which is easier to read then
source <(sed -r 's#.*/(.*)/(.*)#ID=\1; USER=\2#' <<< $NETWORK_PROFILE_URL)

When you end a line with a pipe, you do not need to escape the newline with \.

curl "https://stackexchange.com/users/$ID/$USER?tab=accounts" \
    | grep "account-container" -A1 \
    | grep "http" \
    | cut -d"/" -f3,5 \
    | parse 
# can be changed into
curl "https://stackexchange.com/users/$ID/$USER?tab=accounts" |
    grep "account-container" -A1 |
    grep "http" |
    cut -d"/" -f3,5 |
    parse 

You had a hard job parsing the curl output with different tools. In this situation you won't need to parse large files, the performancy penalty is very small. It is worth the effort for other cases to look for a solution with awk.
The grep -A1 is replaced by setting a flag for parsing the next line.
Fields are found with the delimiter '/'.
Combining substr is only slightly more complex:

curl -s "https://stackexchange.com/users/$ID/$USER?tab=accounts" |
   awk -F "[/]" '
      parse==1 {
         print substr($3,1,index($3,".")-1) ":";
         printf("  url: https://%s\n", $3);
         printf("  id: %s\n", $5);
         parse=0;
      }
      /account-container/ {parse=1}
    '