One of my main goals regarding this script has been to figure out a way to use the "adblock.sources" file available on its main repository and not my fork. That version has some syntax issues with its JSON, preventing it from being directly piped into jq.

After some research, I came across the jq -n -f {file} command that successfully read in the ill-formatted JSON! I have a couple questions on this feature and one on the sed command.

  1. Is jq -n -f {file} being used properly?
  2. Can the two jq commands be blended into one command?
  3. Can the two sed parameters be blended into one parameter?

set -eu

trap 'rm "$sources"' EXIT

curl -s -o "$sources" https://raw.githubusercontent.com/openwrt/packages/master/net/adblock/files/adblock.sources

jq -n -f "$sources" | jq -r 'keys[] as $k | [$k, .[$k].url, .[$k].rule] | @tsv' |
while IFS=$'\t' read key url rule; do
    case $key in
        gaming | oisd_basic | yoyo )
        # Ignore these sources:
        # "gaming" blocks virtually all gaming servers
        # "oisd_basic" is included in "oisd_full"
        # "yoyo" is included in "stevenblack"
        * )
            curl -s "$url" |
                case $url in
                    *.tar.gz) tar -xOzf - ;;
                    *) cat ;;
                esac | gawk --sandbox -- "$rule"
done | sed -e 's/\r//g' -e 's/^/ /' | sort -u >| the_blacklist.txt
# use sort over gawk to merge sort multiple temp files instead of using up limited memory

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.