One of my main goals regarding this script has been to figure out a way to use the "adblock.sources" file available on its main repository and not my fork. That version has some syntax issues with its JSON, preventing it from being directly piped into jq
.
After some research, I came across the jq -n -f {file}
command that successfully read in the ill-formatted JSON! I have a couple questions on this feature and one on the sed
command.
- Is
jq -n -f {file}
being used properly? - Can the two
jq
commands be blended into one command? - Can the two
sed
parameters be blended into one parameter?
#!/bin/sh
set -eu
sources=$(mktemp)
trap 'rm "$sources"' EXIT
curl -s -o "$sources" https://raw.githubusercontent.com/openwrt/packages/master/net/adblock/files/adblock.sources
jq -n -f "$sources" | jq -r 'keys[] as $k | [$k, .[$k].url, .[$k].rule] | @tsv' |
while IFS=$'\t' read key url rule; do
case $key in
gaming | oisd_basic | yoyo )
# Ignore these sources:
# "gaming" blocks virtually all gaming servers
# "oisd_basic" is included in "oisd_full"
# "yoyo" is included in "stevenblack"
;;
* )
curl -s "$url" |
case $url in
*.tar.gz) tar -xOzf - ;;
*) cat ;;
esac | gawk --sandbox -- "$rule"
;;
esac
done | sed -e 's/\r//g' -e 's/^/0.0.0.0 /' | sort -u >| the_blacklist.txt
# use sort over gawk to merge sort multiple temp files instead of using up limited memory