Bash script to automate dev environment setup

Question

After blowing up my environment again messing around with installing random programs and packages I decided to write a bash script I can clone from github and use to reinstall everything like I had before

The idea is that I'll be able to keep adding to it as I find more programs to use so that it will always be a stable starting point. Right now it's just a list of whatever I used to reinstall things copy-pasted into a single file.

Do people bother organizing bash scripts? If so what sort of structure do you use? I keep my actual code relatively clean but this whole thing felt like an aside to keep things orderly in the future

echo Email for github?

# get email, assume github username is same and split it off from domain
read email
name=$(echo $email | grep -o '^[^@]*')

mkdir $HOME/GoProjects
mkdir $HOME/GoProjects/src
mkdir $HOME/GoProjects/bin
mkdir $HOME/PyProjects
mkdir $HOME/JSProjects
mkdir $HOME/pemKeys

# basic update and upgrade
sudo apt update && sudo apt -y upgrade

# install Chromium
sudo apt install -y chromium

#install Go
wget https://dl.google.com/go/go1.14.3.linux-amd64.tar.gz
sudo tar xvfz go1.14.3.linux-amd64.tar.gz -C /usr/local/
rm -f go1.14.3.linux-amd64.tar.gz

cat >> $HOME/.profile << EOF
export GOROOT=/usr/local/go
export GOPATH=$HOME/GoProjects
export GOBIN=$HOME/GoProjects/bin
export PATH=$PATH:/usr/local/go:/usr/local/go/bin:$HOME/GoProjects:$HOME/GoProjects/bin
EOF

. $HOME/.profile

go get github.com/lib/pq

# install VScode
curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > packages.microsoft.gpg
sudo install -o root -g root -m 644 packages.microsoft.gpg /usr/share/keyrings/
sudo sh -c 'echo "deb [arch=amd64 signed-by=/usr/share/keyrings/packages.microsoft.gpg] https://packages.microsoft.com/repos/vscode stable main" > /etc/apt/sources.list.d/vscode.list'

sudo apt install -y apt-transport-https
sudo apt update -y
sudo apt install -y code

rm -f packages.microsoft.gpg

# install postgres
wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
RELEASE=$(lsb_release -cs)
echo "deb http://apt.postgresql.org/pub/repos/apt/ ${RELEASE}"-pgdg main | sudo tee  /etc/apt/sources.list.d/pgdg.list
sudo apt update -y
sudo apt -y install postgresql-11

sudo apt install -y build-essential

#install docker
sudo apt install -y apt-transport-https ca-certificates curl gnupg2 software-properties-common
curl -fsSL https://download.docker.com/linux/debian/gpg | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable"
sudo apt update -y
apt-cache policy docker-ce
sudo apt install -y docker-ce
sudo chmod 666 /var/run/docker.sock

#Add anaconda 
sudo apt install -y libgl1-mesa-glx libegl1-mesa libxrandr2 libxrandr2 libxss1 libxcursor1 libxcomposite1 libasound2 libxi6 libxtst6
wget https://repo.anaconda.com/archive/Anaconda3-2020.02-Linux-x86_64.sh
bash Anaconda3-2020.02-Linux-x86_64.sh -b -p
rm -f Anaconda3-2020.02-Linux-x86_64.sh

sudo apt install -y libpq-dev python3-dev
pip install psycopg2

#Add protobuff compiler and grpc
sudo apt install -y protobuf-compiler
export GO111MODULE=on
go get github.com/golang/protobuf/protoc-gen-go
go get -u google.golang.org/grpc

#Need to add NPM
sudo apt install  -y nodejs
sudo apt install -y npm

#setup github
git config --global user.name $name
git config --global user.email $email
git config --global color.ui true

yes "" | ssh-keygen -t rsa -C $email
cat $HOME/.ssh/id_rsa.pub

rm -rf ./SetupDebianDevEnv

Answer 1

I think you could benefit from using Ansible in this scenario then you can easily deploy your configuration to other machines and you will have learned a valuable IT skill.

And rather than use bash scripts, you use yaml configuration files.

As for your script: there is a fair amount of repetitive code. Rather than repeat the wget or curl, you could gather your sources into an array, then run wget/curl in a loop, because you are performing multiple operations that are similar. Upside: more concise code. Downside: less separation between the various steps. I can understand you did it this way. But if your file grows because you keep adding sources, you'd have to reconsider the approach and start using loops.

This is something you could have done here too:

mkdir $HOME/GoProjects
mkdir $HOME/GoProjects/src
mkdir $HOME/GoProjects/bin
mkdir $HOME/PyProjects
mkdir $HOME/JSProjects
mkdir $HOME/pemKeys

An array would make sense. As mentioned above, don't forget to quote !

Same for the apt install, but since you are adding some repo sources, that needs to be done beforehand, and it would require reorganization of your script.

The other issue is you are downloading specific software versions that will probably be outdated by the time you reinstall your computer. Thus, an outdated script becomes less valuable, if you have yet to update or patch your install to have a setup that is current.

As an example:

#install Go
wget https://dl.google.com/go/go1.14.3.linux-amd64.tar.gz
sudo tar xvfz go1.14.3.linux-amd64.tar.gz -C /usr/local/
rm -f go1.14.3.linux-amd64.tar.gz

I would use variables instead, so that the package name is defined only once:

#install Go
go_file="go1.14.3.linux-amd64.tar.gz"
wget "https://dl.google.com/go/$go_file"
sudo tar xvfz "$go_file" -C /usr/local/
rm -f "$go_file"

It is a minor change that will make maintenance of this file more manageable.

Also, you could move the code to dedicated functions. That makes it a bit easier to separate the various steps you are performing. Also, you can easily disable certain items by simply commenting out the function call (= changing only 1 line). In fact it would be nice if you could run your script à la carte, it is perfectly possible that you will want to run it again, but discard certain items that are already installed. For instance if your script crashed in the middle and you don't want to waste time reinstalling everything from the beginning, which is wasteful.

The other benefit of calling functions in sequence, is that you can easily reorder them. At some point you may find out that a certain piece of software should or must be installed before another, because of some dependencies.

I would install software packages from the regular OS repositories whenever possible. Do you really need to install Postgresql from source ?

I would also avoid PIP and install the Python packages from apt-get instead, unless they are not available for your OS.

Answer 2

First of all, I would suggest using shellcheck when writing bash https://www.shellcheck.net/, it will point out many many errors, some trivial and some not so trivial.

Line 7:
mkdir $HOME/GoProjects
      ^-- SC2086: Double quote to prevent globbing and word splitting.

Simple enough, if your $HOME has a space in it say, the mkdir won't work as you expect.

More subtle

Line 4:
read email
^-- SC2162: read without -r will mangle backslashes.

Maybe not a problem for you, but no harm in doing it right.

You can use shellcheck in editors via plugins etc, there's a cli you can use. Really nice, bash is very easy to make mistakes in as we all know and this will just generally help you.

Do people organise bash scripts, yes! I've known some old hands who are very skilled with bash. As far as I know there's no consensus on the structure to follow, but generally I think you want to be following good programming guidelines, split things into functions, make it modular, etc. It's good that you have comments, but I think better still to replace the comment with a function with a sensible name which does the part the comment alludes to.

That being said, what you have is basically as far as I'm concerned one of the canonical applications of a bash script, i.e. not doing much complicated, just running a bunch of commands in order. You probably use it one off infrequently. Is it worth making it any better? Probably not.

You have no error handling, and various things are hard coded assumptions about the machine you will run this on, your script isn't idempotent, when something does go wrong, when you run it the second time it may have funny results. Idempotency is another good thing to aim for. Is that any of this is missing bad, not really.

Personally, I don't like bash scripts. I prefer to use a programming language which is a bit more verbose but allows me to be more confident of what I'm doing. For this reason I write my scripts in node or python when I can, as these allow me to accumulate a collection of functions which I find easier to reason about, do error handling with, and externalise configuration.