Creating a proper workflow through version control and a Makefile

Question

disclaimer: I've posted this around but the question keeps being put on hold for being inherently opinionated which I understand, just looking for some vague direction as to how a more experienced developer would approach this problem. I think this is the right place based on what I'm asking and I apologize if not.

So I've been working on creating a workflow for my buddy and I to use across the 2 projects and few servers we have. While at first I started with a shared group ownership option of the project files, I found this to be annoying and prone to human error.

I decided to convert the project to an svn controlled directory on a separate server where my production server and ourselves can pull from as needed, we can do local builds/testing and then tell the server to pull changes when we are ready to push to live, the concept is nice and seems pretty simple.

I have the svn system built out how I like it however I am struggling with how to set up my Makefile, my plan is to have a few commands:

• clean - go and delete all extra artifacts created as you work
• dev - spin up a local flask instance for testing purposes
• venv - destroy and rebuild the virtual environment, used mainly for new setups when you first pull the project
• install - tell the production server to pull the most recent changes and then restart apache through an 'svn' user we all have access to

This is the simplest Makefile I can think of to do the jobs I want, as well as easily allowing for more support as time goes on if need be.

My problem is that I'm not exactly sure what the best way to approach 'make venv' and 'make install'. I have it set up manually right now and it works but it's just not graceful and I know it's becuase of my lack of knowledge in building a large python project like this. I've seen people use setuptools but I'm not sure if that's totally overkill for my project or not.

Does anyone have a similar workflow and/or have any suggestions on how I can improve my setup? I have my Makefile and install.sh script below for reference.

install.sh:

svn up

sudo /etc/init.d/apache2 restart

Makefile:

clean:
    find . -name \*~ -delete
    find . -name \*.pyc -delete

test:
    FLASK_APP=octlive/__init__.py \
    FLASK_DEBUG=1 \
    ../venv/bin/flask run --host=0.0.0.0

venv:
    virtualenv ../venv
    ../venv/bin/pip install -r require.txt

relclean: clean 
    rm -rf ../venv

dev: relclean venv test

install:
    ssh user@server '/var/www/app/install.sh'

and just in case it's useful, my require.txt:

Flask
Flask-session
Flask-bootstrap
WTForms

I know this is not the best practice how I have my Makefile set up, specifically how I am using make to call install.sh on the production server. Has anyone taken a similar approach to what I've done and if so what suggestions do you have? I feel kind of lost in how to make this better while knowing it isn't that good to begin with.

Answer 1

I don't know anything about Flask, but there are definitely problems with the Makefile:

dev: relclean venv test

In a parallel build, the building of the relclean target can overlap with that of venv and test; you might want to add some ordering there (or put .NOTPARALLEL: as a target).

The rules for clean and dev don't create their targets, so need to be marked with .PHONY.

All makefiles should have .DELETE_ON_ERROR to avoid partially-written files satisfying the dependencies.

The install target doesn't appear to depend on anything. So make install may well install an out-of-date product. On further reading, the install target doesn't look like a conventional install step - perhaps it would be better named as update-server or something?

Overall, it looks more like a job for a script than a Makefile, unless you are actually going to make use of Make's dependency engine. Perhaps something like this:

#!/bin/bash

# Definitions
server=server
user=${USER:-user}

export FLASK_APP=octlive/__init__.py
export FLASK_DEBUG=1

# Actions
action()
{
do case "$1" in
       clean)
           find . -name \*~ -delete
           find . -name \*.pyc -delete
           ;;
       test)
           ../venv/bin/flask run --host=0.0.0.0
           ;;
       venv)
           virtualenv ../venv
           ../venv/bin/pip install -r require.txt
           ;;
       relclean)
           action clean 
           rm -rf ../venv
           ;;
       dev)
           action relclean
           action venv
           action test
           ;;
       install)
           ssh "$user@$server" /var/www/app/install.sh
           ;;
   esac
}


for arg in "$@"
do action "$arg"
done