In order to avoid having to make any real progress (at least for a short time), I wrote a script that measures how much progress I have made so far, instead. Specifically, this script sums up the file sizes of files in all commits of a git repository, filtering files either by their endings, a whitelist or not at all, and plots this size against the commit dates.

This script needs gitpython to work.

import argparse
import git
import matplotlib.pyplot as plt
import numpy as np
import os

def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("--dir", help="Directory of the git repository to analyze (default: '.')", default=os.curdir)
    parser.add_argument("--pattern", help="Count only files ending in pattern")
    parser.add_argument("--files", help="Only count these files", nargs='+')
    return parser.parse_args()

def filter_pattern(pattern):
    """Returns a function that returns True if the file name ends with pattern"""
    return lambda file: file.name.endswith(pattern)

def filter_whitelist(whitelist):
    """Returns a function that returns True if the file name is in whitelist"""
    whitelist = set(whitelist)
    return lambda file: file.name in whitelist

def get_file_sizes(commit, filter_func=None):
    """Return the sum of the file sizes in the commit.

    Counts only files passing filter_func.
    return sum(file.size
               for file in filter(filter_func, commit.tree.traverse()))

def get_file_sizes_per_commit(repo, filter_func=None):
    """Get the filesizes in all commits of repo.

    Counts only files passing filter_func.
    Ordered from oldest to newest commit.
    return np.array([get_file_sizes(commit, filter_func)
                     for commit in repo.iter_commits()])[::-1]

def get_commit_dates(repo):
    """Get the commit dates of all commits in repo.
    Ordered from oldest to newest.
    return np.array([commit.committed_datetime
                     for commit in repo.iter_commits()])[::-1]

def plot_repo_size(repo, filter_func):
    sizes = get_file_sizes_per_commit(repo, filter_func)
    dates = get_commit_dates(repo)

    plt.plot(dates, sizes)
    plt.ylabel("Repository size")
    plt.xlabel("Commit date")

if __name__ == "__main__":
    args = parse_args()
    filter_func = filter_whitelist(args.files) if args.files else filter_pattern(args.pattern) if args.pattern else None

    repo = git.Repo(args.dir)
    plot_repo_size(repo, filter_func)

I am looking for general feedback as well as how to improve setting the filter function from the parsed arguments.

I am aware that it is not strictly necessary for the data I'm plotting to be numpy.arrays, but I wanted to be able to expand this script in the future and they are just easier to work with (I might even make it a pandas.DataFrame to be able to restrict the date range, interpolate to the future, etc).

With the repository I am using this on, this produces the following image when run with python plot_progress.py --pattern .tex:

enter image description here

  • \$\begingroup\$ Say you have 2 files. One of them doubles in size and the other loses half it's size. What is your actual and expected output? As in, is it already counting absolute values or will it try to tell you you've got 0 progress? \$\endgroup\$ – Mast Oct 18 '18 at 19:41
  • \$\begingroup\$ @Mast In that case it would depend on the original sizes of the two files, since the file sizes are counted in bytes (and not relative to something). But yes, it could look like I did not make any progress at all in a case like this. However, I am fine with that and aware that number of bytes is not a good metric in general (especially for code). But since I am tracking progress in writing a large latex document, more file content tends to mean more progress. \$\endgroup\$ – Graipher Oct 18 '18 at 21:00

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.