Developing an investment strategy based on stock movements

Question

Below is my code

import pandas as pd
import math
import csv

fund = 10000
print("investment",fund)

pval = 0
oldportfolio = []

dts = ["06 Feb 2017", "07 Feb 2017", "08 Feb 2017", "09 Feb 2017", "10 Feb 2017", "13 Feb 2017", "14 Feb 2017", "15 Feb 2017", "16 Feb 2017", "17 Feb 2017",
        "20 Feb 2017", "21 Feb 2017", "22 Feb 2017", "23 Feb 2017", "27 Feb 2017"]
for dt in dts:
    files = ["stocklistcustom.csv"]
    for file in files:
        df = pd.read_csv(file, header=None)
        i = 0
        filecount = len(df)
        result = []
        while i < filecount:
        # while i < 10:
            name = df[0][i]
            link = df[1][i]
            mcsym = df[2][i]
            i = i + 1
            filepath = "data/nse/his/" + mcsym + ".csv"
            try:
                sp = pd.read_csv(filepath, header=None)
                endrow = sp[sp[0] == dt].index[0] + 1
                parray = []
                tarray = []
                starray = []
                intdate = []
                p1 = 0
                p2 = 0
                p3 = 0
                p4 = 0
                j = 0
                mavg15 = ''
                mavg60 = ''
                olddiff = 0
                days = 2
                strtrow = endrow - days - 60
                for k in range (strtrow, endrow):
                    date = sp[0][k]
                    price = float(sp[4][k])
                    k = k + 1
                    parray.append(price)
                    j = j + 1
                    strtavg = j - 15
                    mavg15 = sum(parray[strtavg:j]) / 15
                    strtavg = j - 60
                    mavg60 = sum(parray[strtavg:j]) / 60
                    # buy criteria
                    if j > 59:
                        diff = mavg60 - mavg15
                        if diff < 0 and olddiff > 0:
                            trigger = 1
                            intdate.append(date)
                        else:
                            trigger = 0
                        tarray.append(trigger)
                        olddiff = diff
                    # sell criteria
                    if j == (days + 60):
                        pricep = (price - p1) * 100 / p1
                        p1p = (p1 - p2) * 100 / p2
                        p2p = (p2 - p3) * 100 / p3
                        p3p = (p3 - p4) * 100 / p4
                        if pricep < -5 or pricep > 8:
                            sell = 1
                        if price < p1 and p1 < p2 and p2 < p3:
                            sell = 1
                        else:
                            sell = 0
                    p4 = p3
                    p3 = p2
                    p2 = p1
                    p1 = price
                if sum(tarray) > 0:
                    result.append([name,mcsym,"buy",price])
                if sell > 0:
                    result.append([name,mcsym,"sell",price])
            except:
                # print(name,"not found")
                pass

    # print(result)

    output = "output/triggers/"+dt+"trigger.csv"
    with open(output, "wb") as f:
        writer = csv.writer(f)
        writer.writerows(result)
        print(output,"exported")

    # Code for calculating investment
    portfolio = []
    for row in result:
        if row[2] == "sell" and len(oldportfolio) > 0:
            pindex = 0
            for buys in oldportfolio:
                bindex = 0
                for stock in buys:
                    if row[0] == stock[0]:
                        sellqty = stock[2]
                        sellp = row[3]
                        sellval = sellqty * sellp
                        purchasep = stock[1]
                        sellcost = purchasep * sellqty
                        print(dt,"selling",row[0],row[1],sellp,sellqty,sellval)
                        # print(oldportfolio)
                        del oldportfolio[pindex][bindex]
                        # print(oldportfolio)
                        fund = fund + sellval
                        pval = pval - sellcost
                    bindex = bindex + 1
                pindex = pindex + 1
    # print("op", oldportfolio)
    # print(dt,"fund after selling",fund)
    buycount = sum(1 for row in result if row[2]==("buy"))
    if buycount > 0:
        maxinvest = fund / buycount
    for row in result:
        if row[2] == "buy":
            name = row[0]
            price = row[3]
            qty = math.floor(maxinvest / price)
            if qty > 0:
                val = qty * price
                print(dt,"buying",name,row[1],price,qty,val)
                portfolio.append([name,price,qty,val])
                fund = fund - val

    # print("portfolio",portfolio)
    pval = pval + sum(row[3] for row in portfolio)
    print(dt,"cash",fund,"portfolio value",pval,"total",fund+pval)
    oldportfolio.append(portfolio)

print(oldportfolio)

It computes the value of portfolio on each day after implementing a specific buy and sell strategy.

But it takes huge time in execution. How to speed it up?

Answer 1

How to speed it up?

You might focus on the assignments to df and sp, which parse a potentially very large number of rows. Otherwise, there's nothing that is obviously slow.

The code as written is illegible, as the outer for loop has well over a hundred lines of code with more than half a dozen levels of nesting. As an aid to everyone, especially yourself, you really want to break out several helper functions.

For the community to be able to offer more help, you should post timings from a profiling run, as well as the example data that Bryce Guinta mentioned.

Answer 2

Readability could be improved (Besides what @J_H mentioned), by giving meaningful variable names (e.g. j, p1, p2 etc. could have more descriptive names).

Regarding performance, I noticed that the same csv file (stocklistcustom.csv) is read again and again for each date. It would be better to read it once and keep contents in memory.