3
\$\begingroup\$

I have a correlation matrix containing 4 time points, each with multiple samples. Each sample is identified with a time point with its name. What I am trying to accomplish here is to create a directed graph using Python's (2.7) Networkx with edges connecting nodes from the last time point to the first time point (6h --> 4h --> 2h --> 1h) given that the correlation value is above a certain threshold. On top of this I would only like to draw edges to nodes only if they have an already existing edge (with exception to the last time point). This will narrow the network the further the path gets.

Additionally since there are only 4 time points, I would like to create a sort of x-axis with each time point and with the respective nodes listed vertically above the x-axis tick marks. Ultimately I would like the graph to look something like a horizontal Christmas tree.

The code I have written works but it was written sloppy and I'm trying my best to compact it. I have listed my code below:

import networkx as nx
import matplotlib.pyplot as plt
from pandas import DataFrame

DG=nx.DiGraph()

corr=DataFrame.from_csv('Correlation_LPS-timecourse.txt', header=0, sep='\t')

timepoint1_allCols=corr.filter(regex=r'(?i)_1h_', axis=0)
timepoint2_allCols=corr.filter(regex=r'(?i)_2h_', axis=0)
timepoint3_allCols=corr.filter(regex=r'(?i)_4h_', axis=0)
timepoint4_allCols=corr.filter(regex=r'(?i)_6h_', axis=0)

timepoint_12=timepoint1_allCols.filter(regex=r'(?i)_2h_', axis=1)
timepoint_23=timepoint2_allCols.filter(regex=r'(?i)_4h_', axis=1)
timepoint_34=timepoint3_allCols.filter(regex=r'(?i)_6h_', axis=1)

threshold = 0.98

for idx, row in timepoint_34.iterrows():
    for i,entry in enumerate(row):
        if entry > threshold:
            DG.add_edge(row.index[i], idx, weight=entry)

for idx, row in timepoint_23.iterrows():
    for i,entry in enumerate(row):
            if entry > threshold and DG.degree(row.index[i]):
                DG.add_edge(row.index[i], idx, weight=entry)

for idx, row in timepoint_12.iterrows():
    for i,entry in enumerate(row):
            if entry > threshold and DG.degree(row.index[i]):
                DG.add_edge(row.index[i], idx, weight=entry)

nx.draw(DG, pos = nx.spring_layout(DG), with_labels=True)
plt.show()

If you're interested in playing with the real data that I am using I have uploaded it here.

\$\endgroup\$
1
\$\begingroup\$

This is an awkward way of storing data. You have extremely similar variable names with similar methods of getting the data. You should put them in a list instead, and then you can get them with a list comprehension:

timepoint1_allCols=corr.filter(regex=r'(?i)_1h_', axis=0)
timepoint2_allCols=corr.filter(regex=r'(?i)_2h_', axis=0)
timepoint3_allCols=corr.filter(regex=r'(?i)_4h_', axis=0)
timepoint4_allCols=corr.filter(regex=r'(?i)_6h_', axis=0)

could be

timepoint_allCols = [corr.filter(regex=r'(?i)_{}h_'.format(hour), axis=0)
                     for hour in (1, 2, 4, 6)]

The same with your follow up variables. It's easier to understand and will avoid accidental typos leading you astray. Then it also makes looping over the three results easier:

for idx, row in timepoint_34.iterrows():
    for i,entry in enumerate(row):
        if entry > threshold:
            DG.add_edge(row.index[i], idx, weight=entry)

for idx, row in timepoint_23.iterrows():
    for i,entry in enumerate(row):
            if entry > threshold and DG.degree(row.index[i]):
                DG.add_edge(row.index[i], idx, weight=entry)

for idx, row in timepoint_12.iterrows():
    for i,entry in enumerate(row):
            if entry > threshold and DG.degree(row.index[i]):
                DG.add_edge(row.index[i], idx, weight=entry)

You do need to make some variations but I think it's better to not repeat yourself.

for num, timepoint in enumerate(timepoints[::-1]):
    for idx, row in timepoint.iterrows():
        for i, entry in enumerate(row):
            if entry > threshold and (num == 0 or DG.degree(row.index[i])):
                DG.add_edge(row.index[i], idx, weight=entry)

To explain some things, I'm looping backwards by using the slicing operator [::-1]. I'm also using enumerate along with num == 0 to check if it's your first timepoint so that you know whether or not to use the DG.degree(row.index[i]) test. This is a bit of a hacky solution, but as you're more familiar with the data and process, you could come up with a better and clearer test to determine if the test is necessary to run. The point is, that collapsing it into one loop (even if you need to test for one exception condition) is clearer and less code because the loops were so similar.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.