Finding columns of a matrix that are the same for combinations of events

How I can change this multi-loop to reduce the computation time?

A is a matrix (5 × 10000) where the fifth line contains values between 1 and 50 corresponding to different events of an experiment. My goal is to find the columns of the matrix which are the same for all possible combinations of 7 different events.

data = A(1:4,:);
exptEvents = A(5,:);

% Find repeats
[b,i,j] = unique(data', 'rows');

% Organize the indices of the repeated columns into a cell array
reps = arrayfun(@(x) find(j==x), 1:length(i), 'UniformOutput', false);

% Find events corresponding to these repeats
reps_Events = cellfun(@(x) exptEvents(x), reps, 'UniformOutput', false);

U = cellfun(@unique, reps_Events, 'UniformOutput', false);
repeat_counts = cellfun(@length, U);

kk=1;

for i1=1:50
for i2=1:50
for i3=1:50
for i4=1:50
for i5=1:50
for i6=1:50
for i7=1:50
if i1~=i2 && i2~=i3 && i3~=i4 && i4~=i5 && i5~=i6 && i6~=i7
myEvents = [i1 i2 i3 i4 i5 i6 i7];
v= b(cellfun(@(x)all(ismember(myEvents,x)),U),:);
intersection(1,kk)=v(1);
intersection(2,kk)=v(2);
intersection(3,kk)=v(3);
intersection(4,kk)=v(4);
intersection(5,kk)=i1;
intersection(6,kk)=i2;
intersection(7,kk)=i3;
intersection(8,kk)=i4;
intersection(9,kk)=i5;
intersection(10,kk)=i6;
intersection(11,kk)=i7;
kk=kk+1;
end
end
end
end
end
end
end
end

• I'm not sure I understand your problem clearly - does the 5th row of A contain a cell for each column, each with 7 values (1-50) in it? – aganders3 Dec 14 '11 at 22:19
• I think the problem is the need to avoid 'for' loops. otherwise you can see my last question (Intersection of several sets of results corresponding to different events in matlab) for an example (5x20) of the matrix A. – bzak Dec 14 '11 at 22:34
• So does the code you pasted here give the desired output, it just takes too long? – aganders3 Dec 14 '11 at 22:49
• yes this is the case for code – bzak Dec 14 '11 at 23:16

There's quite a few things that you can do to reduce computation time.

First of all, if you know that an outcome has to be the combination of seven events, you can throw out whatever has fewer of them.

Then, you can preassign the output array (growing in a loop is usually not very fast).

Finally, you want to generate all possible subsets if there are, say, 9 events from which you can choose 7.

So you'd start

%# throw out useless stuff
tooFewIdx = repeat_counts < 7;

b(tooFewIdx,:) = [];
U(tooFewIdx) = [];
repeat_counts(tooFewIdx) = [];

%# estimate how many combinations you will get
nPerms = arrayfun(@(x)nchoosek(x,7),repeat_counts);

intIdx = [0;cumsum(nPerms(:))];

%# pre-assign output
%# If this runs out of memory, try using integer formats
%# instead, e.g. zeros(...,'uint8') if no value is above 255
intersection = zeros(intIdx(end),11);

%# loop to fill intersection
for i=1:length(U)

%# fill in the experiment information
intersection((intIdx(i)+1):intIdx(i+1),1:4) = repmat(b(i,:),nPerms(i),1);

%# add all combinations of events
intersection((intIdx(i)+1):intIdx(i+1),5:11) = combnk(U{i},7);

end

• ??? Maximum variable size allowed by the program is exceeded. Error in ==> intersection = zeros(intIdx(end),11); I have values ​​that are above 1000 – bzak Dec 14 '11 at 23:58
• @bzak: If you are trying to choose 7 out of 50, then you're going to do a lot of combinations, which means that you will not have enough memory to create the variable. Maybe you may have to rethink your approach to whatever problem it is you're trying to solve. – Jonas Dec 15 '11 at 5:09
• I reduced the matrix to a size of (5x2000) and tried to see the results for all 3 possible combinations. I found the first error: ??? Subscript indices must either be real positive integers or logicals. Error in ==> intersection((intIdx(i-1)+1):intIdx(i),1:4) = repmat(b(i,:),nPerms(i),1); So I replaced (for i = 1: length (U)) by (for i = 2: length (U)) but I got the second error: ??? Subscripted assignment dimension mismatch. Error in ==> intersection((intIdx(i-1)+1):intIdx(i),1:4) = repmat(b(i,:),nPerms(i),1); – bzak Dec 15 '11 at 12:27
• @user319336: turns out that I made an error with the indices. This should hopefully work. – Jonas Dec 15 '11 at 19:34