1
\$\begingroup\$

I have two matrices a and b that have, on each line, some values (null or not) and then a bunch of zeros at the end.

I want to construct a matrix c that for each line has the corresponding line of a from the first element till to the last not-null element, concatenated with the corresponding line of b from the first element till to the last not-null element. Like in the following example:

>> a=[1 NaN 3 0; 1 0 0 0]

a =

     1   NaN     3     0
     1     0     0     0

>> b=[1 0 1 0; 0 0 0 0]

b =

     1     0     1     0
     0     0     0     0

>> c=NaN(2,8) %same dimension as a and b   

c =

   NaN   NaN   NaN   NaN   NaN   NaN   NaN   NaN
   NaN   NaN   NaN   NaN   NaN   NaN   NaN   NaN

        for ii=1:size(a,1)
        [v,x]=find(a(ii,:));
        if numel(x)>0 %it could be all null
            x1=a(ii,1:x(end)); %x(end) because I want to retain the zeros before the last not-null element
        end
        [v,x]=find(b(ii,:));
        if numel(x)>0
            x1=[x1 b(ii,1:x(end))];
        end
        c(ii,1:numel(x1))=x1;
        end
>> c

c =

     1   NaN     3     1     0     1   NaN   NaN
     1   NaN   NaN   NaN   NaN   NaN   NaN   NaN

How can I improve this code? a and b always have the same dimensions (that can be different from the example though). a and b values can be positive/negative finite or infinite doubles or NaNs. In my case I am interested only to merge two matrices.

Are there "standard" ways to vectorize a code? What do you look at when thinking to how vectorize a code?

\$\endgroup\$
  • 1
    \$\begingroup\$ Out of curiosity — what do a and b represent, and why do you want to do this operation? \$\endgroup\$ – 200_success Jul 6 '16 at 10:17
  • \$\begingroup\$ @200_success It comes out during a messy data post-processing. I have a matrix of the dimensions of c containing an index that determines what I have to do with the values in a matrix A with the same dimensions. Unfortunately A got split ina and b during the data analysis. The fastest solution is now to merge them again. The data are temperature and stratification data of ocean water columns. \$\endgroup\$ – shamalaia Jul 6 '16 at 10:22
  • \$\begingroup\$ @StewieGriffin unfortunately I think that the only thing that it cannot be is -Inf \$\endgroup\$ – shamalaia Jul 7 '16 at 7:20
4
\$\begingroup\$

I see two major way to simplify and speed this up. First, rather than copying a to c at every step of the loop, you can define c to include a from the beginning. Second, you can find all the nonzero, non-NaN values in a vectorized manner at the very beginning This reduces the time for me be about 1/2 for a large (~10000 row) random data set.

function c = catnonzero_2(a, b)
    c=[a, NaN(size(b))];
    agood = a~=0;
    bgood = b~=0;
    for ii=1:size(a,1)
        ailast=find(agood(ii,:), 1, 'last');
        if isempty(ailast)
            ailast = 0;
        end
        bilast=find(bgood(ii,:), 1, 'last');
        if ~isempty(bilast)
            c(ii, ailast+1:ailast+bilast) = b(ii, 1:bilast);
        else
            c(ii, ailast+1:end) = NaN;
        end
    end
end

This can be improved even more by separating out the rows that can be vectorized from those that can't.

  1. I find all the rows of a and b with some non-null values ("good" rows). Rows that aren't good are then filled with NaN.
  2. I find all the rows of a and b where the last value is non-null ("perfect" rows).
  3. For rows that are good but not perfect ("imperfect" rows):
    1. For b I can loop over them an put the NaN values at the end. This allows me to treat all b rows as either "perfect" or "bad", since the "imperfect" rows already have the NaN values at the end and thus can be used in the entirety.
    2. For a, I can loop over the rows where a are imperfect and b are not good and put NaN at the end of those. Where b is good, b will be ovewriting those anyway so it isn't necessary as long as b is at least as long as a.
  4. I make c like in the previous example.
  5. In cases where a is perfect and b is good, I can just write b to c at the point where a ends.
  6. In the cases where a is bad and b is food, I can just write b from the start of c.
  7. In the remaining cases, I have to calculate where b starts, and then write b.

  8. 3.

This reduces the time for me by about another 1/4:

function c = catnonzero_3(a, b)
    alen = size(a, 2);
    blen = size(b, 2);

    agood = a~=0;
    bgood = b~=0;

    agoodrows = any(agood, 2);
    bgoodrows = any(bgood, 2);

    aperfectrows = agood(:,end);
    bperfectrows = bgood(:,end);
    aimperfectrows = agoodrows & ~aperfectrows;

    a(~agoodrows, :) = NaN;
    b(~bgoodrows, :) = NaN;

    if alen <= blen
       apadtarg = aimperfectrows & ~bgoodrows;
    else
       apadtarg = aimperfectrows;
    end

    for ii=find(apadtarg)'
        a(ii, ailast(ii)+1:end) = NaN;
    end
    for ii=find(bgoodrows & ~bperfectrows)'
        b(ii, bilast(ii)+1:end) = NaN;
    end

    c=[a, NaN(size(b))];

    ailast = @(ii) find(agood(ii,:), 1, 'last');
    bilast = @(ii) find(bgood(ii,:), 1, 'last');

    aperfbgood = aperfectrows & bgoodrows;
    c(aperfbgood, size(a, 2)+1:end) = b(aperfbgood, :);

    abadbgood = ~agoodrows & bgoodrows;
    c(abadbgood, 1:blen) = b(abadbgood, :);

    for ii=find(aimperfectrows & bgoodrows)'
        ajj = ailast(ii);
        c(ii, ajj+1:ajj+blen) = b(ii, :);
    end
end

And here is the data set I used to test (your version is catnonzero_1:

a=randi([-11, 10], 100000,80);
b=randi([-11, 10], 100000,100);

a(a==-11) = NaN;
b(b==-11) = NaN;

timeit(@() catnonzero_1(a, b))
timeit(@() catnonzero_2(a, b))
timeit(@() catnonzero_3(a, b))

And the result:

>> run_w_time

ans =

    0.889284394000000


ans =

    0.411362394000000


ans =

    0.102184394000000


ans =

    1


ans =

    1


c1 = catnonzero_1(a, b);
c2 = catnonzero_2(a, b);
c3 = catnonzero_3(a, b);

all(all(c1(~isnan(c1))==c2(~isnan(c2))))
all(all(c1(~isnan(c1))==c3(~isnan(c3))))
\$\endgroup\$
  • \$\begingroup\$ You had my upvote almost a month ago, but I just re-read the answer. I just want to say: This is a very good answer in my eyes =) \$\endgroup\$ – Stewie Griffin Aug 4 '16 at 5:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.