Horizontally concatenating the entries of two matrices, and padding with NaNs

Question

I have two matrices a and b that have, on each line, some values (null or not) and then a bunch of zeros at the end.

I want to construct a matrix c that for each line has the corresponding line of a from the first element till to the last not-null element, concatenated with the corresponding line of b from the first element till to the last not-null element. Like in the following example:

>> a=[1 NaN 3 0; 1 0 0 0]

a =

     1   NaN     3     0
     1     0     0     0

>> b=[1 0 1 0; 0 0 0 0]

b =

     1     0     1     0
     0     0     0     0

>> c=NaN(2,8) %same dimension as a and b   

c =

   NaN   NaN   NaN   NaN   NaN   NaN   NaN   NaN
   NaN   NaN   NaN   NaN   NaN   NaN   NaN   NaN

        for ii=1:size(a,1)
        [v,x]=find(a(ii,:));
        if numel(x)>0 %it could be all null
            x1=a(ii,1:x(end)); %x(end) because I want to retain the zeros before the last not-null element
        end
        [v,x]=find(b(ii,:));
        if numel(x)>0
            x1=[x1 b(ii,1:x(end))];
        end
        c(ii,1:numel(x1))=x1;
        end
>> c

c =

     1   NaN     3     1     0     1   NaN   NaN
     1   NaN   NaN   NaN   NaN   NaN   NaN   NaN

How can I improve this code? a and b always have the same dimensions (that can be different from the example though). a and b values can be positive/negative finite or infinite doubles or NaNs. In my case I am interested only to merge two matrices.

Are there "standard" ways to vectorize a code? What do you look at when thinking to how vectorize a code?

Answer 1

I see two major way to simplify and speed this up. First, rather than copying a to c at every step of the loop, you can define c to include a from the beginning. Second, you can find all the nonzero, non-NaN values in a vectorized manner at the very beginning This reduces the time for me be about 1/2 for a large (~10000 row) random data set.

function c = catnonzero_2(a, b)
    c=[a, NaN(size(b))];
    agood = a~=0;
    bgood = b~=0;
    for ii=1:size(a,1)
        ailast=find(agood(ii,:), 1, 'last');
        if isempty(ailast)
            ailast = 0;
        end
        bilast=find(bgood(ii,:), 1, 'last');
        if ~isempty(bilast)
            c(ii, ailast+1:ailast+bilast) = b(ii, 1:bilast);
        else
            c(ii, ailast+1:end) = NaN;
        end
    end
end

This can be improved even more by separating out the rows that can be vectorized from those that can't.

1. I find all the rows of a and b with some non-null values ("good" rows). Rows that aren't good are then filled with NaN.
2. I find all the rows of a and b where the last value is non-null ("perfect" rows).
3. For rows that are good but not perfect ("imperfect" rows):
   1. For b I can loop over them an put the NaN values at the end. This allows me to treat all b rows as either "perfect" or "bad", since the "imperfect" rows already have the NaN values at the end and thus can be used in the entirety.
   2. For a, I can loop over the rows where a are imperfect and b are not good and put NaN at the end of those. Where b is good, b will be ovewriting those anyway so it isn't necessary as long as b is at least as long as a.
4. I make c like in the previous example.
5. In cases where a is perfect and b is good, I can just write b to c at the point where a ends.
6. In the cases where a is bad and b is food, I can just write b from the start of c.

7. In the remaining cases, I have to calculate where b starts, and then write b.

3.

This reduces the time for me by about another 1/4:

function c = catnonzero_3(a, b)
    alen = size(a, 2);
    blen = size(b, 2);

    agood = a~=0;
    bgood = b~=0;

    agoodrows = any(agood, 2);
    bgoodrows = any(bgood, 2);

    aperfectrows = agood(:,end);
    bperfectrows = bgood(:,end);
    aimperfectrows = agoodrows & ~aperfectrows;

    a(~agoodrows, :) = NaN;
    b(~bgoodrows, :) = NaN;

    if alen <= blen
       apadtarg = aimperfectrows & ~bgoodrows;
    else
       apadtarg = aimperfectrows;
    end

    for ii=find(apadtarg)'
        a(ii, ailast(ii)+1:end) = NaN;
    end
    for ii=find(bgoodrows & ~bperfectrows)'
        b(ii, bilast(ii)+1:end) = NaN;
    end

    c=[a, NaN(size(b))];

    ailast = @(ii) find(agood(ii,:), 1, 'last');
    bilast = @(ii) find(bgood(ii,:), 1, 'last');

    aperfbgood = aperfectrows & bgoodrows;
    c(aperfbgood, size(a, 2)+1:end) = b(aperfbgood, :);

    abadbgood = ~agoodrows & bgoodrows;
    c(abadbgood, 1:blen) = b(abadbgood, :);

    for ii=find(aimperfectrows & bgoodrows)'
        ajj = ailast(ii);
        c(ii, ajj+1:ajj+blen) = b(ii, :);
    end
end

And here is the data set I used to test (your version is catnonzero_1:

a=randi([-11, 10], 100000,80);
b=randi([-11, 10], 100000,100);

a(a==-11) = NaN;
b(b==-11) = NaN;

timeit(@() catnonzero_1(a, b))
timeit(@() catnonzero_2(a, b))
timeit(@() catnonzero_3(a, b))

And the result:

>> run_w_time

ans =

    0.889284394000000


ans =

    0.411362394000000


ans =

    0.102184394000000


ans =

    1


ans =

    1


c1 = catnonzero_1(a, b);
c2 = catnonzero_2(a, b);
c3 = catnonzero_3(a, b);

all(all(c1(~isnan(c1))==c2(~isnan(c2))))
all(all(c1(~isnan(c1))==c3(~isnan(c3))))