# Reduction of time for the code of sorting RGB and Depth images according to timestamp values

I have RGB Images, Depth images and OpenPose Images(generated from RGB Images). The number of RGB and Depth images are different so I have to find the exactly which RGB images corresponds to which depth images. This is done using the time stamp values I have for both RGB and Depth Images.

The problem here is this code I have written takes around 7-10 minutes to execute which I feel is a lot. How can the time be reduced for the same? Here mainly imwrite is taking lot of time.

function image_sorting(color_img, depth_img, Openpose, selected_color, selected_depth)

filePattern = fullfile(depth_img, '*.timestamp');
file = dir(filePattern);

filePattern2 = fullfile(color_img, '*.timestamp');
file2 = dir(filePattern2);

filePattern3 = fullfile(depth_img, '*.bmp');
file3 = dir(filePattern3);

filePattern4 = fullfile(Openpose, '*.png');
file4 = dir(filePattern4);

for k = 1:length(file)
depthTimestampBase = file(k).name;
depthTimestamp = fullfile(depth_img, depthTimestampBase);
fileID = fopen(depthTimestamp,'r');
% format longG
A(k,:) = textscan(fileID,'%d64') ;
fclose(fileID);
end

for m = 1:length(file2)
rgbTimestampBase = file2(m).name;
rgbTimestamp = fullfile(color_img, rgbTimestampBase);
fileID2 = fopen(rgbTimestamp,'r');
%format longG
B(m,:) = textscan(fileID2,'%d64') ;
fclose(fileID2);
end

%%%%%% Here there are two parts. Use any of them according to conditions
if length(file2) <= length(file)
%PART 1:- If RGB images are less than depth then use the following code

for m = 1:length(file2)
for k = 1:length(file)

C{k,m} = A{k,1} - B{m,1};
if C{k,m}<0
C{k,m} = -C{k,m};
end
end
end

[V,X] = min(cell2mat(C),[],1); % Is a row vector containing the minimum value of columns
% V gives minimum value and X gives Index

m = 1;
for w= X

depthDataBase = file3(w).name;
rgbDataBase = file4(m).name;

depthData = fullfile(depth_img, depthDataBase);
rgbData = fullfile(Openpose, rgbDataBase);

depthData2 = fullfile(selected_depth, depthDataBase);
imwrite(imageArrayy, depthData2);
rgbData2 = fullfile(selected_color, rgbDataBase);
imwrite(imageArrayy2, rgbData2);

m = m+1; % for part 2 m = m+1
end

%
% PART 2:- If RGB images are more than depth then use the following code
else

for m = 1:length(file2)
for k = 1:length(file)

C{m,k} = A{k,1} - B{m,1};
if C{m,k}<0
C{m,k} = -C{m,k};
end
end
end

[V,X] = min(cell2mat(C),[],1); % Is a row vector containing the minimum value of columns
% V gives minimum value and X gives Index

w = 1;
for m= X

depthDataBase = file3(w).name;
rgbDataBase = file4(m).name;

depthData = fullfile(depth_img, depthDataBase);
rgbData = fullfile(Openpose, rgbDataBase);

depthData2 = fullfile(selected_depth, depthDataBase);
imwrite(imageArrayy, depthData2);
rgbData2 = fullfile(selected_color, rgbDataBase);
imwrite(imageArrayy2, rgbData2);

w = w+1;
end
end


Update 1:- Screenshot of Profile execution time.

• Have you used the profiler to determine imwrite is taking a lot of time? If so, a screen shot of the profiler output might be helpful (it also might not be). How many images and what size are you working with? – David Dec 11 '19 at 22:09
• And also if you could clarify A and B which are first defined as arrays, but then used as cell arrays when calculating C, should they be cell arrays or normal arrays? – David Dec 11 '19 at 22:27
• @David I have added screenshot of profile. Also about the arrays, First I am reading timestamp values of both RGB and Depth images in B and A respectively. Now to find the frames which were taken on almost same time, I am subrtacting each value of B with one value of A for all the values of A and then finally taking that value of A and B which has the minimum difference value. So I think it has to be used as cell arrays. – Ankit Jaiswal Dec 12 '19 at 10:41
• Wow imwrite really is taking all the time! Given it's a builtin mex file I can't think of anything you'll be able to do given you can't reduce the number of images to save. – David Dec 12 '19 at 22:02
• I have rolled back your latest edit. Please do not update the code in your question to incorporate feedback from answers, doing so goes against the Question + Answer style of Code Review. This is not a forum where you should keep the most updated version in your question. Please see what you may and may not do after receiving answers. – Heslacher Jan 6 at 16:42

You are reading in an image and then writing it back out again without making any changes. If the purpose is to copy the file, you are better off directly copying the file. You can use the function copyfile for that.

The code is fairly readable. There are some improvements you can make:

filePattern = fullfile(depth_img, '*.timestamp');
file = dir(filePattern);

filePattern2 = fullfile(color_img, '*.timestamp');
file2 = dir(filePattern2);

filePattern3 = fullfile(depth_img, '*.bmp');
file3 = dir(filePattern3);

filePattern4 = fullfile(Openpose, '*.png');
file4 = dir(filePattern4);


The four filePatternN variables are never used again. Why use four different variables, rather than repeat the same name? Why not omit the temporary variable and put these into single lines?

file = dir(fullfile(depth_img, '*.timestamp'));


Next, you have two identical loops, one reads from file, one from file2. One writes to A, one to B. Why not make this into a function? You can write local functions at the end of your M-file. You'd simplify this bit to:

A = get_time_stamps(file);
B = get_time_stamps(file2);


Another difference between these two loops is that one uses fileID and one uses fileID2. There is again no need to use two different names for local variables that you don't use outside the loop.

You can improve your code by making A and B numeric arrays rather than cell arrays:

tmp = textscan(fileID,'%d64');
A(k) = tmp{1};


Be sure to preallocate the arrays!

Next you have again two large blocks of mostly repeated code. if length(file2) <= length(file). You could put this into a function and call it do_stuff(file,file2) or do_stuff(file2,file) depending on which of the two is larger. But really, this might not be necessary at all with some better logic.

To find which time stamps are closest to each other, you don't need a O(n2) algorithm. You could instead sort the two arrays of time stamps, and walk through them one by one. The algorithm might be slightly more complex, but you wouldn't need to duplicate the code.

The other alternative is to use vectorized processing to do the O(n2) computation without using loops at all. But in modern versions of MATLAB, loops are not so slow any more, so this is not something you should be worrying about anyway. The way to speed up this loop is to preallocate the array C.

You can also simplify it by writing C{m,k} = abs(A{k,1} - B{m,1});, and removing the if statement.

• Thank you. I have reduced the time by 1/10th by using the copyfile hint. Also I have changed the first part of my code related to Paths. – Ankit Jaiswal Jan 6 at 16:05

As suggested by Cris, did the changes in path and used the hint of copy files and now the it takes around 21 seconds. Here is the solution

function image_sorting(color_img, depth_img, Openpose, selected_color, selected_depth)

file = dir(fullfile(depth_img, '*.timestamp'));

file2 = dir(fullfile(color_img, '*.timestamp'));

file3 = dir(fullfile(depth_img, '*.bmp'));

file4 = dir(fullfile(Openpose, '*.png'));

for k = 1:length(file)
fileID = fopen(fullfile(depth_img, file(k).name),'r');
% format longG
A(k,:) = textscan(fileID,'%d64') ;
fclose(fileID);
end

for m = 1:length(file2)

fileID2 = fopen(fullfile(color_img, file2(m).name),'r');
%format longG
B(m,:) = textscan(fileID2,'%d64') ;
fclose(fileID2);
end

%%%%%% Here there are two parts. Use any of them according to conditions
if length(file2) <= length(file)
%PART 1:- If RGB images are less than depth then use the following code

for m = 1:length(file2)
for k = 1:length(file)

C{k,m} = A{k,1} - B{m,1};
if C{k,m}<0
C{k,m} = -C{k,m};
end
end
end

[V,X] = min(cell2mat(C),[],1); % Is a row vector containing the minimum value of columns
% V gives minimum value and X gives Index

m = 1;
copyfile(Openpose, selected_color)
for w= X

depthDataBase = file3(w).name;
%rgbDataBase = file4(m).name;

depthData = fullfile(depth_img, depthDataBase);
%rgbData = fullfile(Openpose, rgbDataBase);

depthData2 = fullfile(selected_depth, depthDataBase);
imwrite(imageArrayy, depthData2);
%rgbData2 = fullfile(selected_color, rgbDataBase);
%imwrite(imageArrayy2, rgbData2);
%movefile(file4, file5)

m = m+1; % for part 2 m = m+1
end

%
% PART 2:- If RGB images are more than depth then use the following code
else

for m = 1:length(file2)
for k = 1:length(file)

C{m,k} = A{k,1} - B{m,1};
if C{m,k}<0
C{m,k} = -C{m,k};
end
end
end

[V,X] = min(cell2mat(C),[],1); % Is a row vector containing the minimum value of columns
% V gives minimum value and X gives Index

copyfile(depth_img, selected_depth)
for m= X

%depthDataBase = file3(w).name;
rgbDataBase = file4(m).name;

%depthData = fullfile(depth_img, depthDataBase);
rgbData = fullfile(Openpose, rgbDataBase);