# Accelerate computing a Gram-like matrix with large files

I have N(>1000) .npy files, each has ~80MB, each contains a vector of high dimension (P).

I have to deal with these data with the code below:

G = np.zeros((N, N))
for i in range(N):
for j in range(i, N, 1):
G[i, j] = process(vi, vj)
G[j, i] = G[i, j]

post_process(G)


It takes a lot of time depending on reading files and I'd like to accelerate it.

Is there any suggestion?