.npy files, each has ~80MB, each contains a vector of high dimension (
I have to deal with these data with the code below:
G = np.zeros((N, N)) for i in range(N): vi = np.load(str(i)+'.npy') for j in range(i, N, 1): vj = np.load(str(i)+'.npy') G[i, j] = process(vi, vj) G[j, i] = G[i, j] post_process(G)
It takes a lot of time depending on reading files and I'd like to accelerate it.
Is there any suggestion?