I have a matrix of fewer than 1021 rows. I'd like to take its transpose, but it is too big for GNU `datamash` and `awk` solutions to fit into memory.

To keep memory overhead low, my thought is that I keep a list of read-only file pointers to the start of each row. I then read out bytes from each pointer until I hit a delimiter (tab or newline). 

Because I have fewer than 1021 rows, I won't hit the usual 1024 OS-based file pointer limit.

Once I have read out a field from all file pointers, I write a newline character and start over, reading a field from all file pointers, and again, until no more bytes are available:

    #!/usr/bin/env python                                                                                                                                                                                                                               
    
    import sys
    import os
    
    try:
      in_fn = sys.argv[1]
    except ValueError as ve:
      sys.exit(-1)
    
    def get_size(fn):
      st = os.stat(fn)
      return st.st_size
    
    #                                                                                                                                                                                                                                                   
    # 1) Read in file offsets to start of new line                                                                                                                                                                                                      
    # 2) Open up a file pointer to that offset                                                                                                                                                                                                          
    # 3) Process each file pointer to get a value until a delimiter is hit, then write it as a row of output                                                                                                                                            
    #                                                                                                                                                                                                                                                   
    
    size = get_size(in_fn)
    
    fps = []
    new_fp = open(in_fn, 'r')
    new_fp.seek(0, 0)
    fps.append(new_fp)
    
    with open(in_fn, 'r') as f:
      byte = f.read(1)
      while byte:
        if byte == '\n':
          new_offset = f.tell()
          if new_offset < size:
            new_fp = open(in_fn, 'r')
            new_fp.seek(new_offset, 0)
            fps.append(new_fp)
          else:
            break
        byte = f.read(1)
    
    while size > 0:
      for fi, f in enumerate(fps):
        byte = f.read(1)
        size -= 1
        while byte:
          if byte == '\t' or byte == '\n':
            if fi != len(fps) - 1: sys.stdout.write('\t')
            break
          sys.stdout.write('%s' % (byte))
          byte = f.read(1)
          size -= 1
      sys.stdout.write('\n')
    
    for f in fps:
      f.close()

Is there anything I can do to improve the performance of this? Reading and processing a set of file pointers one byte at a time seems quite expensive. However, I need to find newline characters to build offsets and file pointers. Is there a cleverer/faster way (in Python) to find the byte offsets of newlines?