Using Python/Cython, I am trying to construct an efficient way to index boolean values. I have been trying to use the bitarray and/or bitstring packages to construct a fast, non memory intensive way to hold a lot of booleans. My goal was for a list like object that could be sliced as such, and actually managed the bool values in real bits of data:
>>> from bitarray import bitarray
>>> array = bitarray(1000000) # creates 1000000 allocated bits
>>> array.setall(False)
>>> array[4] # holds boolean values
False
>>> from timeit import timeit
>>> timeit(setup = 'from __main__ import array',
stmt = 'array[500000]')
0.21002184844218164
# one million iterations, making each iteration 210 ns
>>> from bitstring import BitArray
>>> array = BitArray(1000000) # just as we did in the first example
>>> array[6] # all are False by default
False
>>> from timeit import timeit
>>> timeit(setup = 'from __main__ import array',
stmt = 'array[500000]')
2.6748261753875795
# oh no, this one is very slow, each iteration takes 2674 ns
As you can see, there are already packages that construct this binary data and allow you to index it at random. Would it be possible to get to an even lower level to the RAM myself? 210ns is fast, but I would like to get indexing below 40ns on a 1000000 bit array. How can this be done?
Edit:
Individual item assignment to the boolean array after it is created, would be nice, but it is not required.