A program that reads 8763x2 values from a CSV file and initialises an array and few other variables

Question

Relatively new to Python. Have some experience in C++.

I have written a small program that reads a CSV file (SAMPLE) and initialises two arrays and few values. There are 2 values in each line, and there are 8763 such lines. The first value of each line is put into one array and the second value is put into the second array. The last three values of each array (total 6) are then given to another variable.

I want further calculations as accurate as possible hence I've tried the decimal.Decimal approach for Float Point Arithmetic, but for some reason, it's not that accurate. I think I'm using it wrong. It is still in an acceptable tolerance.

Also, would like to know if there is a better/efficient way to initialise the array and/or variables.

Here is code:

    alpha = np.zeros(8763, dtype='float32')  # Edited in later for clarity
    gamma = np.zeros(8763, dtype='float32')  # Edited in later for clarity
    rf = 100
    counter = 0
    with open('CSVData.csv', 'r') as csv_file_in:
        csv_reader = csv.reader(csv_file_in)
        for line in csv_reader:
            alpha[counter] = (float(decimal.Decimal(line[0])))  # Alpha Angle Initialized
            gamma[counter] = (float(decimal.Decimal(line[1])))  # Gamma Angle Initialized
            if counter == 8762:
                break
            counter = counter + 1
    csv_file_in.close()

    # Initializing last 6 parameters
    C_NSX = rf * alpha[8760]
    C_NSY = rf * gamma[8760]
    C_EWX = rf * alpha[8761]
    C_EWY = rf * gamma[8761]
    NSC = int(alpha[8762])
    EWC = int(gamma[8762])

Here the value of C_NSX should be 2790 but is 2789.9999185. (Debug Image)

EDIT: ADDED INFO ON THE PURPOSE OF THIS CODE

Following is my problem statement:

There is a matrix of octagons, for example, 4 octagons in a row and 3 such rows. So 4 columns and 3 rows of octagons. But they are not arranged in a perfect rectangle form.

Case1:

The first Octagon, O(0,0) has coordinates(0,0), while the one at its right and the one at the bottom are at a bit offset. So O(0,1) has coordinates (15,50) and O(1,0) has coordinates (30,15). This information is enough to define the whole array of octagons. Now next requirement is to find how much is the net visible area of these octagons. In the first example, as the coordinates are far away, there would be no overlapping and total visible area is simply...

Total_Visible_Area = Area_of_Single_Octagon * No_Of_Rows * No_Of_Columns

But when the coordinates are a bit complex, for example, Case 2: Here, the distance between the adjacent octagons is less than their width and hence they are overlapping. Coordinates also are negative (which does not matter much actually, just something to note). Now to determine the visible area for this I wrote the following function.

The piece of code that I have written above is the first step to calculate all these. Each case is defined using Alpha, Gamma, C_NSX, C_NSY, C_EWX, C_EWY, NSC and EWC. Among these Alpha and Gamma have 8760 different values (mostly zeroes) and all others (capitals) are constants. I have to find the overlapping area in each of the 8760 cases. The code for finding those I had discussed here earlier, and thought won't be necessary to bring out again. After that last question, I have moved to python from CPP and started learning OpenCV.

Answer 1

You don't really mention what are the values used for. You'll always have a tradeoff between speed and memory (in this case memory amounts to how many digits you have).

Some issues with your code:

1) You're casting your decimal.Decimal(line[0]) as float, that should negate the advantage of using Decimal. And this might be a reason why you see the unexpected inconsistency. You have two options (in my opinion), you drop the use of numpy in favor of array from the standard library or you drop the pretense of using Decimal and use a pure numpy solution. But this will depend on what exactly you're doing.

2) Looping over the CSV Reader is a way to read out the data, but numpy implements better ways to load a csv. See the last example in the documentation here.

Thus, in the end the question back is, what for do you want the numbers to be a accurate as possible? Keep in mind that the values in the file are already "wrong" as they most likely come already from a float type, so in my opinion, you don't gain anything by using Decimal in this case. A different scenario would be if the values are integers and then in your program you're doing some divisions or multiplications, in that case, you have from the beginning control of the floating-point values and then using Decimal might be what you want to do.

Keep in mind, that highly sophisticated scientific research and their calculations use 64-bit floats and they do just fine.