2
\$\begingroup\$

Relatively new to Python. Have some experience in C++.

I have written a small program that reads a CSV file (SAMPLE) and initialises two arrays and few values. There are 2 values in each line, and there are 8763 such lines. The first value of each line is put into one array and the second value is put into the second array. The last three values of each array (total 6) are then given to another variable.

I want further calculations as accurate as possible hence I've tried the decimal.Decimal approach for Float Point Arithmetic, but for some reason, it's not that accurate. I think I'm using it wrong. It is still in an acceptable tolerance.

Also, would like to know if there is a better/efficient way to initialise the array and/or variables.

Here is code:

    alpha = np.zeros(8763, dtype='float32')  # Edited in later for clarity
    gamma = np.zeros(8763, dtype='float32')  # Edited in later for clarity
    rf = 100
    counter = 0
    with open('CSVData.csv', 'r') as csv_file_in:
        csv_reader = csv.reader(csv_file_in)
        for line in csv_reader:
            alpha[counter] = (float(decimal.Decimal(line[0])))  # Alpha Angle Initialized
            gamma[counter] = (float(decimal.Decimal(line[1])))  # Gamma Angle Initialized
            if counter == 8762:
                break
            counter = counter + 1
    csv_file_in.close()

    # Initializing last 6 parameters
    C_NSX = rf * alpha[8760]
    C_NSY = rf * gamma[8760]
    C_EWX = rf * alpha[8761]
    C_EWY = rf * gamma[8761]
    NSC = int(alpha[8762])
    EWC = int(gamma[8762])

Here the value of C_NSX should be 2790 but is 2789.9999185. (Debug Image)

EDIT: ADDED INFO ON THE PURPOSE OF THIS CODE

Following is my problem statement:

There is a matrix of octagons, for example, 4 octagons in a row and 3 such rows. So 4 columns and 3 rows of octagons. But they are not arranged in a perfect rectangle form.

Case1:

Matrix of Octagons

The first Octagon, O(0,0) has coordinates(0,0), while the one at its right and the one at the bottom are at a bit offset. So O(0,1) has coordinates (15,50) and O(1,0) has coordinates (30,15). This information is enough to define the whole array of octagons. Now next requirement is to find how much is the net visible area of these octagons. In the first example, as the coordinates are far away, there would be no overlapping and total visible area is simply...

Total_Visible_Area = Area_of_Single_Octagon * No_Of_Rows * No_Of_Columns

But when the coordinates are a bit complex, for example, Case 2: enter image description here Here, the distance between the adjacent octagons is less than their width and hence they are overlapping. Coordinates also are negative (which does not matter much actually, just something to note). Now to determine the visible area for this I wrote the following function.

The piece of code that I have written above is the first step to calculate all these. Each case is defined using Alpha, Gamma, C_NSX, C_NSY, C_EWX, C_EWY, NSC and EWC. Among these Alpha and Gamma have 8760 different values (mostly zeroes) and all others (capitals) are constants. I have to find the overlapping area in each of the 8760 cases. The code for finding those I had discussed here earlier, and thought won't be necessary to bring out again. After that last question, I have moved to python from CPP and started learning OpenCV.

\$\endgroup\$
3
  • \$\begingroup\$ @JanneKarila Both are dtype='float32' \$\endgroup\$ Jan 13, 2020 at 7:41
  • \$\begingroup\$ @AdityaSoni, On one hand you said that "decimal.Decimal approach but for some reason, it's not that accurate", on the other hand - you concerned with should be 2790 but is 2789.9999185 . Contradiction? \$\endgroup\$ Jan 13, 2020 at 8:04
  • \$\begingroup\$ @RomanPerekhrest I saw that Float Point issues are solved with the decimal approach, hence I tried it. But it is not working. I may be doing something wrong. It is still not accurate. It should be 2790 but is 2789.9999. I may have sentenced something wrong, but I'm not seeing any contradiction. In the end, I did say that this is still an acceptable tolerance but I posted this question as I want to make it better. \$\endgroup\$ Jan 13, 2020 at 8:14

1 Answer 1

3
\$\begingroup\$

You don't really mention what are the values used for. You'll always have a tradeoff between speed and memory (in this case memory amounts to how many digits you have).

Some issues with your code:

1) You're casting your decimal.Decimal(line[0]) as float, that should negate the advantage of using Decimal. And this might be a reason why you see the unexpected inconsistency. You have two options (in my opinion), you drop the use of numpy in favor of array from the standard library or you drop the pretense of using Decimal and use a pure numpy solution. But this will depend on what exactly you're doing.

2) Looping over the CSV Reader is a way to read out the data, but numpy implements better ways to load a csv. See the last example in the documentation here.

Thus, in the end the question back is, what for do you want the numbers to be a accurate as possible? Keep in mind that the values in the file are already "wrong" as they most likely come already from a float type, so in my opinion, you don't gain anything by using Decimal in this case. A different scenario would be if the values are integers and then in your program you're doing some divisions or multiplications, in that case, you have from the beginning control of the floating-point values and then using Decimal might be what you want to do.

Keep in mind, that highly sophisticated scientific research and their calculations use 64-bit floats and they do just fine.

\$\endgroup\$
4
  • \$\begingroup\$ How do I show what I'm doing with these numbers, comment here or edit the question? \$\endgroup\$ Jan 15, 2020 at 13:06
  • \$\begingroup\$ @AdityaSoni make an edit to the question, but for what I wrote is not really relevant as I described both cases already. It still would be a good idea for completeness of your question to add that info. \$\endgroup\$ Jan 15, 2020 at 13:15
  • \$\begingroup\$ added some info, hope I have not made it more complicated. \$\endgroup\$ Jan 16, 2020 at 5:26
  • \$\begingroup\$ the first 8760 values are angles and won't make much difference. But the last 3x2 values are constants and need to be accurate. And I'm having trouble only in that. I guess rounding that up into an int would be my best solution right now. The NumPy method of reading the CSV looks great, I'm going to give that a try and start using it. Also, I'm thinking of directly reading the excel sheet from where all these numbers are coming. Thanks for your inputs. You've been a great help. \$\endgroup\$ Jan 16, 2020 at 5:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.