2
\$\begingroup\$

My code is functioning the way I'm expecting, with the if. However, it is not pythonic to me, and that bugs me.

The multiple if in the code below are supposed to update a property_id field on a database when reading the csv files, properties.csv. if property_id in the file is equal to '2502', it should update the table tenants of the database with the property_id '95560' instead of '2502' and etc (see code below).

# -*- coding: utf-8 -*-

import csv
import io
import os
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from io import StringIO

import boto3
import psycopg2
from botocore.exceptions import ClientError

import replace as rep

LOW = 30.0
NEUTRAL = 70.0

CAT = ['OTHERS', 'HSS', 'UTILS', 'TBD', 'IT']

tables_order = ['properties', 'tenants', 'leases', 'managers', 'tenant_area',
                'budget', 'tenant_property_occupied_area', 'tenant_client_occupied_area', 'schedule']

tables_fields = {
    'leases': ['property_id', 'lease_id', 'break_option', 'charges_amount', 'currency', 'debtor', 'end_date', 'erv_pa', 'floor_area', 'floor_index', 'floor_name', 'lease_status', 'occupational_status', 'rent_amount', 'start_date', 'tenant_id', 'tenant_name'],
    'tenants': ['tenant_id', 'tenant_name', 'address', 'total_area', 'debtor', 'currency', 'mobile_phone', 'direct_phone', 'phone', 'contact_name', 'mail', 'property_id'],
    'properties': ['property_id', 'comment', 'is_listed', 'property_name', 'address', 'total_area', 'debtor', 'rent_amount', 'charges_amount', 'currency', 'client_name', 'year_constructed', 'budget_used', 'budget_used_date', 'annual_budget', 'year_last_refurbished', 'latitude', 'longitude'],
    'managers': ['property_id', 'name', 'role', 'mail', 'phone', 'direct_phone', 'mobile_phone'],
    'tenant_area': ['property_id', 'tenant_id', 'area', 'rent_amount', 'rent_exposure', 'area_exposure'],
    'budget': ['property_id', 'category', 'budget'],
    'tenant_property_occupied_area': ['tenant_id', 'property_id', 'area'],
    'tenant_client_occupied_area': ['tenant_id', 'client_name', 'area'],
    'schedule': ['Property Ref', 'client_parent_ref', 'Client Ref', 'Client Name', 'Property Name', 'Property Town', 'vat_opted', 'Unit ref', 'Unit Name', 'Tenant Ref', 'Tenant', 'property_type_code', 'Property Manager', 'property_status_code', 'lease ref', 'Lease status', 'ref', 'passing_rent', 'annual_s_c', 'Lease years', 'Lease months', 'Lease days', 'Monthly Rent', 'term_start_date', 'term_end_date', 'next_review_date', 'review_basis', 'outside_1954_act', 'holding_over', 'expiry_date', 'managed_by', 'valuation_date', 'valuation_amount', 'floor_area', 'prime_record', 'measured_in', 'comments', 'credit_controller', 'valuation_effective_date', 'client_type', 'CF team', 'FM team', 'PM team', 'Health & Safety', 'SC auditors', 'Utility recovery', 'Debtor ref- current tenancy', 'Debtor name', 'tenancy.start_date', 'occupation_start_date', 'Review frequency', 'breach status', 'Erv_Psf']
}


temp2nd_lst = ['95560','2502','21489','660133','921010','580000','921600','673301','3953','3751','2514','4900']

class Service:

    def __init__(self, dao):
        self.dao = dao

    def process(self):
        print('Starting')

        exceptions = io.StringIO()
        is_there_exceptions = False

        properties_area_and_rent = {}

        try:

            properties_ids = []

            for table_name in tables_order:
                print(table_name)
                with open('/home/andykw/cloned_projects/dadoo/dadoo-ingest-repmdata/mockdata_fr/' + table_name + '.csv', 'r') as file:
                    # it opens csv files in the directory
                    s = file
                    if os.environ['ORIGIN'] == 'ie':
                        delimiter = ';'
                    else:
                        delimiter = ','
                    reader = csv.DictReader(s, delimiter=delimiter)
                    for row in reader:
                        if table_name == 'properties':
                            if row['property_id'] not in properties_ids:
                                print('toto')
                            continue
                        elif table_name == 'tenants': 
                        # The condition above
                        # check if the name of the table is tenants
                        # if it is the case, the if are doing their work
                            print('tato')
                            if row.get('property_id') == '2502':
                                 row.update({'property_id': '95560'})
                            elif row.get('property_id') == '21489':
                                row.update({'property_id': '2502'})
                            elif row.get('property_id') == '660133':
                                row.update({'property_id': '21489'})
                            elif row.get('property_id') == '921010':
                                row.update({'property_id': '660133'})
                            elif row.get('property_id') == '580000':
                                row.update({'property_id': '921010'})
                            elif row.get('property_id') == '921600':
                                row.update({'property_id': '580000'})
                            elif row.get('property_id') == '673301':
                                row.update({'property_id': '921600'})
                            elif row.get('property_id') == '3953':
                                row.update({'property_id': '673301'})
                            elif row.get('property_id') == '3751':
                                row.update({'property_id': '3953'})
                            elif row.get('property_id') == '2514':
                                row.update({'property_id': '3751'})
                            elif row.get('property_id') == '4900':
                                row.update({'property_id': '2514'})
                            elif row.get('property_id') == '21065':
                                row.update({'property_id': '4900'})
                        self.dao.insert(table_name,
                                        clean(row), tables_fields[table_name])


            d = self.dao.get_most_recent_date_modified(tables_order)
            self.dao.insert('settings',
                            {'date': d, 'name': 'data_date'}, ['date', 'name'])
        except Exception as e:
            print(e)

        if is_there_exceptions:
            print('toto')

        print('Finished')

As I said, the ifis doing what it should do e.g, catching the property_id like 2502 and row updating it with the property_id that I want. There are only a dozen of examples in this case.

However as I said above, I'm looking for something more pythonic and using the list temp2nd_lst.

I tried something like the code below

properties_ids = []
temp_1st_lst = ['2502','21489','660133','921010','580000','921600','673301','3953','3751','2514','4900','21065']

temp_2nd_lst = ['95560','2502','21489','660133','921010','580000','921600','673301','3953','3751','2514','4900']


i_temp = 1
if table_name == 'tenants':
    for prop_id in temp_1st_lst:
        row.update({'property_id': temp_2nd_lst[i_temp]})
        i_temp = += 1

Issue is it is looping 12 times, and updating my records with the last property_id of the list, temp_2nd_list, 4900, instead of shuffling the property id like with the if.

Any ideas are more than welcomed as I'm trying to figure out how to do in a more pythonic way than using the if.

\$\endgroup\$
  • \$\begingroup\$ Is your code working as intended? see our help center for more info about how to ask a good question: codereview.stackexchange.com/help/dont-ask \$\endgroup\$ – dfhwze Aug 17 '19 at 17:05
  • 1
    \$\begingroup\$ Hi @dfhwze, my code is working as intended. \$\endgroup\$ – Andy K Aug 17 '19 at 17:10
  • 1
    \$\begingroup\$ I'm not sure why this is being closed for the reasons it is. I can't see anywhere where they indicate that the code isn't working as expected. The very last part isn't working, but I don't think that invalidates it as it isn't the main code up for review. \$\endgroup\$ – Carcigenicate Aug 17 '19 at 17:41
  • \$\begingroup\$ @Carcigenicate and last but not least, I said explicitly that the last part, was not working and I was not sure what to do. :) \$\endgroup\$ – Andy K Aug 17 '19 at 17:50
4
\$\begingroup\$

Create a dictionary mapping the old property IDs to the new ones, then do a lookup:

# I'm only showing few pairs of transformations for brevity.
# You'll need to expand this to include the rest
OLD_TO_NEW_IDS = {'2502': '95560',
                  '21489': '2502',
                  '660133': '21489',
                  '921010': '660133'}  

# Get the old ID
old_id = row.get('property_id')

# Do a lookup and get the new ID
new_id = OLD_TO_NEW_IDS.get(old_id, None)  # Will return None if the lookup failed to find the old ID

if new_id:  # Only update if the new ID is not None
    row.update({'property_id': new_id})

Having a long dictionary like that in your code is unfortunate though. You may want to consider storing it as a JSON or similar format to file, and loading/parsing it as needed.

| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ I'd also make a log entry when an id gets mapped to a new id. Mr. Murphy is known to make typos when updating mappings. \$\endgroup\$ – RootTwo Aug 18 '19 at 1:43
  • \$\begingroup\$ Hi @Carcigenicate, you rock! \$\endgroup\$ – Andy K Aug 18 '19 at 13:31
  • \$\begingroup\$ So simple, yet so powerful. I've never seen the lookup as such before. Ty. \$\endgroup\$ – Andy K Aug 18 '19 at 13:32
  • \$\begingroup\$ @RootTwo I'm not sure what you're referring to. Who's "Mr Murphy"? \$\endgroup\$ – Carcigenicate Aug 18 '19 at 14:21
  • 1
    \$\begingroup\$ @AndyK Ya, dictionaries are incredibly useful. Whenever you have two associated pieces of data and need to find the pair of data based one one of the pieces, you should probably be using a dictionary. That scenario is far more common than it sounds. \$\endgroup\$ – Carcigenicate Aug 18 '19 at 14:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.