Display the difference between DataFrames' dtypes?

Question

I recently updated Pandas and noticed that it is no longer possible to merge DataFrames with mismatched dtypes. I have a script which relied on merging and seemed to work in the past despite having mismatched dtypes. I need to display to the user which columns in two dataframes are causing problems, so the user can then adjust the types accordingly. (Specifically, one dataframe is read in from a database and represents what is currently in the DB, while the second dataframe includes any changes/new data to be applied to the database. Once the the user finds the problem columns they can determine if the DataFrame that contains the changes was meant to change the database's type or whether there is an error in the changes). The following code appears to work, but I feel like pandas must have a built in better way to deal with this problem.

def get_mismatched_dtypes(self, df1, df2):
    mismatch = {}
    for key, val in df1.dtypes.iteritems():
        if key in df2.dtypes and df2.dtypes[key] != val:
            mismatch[key] = (f"df1:{val}, df2: {df2.dtypes[key]}")
    print(mismatch)
    return mismatch

Answer 1

self

Why put this method on a class? the lack of use of self in the method should act as a flag

string result

you format the mismatch as a string (f"df1:{val}, df2: {df2.dtypes[key]}"). This way you can not do anything about it any more. Better would be to use a tuple or dict here

generator

generally, when I see

def method():
    result = {}
    for key in ...:
        result[key] = ...
    return key

it is more clear to work with a generator

def method():
    for key in ...:
        yield key, ...

If the user of the method needs a dict, he can do dict(method()). If he wants to iterate over the items, there is no need to instantiate the dict

alternative approach

you can use sets to see which types are different:

def get_mismatched_dtypes(df1, df2):
    df1_types = set(df1.dtypes.items())
    df2_types = set(df2.dtypes.items())
    for column_name, df1_type in df1_types - df2_types:
        yield column_name, (df1_type, df2.dtypes[column_name])



df1 = pd.DataFrame({"col1": [1, 2, 3], "col2": [4, 5, 6]}, index=list(range(3)))
df2 = pd.DataFrame({"col1": list("abc"), "col2": [4, 5, 6]}, index=list(range(3,6)))
dict(get_mismatched_dtypes(df1, df2))

{'col1': (dtype('int64'), dtype('O'))}

for clarification: df1_types - df2_types is {('col1', dtype('int64'))}

If you want it formatted like in your code:

{
    column_name: f"df1: {df1_type}, df2: {df2_type}"
    for column_name, (df1_type, df2_type) in get_mismatched_dtypes(df1, df2)
}

{'col1': 'df1: int64, df2: object'}