3
\$\begingroup\$

I am doing a Pandas project on alcohol consumption. For your information, the dataset has the following columns:

| Continent | Country | Beer | Spirit | Wine |

The following is my code:

# Separating data by continent
# ----------------------------
data_asia   = data[data['Continent'] == 'Asia']
data_africa = data[data['Continent'] == 'Africa']
data_europe = data[data['Continent'] == 'Europe']
data_north  = data[data['Continent'] == 'North America']
data_south  = data[data['Continent'] == 'South America']
data_ocean  = data[data['Continent'] == 'Oceania']

top_5_asia_beer = data_asia.nlargest(5, ['Beer Servings'])[['Country', 'Beer Servings']]
top_5_asia_spir = data_asia.nlargest(5, ['Spirit Servings'])[['Country', 'Spirit Servings']]
top_5_asia_wine = data_asia.nlargest(5, ['Wine Servings'])[['Country', 'Wine Servings']]
top_5_asia_pure = data_asia.nlargest(5, ['Total Litres of Pure Alcohol'])[['Country', 'Total Litres of Pure Alcohol']]

top_5_africa_beer = data_africa.nlargest(5, ['Beer Servings'])[['Country', 'Beer Servings']]
top_5_africa_spir = data_africa.nlargest(5, ['Spirit Servings'])[['Country', 'Spirit Servings']]
top_5_africa_wine = data_africa.nlargest(5, ['Wine Servings'])[['Country', 'Wine Servings']]
top_5_africa_pure = data_africa.nlargest(5, ['Total Litres of Pure Alcohol'])[['Country', 'Total Litres of Pure Alcohol']]

top_5_europe_beer = data_europe.nlargest(5, ['Beer Servings'])[['Country', 'Beer Servings']]
top_5_europe_spir = data_europe.nlargest(5, ['Spirit Servings'])[['Country', 'Spirit Servings']]
top_5_europe_wine = data_europe.nlargest(5, ['Wine Servings'])[['Country', 'Wine Servings']]
top_5_europe_pure = data_europe.nlargest(5, ['Total Litres of Pure Alcohol'])[['Country', 'Total Litres of Pure Alcohol']]

top_5_north_beer = data_north.nlargest(5, ['Beer Servings'])[['Country', 'Beer Servings']]
top_5_north_spir = data_north.nlargest(5, ['Spirit Servings'])[['Country', 'Spirit Servings']]
top_5_north_wine = data_north.nlargest(5, ['Wine Servings'])[['Country', 'Wine Servings']]
top_5_north_pure = data_north.nlargest(5, ['Total Litres of Pure Alcohol'])[['Country', 'Total Litres of Pure Alcohol']]

top_5_south_beer = data_south.nlargest(5, ['Beer Servings'])[['Country', 'Beer Servings']]
top_5_south_spir = data_south.nlargest(5, ['Spirit Servings'])[['Country', 'Spirit Servings']]
top_5_south_wine = data_south.nlargest(5, ['Wine Servings'])[['Country', 'Wine Servings']]
top_5_south_pure = data_south.nlargest(5, ['Total Litres of Pure Alcohol'])[['Country', 'Total Litres of Pure Alcohol']]

top_5_ocean_beer = data_ocean.nlargest(5, ['Beer Servings'])[['Country', 'Beer Servings']]
top_5_ocean_spir = data_ocean.nlargest(5, ['Spirit Servings'])[['Country', 'Spirit Servings']]
top_5_ocean_wine = data_ocean.nlargest(5, ['Wine Servings'])[['Country', 'Wine Servings']]
top_5_ocean_pure = data_ocean.nlargest(5, ['Total Litres of Pure Alcohol'])[['Country', 'Total Litres of Pure Alcohol']]

I understand the ridiculousness of my code in terms of duplicity and repetitiveness. Can anyone please share tips and tricks to refactor the code?

\$\endgroup\$
1
  • 2
    \$\begingroup\$ Welcome to Code Review! The current question title, which states your concerns about the code, is too general to be useful here. Please edit to the site standard, which is for the title to simply state the task accomplished by the code. Please see How to get the best value out of Code Review: Asking Questions for guidance on writing good question titles. \$\endgroup\$ Feb 25 '20 at 11:28
1
\$\begingroup\$

Depends on what you want to do with it. It seems a bit odd to store each top 5 in its own variable.

For starters, you can slice a DataFrame by continent using .groupby:

for continent, continent_data in data.groupby("Continent"):
    # `continent` is now the name of the continent (you don't have to type the continent names manually)
    # `continent_data` is a dataframe, being a subset of the `data` dataframe

Edit based on first comment: if you want to plot the variables, it's definitely not a good idea to store them each in a separate variable. Do you already know how you want to visualize your data? That's something you will need to work toward. I can't really see a top 5 countries for each type of alcoholic beverage for each continent in one plot.

continents = []
top5s = {}
for continent, continent_data in data.groupby("Continent"):
    continents.append(continent)
    for beverage_column in ["Beer Servings", "Spirit Servings", "Wine Servings"]:
        topcountries = continent_data.nlargest(5, beverage_column)
        # do something with the data, such as:
        print(f"Top 5 countries in {continent} for {beverage}:")
        for row in topcountries.iterrows():
            print(f"- {row.Country}: {row['beverage_column']} servings")

To be very exact: groupby() doesn't return an iterable of tuples, but actually just a GroupBy object that implements iterability (i.e. the __iter__() method).

\$\endgroup\$
3
  • \$\begingroup\$ Oh, I now realised that the groupby() function returns a tuple of categories and dataframe subsets. Thanks. Sorry for not informing more, I actually intend to later make a graph for the top 5 of each drink category for each continent. Any advice for this? \$\endgroup\$ Feb 25 '20 at 9:36
  • \$\begingroup\$ See my edits :) \$\endgroup\$
    – Lewistrick
    Feb 26 '20 at 11:20
  • \$\begingroup\$ Yeah, I found out about GroupBy object in the documentation. Silly me for mistaking tuples. Thanks. \$\endgroup\$ Feb 26 '20 at 18:46
0
\$\begingroup\$

sample_data

np.random.seed(42)

drinks = ["Beer", "Spirit", "Wine"]
continents = [
    "Asia",
    "Africa",
    "Europe",
    "North America",
    "South America",
    "Oceania",
]
countries = [f"country_{i}" for i in range(10)]
index = pd.MultiIndex.from_product(
    (continents, countries), names=["continent", "country"]
)
data = np.random.randint(1_000_000, size=(len(index), len(drinks )))

df = pd.DataFrame(index=index, columns=columns, data=data).reset_index()

data structures

The most jarring about this, is that each datapoint has it's own variable.

A first step would be to use dictionaries:

data_by_continent = {
    continent: df.loc[df["continent"] == continent]
    for continent in continents
}

Note that I used .loc to explicitly make a copy instead of a view to prevent changes in one part of the code contaminating another.

Then the spirit consumption per continent is:

spirit_per_continent = {
    continent: data.loc[
        data["Spirit"].nlargest(5).index, ["country", "Spirit"]
    ]
    for continent, data in data_by_continent.items()
}

and nested per beverage

consumption_per_drink_continent = {
    drink: {
        continent: data.loc[
            data[drink].nlargest(5).index, ["country", drink]
        ]
        for continent, data in data_by_continent.items()
    }
    for drink in drinks
}

pandas groupby

If you reform your dataframe into a tidy format, you can use an easy groupby.

pandas.melt is a very handy method to shape a dataframe

df2 = pd.melt(
    df,
    id_vars=["continent", "country"],
    var_name="drink",
    value_name="consumption",
)
  continent   country     drink   consumption
....
175   Oceania     country_5   Wine    456551
176   Oceania     country_6   Wine    894498
177   Oceania     country_7   Wine    899684
178   Oceania     country_8   Wine    158338
179   Oceania     country_9   Wine    623094

groupby

now you can use groupby, and then later join on the index of df2 to introduce the country

(
    df2.groupby(["continent", "drink"])["consumption"]
    .nlargest(5)
    .reset_index(["continent", "drink"])
    .sort_values(
        ["continent", "drink", "consumption"], ascending=[True, True, False]
    )
    .join(df2["country"])
)
      continent   drink   consumption     country
17    Africa  Beer    953277  country_7
19    Africa  Beer    902648  country_9
15    Africa  Beer    527035  country_5
13    Africa  Beer    500186  country_3
14    Africa  Beer    384681  country_4
...   ...     ...     ...     ...
162   South America   Wine    837646  country_2
160   South America   Wine    742139  country_0
167   South America   Wine    688519  country_7
161   South America   Wine    516588  country_1
166   South America   Wine    136330  country_6

90 rows × 4 columns
\$\endgroup\$
1
  • \$\begingroup\$ The melt() function seems useful. Thanks. \$\endgroup\$ Feb 26 '20 at 18:48

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.