Accessing a subset of keys from a nested dictionary in Python

Question

I am trying to access data from Coinmarketcap API in Python and came up with the following code:

def fetch_coin_prices(**kwargs):
    """Retrieve cryptocurrency data from CoinMarketCap and return a dictionary
    containing coin names with their current prices.

    Keyword arguments to this function are mapped to the CoinMarketCap API,
    refer to their documentation for their meaning:
    https://coinmarketcap.com/api/
    """
    response = requests.get(
        'https://api.coinmarketcap.com/v2/ticker/',
        params=kwargs
    )
    response.raise_for_status()
    coin_data = response.json()

    return coin_data.get('data', {}).values()


data = fetch_coin_prices(limit=100, start=0, sort='id')

I am trying to write this data to a Redis HMSET that accepts a dictionary and I only want the properties 'id', 'name', 'symbol', 'rank', 'price' and 'volume_24h'. The price and volume properties are nested and therefore I haven't found a way to get everything in 1 go. My redis hmset needs to store data in key value form where key is the coin id and value is the CSV of the current coin.

hmset = {}
fieldnames = ['id', 'name', 'symbol', 'rank' ,'price', 'volume_24h']
for row in data:
    subset = {}
    subset['id'] = row.get('id', None)
    subset['name'] = row.get('name', None)
    subset['symbol'] = row.get('symbol', None)
    subset['rank'] = row.get('rank', None)
    subset['price'] = row.get('quotes', {}).get('USD', {}).get('price', None)
    subset['volume_24h'] = row.get('quotes', {}).get('USD', {}).get('volume_24h', None)

    if all([subset[key] for key in subset]):
        csv_string = io.StringIO()
        csv_writer = csv.DictWriter(csv_string, fieldnames=fieldnames, extrasaction='ignore' , lineterminator=':')
        csv_writer.writerows(data_subset)
        hmset[subset['id']] = csv_string.getvalue()

This is what I have come up with so far but it looks very ugly in my opinion to access keys. Is there a better way to do this?

Answer 1

The way using dict.get certainly works, but can become a bit unreadable if you need to chain them.

An alternative is to use exception handling in this case.

subset['volume_24h'] = row.get('quotes', {}).get('USD', {}).get('volume_24h', None)

is equivalent to

try:
    subset['volume_24h'] = row['quotes']['USD']['volume_24h']
except KeyError:
    subset['volume_24h'] = None

(This would be a bit neater if PEP-463 had not been rejected:)

subset['volume_24h'] = row['quotes']['USD']['volume_24h'] except KeyError: None

Answer 2

I previously have written a helper for this, as I've come across it too. If you pass the object and the keys to the object then you can do the same.

def get_nested(obj, keys)
    try:
        for key in keys:
            obj = obj[key]
    except KeyError:
        return None
    return obj

This makes usage:

subset['price'] = get_nested(row, 'quotes USD price'.split())

This allows you to extend the keys argument to a minilanguage for other common methods you'd want to apply to returned objects. One case I had was since JavaScript doesn't like maps much (not objects) some APIs tend to convert dictionaries to arrays, and so I had to search a list of objects for which object I wanted, as I couldn't index the list as I'd get incorrect data at times.