3
\$\begingroup\$

I am trying to access data from Coinmarketcap API in Python and came up with the following code:

def fetch_coin_prices(**kwargs):
    """Retrieve cryptocurrency data from CoinMarketCap and return a dictionary
    containing coin names with their current prices.

    Keyword arguments to this function are mapped to the CoinMarketCap API,
    refer to their documentation for their meaning:
    https://coinmarketcap.com/api/
    """
    response = requests.get(
        'https://api.coinmarketcap.com/v2/ticker/',
        params=kwargs
    )
    response.raise_for_status()
    coin_data = response.json()

    return coin_data.get('data', {}).values()


data = fetch_coin_prices(limit=100, start=0, sort='id')

I am trying to write this data to a Redis HMSET that accepts a dictionary and I only want the properties 'id', 'name', 'symbol', 'rank', 'price' and 'volume_24h'. The price and volume properties are nested and therefore I haven't found a way to get everything in 1 go. My redis hmset needs to store data in key value form where key is the coin id and value is the CSV of the current coin.

hmset = {}
fieldnames = ['id', 'name', 'symbol', 'rank' ,'price', 'volume_24h']
for row in data:
    subset = {}
    subset['id'] = row.get('id', None)
    subset['name'] = row.get('name', None)
    subset['symbol'] = row.get('symbol', None)
    subset['rank'] = row.get('rank', None)
    subset['price'] = row.get('quotes', {}).get('USD', {}).get('price', None)
    subset['volume_24h'] = row.get('quotes', {}).get('USD', {}).get('volume_24h', None)

    if all([subset[key] for key in subset]):
        csv_string = io.StringIO()
        csv_writer = csv.DictWriter(csv_string, fieldnames=fieldnames, extrasaction='ignore' , lineterminator=':')
        csv_writer.writerows(data_subset)
        hmset[subset['id']] = csv_string.getvalue()

This is what I have come up with so far but it looks very ugly in my opinion to access keys. Is there a better way to do this?

\$\endgroup\$
  • 2
    \$\begingroup\$ what do you mean exactly with The price and volume properties are nested. please share some example data, and what you expect \$\endgroup\$ – Maarten Fabré Jun 26 '18 at 7:40
  • \$\begingroup\$ @MaartenFabré thank you for reviewing this question, at the very top I have added a link to the coinmarketcap API containing real data \$\endgroup\$ – PirateApp Jun 26 '18 at 9:08
2
\$\begingroup\$

The way using dict.get certainly works, but can become a bit unreadable if you need to chain them.

An alternative is to use exception handling in this case.

subset['volume_24h'] = row.get('quotes', {}).get('USD', {}).get('volume_24h', None)

is equivalent to

try:
    subset['volume_24h'] = row['quotes']['USD']['volume_24h']
except KeyError:
    subset['volume_24h'] = None

(This would be a bit neater if PEP-463 had not been rejected:)

subset['volume_24h'] = row['quotes']['USD']['volume_24h'] except KeyError: None
\$\endgroup\$
  • \$\begingroup\$ upvoted! thank you for the analysis, one curious question is how do you manage keynames like quotes USD price etc in your code, do you hardcode them, do you have them as environment variables so that you can modify them, some other technique perhaps? \$\endgroup\$ – PirateApp Jun 26 '18 at 11:14
  • 1
    \$\begingroup\$ @PirateApp: That depends. If it is one-off code just hard-code them. On the other hand if the possibility exists that you might have some other data source with different structure, you might need to outsource it to a class that gets the data and translates it into some canonical format. \$\endgroup\$ – Graipher Jun 26 '18 at 11:15
  • 2
    \$\begingroup\$ Damn, PEP-463 would have made so much Python code sick. There'd also be no need for get with it too... :O \$\endgroup\$ – Peilonrayz Jun 26 '18 at 11:25
4
\$\begingroup\$

I previously have written a helper for this, as I've come across it too. If you pass the object and the keys to the object then you can do the same.

def get_nested(obj, keys)
    try:
        for key in keys:
            obj = obj[key]
    except KeyError:
        return None
    return obj

This makes usage:

subset['price'] = get_nested(row, 'quotes USD price'.split())

This allows you to extend the keys argument to a minilanguage for other common methods you'd want to apply to returned objects. One case I had was since JavaScript doesn't like maps much (not objects) some APIs tend to convert dictionaries to arrays, and so I had to search a list of objects for which object I wanted, as I couldn't index the list as I'd get incorrect data at times.

\$\endgroup\$
  • \$\begingroup\$ upvoted! thank you for sharing code! i searched for a few questions right here on codereview on flattening nested json and i saw there are a lot of intricacies involved like handling empty keys and any hashable type as opposed to mere string keys, this approach is something i never imagined :) \$\endgroup\$ – PirateApp Jun 26 '18 at 11:40

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.