4
\$\begingroup\$

I request data from a table in a database and each line comes as a dictionary like this one :

{
    "timestamp" : 1234657890,
    "prices" : {
                   "AAA" : 111,
                   "BBB" : 222,
                    ...
                   "ZZZ" : 999
               }
}

From all those lines i wanted to create a dataframe like this:

Timestamp    AAA    BBB    ...   ZZZ
1234657890   111    222    ...   999
1234567891   110    223    ...   997
   ...
1324657899   123    208    ...  1024

So i did :

rawData = database_request()
listPrices = []
for row in rawData
    tmp = {'timestamp': row['timestamp']}
    tmp.update({name : price for name,price in row['prices'].items()})
    listPrices.append(tmp)
df = pd.DataFrame(listePrices)

So i was wondering if there were a more pythonic way to do this ?

\$\endgroup\$
3
  • 1
    \$\begingroup\$ Does it only have prices and timestamps in the dictionary? \$\endgroup\$ – Aryan Parekh Oct 14 '20 at 9:40
  • \$\begingroup\$ yes only those two fields \$\endgroup\$ – TmSmth Oct 14 '20 at 9:42
  • 1
    \$\begingroup\$ You need pd.json_normalize here. \$\endgroup\$ – GZ0 Oct 14 '20 at 12:05
3
\$\begingroup\$

Your rawData (which should be ideally named raw_data, python suggests a style guide to name variables and functions in lower_snake_case) is already in a list structure. You can manipulate this in place, without having to process the whole dataset manually.

for row in raw_data:
    row.update(row.pop("prices"))
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.