New answers tagged pandas
1
One of my friends on Discord helped me out. He didn't like me attempt either! =oP
I created this function:
import datetime
def str2date(string):
try:
datetime_holder = datetime.datetime.strptime(string, "%Y-%m-%d")
return datetime_holder.date()
except ValueError:
return string
except TypeError:
return string
...
0
You need to merge dataframes like done here in Pandas Merge. You also need to read on dataframe joins which is a common topic in learning databases.Here I have not done any tweaking with indices i.e, whether to keep left ones or right ones but you can check the docs for better information on here Merge Docs Pydata
import pandas as pd
columns = "a b c d ...
2
Yes. Try to automate converting a URL to a name, instead of hardcoding the mapping. With only two URLs, your approach is doable, but as soon as you have to handle lots of different URLs, it will quickly become very painful.
Also, avoid first copying a column and then replacing every item in it. Rather, construct the column with the right contents directly. ...
2
I think you can do two things. First, you should be able to directly use the output of applying the outer function. No need for this output_list business. Next thing, you should vectorize your inner function. You actually don't need it at all, you can just use numpy.searchsorted to find how many rows you need.
import numpy as np
import pandas as pd
def ...
4
I agree with the accepted answer. An alternative way to frame this is a multi-index, with indices of id and variable.
You will then effectively have three-dimensional data, where the first dimension is an integral ID, the second dimension is a categorical variable name, and the third dimension is your value.
You could extend this concept even further, with ...
6
iterrows
Of all the ways to iterate over a pandas DataFrame, iterrows is the worst. This creates a new series for each row. this series also has a single dtype, so it gets upcast to the least general type needed. This can lead to unexpected loss of information (large ints converted to floats), or loss in performance (object dtype).
Slightly better is ...
0
else-after-continue
This:
if -threshold < cum_return < threshold:
# this possibility will probably occur the most frequently
continue
elif cum_return > threshold:
times.append(cum_return)
up_threshold += threshold
down_threshold += threshold
else:
times.append(cum_return)
up_threshold -= threshold
...
1
Else-after-return
Some people consider this a stylistic choice, but this:
if check:
return FileResponse(
data,
as_attachment=True,
filename=config['DOWNLOAD']['DOWNLOAD_FILE_AS'])
else:
return HttpResponse(data)
can be
if check:
return FileResponse(
...
1
I think there is quite a lot that could be improved on in your approach.
My main piece of advice is to try and process each line in the data only once, since each line is independent you should be able to do this.
I'm not too familiar with pandas but it seems like there are two main areas of concern.
The section where you clean up the data and filter out ...
2
If your series has many more rows than you have names to check (which should be the case), you should use the vectorized string functions in pandas.
names = ["Mary", "Alan"]
names_re = "|".join(names)
df = pd.DataFrame({"A": ["Mary herself","Mary is mine","John himself","John is mine&...
4
elif
Your code doesn't look like it will work the way you intend it to:
if "Mary" in name:
print(name)
if "Alan" in name:
print(name)
else:
print("no")
If name contains "Mary", but does not contain "Alan", you will have two lines printed, due to the first print(name) and the print("no&...
5
Your code follows this pseudo-code pattern:
if any of these strings are in name:
do x
The simplest way to express that in Python is to invert the condition:
if name in any of these strings:
do x
Or as actual Python:
if name in ["Alan", "Mary"]:
print(name)
1
Instead of using tail, if you only need one item there are the first and last methods, which do exactly what you think they would do with grouped dataframes:
df_example.groupby(['id', 'date']).last()
I doubt there is a faster way to create the activity column. And you have to create a new one because of your requirement with what counts to which day.
But ...
2
flag as boolean
If you change the flags from 'Y' and 'N' to True and False You can use boolean indexing. This should speed up a lot of things already
set
You check for each combination word in dest_words for word in source_words on a list of words. If the check matches, you convert to a set. The containment check would be sped up by checking against a list, ...
Top 50 recent answers are included
