The tweepy.models.Status class creates a 'tweet' object, it has attribute .created_at which is the posting time of the tweet in the form of Python datetime object.

Here is the case, I have a function time_distribution(tweets, **kwargs) which calculates the distribution of the tweets by time. The function is at the end of this post.

Here is an example implementation, we may use the samples from statistweepy which is about 3000 tweets of @CocaCola_ID twitter account, stored in a .npy file. Each 'tweet' is a tweepy.models.Status object.


import numpy
import matplotlib.pyplot as plt

data = list(numpy.load('CocaCola_ID_3093samples_hour_min_21_50_date_24_6_2018.npy'))

dist_only = time_distribution(data, unit = 'hour')

fig, ax = plt.subplots(1, 1)
dist_and_plot = time_distribution(data, cont_hist = (ax, 20), unit = 'hour')

With argument cont_hist = (ax, n), the function also returns a histogram with n number of bins to be plotted on top of ax.

My question :

  • How to write it better? (sustainable, convenience, readable, compact)
  • How to optimize the performance?

import operator
import itertools

def time_distribution(tweets, unit = 'hour', output = 'frequency', cont_hist = False):

    if not any([unit == 'year', unit == 'month', unit == 'day', unit == 'hour']):
        raise AssertionError(' The argument unit must be one of \'year\',  \'month\',  \'day\', or \'hour\'. ')

    unicity_key = operator.attrgetter('created_at.'+unit)
    tweets = sorted(tweets, key=unicity_key)
    distribution = {}

    for time, time_tweets in itertools.groupby(tweets, key=unicity_key):

        time_tweets = list(time_tweets)

        if output == 'frequency':

            distribution[time] = len(time_tweets)
            fy = lambda x: x

        elif output == 'tweets':

            distribution[time] = time_tweets
            fy = lambda x: len(x)


            raise AssertionError(' The argument output must be either \'frequency\', or \'tweets\'. ')

    if cont_hist:

        axes = cont_hist[0]

        if unit == 'year':

            xt = [tweet.created_at.year + (tweet.created_at.month + (tweet.created_at.day + tweet.created_at.hour/24)/30)/12 for tweet in tweets]

        elif unit == 'month':

            xt = [(tweet.created_at.month + (tweet.created_at.day + tweet.created_at.hour/24)/30) for tweet in tweets]

        elif unit == 'day':

            xt = [(tweet.created_at.day + tweet.created_at.hour/24) for tweet in tweets]

        elif unit == 'hour':

            xt = [tweet.created_at.hour + (tweet.created_at.minute + tweet.created_at.second/60)/60 for tweet in tweets]

        a = int(min(xt)) + 1
        b = int(max(xt)) + 1

        ticks = range(a-1, b+2)

        histogram = axes.hist(xt, bins = cont_hist[1])
        axes.set_xticklabels([str(i) for i in ticks])

        return distribution, histogram

    return distribution

Plot result :

enter image description here


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.