1
\$\begingroup\$

I work in marketing and I am currently trying to aggregate data for 50 facebook pages on the same Business Page.

I have a loop that tries a series of requests to grab all data. I am not sure I am handling exceptions in the best way possible, so I was wondering if somebody had any idea on how to improve the situation.

Main problem is that, for a reason or another, the calls for page n°40 out of my 50 pages break the API Limit 8th fold. Not sure what's causing the issue, but I imagine it has to do with this part of my code.

I can post the whole function if necessary.

counter = 0

while True:

    try:

        if ('next' in s['paging']):

            next_s   =   s['paging']['next']
            s        =   requests.get(next_s).json()
            s_total.extend(s['data'])
            counter += 1

            print("Getting Page. Current progress: {}".format(counter))


        elif ('after' in s['paging']['cursors']):

            next_s       =   s['paging']['cursors']['after']
            new_page_url = next_request+next_s
            s            =   requests.get(new_page_url).json()
            s_total.extend(s['data'])
            print("Getting After. Current progress: {}".format(counter))


        elif ('32' in str(s['error']['code'])):

            print('Faced error {} in: {} '.format(str(s['error']['code']), username))
            print('Sleeping for 1 hour')
            sleep(3605)

            next_s   =   s['paging']['next']
            s        =   requests.get(next_s).json()
            s_total.extend(s['data'])
            counter += 1



            print('Continuing...')
            continue


        elif ('4' in str(s['error']['code'])):

            print('Faced error {} in: {} '.format(str(s['error']['code']), username))
            print('Sleeping for 1 hour')
            sleep(3605)

            next_s   =   s['paging']['next']
            s        =   requests.get(next_s).json()
            s_total.extend(s['data'])
            counter += 1
            print('Continuing...')
            continue

        else:
            print(s['error'])
            break

    except KeyError:

        print('Finished page : {} '.format(username))

        break
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.