I have been typically adding the @transaction.atomic decorator to my endpoints and appreciate if a request fails at any point the data completely rolls back. However, some of my endpoints will make an API call to another API service. If my token for that API service is invalid, I refresh the token. This is all well and good until an exception occurs after the API token has been refreshed. The transaction is rolled back including the token information which then will be stale. This includes the new refresh token information so I'm now unable to refresh the auth token.
I attempted to get around this situation by distributing the token refresh to a celery task and waiting for the result as a way of isolating that database change. Even if the requests fails later on, the token information remains while the rest is rolled back.
Here is a simplified example
@transaction.atomic() def post(request): response = api_call() db_operation(response['result']) # Could potentially have an exception def api_call(allow_refresh=True) response = requests.post(url, params) if response.status_code == 401: if allow_refresh: refresh_token_task.delay().get(timeout=10) response = api_call(allow_refresh=False) else: raise Exception('Invalid authorization') # In the case refreshing token doesn't resolve authorization issues, this prevents infinite loop. return response
Phew! I thought I was in the clear except... sometimes I distribute api calls. It didn't occur to me but I realized calling celery tasks from celery tasks easily risks locking. So now I've written out a solution that involves chaining.
def post(request): chain(api_call_task.s(), refresh_token_task.s(), api_call_task.s(), db_operation_task.s())() @task def api_call_task(initial_response=None) if initial_response and initial_response.status_code != 401: return response response = requests.post(url, params) return response @task def refresh_token_task(response): if response.status_code == 401: refresh_token() return response @transaction.atomic() @task def db_operation_task(response): if response.status_code == 401: raise Exception('Invalid authorization') db_operation(response['result'])
I think this works to a degree but gosh what a confusing mess. By chaining the functions, the response is continuously passed through which determines if the token needs to be refreshed and/or if the call needs to be made again and then finally to handle the db operation based upon the response to the api.
Is there a simpler way? Marginal improvement? Bad practice involved? Would it better to just let go of transaction.atomic? Let me know if my simplified pseudo code can use clarification.