ThreadPoolExecutor submit refactor

Question

Currently I need to download thousand of images, I can use 2 functions to download the images: LoadUrl or LoadUrlProxy.

Each function has different parameters:

LoadUrl(url, timeout)
LoadUrlProxy(url, proxy_client, proxy_params)

As I don't want to duplicate the code, I'm looking to simplify the way I'm passing a function reference and its parameters, not sure If below is the best way or I can improve this code?

     # func(Function): Function reference which defines how to download images
      func = LoadUrl
      if use_proxy:
        proxy_client = CreateProxyClient()
        proxy_params = GetProxyParams()        
        func = LoadUrlProxy
      images = []

      with futures.ThreadPoolExecutor(max_workers=_MAX_WORKERS) as executor:
        # Start the load operations and mark each future with its URL
        if use_proxy:
          future_to_url = {
              executor.submit(func, image, proxy_client, proxy_params): image
              for image in image_list
          }
        else:
          future_to_url = {
              executor.submit(func, image, _EXECUTOR_TIMEOUT): image
              for image in image_list
          }
        for future in futures.as_completed(future_to_url):
          image = future_to_url[future]
          try:
            images.append((image.master_id, future.result()))
          except Exception as exc: 
            logging.exception('%r Generated an exception: %s', image.url, exc)

Answer 1

Why not use functools.partial to simplify further and avoid the if use_proxy a second time?

Something along the lines of

if use_proxy:
    func = functools.partial(
            LoadUrlProxy,
            proxy_client=CreateProxyClient(),
            proxy_params=GetProxyParams())
else:
    func = functools.partial(LoadUrl, timeout=_EXECUTOR_TIMEOUT)

with futures.ThreadPoolExecutor(max_workers=_MAX_WORKERS) as executor:
    future_to_url = {
            executor.submit(func, image): image
            for image in image_list
    }

    for future in futures.as_completed(future_to_url):
        image = future_to_url[future]
        try:
            images.append((image.master_id, future.result()))
        except Exception as exc: 
            logging.exception('%r Generated an exception: %s', image.url, exc)