urllib proxy access function

Question

Several of my Python 2.7 programs need to access server tasks via our server Apache instance. The 'client' programs will run on Windows in various environments. They may be run on systems that do not use a proxy, or on systems that use a proxy with no authentication, or enterprise systems that use an authenticated proxy server.

The function below is meant to address all three proxy situations.

My main question is whether I have coverage for requirement for all scenarios - i.e. for all possible proxy configurations. I am not familiar with, for example, the kind of proxy authentication that might be used on Windows Active Directory systems - is it different than what my code can handle?

I'm also interested in suggestions for general improvement of the code. The exception handlers are incomplete - I'll work on them once I'm sure the code is ok.

def openUrl(url, data=None):
    """ General purpose URL opener
    Handles no proxy - un-authenticated proxy and authenticated proxy

    urlencoding of the data argument to be performed prior to invoking this function

    :param url: The url to access
    :param data: Post data to accompany the url request
    :return: an open urllib object
    """
    # case 1: Proxy not required as no PROXY_URL found in environment.  Fall through to urlopen
    if os.getenv('PROXY_URL') is None:
        pass

    # case 2: un-authenticated proxy, just load the proxy handler and then fall through to urlopen
    elif os.getenv('PROXY_USER') is None:
        proxy_support = urllib2.ProxyHandler({})
        opener = urllib2.build_opener(proxy_support)
        urllib2.install_opener(opener)

    # case 3:  Authenticated proxy.  Load password manager and proxyHandler, the fall through to urlopen
    else:
        password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
        password_mgr.add_password(None, os.getenv('PROXY_URL'), os.getenv('PROXY_USER'), os.getenv('PROXY_PASSWORD'))
        auth_handler = urllib2.ProxyBasicAuthHandler(password_mgr)
        opener = urllib2.build_opener(auth_handler)
        urllib2.install_opener(opener)

    # Access url using proxy defined above (or no proxy at all)
    try:
        connection = urllib2.urlopen(url, data)
    except urllib2.URLError, err:
        print(repr(err))
        exit()
    except urllib2.HTTPError, err:
        print(repr(err))
        exit()

    return connection

Answer 1

• The function should be called open_url according to Python coding conventions.
• The if/pass combination isn't so nice, it's just more to read. I'd nest the conditions instead.
• I don't get the exception handler. Why exit hard? Maybe there's a short network outage and then the program just shuts down? Let other parts of the program handle those scenarios. They need to deal with urllib2 exceptions anyway.
• PROXY_PASSWORD isn't verified, does None work as a value?

Otherwise looks fine; I'd suggest caching those handlers and only installing them once, e.g.:

def install_open_url_handlers():
    # case 1: Proxy not required as no PROXY_URL found in environment.  Fall through to urlopen,
    # otherwise:
    proxy_url = os.getenv('PROXY_URL')
    if proxy_url is not None:
        proxy_user = os.getenv('PROXY_USER')

        # case 2: un-authenticated proxy, just load the proxy handler and then fall through to urlopen
        if proxy_user is None:
            handler = urllib2.ProxyHandler({})
        # case 3: Authenticated proxy.  Load password manager and proxyHandler, the fall through to urlopen
        else:
            password_mgr = urllib2.HTTPPasswordMgrWithDefaultRealm()
            password_mgr.add_password(None, proxy_url, proxy_user, os.getenv('PROXY_PASSWORD'))
            handler = urllib2.ProxyBasicAuthHandler(password_mgr)

        urllib2.install_opener(urllib2.build_opener(handler))


HANDLER_INSTALLED = False


def open_url(url, data=None):
    """ General purpose URL opener
    Handles no proxy - un-authenticated proxy and authenticated proxy

    urlencoding of the data argument to be performed prior to invoking this function

    :param url: The url to access
    :param data: Post data to accompany the url request
    :return: an open urllib object
    """
    global HANDLER_INSTALLED
    if not HANDLER_INSTALLED:
        HANDLER_INSTALLED = False
        install_open_url_handlers()

    # Access url using proxy defined above (or no proxy at all)
    return urllib2.urlopen(url, data)

Answer 2

I know you said the error handlers weren't done, but don't use except Exception, err. It's syntax that was removed in Python 3.x because it muddied the waters when you want to accept multiple exceptions, ie. except ValueError, IndexError. It is possible to do that, but you have to manually wrap them in parentheses to prevent the ambiguity. When you want to except an error and store it with a name, use the explicit syntax except Exception as err, so it's fully clear what you mean.

Also, you might be able to just catch both exceptions in one go, if you don't intend to treat them differently. As I said above, you just need to put both exceptions in a tuple, like this:

except (urllib2.URLError, urllib2.HTTPError) as err:
    print(repr(err))
    exit()