I'm a little frustrated with the state of url parsing in python, although I sympathize with the challenges. Today I just needed a tool to join path parts and normalize slashes without accidentally losing other parts of the URL, so I wrote this:
from urlparse import urlsplit, urlunsplit def url_path_join(*parts): """Join and normalize url path parts with a slash.""" schemes, netlocs, paths, queries, fragments = zip(*(urlsplit(part) for part in parts)) # Use the first value for everything but path. Join the path on '/' scheme = next((x for x in schemes if x), '') netloc = next((x for x in netlocs if x), '') path = '/'.join(x.strip('/') for x in paths if x) query = next((x for x in queries if x), '') fragment = next((x for x in fragments if x), '') return urlunsplit((scheme, netloc, path, query, fragment))
As you can see, it's not very DRY, but it does do what I need, which is this:
>>> url_path_join('https://example.org/fizz', 'buzz') 'https://example.org/fizz/buzz'
>>> parts=['https://', 'http://www.example.org', '?fuzz=buzz'] >>> '/'.join([x.strip('/') for x in parts]) # Not sufficient 'https:/http://www.example.org/?fuzz=buzz' >>> url_path_join(*parts) 'https://www.example.org?fuzz=buzz'
Can you recommend an approach that is readable without being even more repetitive and verbose?