I've hodgepodged together an attempt to extract all phone numbers from all CSVs in a directory, regardless of where they are and what format they're in. I want all phone numbers to be printed to a compilation CSV with no repeat elements in the format
8885551234 with NO leading ones. E.g.,
1 (510) 533-2134 would become
5105332134, as would
510-5332134 (and the latter would count as a dupe and fail to be written).
My challenges are that the numbers are in different formats, in different columns (and sometimes in several columns), and that the files in the directory are, in total, many gigabytes in size.
What I came up with, first, fails to weed out numbers that start with 0 or 1 (though it does seem to "skip" the 1 in 11-digit numbers); second, I need checked by a regexpert to see if it is even doing what I want (identifying and printing phone numbers); and third, is far too slow:
import csv import re import glob import string with open('phonelist.csv', 'wb') as out: seen = set() output =  out_writer = csv.writer(out) csv_files = glob.glob('*.csv') #csv_files2 = glob.glob('*.csv') for filename in csv_files: with open(filename, 'rbU') as ifile: read = csv.reader(ifile) for row in read: for column in row: s1 = column.strip() result = re.match( r'.*(\+?[2-9]?[0-9]?[0-9]?-?\(?[0-9][0-9][0-9]\)? ?[0-9][0-9][0-9]-?[0-9][0-9][0-9][0-9]).*', s1) or re.match( r'.*(\+?[2-9]?[0-9]?[0-9]?-?\(?[0-9][0-9][0-9]\)?-?[0-9][0-9][0-9]-?[0-9][0-9][0-9][0-9]).*', s1) if result: tempStr = result.group(1) for ch in ['(', ')', '-', ' ']: tempStr = tempStr.replace(ch, '') if tempStr not in seen: seen.add(tempStr) output.append(tempStr) for val in output: out_writer.writerow([val])