I wrote this program as part of a work-related problem but with a focus on improving my Python skills. The program was needed to do a word count PDF files. The problem was there were a large number of PDF files (over 1000) which were scattered over a drive. I made use of a number of modules to achieve this.
The program uses
glob.iglob to search through a directory tree for PDF files. Once it has a PDF file it extracts the content to a list
len it works out the number of pages in the PDF. It then uses the
slip() method to do a word count for the full document. It returns to the top of the loop and repeats till all PDF files have been counted.
I have heavily commented in my code.
Import, file paths input/output
import PyPDF2 import glob import xlsxwriter # ------------------------ Input file path ------------------------ # input_file_path = r"\\...pdf" input_file_count_path = r"\\.." # ------------------------ Output file path ----------------------- # output_file_path = "word_count.xlsx"
def main(input_file_path, output_file_path, file_count): # lists total_words_list =  # word count total_pages_list =  # page count file_name_list =  print('PyPDF2 import complete') for pdfFileObj in glob.iglob(input_file_path): try: # save the file path to list file_name_list.append(pdfFileObj) # To get a PdfFileReader object that represents this PDF, call PyPDF2.PdfFileReader() and pass it pdfFileObj. Store this PdfFileReader object in pdfReader. pdfReader = PyPDF2.PdfFileReader(pdfFileObj) number_of_pages = pdfReader.getNumPages() saved_text =  # loop to count number pages in document for page_number in range(number_of_pages): page = pdfReader.getPage(page_number) # Once you have your Page object, call its extractText() method to return a string of the page’s text ❸. The text extraction isn’t perfect. page_content = page.extractText() # appned to list saved_text.append(page_content) # count the total number of pages on the document total_pages = len(saved_text) # add number of pages from document 'total_pages' to total_pages_list which will keep record for all documents. total_pages_list.append(total_pages) # print number of pages print("total number of pages are: ", total_pages) # create new var word_count_total = 0 # loop count words and output total words on document for i in saved_text: #The split() method splits a string into a list. which means 'len' can be used to count the number of words per page. word_count = len(i.split()) # Might be some counting issues, need to test futher. #Totals up the count of words for all pages in the PDF. word_count_total = word_count + word_count_total # takes total word count for each document and adds to a list total_words_list.append(word_count_total) print("Total word count for your file is: ", word_count_total) print(pdfFileObj) except: print("------there was a problem with the file, it will be skipped-------") continue # Deals with ENCRYPTED files. return(total_words_list, total_pages_list, file_name_list, input_file_path, output_file_path)
There is a separate function to write the data lists to an excel file using
# ----------------- function to save to file ------------------------ # def save_to_file (total_words_list, total_pages_list, file_name_list, input_file_path, output_file_path): print("output to excel started") # makes a new excel file workbook = xlsxwriter.Workbook(output_file_path) # add_worksheet method called on workbook object worksheet = workbook.add_worksheet() # start from first cell row = 1 # loop word count, page count and file location then write to file for a, b, c in zip(total_pages_list, total_words_list, file_name_list): # write to file using write method worksheet.write(row, 0, a ) worksheet.write(row, 1, b ) worksheet.write(row, 2, c ) row += 1 workbook.close() print("---output to excel file complete, program all finished---") main(input_file_path, output_file_path, file_count)
Areas of concern:
- Is my code clean? could it be more concise?
- Being new to Python I am sure to overlooked something simple..?
- Program is very slow when running on deep directory trees e.g. depth 4+