Trying to use global variables for referencing directories in Python 2.5

Question

I am writing a small utility that reads a number of print files to extract out financial data, and then build a spreadsheet populating the data in specific cells.

I have it working, but it's not elegant and certainly not what you would call Pythonic, so I am rewriting to make it more efficient and improve my Python skills.

First hurdle...

I'd like to use a procedure to write out to a log-file but I'm not sure I've got my head around how global functions and variables work in python.

Reading the question https://stackoverflow.com/questions/13034496/using-global-variables-between-files-in-python suggests that I need to use a separate file to hold the global variables, so I have this in config.py:

def init():
    global DEBUG
    DEBUG = True
    global directoryBase
    directoryBase = 'C:/Users/djehan1/workspace/PeriodEnd/'
    global directoryLogFile
    directoryLogFile = dirBase + ''
    global directoryReportSource
    directoryReportSource = directoryBase + 'reports/'
    global directoryExcelOutput
    directoryExcelOutput = directoryBase + 'output/'
    global directoryGeneralLedger
    directoryGeneralLedger = directoryBase + 'resource/'
    global directoryListOfReports
    directoryListOfReports = directoryBase + 'resource/'
    global gl_reporting_year
    gl_reporting_year = ''
    global gl_reporting_period
    gl_reporting_period = ''
    global reportType
    reportType=''
    global logFileHandle
    logFileHandle=None

I then have a functions.py file that has the following procedure for writing out to the log file:

def write_log(line):
    if line.strip() != '':
        import datetime
        now = datetime.datetime.now()
        config.logFileHandle.write(now.strftime('%H:%M:%S') + ' - ' + line + '\n')
    else:
        config.logFileHandle.write('\n')

Then in my main.py I have:

import config
import datetime

from functions import *
config.init()

config.logFileHandle = open(config.directoryLogFile + 'period_end_process.log', 'w')

As you can see from the last line of code, I have to prefix the logFileHandle with config. While this is fine for this one procedure, I will need to access the other directory variables in other parts of the program and thus use the same config.directoryName type syntax. You will also see that I have to import the datetime library into the main.py and the function.py - this I don't get as I would assume that having loaded the library at the start of the main program, before the import of the functions file, the datetime library would be available to all?

In order to make the project manageable, I intend to have sub-files that contain specific related functions - so, the process to create the Excel document will be encapsulated into one python file that will be called from the main program, for example.

Then functions (like writing the log file) are centralised in the functions file, and can be called from wherever as needed.

So, my questions are: 1. Is there a way to have the datetime available to all sub modules? 2. Have I understood the concept of the config file and is this the best way to implement?

UPDATE: I have reworked the code as follows:

main.py

import config
from functions import *

# Initialise globals
config.init()

# Create the log file handle
create_log_file_handle()

# Get current time
now = get_now()

# Start...
write_log('Starting run ' + now)

functions.py now has

import config

def write_log(line):
    if line.strip() != '':
        now = get_now()
        config.logFileHandle.write(now + line + '\n')
    else:
        config.logFileHandle.write('\n')

def create_log_file_handle():
    global logFileHandle
    config.logFileHandle = open(config.directoryLogFile + 'period_end_process.log', 'w')

def get_now():
    import datetime
    now = datetime.datetime.now()
    return now.strftime('%d/%m/%y - %H:%M:%S')

This works fine and now means that I am not importing datetime in multiple places!

It will still mean that if I need to access directoryReportSource that I am having to do config.directoryReportSource, so I guess the question is still - is this the way to do this?

Answer 1

"Have I understood the concept of the config file and is this the best way to implement?"

Take everything in this:

def init():

and remove the global declarations, delete the def line, and dedent the code. Then there will be no need to "initialize" the module.

"Is there a way to have the datetime available to all sub modules?"

Import the module into the namespace of the modules that you use it in. Do all your imports at the top of the file. The below is wrong:

def write_log(line):
    if line.strip() != '':
        import datetime
        now = datetime.datetime.now()
        config.logFileHandle.write(now.strftime('%H:%M:%S') + ' - ' + line + '\n')
    else:
        config.logFileHandle.write('\n')

Instead of this function, you should use the logging module.

import logging
logging.basicConfig(level=logging.DEBUG, # or other level
                    filename='myfilename.log',
                    format='%(asctime)s %(message)s',
                    datefmt='%m/%d/%Y %I:%M:%S %p',
                    )
logger = logging.getLogger(__name__)
logger.info('here is some info')
logger.debug('here is a debug statement')
logger.warn('here is a warning')
logger.error('here is an error')
logger.critical('a critical problem happened')

and you'll get the following:

$ cat myfilename.log 
09/25/2015 12:25:05 AM here is some info
09/25/2015 12:25:05 AM here is a debug statement
09/25/2015 12:25:05 AM here is a warning
09/25/2015 12:25:05 AM here is an error
09/25/2015 12:25:05 AM a critical problem happened

"Is this the way to do this?"

No, do not import *, do not "initialize" modules, do not write your own logger. You can have globals in a config module, though.