I learnt to code a long, long time ago.
Concerns
Existing approach is messy and will be hard to maintain. System errors are not trapped and the error messages could be better implemented. Final hashcode calculation occurs after the data file is written.
I have been through a few python tutorials some time ago but it is hard without a real-world problem. I'd like to use this as an OOP learning opportunity. This code actually does the job and is well on the way to finalising a file validation feature yet to be implemented.
Everything is up for review here.
Situation
Data files are created by a road model and this program rapidly creates scenarios for option testing. It involves overwriting two 'barrier' fields and and one of two overtaking lane fields in the road (ROD) file. I don't understand or need to care about the many other fields but they must not be changed as some involve complex calculations. A hashcode is being introduced to prevent inadvertent ROD file changes but they must remain human readable. Two other files form part of the data set but the changes are trivial - renaming both files and replacing the first line of one with the filename.
So, a database file properties are a filename, with the filename and description on the first two lines then eight other lines before the 109 character records start. The first six characters in each record are the distance (starting at some arbitrary value near 0.0) incremented by 0.1 in each record. Whole numbers have no decimal or trailing zero. There are two other files in the dataset with different extensions.
The program provides a service of creating a new option from an existing road file set. (New option could be updating existing dataset.) The file and command line are checked and the road file data structure analysed. The hashcode register provides a validation and updating service.
#!/usr/bin/python
# With existing road file create new overtaking lane option file(s).
#
# Copy file, insert overtaking lane and barrier line codes for specified
# chainage, replace first line with new filename, second line with description
# and update MD5 HashCode (future).
#
# eg python OptionEditor.py Demo.ROD 1.5 2.2 P Option.ROD Destination_Description
#
# By Neale Irons version 25/02/2018 (CC BY-SA 4.0)
import sys
import os
import shutil
import tempfile
import hashlib
def main():
def copyFile(src, dest):
# From https://www.pythoncentral.io/how-to-copy-a-file-in-python-with-shutil/
try:
shutil.copy(src, dest)
# eg. src and dest are the same file
except shutil.Error as e:
print('Error: %s' % e)
# eg. source or destination doesn't exist
except IOError as e:
print('Error: %s' % e.strerror)
return
def HashCode_register(HASHFILE, src, action):
# with open (hashfile) as hf:
# hash, fn = (open(HASHFILE).read().split('\n').split(' *')
hashcode = hashlib.md5(open(src, 'rb').read()).hexdigest()
print hashcode+ " *" + src # Debug ****
lines = open(HASHFILE).read().split('\n')
for item in lines:
if action == 'validate':
if hashcode in item: # Maybe uppercase compare
return True
elif action == "update":
if src in item: # Maybe uppercase compare
print lines # Debug - no-op
# if item_number_hashcode != hashcode: # FIX THIS ***
# item_hashcode(update_number) = hashcode # need write access here FIX THIS ***
# else:
# write_append_HASFILE(hashcode + " *" + src) # need write access here FIX THIS ***
# return True
return False
def replace_textfile_line(text_filename, new_text, description):
# Replace first line with new_text.
#
# Adapted from ReplFHdr.py
# By Neale Irons version 14/02/2018 (CC BY-SA 4.0)
# import sys, os, tempfile
with open(text_filename) as src:
with tempfile.NamedTemporaryFile('w',
dir=os.path.dirname(text_filename), delete=False) as dst:
line = src.readline() # Read first line
dst.write(new_text + '\n') # Replace first line
if description: # If description not empty
line = src.readline() # Read second line
dst.write(description + '\n') # Replace second line
shutil.copyfileobj(src, dst) # Copy rest of file
os.unlink(text_filename) # remove old version
os.rename(dst.name, text_filename) # rename new version
return()
#Constants
ODO_STEP_KM = 0.1
RECORD_SIZE = 109
BARRIER_COLUMN1 = 11
BARRIER_COLUMN2 = 16
BARRIER_VALUE = '+' # Debug - use '+' instead of '-'
OTLANE_COLUMN1 = 22
OTLANE_COLUMN2 = 27
OTLANE_VALUE = 'Y' # Debug - use 'Y' instead of 'T'
OTLANE_DIRECTION_CODES = "PC"
HASHFILE = "Hashcode.MD5"
True = not False
# Validate command line
if len(sys.argv) < 6 or len(sys.argv) > 7:
format_string = (' Usage: python {} Source.ROD Start_km End_km P|C Destination.ROD ["Destination_Description"]')
print(format_string.format(sys.argv[0]))
print(' Use existing road files to create new overtaking lane option and optionally insert second line description.)')
sys.exit()
src = sys.argv[1]
otl_start_km = float(sys.argv[2])
otl_end_km = float(sys.argv[3])
otl_direction = sys.argv[4]
dest = sys.argv[5]
if len(sys.argv) < 6:
description = sys.argv[6]
else:
description = ''
if otl_start_km > otl_end_km:
print("Invalid start/end overtaking lane chainage - start must be less than end. Exiting...")
sys.exit()
if otl_direction not in OTLANE_DIRECTION_CODES:
print("Direction {} is invalid, must be P or C. Exiting..."
.format(sys.argv[4]))
sys.exit()
# Validate file
if not os.path.isfile(src):
print("File path {} does not exist. Exiting..."
.format(src))
sys.exit()
if src <> dest:
# Validate support files
src_name, src_ext = os.path.splitext(src)
if not os.path.isfile(src_name+'.MLT'):
print("Requires support file {}. Exiting..."
.format(src_name+'.MLT'))
sys.exit()
if not os.path.isfile(src_name+'.OBS'):
print("Requires support file {}. Exiting..."
.format(src_name+'.OBS'))
sys.exit()
# Validate HashCode
if not HashCode_register(HASHFILE, src, "validate"):
print("Invalid file - failed HashCode. Exiting...")
# Read header
with open(src) as fp:
# *** if fails raise "Invalid file format - first 10 lines must be file header
for x in range(1, 10):
line = fp.readline()
header_length=fp.tell() # Calculate header length
fp.seek(0,2)
file_length = fp.tell()
fp.close()
if (file_length - header_length) % RECORD_SIZE != 0:
print("Invalid file format - must be 10 lines and {} character records. Exiting..."
.format((file_length - header_length) % RECORD_SIZE))
sys.exit()
nrecs = (file_length - header_length) // RECORD_SIZE
if nrecs < 1:
print("Invalid file format - file must contain at least one record. Exiting...")
sys.exit()
# dest_tmp = NamedTemporaryFile(delete=False)
dest_tmp = src + ".tmp"
# print("Temp file: {}".format(dest_tmp)) # Debug
copyFile(src, dest_tmp)
# Open file for random read/write
with open(dest_tmp, 'r+b') as fp:
fp.seek(header_length,0)
odo_start_km = float((fp.read(6)))
fp.seek(RECORD_SIZE * -1, 2)
odo_end_km = float((fp.read(6)))
if otl_start_km <= odo_start_km:
print("Invalid start overtaking lane chainage - Overtaking lane can't start before first road chainage. Exiting...")
sys.exit()
if otl_end_km >= odo_end_km:
print("Invalid end overtaking lane chainage - Overtaking lane can't ends after last road chainage. Exiting...")
sys.exit()
if int(round((odo_end_km - odo_start_km)/ODO_STEP_KM) + 1) != nrecs:
print("Invalid file format - incorrect number of lines between start and end distances. Exiting...")
sys.exit()
while otl_start_km < otl_end_km:
fpos=int(round(header_length + ((otl_start_km - odo_start_km) / ODO_STEP_KM * RECORD_SIZE) - 1))
fp.seek(BARRIER_COLUMN1 + fpos)
fp.write(BARRIER_VALUE)
fp.seek(BARRIER_COLUMN2 + fpos)
fp.write(BARRIER_VALUE)
if otl_direction == "P":
fp.seek(OTLANE_COLUMN1 + fpos)
elif otl_direction == "C":
fp.seek(OTLANE_COLUMN2 + fpos)
fp.write(OTLANE_VALUE)
otl_start_km += ODO_STEP_KM
fp.close()
if dest <> src :
# Create new supporting files
dest_name, dest_ext = os.path.splitext(dest)
copyFile(src_name+'.MLT', dest_name+'.MLT')
replace_textfile_line(src_name+'.MLT', src_name+'.MLT', '')
copyFile(src_name+'.OBS', dest_name+'.OBS')
replace_textfile_line(dest_tmp, dest, description)
HashCode_register(HASHFILE, src, "update") # Not working yet ****
from shutil import move
move(dest_tmp, dest)
if __name__ == '__main__':
main()
Data record snippet (not much happening in this sample):
0.8 -1 -1 F F 9 9
0.9 -1 -1 F F 10 201
1 -1 -1 F F 11 202
1.1 -1 -1 F F 12 12
1.2 1 -1 F F 13 13
1.3 1 -1 F F 14 14
1.4 1 -1 F F 15 15
1.5 -1 1 F F 16 16
1.6 -1 1 F F 17 17
1.7 -1 1 F F 18 18
1.8 -1 -1 F F 19 19
1.9 -1 -1 F F 20 20
2 -1 -1 F F 21 21
2.1 -1 -1 F F 22 22
2.2 -1 -1 F F 23 23
2.3 -1 -1 F F 24 24
Hashcode.MD5:
8d443f2e93a3f0b67f442e4f1d5a4d6d *md5.exe
eb574b236133e60c989c6f472f07827b *md5sum.exe
c1f62b08d050f2a30790c5b2f3d29f5c *Demo.ROD
9f036e40755daaa5a5fc141d3cbfbb99 *nGetPid.exe
FIX THIS
andNot working yet
make it seem that this code isn't ready for review yet. \$\endgroup\$