5
\$\begingroup\$

I am trying to develop a simple inventory system that takes JSON data gathered from a Powershell script, inputs the data into a MongoDB, and displays the info in an interesting manner for the user. It works well at the moment, but I do have a few interesting questions about how I did this, i.e. should I make the DB connection once at the top of my server.py file or every time I call a method? In a future edition, I'll try to have SSL and LDAP SSO integrated.

1. Gather the information

Right now, I am using Powershell + WMIC to get the information. I am not sure if there is a better way to get it (maybe an agent on a machine?), but here is the code and output for that:

fingerprint.ps1

<##################################################################### 
 Fingerprint by jeff Tindell
#####################################################################>
<#####################################################################
            Global Variables 
#####################################################################>

#dictionaries
#Drive type dictionary (from http://msdn.microsoft.com/en-us/library/aa394173(v=vs.85).aspx)
$driveTypeDictionary = "Unknown","No Root Directory", "Removable Disk", "Local Disk", "Network Drive", "Compact Disk", "RAM Disk" 
#OS Version number dictionary (from http://msdn.microsoft.com/en-us/library/windows/desktop/ms724832(v=vs.85).aspx)
$osProductTypeDictionary = "Unknown","Client","Domain Controller","Server"
#Windows Client Dictionaries
$winClientDictionary = @{"6.3" = "Windows 8.1"; "6.2" = "Windows 8"; "6.1" = "Windows 7"; "6.0" = "Windows Vista"; 
                         "5.2" = "Windows XP 64-Bit"; "5.1" = "Windows XP"}
$winServerDictionary = @{"6.3" = "Windows Server 2012 R2"; "6.2" = "Windows Server 2012"; "6.1" = "Windows Server 2008 R2";
                         "6.0" = "Windows Server 2008"; "5.2" = "Windows Server 2003 R2"; "5.1" = "Windows Server 2003"}

#variables
$masterOutput = @{}



<#####################################################################
            Input Variables
#####################################################################>




<#####################################################################
            WMI Calls
#####################################################################>

$computerSystemInfo = gwmi Win32_computerSystem
$motherboardInfo = gwmi Win32_baseboard 
$cpuInfo = gwmi Win32_processor 
$winInfo = gwmi Win32_operatingSystem
$memoryInfo = gwmi Win32_PhysicalMemory
$drivesInfo = gwmi Win32_LogicalDisk
$networkInfo = (gwmi win32_networkadapterconfiguration | where IPAddress -NE $NULL)


<#####################################################################
            Logic
#####################################################################>




#Computer System Info

$domain = $computerSystemInfo.Domain
$manufacturer = $computerSystemInfo.Manufacturer
$model = $computerSystemInfo.Model
$computerName = $computerSystemInfo.Name
#computer system output
$systemInfoOutput = @{"computer_domain"= "$domain";"computer_name" = "$computerName"; "system_make"=$manufacturer; "system_model"=$model}
$masterOutput.Add("system_info", $systemInfoOutput)

#Motherboard Info
$mbMake = $motherboardInfo.Manufacturer
$mbSerial = $motherboardInfo.SerialNumber
#motherboard output
$mbOutput = @{"make" = "$mbMake"; "serial" = "$mbSerial"}
$masterOutput.Add("motherboard_info", $mbOutput)

#CPU Info
$cpuManufacturer = $cpuInfo.Manufacturer
$cpuSpeed = "{0:N1}" -f ($cpuInfo.MaxClockSpeed/1000) + "GHz"
$cpuName = $cpuInfo.Name
#cpuOutput
$cpuInfoOutput = @{"name" = "$cpuName"; "make" = "$cpuManufacturer"; "speed" = "$cpuSpeed"}
$masterOutput.Add("cpu_info", $cpuInfoOutput)


#os Info
$osProdType = $osProductTypeDictionary[($winInfo.ProductType)]
$osBit = $winInfo.OSArchitecture
$osMajorVersion = $winInfo.Version.Substring(0,3)
$winVersionName =""
switch ($osProdType) {
    "Unknown" {$winVersionName = $winClientDictionary.get_Item($osMajorVersion)}
    "Client" {$winVersionName = $winClientDictionary.get_Item($osMajorVersion)}
    default {$winVersionName = $winServerDictionary.get_Item($osMajorVersion)}
}
$osVersionNumber = $winInfo.Version
#OS output
$osInfoOutput = @{"name" = "$winVersionName"; "osBit"= "$osBit"; version_number= "$osVersionNumber"; "type" = "$osProdType"}
$masterOutput.Add("os_info", $osInfoOutput)

#physical memory info
$memorySize =  $memoryInfo.Capacity/1073741824 #in GB
$memorySpeed = $memoryInfo.Speed
#memory Output
$memoryOutput = @{"size" = "$memorySize"; "speed" = "$memorySpeed"}
$masterOutput.Add("memory_info", $memoryOutput)



#Hard drive Info
#drive Output

$drivesOutput = @{}
$currentDrive=1
foreach ($drive in $drivesInfo){
    $currentDriveHash = @{}

    $currentDriveHash.Add("device_id", $drive.DeviceID)
    $currentDriveHash.Add("name", $drive.VolumeName)
    $currentDriveHash.Add("type" , ($driveTypeDictionary[($drive.DriveType)]))
    $currentDriveHash.Add("free", $drive.FreeSpace)
    $currentDriveHash.Add("used", ($drive.size - $drive.FreeSpace))
    $currentDriveHash.Add("max_size", $drive.Size)


    $drivesOutput.add("drive_$currentDrive", $currentDriveHash)
    $currentDrive++

}

$masterOutput.Add("drive_info", $drivesOutput)

$networkOutput = @{}
$currentNetDevice = 1
foreach ($netDevice in $networkInfo){
    $currentIP = ($netDevice.IPAddress | Where-Object {$_ -like "*.*.*.*"})
    $currentNetHash = @{}
    $currentNetHash.Add("name", $netDevice.Description)
    $currentNetHash.Add("dhcp", $netDevice.DHCPEnabled)
    $currentNetHash.Add("ip", $currentIP)
    $currentNetHash.Add("gw", $netDevice.defaultIPGateway)
    $networkOutput.add("interface_$currentNetDevice", $currentNetHash)
    $currentNetDevice++


}

$masterOutput.Add("net_info", $networkOutput)


<#####################################################################
            Output
#####################################################################>

$jsonOutput = $masterOutput | ConvertTo-Json
Set-Content -Path "././$Computername.json" -Encoding UTF8 -Value $jsonOutput



<#####################################################################
            import into mongodb
#####################################################################>

This would produce a sample output file (template.json):

{
"device_name":"COMPUTERNAME",
"cpu_make": "GenuineIntel",
"cpu_name": "Intel(R) Core(TM) i7-3517U CPU @ 1.90GHz",
"cpu_speed": "2.4GHz",
"type" : "server",
"system_computer_domain": "WORKGROUP",
"system_computer_name": "COMPUTERNAME",
"system_make": "LENOVO",
"system_model": "20175",
"os_type": "Client",
"os_bit": "64-bit",
"os_name": "Windows 8.1",
"os_version_number": "6.3.9600",
"mb_serial": "EB32244241",
"mb_make": "LENOVO",
"eth0_gw": "192.168.101.1",
"eth0_dhcp": true,
"eth0_name": "ASIX AX88772B USB2.0 to Fast Ethernet Adapter",
"eth0_ip": "192.168.0.4",
"mem_size": "8",
"mem_speed": "1600",
"drive2_device_id": "D:",
"drive2_type": "Local Disk",
"drive2_used": 1830584320,
"drive2_max_size": 4294963200,
"drive2_name": "LENOVO",
"drive2_free": 2464378880,
"drive1_device_id": "C:",
"drive1_type": "Local Disk",
"drive1_used": 39408615424,
"drive1_max_size": 99026464768,
"drive1_name": "Windows8_OS",
"drive1_free": 59617849344
}

2. Import the Data

Next I import the data using the CL of mongoimport:

mongoimport --host localhost --db pacman --collection {servers | net_devices} .\{FILENAME}

3. Display the data

Finally I display the data in a WebApp using bottle and pymongo. Here is my dir structure for the project (called "pacman" for performance and capacity management):

PACMAN
|--static
   |--css
      |--style.css 
   |--img
      |--delete.png
      |--edit.png
|--views
   |--file_control
      |--add_file_to_existing.tpl
      |--edit_existing_filename.tpl
   |--device_view.tpl
   |--home.tpl
|--server.py

Server.py

import copy
import mimetypes
import bottle
from bson import ObjectId
import gridfs
import pymongo
import sys

__author__ = 'Jeff Tindell'



# Preformance And Capacity MANagment (system)

# The point of this program is to take json documents consisting of server or network devices basic configuration
# and display it on a basic web form.

# It will also hold and link documents regarding the systems (config files or whatever).




#establish a connection to the db:
connection = pymongo.MongoClient("mongodb://localhost")
db = connection.pacman
# get collections of my network devices and servers in the inventory
net_devices_col = db.net_devices
servers_col = db.servers
errors = []
#get gridfs for the two dbs
fs = gridfs.GridFS(db)



# Static Routes
@bottle.get('/<filename:re:.*\.js>')
def javascripts(filename):
    return bottle.static_file(filename, root='static/js')


@bottle.get('/<filename:re:.*\.css>')
def stylesheets(filename):
    return bottle.static_file(filename, root='static/css')


@bottle.get('/<filename:re:.*\.(jpg|png|gif|ico)>')
def images(filename):
    return bottle.static_file(filename, root='static/img')


@bottle.get('/<filename:re:.*\.(eot|ttf|woff|svg)>')
def fonts(filename):
    return bottle.static_file(filename, root='static/fonts')



#home page

@bottle.route('/')
def home_page():
    #copy any errors out of the errors dict, and then empty the dict
    err_list = copy.deepcopy(errors)
    errors[:] = []

    #run a search for all servers and net_devices
    net_devices = net_devices_col.find()
    servers = servers_col.find()



    #send the results to the home page:
    template_args = {'network_devices': net_devices, 'servers': servers, 'errors': err_list}
    return bottle.template('home.tpl', template_args)

@bottle.route('/showDevice')
def show_device():
    # get the url information (passed in as /showDevice?id=35jfjae3...&type=server)
    # type will either be server or net (for now).

    device_id = bottle.request.query.id
    device_type = bottle.request.query.type

    cursor = None
    device = {}
    attached_files = {}

    if device_id: # was an id sent in?
        # if so, search the database for the proper object

        query = {"_id" : ObjectId(device_id)}

        if device_type == "server":
            cursor = db.servers.find(query)
        elif device_type == "net":
            cursor = db.net_devices.find(query)
        else: # couldnt find device type
            errors.append({'text': 'device type not recognized'})
    else: # no id was sent in
        errors.append({'text':'Device not found, No id sent in.'})

    #after the search
    if cursor: #if the search turn up something
        for documents in cursor: # get the dictionaries out of the cursor
            device = documents
    #search the files db for any attached files
        attached_files = db.fs.files.find({"device_id" : ObjectId(device_id)})

    # return the search results
        return bottle.template('device_view.tpl', {'device': device, 'attached_files': attached_files})
    else: #the search was unsucessful
        errors.append({'text': 'search turned up no results'})
    bottle.redirect('/')


@bottle.route('/addDevice')
def add_device():

    return None


@bottle.route('/addFile')
def add_file():
    device_id = bottle.request.query.id
    device_name = bottle.request.query.name
    device_type = bottle.request.query.type
    device={'_id': device_id, 'name': device_name, 'type': device_type}

    return bottle.template('file_control/add_file_to_existing.tpl', {'device':device})


@bottle.route('/upload', method='POST')
def do_upload():
    data = bottle.request.files.data
    did = bottle.request.query.id
    type = bottle.request.query.type
    device_url = '/showDevice?id=' + str(did) + '&type=' + type+'#files'
    raw = data.file.read()  # This is dangerous for big files
    file_name = data.filename
    try:
        newfile_id = fs.put(raw, filename=file_name, device_id = ObjectId(did))
    except:
        return "error inserting new file"

    return bottle.redirect(device_url)



@bottle.route('/download')
def download():
    file_id = ObjectId(bottle.request.query.id)
    if file_id:
        try:
            file_to_download = fs.get(file_id)
        except:
            return "document id not found for id:" + file_id, sys.exc_info()[0]
        file_extension = str(file_to_download.name)[(str(file_to_download.name).index('.')):]

        bottle.response.headers['Content-Type'] = (mimetypes.types_map[file_extension])
        bottle.response.headers['Content-Disposition'] = 'attachment; filename=' + file_to_download.name

        return  file_to_download

@bottle.route('/editFilename')
def edit_page():
    # in comes device id, device type, and file id, and filename
    device_id = bottle.request.query.did
    fid = bottle.request.query.fid
    device_type = bottle.request.query.type
    old_filename = bottle.request.query.ofn
    filedict = {'_id': ObjectId(fid), 'ofn': old_filename}
    device={'_id': ObjectId(device_id), 'type': device_type}

    return bottle.template('file_control/edit_existing_filename.tpl', {'device':device, 'file':filedict})

@bottle.route('/updateFilename', method='POST')
def update_filename():
    # /updateFilename?fid=FILE_ID&did=DEVICE_ID&type=TYPE
    fid= ObjectId(bottle.request.query.fid)
    did= ObjectId(bottle.request.query.did)
    dtype = bottle.request.query.type
    form_dict = bottle.request.forms
    new_name = str(form_dict['new_filename']) + str(form_dict['ext'])

    device_url = '/showDevice?id=' + str(did) + '&type=' + dtype + '#files'
    db.fs.files.update({'_id': fid}, {'$set': {'filename': new_name}})
    return bottle.redirect(device_url)





@bottle.route('/removeFile')
def delete_file():
    # /removeFile?fid=FILE_ID&did=DEVICE_ID&type=TYPE
    fid= ObjectId(bottle.request.query.fid)
    did = ObjectId(bottle.request.query.did)
    dtype = bottle.request.query.type
    device_url = '/showDevice?id=' + str(did) + '&type=' + dtype + '#files'
    fs.delete(fid)
    return bottle.redirect(device_url)

@bottle.route('/removeDevice')
def delete_device():
    # Need to delete any files related to this device, then delete the device
    did = ObjectId(bottle.request.query.did)
    dtype = bottle.request.query.type
    results = db.fs.files.find({'device_id': did})
    for file in results:
        fs.delete(file['_id']) # delete all files associated with this entry
    if dtype == 'net':
        col = db.net_devices
    elif dtype == 'server':
        col = db.servers
    else:
        return bottle.redirect('/')

    col.remove(did)


    return bottle.redirect('/')





bottle.debug(True)
bottle.run(host='localhost', port=8080)

There is a lot more code, but rather than display it all, I'll just put up a img of the homepage and the page after clicking on a device:

Home

home page

Device

device page

I am not a full-time developer, so I look forward to any insight anyone could provide.

\$\endgroup\$
  • 3
    \$\begingroup\$ Providing screenshot is a really nice touch. Thanks for asking well documented question, I wish I could help you :-) \$\endgroup\$ – SylvainD Feb 19 '14 at 16:57
  • \$\begingroup\$ Thanks Josay. I'll attempt to update my code, but to find the most up to date code: github.com/jdell64/IPACS \$\endgroup\$ – Jeff Feb 26 '14 at 13:20
1
\$\begingroup\$

Okay, so this looks quite good!

  1. Connecting to the database at the top makes sense, since you want to be connected to it at all times. This is the best option in bottle, especially if you are not afraid of loosing the connection. In more complete frameworks, the database connections are handled for you, and various options exist, such as persistent connections or connection pools.
  2. Small typo in your screenshot and your code, you meant 'Performance', not 'Preformance'.
  3. You should use docstrings instead of comments to document your functions. You can also look at PEP 8 to see how Python code is generally formatted.
\$\endgroup\$
  • \$\begingroup\$ I'll take a look at docstrings (I'm new to python). Any examples on how to handle all static routes in one method? Would it be like, def incoming_request(url_string)? \$\endgroup\$ – Jeff Feb 27 '14 at 16:33
  • \$\begingroup\$ Didn't notice the root parameter, sorry. Probably not a great advice. \$\endgroup\$ – Quentin Pradet Feb 27 '14 at 21:57

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for?Browse other questions tagged or ask your own question.