# Is it feasible to create a syslog server which writes to a client dataset?

I'm creating a simple syslog server in Delphi XE2 using Indy's TIdSyslogServer. I've decided to make it dump into a TClientDataSet and display in a TDBGrid, but I'm skeptical about how well it would handle if the log got quite big, and what I should expect when it grows to millions of records. This application is for internal use and I don't intend to make any software from it, and just keep the code real simple.

The purpose of the application is for numerous IP surveillance cameras along with various other network based equipment to report their log to one place.

This is a simple application with just 1 form, all the code is directly in the form's unit. The actual application is a separate project (call this my SSCCE).

uMain.pas

unit uMain;

interface

uses
Winapi.Windows, Winapi.Messages, System.SysUtils, System.Variants, System.Classes, Vcl.Graphics,
Vcl.Controls, Vcl.Forms, Vcl.Dialogs,
IdSocketHandle, IdBaseComponent, IdComponent, IdSysLogServer, IdSysLog,
IdSysLogMessage, IdUDPBase, IdUDPServer,
Vcl.StdCtrls, Vcl.Grids, Vcl.DBGrids,
Data.DB, Datasnap.DBClient, MidasLib {to avoid requiring MIDAS.DLL};

type
TForm1 = class(TForm)
Server: TIdSyslogServer;
DS: TClientDataSet;
DSC: TDataSource;
DBGrid1: TDBGrid;
procedure FormCreate(Sender: TObject);
procedure FormDestroy(Sender: TObject);
procedure ServerSyslog(Sender: TObject;
ASysLogMessage: TIdSysLogMessage; ABinding: TIdSocketHandle);
private
procedure PrepareDS;
public

end;

var
Form1: TForm1;

implementation

{\$R *.dfm}

procedure TForm1.FormCreate(Sender: TObject);
var
H: TIdSocketHandle;
begin
PrepareDS;
Server.Bindings.Clear;
H.IP:= '0.0.0.0';     //All IP's
H.Port:= 514;         //syslog standard port 514
Server.Active:= True; //Activate server
end;

procedure TForm1.PrepareDS;
begin
DS.DisableControls;
try
DS.Close;
DS.FieldDefs.Clear;
//Need to convert the next 2 to string
DS.CreateDataSet;
DS.Open;
finally
DS.EnableControls;
end;
end;

procedure TForm1.FormDestroy(Sender: TObject);
begin
Server.Active:= False;
end;

procedure TForm1.ServerSyslog(Sender: TObject;
ASysLogMessage: TIdSysLogMessage; ABinding: TIdSocketHandle);
begin
DS.Append;
DS['timestamp']:= ASysLogMessage.TimeStamp;
DS['pri']:=       ASysLogMessage.Pri;
DS['facility']:=  ASysLogMessage.Facility;
DS['severity']:=  ASysLogMessage.Severity;
DS['hostname']:=  ASysLogMessage.Hostname;
DS['message']:=   ASysLogMessage.Msg.Content;
DS.Post;
end;

end.


uMain.dfm

object Form1: TForm1
Left = 354
Top = 124
Caption = 'Form1'
ClientHeight = 400
ClientWidth = 597
Color = clBtnFace
Font.Charset = DEFAULT_CHARSET
Font.Color = clWindowText
Font.Height = -11
Font.Name = 'Tahoma'
Font.Style = []
OldCreateOrder = False
OnCreate = FormCreate
OnDestroy = FormDestroy
PixelsPerInch = 96
TextHeight = 13
object DBGrid1: TDBGrid
Left = 0
Top = 48
Width = 597
Height = 352
Align = alBottom
Anchors = [akLeft, akTop, akRight, akBottom]
DataSource = DSC
TabOrder = 0
TitleFont.Charset = DEFAULT_CHARSET
TitleFont.Color = clWindowText
TitleFont.Height = -11
TitleFont.Name = 'Tahoma'
TitleFont.Style = []
end
object Server: TIdSyslogServer
Bindings = <>
OnSyslog = ServerSyslog
Left = 120
Top = 168
end
object DS: TClientDataSet
Aggregates = <>
Params = <>
Left = 168
Top = 168
end
object DSC: TDataSource
DataSet = DS
Left = 200
Top = 168
end
end


I'm assuming that at some point I should at least make it dump to a file, then start fresh. That's an obvious feature I will need to add. Along with that of course a way to recall the saved logs. That all comes later, but I'm only worried how the client dataset will handle it when it gets very large, and really how I should determine the maximum before I dump it.

• I'm kinda wondering if this type of question is better off on programmers SE? – Jerry Dodge Dec 6 '13 at 1:18
• For the record, I started building my own syslog using an Indy UDP server, and got it working, but then realized Indy already has a syslog server :-) – Jerry Dodge Dec 6 '13 at 1:22

TClientDataset will work fine for this purpose, except for the "million of records". I don´t believe that any dataset will have a good response for such an amount of data, so I think you will have to work some kind of paging system and keep in the dataset only some hundreds or thoudsands of records, after all, it´s unhuman to handle millions of anything (;-)).
I took a look on your code and noticed that you call Open just after calling CreateDataset. That´s unnecessary given CreateDataset already gets the dataset open.
By the way, you had and excelent idea on including MidasLib in you uses clause.