Chapter 1
Chapter 3

2.- Adding the graphic user interface.

We want the parser log application to have a simple -but nice- user interface that displays records of a log file. But, first of all, what is in a log file?.
The following lines are a sample of the beginning of a typical W3SVC log file (* See foot note):

#Software: Microsoft Internet Information Services 6.0
#Version: 1.0
#Date: 2008-07-13 00:02:28
#Fields: date time s-sitename s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) sc-status sc-substatus sc-win32-status 
2008-07-13 00:02:28 W3SVC1673937396 192.168.16.16 GET /Default.asp - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+5.00;+Windows+98) 200 0 0
2008-07-13 00:02:33 W3SVC1673937396 192.168.16.16 POST /central/noentryaccess.asp - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:02:38 W3SVC1673937396 192.168.16.16 GET /central/data/user.js - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:02:44 W3SVC1673937396 192.168.16.16 GET /central/data/Edition.js - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:02:51 W3SVC1673937396 192.168.16.16 GET /central/data/main.js - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:02:56 W3SVC1673937396 192.168.16.16 GET /central/data/AsyncNotifier.js - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:02 W3SVC1673937396 192.168.16.16 GET /central/data/scormErrorManager.js - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:07 W3SVC1673937396 192.168.16.16 GET /central/data/scormLaunchCourse2.js - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:14 W3SVC1673937396 192.168.16.16 GET /central/data/scormAPI.js - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:19 W3SVC1673937396 192.168.16.16 GET /central/FC4Net_01.asp - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:19 W3SVC1673937396 192.168.16.16 GET /central/FC4Net_MenuMain.asp - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:24 W3SVC1673937396 192.168.16.16 GET /central/blank.htm - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:24 W3SVC1673937396 192.168.16.16 GET /central/DA_1.asp uid=114031 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:30 W3SVC1673937396 192.168.16.16 GET /central/FC4Net.css - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:30 W3SVC1673937396 192.168.16.16 GET /central/images/logo.jpg - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:36 W3SVC1673937396 192.168.16.16 GET /central/images/banner.jpg - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:36 W3SVC1673937396 192.168.16.16 GET /central/images/01_NGroups0.gif - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:41 W3SVC1673937396 192.168.16.16 GET /central/images/01_Acceso0.gif - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:41 W3SVC1673937396 192.168.16.16 GET /central/Data/MC.js - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:46 W3SVC1673937396 192.168.16.16 GET /central/Images/fondo.jpg - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:46 W3SVC1673937396 192.168.16.16 GET /central/Images/MenuTopBgnd.gif - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:51 W3SVC1673937396 192.168.16.16 GET /central/DA_10.asp - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:51 W3SVC1673937396 192.168.16.16 GET /central/DA_100.asp uid=114031 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:57 W3SVC1673937396 192.168.16.16 GET /central/images/MC_Title.jpg - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
2008-07-13 00:03:57 W3SVC1673937396 192.168.16.16 GET /central/Data/MC.js - 80 - 26.26.12.1 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+6.0;+SLCC1;+.NET+CLR+2.0.50727;+Media+Center+PC+5.0;+.NET+CLR+3.0.04506) 200 0 0
        

These log files are pretty self-explaining. There are some lines starting with '#' that are meant to be comments, while the rest of the stuff is records with data.
There is always a line (fourth in this example) that explain what is expected to be found in every field, each field separated by a space character.
Empty fields are marked with a '-' character.

A single log file can contain thousands of lines of messy info. You could need to seek through many of these log files a hint of something not working as expected in a given web app.
These log files are created in a per day/per web site basis. Many different web apps can be published in a web site, and all the messages for all of them are written down to the same log file.
It is like we need some kind of tool to make our lives easier, if we want to find things in log files.
But, what do we want in such a tool?
To start with, we'll be needing to be able to isolate records for the only one app we are worried about.
Then, as we are fine tuning our searches, we'd appreciate the fact of being able of dynamically filter which records are to display. For instance, we'd like to see only the http requests existing for asp pages (not images or javascript files) of the foo web app, and dismiss everything else.
By now you may be thinking that this job is to be solved by applying some regular expressions.

Our python app is going to show the user a main window with several widgets on it:

  • a menu entry wich allows the user to load a log file to parse, via a QFileDialog
  • a menu entry wich allows the user to exit the app
  • a label that will be used to display info to the user
  • a text field for the user to be able to type in regular expressions, which will be applied to the parsing logic
  • a push button that starts the parsing itself
  • a tree widget to display the info parsed in a hierarchical view
  • Why a tree view? Come to think of it, for the sake of organization, of course.
    One of the fields in every record in the log file is the ip address of the client making an http request.
    We can expect -in a first thought- that 2 or more records with the same ip belong to the same user.
    So, to start with, we will put every ip address as an item of top level in the tree widget.
    Another piece of info given in the log file is the user agent (ie. the web browser used).
    It turns out that different people could be sharing ip address (for instance, users accessing the internet through the same proxy).
    Every web browser has its particular footprint, from the point of view of and http server, depending on several factors (wich web browser is being used, wich operating system, service-packs installed in the OS, plugins in the web browser, etc).
    So, if we take together the ip address and user agent, we can be reasonably sure that the different records in the log file which share ip address and user agent were originated by the same client (at least, in most cases it's likely to be like that).
    Therefore, every top level item (ip address) in our tree will have as many children as different browsers are using that ip.
    Finally, every item corresponding to a web browser will have as many children as different records exist in the log file, associated to this ip address and user agent, and that match with the regular expression entered to filter the records to show.
    Information displayed in this way in the tree view would allow us to follow the chain of http requests for each user while accessing the web app, which could be quite useful.

    Enough of the W3SVC files, let's create a Qt form. Back to eric4 ide, select the forms tab in the Project viewer. Right click on it and then select New form from the context menu. Next, select Main window and click on the Ok button. Then, in the save file dialog displayed, select Create new folder and name it ui.
    Note that instead of creating the ui folder at this point, you could have created it as a new package in the Sources tab. There is always more than 1 way to do it :-).
    Finally, name the form LogParser.ui
    The Project viewer shows the new file LogParser.ui. Double-click on it and the QtDesigner is started.
    We will use QtDesigner to design a main window for the app much like shown in the next image:

    Using QtDesigner is pretty straightforward. What I did to get the dialog arranged was:

  • Right click on it and then select Add menu. Then by clicking in the menu you can add as many menu items as you need. I created an item for File/Open log file, a separator and an item for File/Quit. I did also create an item for Help/About...
  • Drag and drop a QLabel widget.
  • Below the QLabel, I drag and dropped a QLineEdit and a QPushButton.
  • Below all of them, I drag and dropped a QTreeWidget.
  • In the previous image, I've annotated which names I have used for every widget. Naming conventions is always an important subject, which we are not going to discuss here. But besides, when using QtDesigner the names also have influence on the signal/slot functions naming.
    Perhaps the most tricky part of the QtDesigner is the layout for the widgets, if you are not used to it. What I did was to click on the QLineEdit, and while holding the shift key pressed, I clicked the QPushButton, so both items are selected. Then I clicked on the Lay out horizontally button. The last thing I did was to right click on a free area of the main window and select Layout/Lay out vertically so all the widgets are included in a vertical layout.

    Save changes and quit the QtDesigner.
    Back to eric4, right click on LogParser.ui and select Compile form. This action will take the .ui file as input (which is xml) and generate python code that displays the form we have just created. A new file is created (ui/UI_LogParser.py), and displayed in the Project viewer, sources tab. This file contains a class (UI_MainWindow) implemented in good python, but it's auto-generated and it will be overwritten everytime we need to do a compilation of the .ui file.
    Therefore, seems like a sensible idea to have a class of our own that extends the autogenerated class functionality, and that is not going to be overwritten by the ui-compiler.
    Fortunately, eric4 allows to do exactly that in a very easy way. Just right click on the .ui file and select Generate dialog code from the context menu.

    In the Forms code generator dialog, click the New... button to create a new class, and in the new dialog shown you can provide the class name, the filename and the path.
    Next, we have to click on the actions we want to implement. The actions, signals and slots names are based on the widgets names. That's why I've provided (in a previous image) the names I've used.
    I've checked the items:

  • on_actionOpen_Log_File_triggered, which will be invoked in response to the user's action File/Open log File
  • on_action_Quit_triggered, invoked in response to File/Quit
  • on_action_About_triggered, for Help/About
  • on_btnParse_clicked, invoked when the users presses the Parse log file button.


  • Important note: all the potentially sensitive data in the tutorial (mainly ip's or id's) have been faked; they aren't real data at all.

    Chapter 1
    Chapter 3