iwla

iwla Commit Details

Date:2014-12-19 11:35:27 (6 years 7 months ago)
Author:Grégory Soutadé
Branch:dev, master
Commit:f92cec4428b15dec25af1058f3df40ee1dc0a672
Parents: a37b661c1fdfd7a970b8b7f6861fa220657c4839
Message:Add Markdown documentation

Changes:
Adocs/main.md (full)

File differences

docs/main.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
iwla
====
Introduction
------------
iwla (Intelligent Web Log Analyzer) is basically a clone of [awstats](http://www.awstats.org). The main problem with awstats is that it's a very monolothic project with everything in one big PERL file. In opposite, iwla has be though to be very modulor : a small core analysis and a lot of filters. It can be viewed as UNIX pipes. Philosophy of iwla is : add, update, delete ! That's the job of each filters : modify statistics until final result.
Nevertheless, iwla is only focused on HTTP logs. It uses data (robots definitions, search engines definitions) and design from awstats. Moreover, it's not dynamic, but only generates static HTML page (with gzip compression option).
Usage
-----
./iwla [-c|--clean-output] [-i|--stdin] [-f FILE|--file FILE] [-d LOGLEVEL|--log-level LOGLEVEL]
-c : Clean output (database and HTML) before starting
-i : Read data from stdin instead of conf.analyzed_filename
-f : Read data from FILE instead of conf.analyzed_filename
-d : Loglevel in ['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL']
Basic usage
-----------
In addition to command line, iwla read parameters in _ default_conf.py _. User can override default values using _conf.py_ file. Each module requires its own parameters.
Main valued to edit are :
analyzed_filename : web server log
domaine_name : domain name to filter
pre_analysis_hooks
post_analysis_hooks
display_hooks
locale
You can then launch iwla. Output HTML files are created in _output_ directory by default. To quickly see it go in output and type
python -m SimpleHTTPServer 8000
Open your favorite web browser at _http://localhost:8000_. Enjoy !
Interesting default configuration values
----------------------------------------
DB_ROOT : Default database directory
DISPLAY_ROOT : Default HTML output directory
log_format : Web server log format (nginx style). Default is what apache log
time_format : Time format used in log format
pages_extensions : Extensions that are considered as a HTML page (or result)
viewed_http_codes : HTTP codes that are cosidered OK
count_hit_only_visitors : If False, doesn't cout visitors that doesn't GET a page but resources only (images, rss...)
multimedia_files : Multimedia extensions (not accounted as downloaded files)
css_path : CSS path (you can add yours)
compress_output_files : Extensions to compress in gzip during display build
Plugins
-------
As previously described, plugins acts like UNIX pipes : final statistics are constantly updated by each plugin to produce final result. We have three type of plugins :
Pre analysis plugins : Called before generating days statistics. They are in charge to filter robots, crawlers, bad pages...
Post analysis plugins : Called after basic statistics computation. They are in charge to enlight them with each own algorithms
Display plugins : They are in charge to produce HTML files from statistics.
To use plugins, just insert their name in pre_analysis_hooks, post_analysis_hooks and display_hooks list.
Create a Plugins
----------------
To create a new plugin, it's necessary to create a derived class of IPlugin (_iplugin.py) in the right directory (_plugins/xxx/your_plugin.py_).
Plugins can defines required configuration values (self.conf_requires) that must be set in conf.py (or can be optional). They can also defines required plugins (self.requires).
For display plugins, a lot of code has been wrote in _display.py_ that simplify the creation on HTML blocks, tables and graphs.
Modules
-------
Optional configuration values ends with *.

Archive Download the corresponding diff file

Branches

Tags