iwla

iwla Commit Details

Date:2020-04-15 14:44:11 (3 months 19 days ago)
Author:Grégory Soutadé
Branch:dev, master
Commit:954b4972a9980f957156d984dbc45fba6b0b9997
Parents: e71f1de7dc04e06f0cade86b3e0722d2a3c6afd9
Message:Update ChangeLog and Documentation

Changes:
MChangeLog (1 diff)
Mdocs/index.md (6 diffs)
Mdocs/modules.md (4 diffs)
Miwla.py (1 diff)

File differences

ChangeLog
1
1
22
33
4
5
6
7
8
9
10
11
12
413
514
615
7
16
17
818
9
19
20
1021
1122
1223
v0.5 (25/05/2017)
v0.5 (15/04/2020)
** User **
Add --dry-run (-D) argument
Add more rules for robot detection :
More than ten 404 pages viewed
No page and no Hit
Pages without hit
New format for (not_)viewed pages/hits and bandwidth that are now recorded by day (in a dictionnary were only element 0 is initialized). Element 0 is the total. WARNING : not backward compatible with previous databases.
Sync data with awstat (develop branch : 7.7+)
Make backup before compressing (low memory servers)
Add top_pages_diff plugin
Add IP exclusion feature
** Dev **
Use cPickle instead of pickle
Don't save all robots requests (only first pass is kept) which allow to save a large amount of memory/disk space
Add one more rule to robot detection : more than ten 404 pages viewed
Handle URLs with empty referer
Don't try to find search engine on robots
** Bugs **
Fix KeyError : geo attribute can not exists
Call post hook plugins even in display only mode
v0.4 (29/01/2017)
** User **
docs/index.md
1111
1212
1313
14
14
1515
1616
1717
......
2020
2121
2222
23
23
24
2425
2526
2627
......
113114
114115
115116
117
116118
117119
118120
......
145147
146148
147149
148
150
151
149152
150153
151154
......
189192
190193
191194
192
193
194
195
196
195
196
197
198
199
197200
198201
199202
......
573576
574577
575578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
576605
577606
578607
Usage
-----
./iwla [-c|--clean-output] [-i|--stdin] [-f FILE|--file FILE] [-d LOGLEVEL|--log-level LOGLEVEL] [-r|--reset year/month] [-z|--dont-compress] [-p]
./iwla [-c|--clean-output] [-i|--stdin] [-f FILE|--file FILE] [-d LOGLEVEL|--log-level LOGLEVEL] [-r|--reset year/month] [-z|--dont-compress] [-p] [-D|--dry-run]
-c : Clean output (database and HTML) before starting
-i : Read data from stdin instead of conf.analyzed_filename
-r : Reset analysis to a specific date (month/year)
-z : Don't compress databases (bigger but faster, not compatible with compressed databases)
-p : Only generate display
-d : Dry run (don't write/update files to disk)
Basic usage
-----------
* plugins/display/top_downloads_diff.py
* plugins/display/top_downloads.py
* plugins/display/top_hits.py
* plugins/display/top_pages_diff.py
* plugins/display/top_pages.py
* plugins/display/top_visitors.py
* plugins/display/track_users.py
analyzed_filename
domain_name
locales_path
compress_output_files*
compress_output_files
excluded_ip
Output files :
DB_ROOT/meta.db
remote_addr =>
remote_addr
remote_ip
viewed_pages
viewed_hits
not_viewed_pages
not_viewed_hits
bandwidth
viewed_pages{0..31} # 0 contains total
viewed_hits{0..31} # 0 contains total
not_viewed_pages{0..31}
not_viewed_hits{0..31}
bandwidth{0..31}
last_access
requests =>
[fields_from_format_log]
None
plugins.display.top_pages_diff
------------------------------
Display hook
Enlight new and updated pages in in top_pages.html
Plugin requirements :
display/top_pages
Conf values needed :
None
Output files :
None
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
plugins.display.top_pages
-------------------------
docs/modules.md
1212
1313
1414
15
1516
1617
1718
......
4445
4546
4647
47
48
49
4850
4951
5052
......
8890
8991
9092
91
92
93
94
95
93
94
95
96
97
9698
9799
98100
......
472474
473475
474476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
475503
476504
477505
* plugins/display/top_downloads_diff.py
* plugins/display/top_downloads.py
* plugins/display/top_hits.py
* plugins/display/top_pages_diff.py
* plugins/display/top_pages.py
* plugins/display/top_visitors.py
* plugins/display/track_users.py
analyzed_filename
domain_name
locales_path
compress_output_files*
compress_output_files
excluded_ip
Output files :
DB_ROOT/meta.db
remote_addr =>
remote_addr
remote_ip
viewed_pages
viewed_hits
not_viewed_pages
not_viewed_hits
bandwidth
viewed_pages{0..31} # 0 contains total
viewed_hits{0..31} # 0 contains total
not_viewed_pages{0..31}
not_viewed_hits{0..31}
bandwidth{0..31}
last_access
requests =>
[fields_from_format_log]
None
plugins.display.top_pages_diff
------------------------------
Display hook
Enlight new and updated pages in in top_pages.html
Plugin requirements :
display/top_pages
Conf values needed :
None
Output files :
None
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
plugins.display.top_pages
-------------------------
iwla.py
5151
5252
5353
54
54
55
5556
5657
5758
analyzed_filename
domain_name
locales_path
compress_output_files*
compress_output_files
excluded_ip
Output files :
DB_ROOT/meta.db

Archive Download the corresponding diff file

Branches

Tags