iwla

iwla Commit Details

Date:2014-12-31 14:09:12 (6 years 6 months ago)
Author:Grégory Soutadé
Branch:dev, master
Commit:8299a7d785c09ba1b668d96704f2bcef410014d4
Parents: 5ed9bbbae8d7b5287ce6b02112fa1c443a2cb780
Message:Update doc

Changes:
Mdocs/index.md (14 diffs)
Mdocs/modules.md (14 diffs)
Miwla.py (1 diff)
Mtools/extract_doc.py (1 diff)

File differences

docs/index.md
110110
111111
112112
113
113114
114115
115116
......
175176
176177
177178
178
179
179
180
180181
181182
182183
183
184
184185
185186
186187
......
189190
190191
191192
192
193193
194194
195195
......
202202
203203
204204
205
206
205
206
207207
208208
209209
210
210
211211
212212
213
213
214214
215215
216
217
218
219
216
220217
221218
222
223
219
224220
225221
226222
......
233229
234230
235231
236
237
232
233
238234
239235
240236
241
237
242238
243239
244
240
245241
246242
247
243
244
245
246
248247
249248
249
250
250251
251252
252253
......
287288
288289
289290
290
291
291
292
292293
293294
294295
295
296
296297
297298
298
299
299300
300301
301
302
302
303
303304
304305
305
306
306307
307308
308309
......
315316
316317
317318
318
319
319
320
320321
321322
322323
323
324
324325
325326
326
327
327328
328329
329
330
330
331
331332
332333
333
334
334335
335336
336337
......
343344
344345
345346
346
347
347
348
348349
349
350
350
351
352
351353
352354
353
355
354356
355357
356
357
358
358359
359360
360361
......
363364
364365
365366
366
367
368
367
369368
370369
371370
372371
373372
374
375
373
374
376375
377
376
378377
379
378
380379
381380
382381
383382
384383
385
386
384
387385
388386
389387
......
392390
393391
394392
395
396
397
393
394
395
396
398397
399398
400399
......
465464
466465
467466
468
469
467
468
470469
471470
472471
473
472
474473
475474
476475
477476
478477
479
478
480479
481480
482481
......
485484
486485
487486
488
489
490
491
487
488
489
492490
493491
494492
......
522520
523521
524522
525
526
523
524
527525
528
526
529527
530
528
531529
532530
533531
534532
535533
534
535
536
537
538
539
540
536541
537542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
538565
539566
540567
......
542569
543570
544571
545
546
547
572
573
574
548575
549576
550577
DB_ROOT/meta.db
DB_ROOT/year/month/iwla.db
OUTPUT_ROOT/index.html
OUTPUT_ROOT/year/_stats.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.all_visits
--------------------------
plugins.display.top_visitors
----------------------------
Display hook
Create All visits page
Create TOP visitors block
Plugin requirements :
None
display_visitor_ip*
Output files :
OUTPUT_ROOT/year/month/all_visits.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.referers
------------------------
plugins.display.all_visits
--------------------------
Display hook
Create Referers page
Create All visits page
Plugin requirements :
post_analysis/referers
None
Conf values needed :
max_referers_displayed*
create_all_referers_page*
max_key_phrases_displayed*
create_all_key_phrases_page*
display_visitor_ip*
Output files :
OUTPUT_ROOT/year/month/referers.html
OUTPUT_ROOT/year/month/key_phrases.html
OUTPUT_ROOT/year/month/all_visits.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.top_visitors
----------------------------
plugins.display.referers
------------------------
Display hook
Create TOP visitors block
Create Referers page
Plugin requirements :
None
post_analysis/referers
Conf values needed :
display_visitor_ip*
max_referers_displayed*
create_all_referers_page*
max_key_phrases_displayed*
create_all_key_phrases_page*
Output files :
OUTPUT_ROOT/year/month/referers.html
OUTPUT_ROOT/year/month/key_phrases.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.top_hits
------------------------
plugins.display.top_downloads
-----------------------------
Display hook
Create TOP hits page
Create TOP downloads page
Plugin requirements :
post_analysis/top_hits
post_analysis/top_downloads
Conf values needed :
max_hits_displayed*
create_all_hits_page*
max_downloads_displayed*
create_all_downloads_page*
Output files :
OUTPUT_ROOT/year/month/top_hits.html
OUTPUT_ROOT/year/month/top_downloads.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.top_downloads
-----------------------------
plugins.display.top_hits
------------------------
Display hook
Create TOP downloads page
Create TOP hits page
Plugin requirements :
post_analysis/top_downloads
post_analysis/top_hits
Conf values needed :
max_downloads_displayed*
create_all_downloads_page*
max_hits_displayed*
create_all_hits_page*
Output files :
OUTPUT_ROOT/year/month/top_downloads.html
OUTPUT_ROOT/year/month/top_hits.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.pre_analysis.page_to_hit
--------------------------------
plugins.display.referers_diff
-----------------------------
Pre analysis hook
Change page into hit and hit into page into statistics
Display hook
Enlight new and updated key phrases in in all_key_phrases.html
Plugin requirements :
None
display/referers
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
None
Output files :
None
None
Statistics update :
visits :
remote_addr =>
is_page
None
Statistics deletion :
None
plugins.pre_analysis.robots
---------------------------
plugins.post_analysis.reverse_dns
---------------------------------
Pre analysis hook
Post analysis hook
Filter robots
Replace IP by reverse DNS names
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
reverse_dns_timeout*
Output files :
None
None
Statistics update :
visits :
remote_addr =>
robot
valid_visitors:
remote_addr
dns_name_replaced
dns_analyzed
Statistics deletion :
None
None
plugins.post_analysis.reverse_dns
---------------------------------
plugins.post_analysis.top_downloads
-----------------------------------
Post analysis hook
Replace IP by reverse DNS names
Count TOP downloads
Plugin requirements :
None
Conf values needed :
reverse_dns_timeout*
None
Output files :
None
None
Statistics update :
valid_visitors:
remote_addr
dns_name_replaced
dns_analyzed
month_stats:
top_downloads =>
uri
Statistics deletion :
None
None
plugins.post_analysis.top_downloads
-----------------------------------
plugins.pre_analysis.robots
---------------------------
Post analysis hook
Pre analysis hook
Count TOP downloads
Filter robots
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
Output files :
None
Statistics creation :
None
Statistics update :
visits :
remote_addr =>
robot
Statistics deletion :
None
plugins.pre_analysis.page_to_hit
--------------------------------
Pre analysis hook
Change page into hit and hit into page into statistics
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
Output files :
None
None
Statistics update :
month_stats:
top_downloads =>
uri
visits :
remote_addr =>
is_page
Statistics deletion :
None
docs/modules.md
1818
1919
2020
21
2122
2223
2324
......
8384
8485
8586
86
87
87
88
8889
8990
9091
91
92
9293
9394
9495
......
9798
9899
99100
100
101101
102102
103103
......
110110
111111
112112
113
114
113
114
115115
116116
117117
118
118
119119
120120
121
121
122122
123123
124
125
126
127
124
128125
129126
130
131
127
132128
133129
134130
......
141137
142138
143139
144
145
140
141
146142
147143
148144
149
145
150146
151147
152
148
153149
154150
155
151
152
153
154
156155
157156
157
158
158159
159160
160161
......
195196
196197
197198
198
199
199
200
200201
201202
202203
203
204
204205
205206
206
207
207208
208209
209
210
210
211
211212
212213
213
214
214215
215216
216217
......
223224
224225
225226
226
227
227
228
228229
229230
230231
231
232
232233
233234
234
235
235236
236237
237
238
238
239
239240
240241
241
242
242243
243244
244245
......
251252
252253
253254
254
255
255
256
256257
257
258
258
259
260
259261
260262
261
263
262264
263265
264
265
266
266267
267268
268269
......
271272
272273
273274
274
275
276
275
277276
278277
279278
280279
281280
282
283
281
282
284283
285
284
286285
287
286
288287
289288
290289
291290
292291
293
294
292
295293
296294
297295
......
300298
301299
302300
303
304
305
301
302
303
304
306305
307306
308307
......
373372
374373
375374
376
377
375
376
378377
379378
380379
381
380
382381
383382
384383
385384
386385
387
386
388387
389388
390389
......
393392
394393
395394
396
397
398
399
395
396
397
400398
401399
402400
......
430428
431429
432430
433
434
431
432
435433
436
434
437435
438
436
439437
440438
441439
442440
443441
442
443
444
445
446
447
448
444449
445450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
446473
447474
448475
......
450477
451478
452479
453
454
455
480
481
482
456483
457484
458485
DB_ROOT/meta.db
DB_ROOT/year/month/iwla.db
OUTPUT_ROOT/index.html
OUTPUT_ROOT/year/_stats.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.all_visits
--------------------------
plugins.display.top_visitors
----------------------------
Display hook
Create All visits page
Create TOP visitors block
Plugin requirements :
None
display_visitor_ip*
Output files :
OUTPUT_ROOT/year/month/all_visits.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.referers
------------------------
plugins.display.all_visits
--------------------------
Display hook
Create Referers page
Create All visits page
Plugin requirements :
post_analysis/referers
None
Conf values needed :
max_referers_displayed*
create_all_referers_page*
max_key_phrases_displayed*
create_all_key_phrases_page*
display_visitor_ip*
Output files :
OUTPUT_ROOT/year/month/referers.html
OUTPUT_ROOT/year/month/key_phrases.html
OUTPUT_ROOT/year/month/all_visits.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.top_visitors
----------------------------
plugins.display.referers
------------------------
Display hook
Create TOP visitors block
Create Referers page
Plugin requirements :
None
post_analysis/referers
Conf values needed :
display_visitor_ip*
max_referers_displayed*
create_all_referers_page*
max_key_phrases_displayed*
create_all_key_phrases_page*
Output files :
OUTPUT_ROOT/year/month/referers.html
OUTPUT_ROOT/year/month/key_phrases.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.top_hits
------------------------
plugins.display.top_downloads
-----------------------------
Display hook
Create TOP hits page
Create TOP downloads page
Plugin requirements :
post_analysis/top_hits
post_analysis/top_downloads
Conf values needed :
max_hits_displayed*
create_all_hits_page*
max_downloads_displayed*
create_all_downloads_page*
Output files :
OUTPUT_ROOT/year/month/top_hits.html
OUTPUT_ROOT/year/month/top_downloads.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.top_downloads
-----------------------------
plugins.display.top_hits
------------------------
Display hook
Create TOP downloads page
Create TOP hits page
Plugin requirements :
post_analysis/top_downloads
post_analysis/top_hits
Conf values needed :
max_downloads_displayed*
create_all_downloads_page*
max_hits_displayed*
create_all_hits_page*
Output files :
OUTPUT_ROOT/year/month/top_downloads.html
OUTPUT_ROOT/year/month/top_hits.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.pre_analysis.page_to_hit
--------------------------------
plugins.display.referers_diff
-----------------------------
Pre analysis hook
Change page into hit and hit into page into statistics
Display hook
Enlight new and updated key phrases in in all_key_phrases.html
Plugin requirements :
None
display/referers
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
None
Output files :
None
None
Statistics update :
visits :
remote_addr =>
is_page
None
Statistics deletion :
None
plugins.pre_analysis.robots
---------------------------
plugins.post_analysis.reverse_dns
---------------------------------
Pre analysis hook
Post analysis hook
Filter robots
Replace IP by reverse DNS names
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
reverse_dns_timeout*
Output files :
None
None
Statistics update :
visits :
remote_addr =>
robot
valid_visitors:
remote_addr
dns_name_replaced
dns_analyzed
Statistics deletion :
None
None
plugins.post_analysis.reverse_dns
---------------------------------
plugins.post_analysis.top_downloads
-----------------------------------
Post analysis hook
Replace IP by reverse DNS names
Count TOP downloads
Plugin requirements :
None
Conf values needed :
reverse_dns_timeout*
None
Output files :
None
None
Statistics update :
valid_visitors:
remote_addr
dns_name_replaced
dns_analyzed
month_stats:
top_downloads =>
uri
Statistics deletion :
None
None
plugins.post_analysis.top_downloads
-----------------------------------
plugins.pre_analysis.robots
---------------------------
Post analysis hook
Pre analysis hook
Count TOP downloads
Filter robots
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
Output files :
None
Statistics creation :
None
Statistics update :
visits :
remote_addr =>
robot
Statistics deletion :
None
plugins.pre_analysis.page_to_hit
--------------------------------
Pre analysis hook
Change page into hit and hit into page into statistics
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
Output files :
None
None
Statistics update :
month_stats:
top_downloads =>
uri
visits :
remote_addr =>
is_page
Statistics deletion :
None
iwla.py
130130
131131
132132
133
133
134134
135135
136136
ANALYSIS_CLASS = 'HTTP'
API_VERSION = 1
IWLA_VERSION = '0.2'
IWLA_VERSION = '0.2-dev'
def __init__(self, logLevel):
self.meta_infos = {}
tools/extract_doc.py
22
33
44
5
6
57
68
79
810
911
12
13
14
15
16
1017
1118
1219
import sys
excludes = ['istats_diff.py']
filename = sys.argv[1]
if filename.endswith('__init__.py'):
sys.exit(0)
for e in excludes:
if filename.endswith(e):
sys.stderr.write('\tSkip %s\n' % (filename))
sys.exit(0)
package_name = filename.replace('/', '.').replace('.py', '')
sys.stdout.write('%s' % (package_name))
sys.stdout.write('\n')

Archive Download the corresponding diff file

Branches

Tags