iwla

iwla Commit Details

Date:2014-12-31 14:52:14 (6 years 6 months ago)
Author:Grégory Soutadé
Branch:dev, master
Commit:2b0fd9fb46e833ae62b5845bd2c1a07722841fcc
Parents: f1fb8cb674e5cb0931f79a051cc35df415d5daab
Message:Generate index table in documentation

Changes:
Mdocs/index.md (11 diffs)
Mdocs/modules.md (11 diffs)
Mtools/extract_doc.py (2 diffs)
Mtools/extract_docs.sh (1 diff)

File differences

docs/index.md
9090
9191
9292
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
93110
94111
95112
......
110127
111128
112129
130
113131
114132
115133
......
233251
234252
235253
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
254
255
264256
265257
266258
267
259
268260
269261
270
262
271263
272264
273
274
265
266
275267
276268
277
269
278270
279271
280272
......
315307
316308
317309
318
319
310
311
320312
321313
322314
323
315
324316
325317
326
318
327319
328320
329
330
321
322
331323
332324
333
325
334326
335327
336328
......
343335
344336
345337
346
347
338
339
348340
349
350
341
342
343
351344
352345
353346
354347
355348
356
357
349
358350
359351
360
352
361353
362354
363355
364356
365357
366
367
368
358
369359
370360
371361
372362
373363
374
375
364
365
376366
377
367
378368
379
369
380370
381371
382
372
383373
384374
385
386
375
387376
388377
389378
......
392381
393382
394383
395
396
397
384
398385
399386
400387
......
437424
438425
439426
440
441
427
428
442429
443430
444431
445
432
446433
447434
448435
449436
450437
451
438
452439
453440
454441
......
457444
458445
459446
460
461
462
447
448
449
450
463451
464452
465453
466454
467455
468
469
456
457
470458
471459
472460
473
461
474462
475463
476464
477465
478466
479
467
480468
481469
482470
......
485473
486474
487475
488
489
490
491
476
477
478
492479
493480
494481
......
522509
523510
524511
525
526
512
513
527514
528515
529516
530
517
531518
532519
533520
......
543530
544531
545532
546
533
547534
548535
549536
550537
551538
552539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
Optional configuration values ends with *.
* iwla.py
* plugins/display/all_visits.py
* plugins/display/referers.py
* plugins/display/top_downloads.py
* plugins/display/top_hits.py
* plugins/display/top_pages.py
* plugins/display/top_visitors.py
* plugins/display/referers_diff.py
* plugins/post_analysis/referers.py
* plugins/post_analysis/reverse_dns.py
* plugins/post_analysis/top_downloads.py
* plugins/post_analysis/top_hits.py
* plugins/post_analysis/top_pages.py
* plugins/pre_analysis/page_to_hit.py
* plugins/pre_analysis/robots.py
iwla
----
DB_ROOT/meta.db
DB_ROOT/year/month/iwla.db
OUTPUT_ROOT/index.html
OUTPUT_ROOT/year/_stats.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.top_visitors
----------------------------
Display hook
Create TOP visitors block
Plugin requirements :
None
Conf values needed :
display_visitor_ip*
Output files :
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
plugins.display.top_pages
-------------------------
plugins.display.top_downloads
-----------------------------
Display hook
Create TOP pages page
Create TOP downloads page
Plugin requirements :
post_analysis/top_pages
post_analysis/top_downloads
Conf values needed :
max_pages_displayed*
create_all_pages_page*
max_downloads_displayed*
create_all_downloads_page*
Output files :
OUTPUT_ROOT/year/month/top_pages.html
OUTPUT_ROOT/year/month/top_downloads.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.top_downloads
-----------------------------
plugins.display.top_pages
-------------------------
Display hook
Create TOP downloads page
Create TOP pages page
Plugin requirements :
post_analysis/top_downloads
post_analysis/top_pages
Conf values needed :
max_downloads_displayed*
create_all_downloads_page*
max_pages_displayed*
create_all_pages_page*
Output files :
OUTPUT_ROOT/year/month/top_downloads.html
OUTPUT_ROOT/year/month/top_pages.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.pre_analysis.page_to_hit
--------------------------------
plugins.display.top_visitors
----------------------------
Pre analysis hook
Change page into hit and hit into page into statistics
Display hook
Create TOP visitors block
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
display_visitor_ip*
Output files :
None
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
visits :
remote_addr =>
is_page
None
Statistics deletion :
None
plugins.pre_analysis.robots
---------------------------
plugins.display.referers_diff
-----------------------------
Pre analysis hook
Display hook
Filter robots
Enlight new and updated key phrases in in all_key_phrases.html
Plugin requirements :
None
display/referers
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
None
Output files :
None
None
Statistics update :
visits :
remote_addr =>
robot
None
Statistics deletion :
None
None
plugins.post_analysis.top_pages
-------------------------------
plugins.post_analysis.reverse_dns
---------------------------------
Post analysis hook
Count TOP pages
Replace IP by reverse DNS names
Plugin requirements :
None
Conf values needed :
None
reverse_dns_timeout*
Output files :
None
None
Statistics update :
month_stats:
top_pages =>
uri
valid_visitors:
remote_addr
dns_name_replaced
dns_analyzed
Statistics deletion :
None
plugins.post_analysis.reverse_dns
---------------------------------
plugins.post_analysis.top_downloads
-----------------------------------
Post analysis hook
Replace IP by reverse DNS names
Count TOP downloads
Plugin requirements :
None
Conf values needed :
reverse_dns_timeout*
None
Output files :
None
None
Statistics update :
valid_visitors:
remote_addr
dns_name_replaced
dns_analyzed
month_stats:
top_downloads =>
uri
Statistics deletion :
None
None
plugins.post_analysis.top_downloads
-----------------------------------
plugins.post_analysis.top_pages
-------------------------------
Post analysis hook
Count TOP downloads
Count TOP pages
Plugin requirements :
None
Statistics update :
month_stats:
top_downloads =>
top_pages =>
uri
Statistics deletion :
None
plugins.pre_analysis.page_to_hit
--------------------------------
Pre analysis hook
Change page into hit and hit into page into statistics
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
Output files :
None
Statistics creation :
None
Statistics update :
visits :
remote_addr =>
is_page
Statistics deletion :
None
plugins.pre_analysis.robots
---------------------------
Pre analysis hook
Filter robots
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
Output files :
None
Statistics creation :
None
Statistics update :
visits :
remote_addr =>
robot
Statistics deletion :
None
docs/modules.md
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
118
219
320
......
1835
1936
2037
38
2139
2240
2341
......
141159
142160
143161
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
162
163
172164
173165
174166
175
167
176168
177169
178
170
179171
180172
181
182
173
174
183175
184176
185
177
186178
187179
188180
......
223215
224216
225217
226
227
218
219
228220
229221
230222
231
223
232224
233225
234
226
235227
236228
237
238
229
230
239231
240232
241
233
242234
243235
244236
......
251243
252244
253245
254
255
246
247
256248
257
258
249
250
251
259252
260253
261254
262255
263256
264
265
257
266258
267259
268
260
269261
270262
271263
272264
273265
274
275
276
266
277267
278268
279269
280270
281271
282
283
272
273
284274
285
275
286276
287
277
288278
289279
290
280
291281
292282
293
294
283
295284
296285
297286
......
300289
301290
302291
303
304
305
292
306293
307294
308295
......
345332
346333
347334
348
349
335
336
350337
351338
352339
353
340
354341
355342
356343
357344
358345
359
346
360347
361348
362349
......
365352
366353
367354
368
369
370
355
356
357
358
371359
372360
373361
374362
375363
376
377
364
365
378366
379367
380368
381
369
382370
383371
384372
385373
386374
387
375
388376
389377
390378
......
393381
394382
395383
396
397
398
399
384
385
386
400387
401388
402389
......
430417
431418
432419
433
434
420
421
435422
436423
437424
438
425
439426
440427
441428
......
451438
452439
453440
454
441
455442
456443
457444
458445
459446
460447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
* iwla.py
* plugins/display/all_visits.py
* plugins/display/referers.py
* plugins/display/top_downloads.py
* plugins/display/top_hits.py
* plugins/display/top_pages.py
* plugins/display/top_visitors.py
* plugins/display/referers_diff.py
* plugins/post_analysis/referers.py
* plugins/post_analysis/reverse_dns.py
* plugins/post_analysis/top_downloads.py
* plugins/post_analysis/top_hits.py
* plugins/post_analysis/top_pages.py
* plugins/pre_analysis/page_to_hit.py
* plugins/pre_analysis/robots.py
iwla
----
DB_ROOT/meta.db
DB_ROOT/year/month/iwla.db
OUTPUT_ROOT/index.html
OUTPUT_ROOT/year/_stats.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.top_visitors
----------------------------
Display hook
Create TOP visitors block
Plugin requirements :
None
Conf values needed :
display_visitor_ip*
Output files :
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
None
Statistics deletion :
None
plugins.display.top_pages
-------------------------
plugins.display.top_downloads
-----------------------------
Display hook
Create TOP pages page
Create TOP downloads page
Plugin requirements :
post_analysis/top_pages
post_analysis/top_downloads
Conf values needed :
max_pages_displayed*
create_all_pages_page*
max_downloads_displayed*
create_all_downloads_page*
Output files :
OUTPUT_ROOT/year/month/top_pages.html
OUTPUT_ROOT/year/month/top_downloads.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.display.top_downloads
-----------------------------
plugins.display.top_pages
-------------------------
Display hook
Create TOP downloads page
Create TOP pages page
Plugin requirements :
post_analysis/top_downloads
post_analysis/top_pages
Conf values needed :
max_downloads_displayed*
create_all_downloads_page*
max_pages_displayed*
create_all_pages_page*
Output files :
OUTPUT_ROOT/year/month/top_downloads.html
OUTPUT_ROOT/year/month/top_pages.html
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
plugins.pre_analysis.page_to_hit
--------------------------------
plugins.display.top_visitors
----------------------------
Pre analysis hook
Change page into hit and hit into page into statistics
Display hook
Create TOP visitors block
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
display_visitor_ip*
Output files :
None
OUTPUT_ROOT/year/month/index.html
Statistics creation :
None
Statistics update :
visits :
remote_addr =>
is_page
None
Statistics deletion :
None
plugins.pre_analysis.robots
---------------------------
plugins.display.referers_diff
-----------------------------
Pre analysis hook
Display hook
Filter robots
Enlight new and updated key phrases in in all_key_phrases.html
Plugin requirements :
None
display/referers
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
None
Output files :
None
None
Statistics update :
visits :
remote_addr =>
robot
None
Statistics deletion :
None
None
plugins.post_analysis.top_pages
-------------------------------
plugins.post_analysis.reverse_dns
---------------------------------
Post analysis hook
Count TOP pages
Replace IP by reverse DNS names
Plugin requirements :
None
Conf values needed :
None
reverse_dns_timeout*
Output files :
None
None
Statistics update :
month_stats:
top_pages =>
uri
valid_visitors:
remote_addr
dns_name_replaced
dns_analyzed
Statistics deletion :
None
plugins.post_analysis.reverse_dns
---------------------------------
plugins.post_analysis.top_downloads
-----------------------------------
Post analysis hook
Replace IP by reverse DNS names
Count TOP downloads
Plugin requirements :
None
Conf values needed :
reverse_dns_timeout*
None
Output files :
None
None
Statistics update :
valid_visitors:
remote_addr
dns_name_replaced
dns_analyzed
month_stats:
top_downloads =>
uri
Statistics deletion :
None
None
plugins.post_analysis.top_downloads
-----------------------------------
plugins.post_analysis.top_pages
-------------------------------
Post analysis hook
Count TOP downloads
Count TOP pages
Plugin requirements :
None
Statistics update :
month_stats:
top_downloads =>
top_pages =>
uri
Statistics deletion :
None
plugins.pre_analysis.page_to_hit
--------------------------------
Pre analysis hook
Change page into hit and hit into page into statistics
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
Output files :
None
Statistics creation :
None
Statistics update :
visits :
remote_addr =>
is_page
Statistics deletion :
None
plugins.pre_analysis.robots
---------------------------
Pre analysis hook
Filter robots
Plugin requirements :
None
Conf values needed :
page_to_hit_conf*
hit_to_page_conf*
Output files :
None
Statistics creation :
None
Statistics update :
visits :
remote_addr =>
robot
Statistics deletion :
None
tools/extract_doc.py
55
66
77
8
9
10
11
12
813
914
1015
......
1419
1520
1621
22
23
24
25
1726
1827
1928
excludes = ['istats_diff.py']
filename = sys.argv[1]
printName = False
if filename == '-p':
filename = sys.argv[2]
printName = True
if filename.endswith('__init__.py'):
sys.exit(0)
sys.stderr.write('\tSkip %s\n' % (filename))
sys.exit(0)
if printName:
sys.stdout.write(' * %s\n' % (filename))
sys.exit(0)
package_name = filename.replace('/', '.').replace('.py', '')
sys.stdout.write('%s' % (package_name))
sys.stdout.write('\n')
tools/extract_docs.sh
66
77
88
9
10
11
12
13
914
10
15
1116
1217
1318
rm -f "${MODULES_TARGET}"
echo "Generate plugins index"
python tools/extract_doc.py -p iwla.py > "${MODULES_TARGET}"
find plugins -name '*.py' -exec python tools/extract_doc.py -p \{\} \; >> "${MODULES_TARGET}"
echo "\n" >> "${MODULES_TARGET}"
echo "Generate doc from iwla.py"
python tools/extract_doc.py iwla.py > "${MODULES_TARGET}"
python tools/extract_doc.py iwla.py >> "${MODULES_TARGET}"
echo "Generate plugins documentation"
find plugins -name '*.py' -exec python tools/extract_doc.py \{\} \; >> "${MODULES_TARGET}"

Archive Download the corresponding diff file

Branches

Tags