iwla

iwla Commit Details

Date:2016-01-18 07:33:48 (5 years 6 months ago)
Author:Grégory Soutadé
Branch:dev, master
Commit:519033e474800f6d93ae54ccac299268f67a4edd
Parents: 5c750e8b0c406f4d7cd3c1aac2282ea4deffe9b3
Message:Do reverse DNS on feeds parsers

Changes:
MChangeLog (2 diffs)
Miwla.py (2 diffs)
Mplugins/post_analysis/reverse_dns.py (1 diff)
Mtools/iwla_convert.pl (1 diff)

File differences

ChangeLog
1
1
22
33
44
......
1515
1616
1717
18
1819
1920
2021
2122
2223
2324
24
25
26
2527
2628
2729
v0.3 (20/12/2015)
v0.3 (17/01/2016)
** User **
Add referers_diff display plugin
Add year statistics in month details
Add gz files support
Add -z option (don't compress databases)
Add own search enfines files
Do reverse DNS on feeds parsers
** Dev **
Add istats_diff interface
Sort documentation output
Add debug traces in robots plugin
Update awstats data
Dont count 'uri' and 'uri/' as differents uri
Remove doule // at the end of URL
Remove final slashes for referrers
** Bugs **
Forgot <body> tag
iwla.py
289289
290290
291291
292
293
294
295
296
297
298
292299
293300
294301
......
596603
597604
598605
599
600
601
602
603
606
607
604608
605609
606610
self.logger.debug("False")
return False
def isValidVisitor(self, hit):
if hit['robot']: return False
if not (conf.count_hit_only_visitors or\
hit['viewed_pages']):
return False
return True
def _appendHit(self, hit):
remote_addr = hit['remote_addr']
self.valid_visitors = {}
for (k,v) in visits.items():
if v['robot']: continue
if not (conf.count_hit_only_visitors or\
v['viewed_pages']):
continue
self.valid_visitors[k] = v
if self.isValidVisitor(v):
self.valid_visitors[k] = v
duplicated_stats['nb_visitors'] = stats['nb_visitors'] = len(self.valid_visitors.keys())
plugins/post_analysis/reverse_dns.py
6464
6565
6666
67
67
6868
6969
70
71
72
7073
7174
7275
return True
def hook(self):
hits = self.iwla.getValidVisitors()
hits = self.iwla.getCurrentVisists()
for (k, hit) in hits.items():
if hit.get('dns_analysed', False): continue
if not hit['feed_parser'] and\
not self.iwla.isValidVisitor(hit):
continue
try:
name, _, _ = socket.gethostbyaddr(k)
hit['remote_addr'] = name.lower()
tools/iwla_convert.pl
107107
108108
109109
110
111
print $FIC "}\n\n";
close($FIC);
print "Done !\n";

Archive Download the corresponding diff file

Branches

Tags