SquidGuard eating memory, blocking all

[mdurakovich]

Hi all, hope you can help.

Running: Fedora 8
Squid Cache: Version 2.6.STABLE22
SquidGuard: 1.2.0 Berkeley DB 4.5.20: (August 18, 2007)

Squid is operational as a transparent proxy. I am trying to add in squidGuard. My results are poor to say the least. Using the most basic configuration, when I test it prior to activating in squid.conf, it is operational both for block and pass, although it takes about 2 minutes to process either way. When I activate it thru squid.conf and restarting squid, it consumes memory rapaciously, and seems to spawn extra processes for itself. Also, no matter whether it is a blocked site or not, no browser can reach any site.

I've documented what I am seeing. Below is my squidGuard.conf, as well as my results for testing the configuration, as my memory as I try it through squid as well. Hope you have ideas/suggestions. Here it all is:


#
# CONFIG FILE FOR SQUIDGUARD
#
# See http://www.squidguard.org/config/ for more examples
#

dbhome /etc/squid/blacklists
logdir /var/log/squid

dest porn{
log porn
domainlist porn/domains
urllist porn/urls
}


acl {
default {
pass !porn all
redirect Not found
}
}

For the config test, I chose the first domain that was listed in my porn/domains file.


[root@Draconia squid]# service squid start
Starting squid: . [ OK ]
[root@Draconia squid]# ps -A | grep squid
633 ? 00:00:00 squid
636 ? 00:00:00 squid
[root@Draconia squid]# echo "http://0--ass-cinema-newsp.da.ru 192.168.1.5/ - - GET" | squidGuard -c /etc/squid/squidGuard.conf -d
2009-03-17 20:44:31 [642] New setting: dbhome: /etc/squid/blacklists
2009-03-17 20:44:31 [642] New setting: logdir: /var/log/squid
2009-03-17 20:44:31 [642] init domainlist /etc/squid/blacklists/porn/domains

...almost two minutes pass...

2009-03-17 20:46:22 [642] init urllist /etc/squid/blacklists/porn/urls
2009-03-17 20:46:23 [642] squidGuard 1.2.0 started (1237337071.709)
2009-03-17 20:46:23 [642] squidGuard ready for requests (1237337183.113)
2009-03-17 20:46:23 [642] source not found
2009-03-17 20:46:23 [642] no ACL matching source, using default
2009-03-17 20:46:23 [642] Request(default/porn/-) Categories 192.168.1.5/- - - REDIRECT
Not found 192.168.1.5/- - -
2009-03-17 20:46:23 [642] squidGuard stopped (1237337183.114)


[root@Draconia squid]# echo "http://www.google.com 192.168.1.5/ - - GET" | squidGuard -c /etc/squid/squidGuard.conf -d
2009-03-17 20:47:17 [646] New setting: dbhome: /etc/squid/blacklists
2009-03-17 20:47:17 [646] New setting: logdir: /var/log/squid
2009-03-17 20:47:17 [646] init domainlist /etc/squid/blacklists/porn/domains

...over two minutes pass...

2009-03-17 20:49:23 [646] init urllist /etc/squid/blacklists/porn/urls
2009-03-17 20:49:23 [646] squidGuard 1.2.0 started (1237337237.851)
2009-03-17 20:49:23 [646] squidGuard ready for requests (1237337363.909)
2009-03-17 20:49:23 [646] source not found
2009-03-17 20:49:23 [646] no ACL matching source, using default

2009-03-17 20:49:24 [646] squidGuard stopped (1237337364.321)

(disk cache decreased by 35 MB after this stopped)

...enabled squidGuard in squid.conf...

prior to restarting squid:


[root@Draconia squid]# cat /proc/meminfo
MemTotal: 968628 kB
MemFree: 372608 kB <--------- starting
Buffers: 94656 kB
Cached: 220108 kB <---------- starting
SwapCached: 4 kB
Active: 378708 kB
Inactive: 151568 kB
HighTotal: 64384 kB
HighFree: 2976 kB
LowTotal: 904244 kB
LowFree: 369632 kB
SwapTotal: 1966072 kB
SwapFree: 1966068 kB
Dirty: 420 kB
Writeback: 0 kB
AnonPages: 155424 kB
Mapped: 69448 kB
Slab: 52276 kB
SReclaimable: 35744 kB
SUnreclaim: 16532 kB
PageTables: 4488 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 2450384 kB
Committed_AS: 498484 kB
VmallocTotal: 110584 kB
VmallocUsed: 4568 kB
VmallocChunk: 105668 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 4096 kB
[root@Draconia squid]# service squid restart
Stopping squid: ................ [ OK ]
Starting squid: . [ OK ]
[root@Draconia squid]# ps -A | grep squid
686 ? 00:00:00 squid
689 ? 00:00:00 squid
690 ? 00:00:15 squidGuard
691 ? 00:00:15 squidGuard
692 ? 00:00:15 squidGuard
693 ? 00:00:15 squidGuard
694 ? 00:00:15 squidGuard

...approx 5-8 minutes later, during which can't reach any pages even allowed...

[root@Draconia squid]# ps -A | grep squid
686 ? 00:00:00 squid
690 ? 00:00:26 squidGuard
691 ? 00:00:27 squidGuard
692 ? 00:00:27 squidGuard
693 ? 00:00:27 squidGuard
694 ? 00:00:27 squidGuard
710 ? 00:00:00 squid
711 ? 00:00:20 squidGuard
712 ? 00:00:12 squidGuard
713 ? 00:00:08 squidGuard
716 ? 00:00:07 squidGuard
717 ? 00:00:18 squidGuard
[root@Draconia squid]# cat /proc/meminfo
MemTotal: 968628 kB
MemFree: 55732 kB <------- decreased drastically,started with 372608
Buffers: 95608 kB
Cached: 525504 kB <------- increased drastically,started with 220108
SwapCached: 4 kB
Active: 571656 kB
Inactive: 272584 kB
HighTotal: 64384 kB
HighFree: 192 kB
LowTotal: 904244 kB
LowFree: 55540 kB
SwapTotal: 1966072 kB
SwapFree: 1966068 kB
Dirty: 72252 kB
Writeback: 580 kB
AnonPages: 162716 kB
Mapped: 70516 kB
Slab: 54408 kB
SReclaimable: 37592 kB
SUnreclaim: 16816 kB
PageTables: 5048 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 2450384 kB
Committed_AS: 512188 kB
VmallocTotal: 110584 kB
VmallocUsed: 4568 kB
VmallocChunk: 105668 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 4096 kB
[root@Draconia squid]#


...stopped all (had to manually kill squidGuard) and waited another 5-8 minutes...


[root@Draconia squid]# cat /proc/meminfo
MemTotal: 968628 kB
MemFree: 255536 kB
Buffers: 92768 kB
Cached: 347616 kB
SwapCached: 4 kB
Active: 410484 kB
Inactive: 239860 kB
HighTotal: 64384 kB
HighFree: 10392 kB
LowTotal: 904244 kB
LowFree: 245144 kB
SwapTotal: 1966072 kB
SwapFree: 1966068 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 149932 kB
Mapped: 68528 kB
Slab: 49408 kB
SReclaimable: 33012 kB
SUnreclaim: 16396 kB
PageTables: 4380 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 2450384 kB
Committed_AS: 490472 kB
VmallocTotal: 110584 kB
VmallocUsed: 4568 kB
VmallocChunk: 105668 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 4096 kB


At this point I'm frustrated and shut the whole damn thing off.

[mdurakovich]

I guess I should have added that the only thing this box does is act as my firewall/squid cache, and it has 1 GB physical memory. Don't know how to get the CPU info, but it's not a slouch.

[mdurakovich]

Bump...anything? Anyone?

[jledhead]

what if you make a custom blacklist and url with a small amount of stuff and then have squidguard load just that? The thinking is maybe one or more of your blocklists and/or url filters are corrupted in some way and choking squidguard.

and they are just text files in case you are wondering.

additionally, what if you change it to pass all

pass all

just to make sure you are getting out using squidguard.

[mdurakovich]

additionally, what if you change it to pass all

That works.

what if you make a custom blacklist and url with a small amount of stuff and then have squidguard load just that?

This works too. So it looks like it was choking on the porn blacklist I downloaded.

Thanks for the ideas. I appreciate it!

Now I can try again.