Squid 2.5 load cache on ramdisk

Support section for FREESCO v0.4.x 3rd Party Packages

Squid 2.5 load cache on ramdisk

Postby twentyninehairs » Wed Apr 06, 2011 1:16 am

Squid & running a large ramdisk from a HD install doesn't seem to be as well documented as other things, I would like to share some experiences I had. I also have a couple questions.

I stopped using Freesco for a long time because the HD in the system I was using several years ago crashed, and I simply hadn't had time to rebuild it. About 6 months ago, my wife's P4 1G RAM computer's HD burned up, instead of fixing it, I replaced her computer. I rebuilt this for my home a few days ago with a 1G CF card formatted FAT 32 for the HD. As far as CF cards go, this one is terribly slow, but works well for what I'm using it for. I have a need for filtering content from the Kid's computers in their bedrooms. Right now, they just don't have access to almost anything from their bedrooms. I also have a 1.5M internet connection, and when everybody is running downloads, doing taxes online, trying to watch youtube & netflix, and playing online flash games at the same time, we have a serious problem. I'm tried Squid 2.5, but especially with the FAT filesystem, CF is slow enough that web pages are very slow loading, if the content isn't in the cache, and much of the content on web pages time out, or doesn't load at all. I eventually created a 128mb RAM disk, and rebuilt the cash on the ramdisk on every reboot. This is working extremely well, I'm sure there is a small amount of lag the first time a page loads, but I can't tell the difference. I'm also using the hosts package to filter some of the garbage. Google's ads seem to be blocked with hosts; may cause some minor problems. Overall, I give the system a thumbs up.

Every time I make a major change, I shutdown the router, pull the CF card, back it up, put it back in & reload. Probably not good for a large production environment, but it is the best thing that I have ever been able to do for my house network. I've already loaded backups several times after failed configuration attempts.

These are the steps I took to get Squid running on the ramdisk:
-Put the ramd128 file provided in the ramdisks folder downloaded with the FREESCO file.
-Changed the router.bat file's ramdisk file sizes to 131072.
(This is mentioned in the freesco documentation as the syslinux.cfg file, but I couldn't find it mentioned any place for a HD install. I didn't do this at first, because I couldn't find it. When booting with this configured incorrectly, the RAMDISK mounted & loaded without errors, but when attempting to read & write to it, it would kick out corruption errors. Took me a while to figure out my ramdisk was crashing all the time because this wasn't changed)
-Added the following lines to Fn1 in the rc_squid file:Attached are the full config files with .txt added to the end of the filename.
rc_squid.txt
squid.conf.txt
(As of yesterday, I am brand new to SQUID, and my ACL in the squid file shows that I'm testing acl commands. Nonetheless, it's really cool stuff.)

mkdir /tmp/squid
mkdir /tmp/squid/cache
chmod 700 /tmp/squid/cache
chown nobody /tmp/squid/cache
chgrp nobody /tmp/squid/cache
/mnt/bootdev/router/pkg/usr/local/squid/sbin/squid -z
-Then I added the following lines to the squid.conf file: (This is a 100mb cache for the 128mb RAM drive)
cache_dir ufs /tmp/squid/cache 100 16 256

This loads and runs correctly on every reboot. However, the rc_squid file's directory builder script is not very clean & kicks out a lot of non-harmful errors because it trys to rebuild the directory when the directory already exists alot.

I have two questions:
-I've used linux for a long time, but I'm not really good at writing scripts. I was wondering if anybody had some suggestions about cleaning the above directory builder this up? What would be a better way of doing this?
-I would like to create a bigger RAM disk. Are there any 512Mb ram disk files for Freesco? If not, how do I go about making one?
You do not have the required permissions to view the files attached to this post.
User avatar
twentyninehairs
Newbie
 
Posts: 2
Joined: Thu Jun 21, 2007 10:42 am
Location: Idaho

Re: Squid 2.5 load cache on ramdisk

Postby twentyninehairs » Wed Apr 06, 2011 6:36 am

I already have a problem with the Squid forwarding system. The state tax website uses port 80 for a weird remote desktop login system to IP 209.123.25.18. Of course this connection is hung up in my proxy server. I have the following lines in Fn1 of my rc_squid file. I'm trying to bypass forwarding to port 3128 for this specific destination ip. Connection works if I remove all reference of the 80->3128 redirection for my subnet, and don't have the entry for the single dest ip. This config doesn't allow the remote desktop protocol to forward. I also tried removing the port 80 reference, and I also tried redirecting 80 -> 80 for this single ip. No luck. What am I doing wrong?

Also, I work nights, so it might take me a while to respond if I'm sleeping when you reply.

ipfwadm -I -i accept -P tcp -S 192.168.56.0/24 -D 0/0 80 -r 3128
ipfwadm -I -i accept -P udp -S 192.168.56.0/24 -D 0/0 80 -r 3128

ipfwadm -I -i accept -P tcp -S 192.168.57.0/24 -D 0/0 80 -r 3128
ipfwadm -I -i accept -P udp -S 192.168.57.0/24 -D 0/0 80 -r 3128

ipfwadm -I -i accept -P tcp -S 192.168.3.0/24 -D 0/0 80 -r 3128
ipfwadm -I -i accept -P udp -S 192.168.3.0/24 -D 0/0 80 -r 3128

ipfwadm -I -i accept -P tcp -S 192.168.55.0/24 -D 0/0 80 -r 3128
ipfwadm -I -i accept -P udp -S 192.168.55.0/24 -D 0/0 80 -r 3128

#Government taxes site remote access for their weird system
ipfwadm -F -i accept -P tcp -S 0/0 -D 209.123.25.182 80 -m
ipfwadm -F -i accept -P udp -S 0/0 -D 209.123.25.182 80 -m
User avatar
twentyninehairs
Newbie
 
Posts: 2
Joined: Thu Jun 21, 2007 10:42 am
Location: Idaho

Re: Squid 2.5 load cache on ramdisk

Postby Lightning » Thu Apr 28, 2011 7:35 pm

Sorry it took so long to reply, but I did not have time when I first saw this post because of the depth of my reply and then I forgot about it.
I have two questions:
-I've used linux for a long time, but I'm not really good at writing scripts. I was wondering if anybody had some suggestions about cleaning the above directory builder this up? What would be a better way of doing this?
-I would like to create a bigger RAM disk. Are there any 512Mb ram disk files for Freesco? If not, how do I go about making one?
For the rc_squid directory builder. Put all of your scripting in the $boot) section of the script so that it is only done once at boot time.

How to make a ramdisk of any size that you want. Here is the details on how to do it. The first thing that is required is the ext2 package so that you have the ext2 tools or a install of Slackware 3.9 or equivalent libc5 system (A new Linux system will NOT work).
It is possible to download the ext2 package and manually un-compress the binaries into the /pkg/ directory structure if you are using a umsdos file system. The "mc" package will make this task much easier than having to do it all from the command line and all you need are the binaries in the package /sbin/ directory copied to the /pkg/sbin/ directory of your system. You will also need the "utils" package to get the gzip binary in FREESCO 043 and less systems. To do this in "mc" just highlight the package name and hit "enter" then highlight the ext2-0.4.x.tgz file and hit "enter" again and move the highlighter to the sbin directory. Then hit tab and move the other window to the /pkg/sbin directory, tab again and hit F5.

Then on the command line in run these commands to create a 512MB ramdisk
Code: Select all
mkdir -p /boot/ramdisk/ram
dd if=/dev/zero of=/boot/ramdisk/ramd512  bs=1024  count=524288
mke2fs /boot/ramdisk/ramd512
e2fsck /boot/ramdisk/ramd512
tune2fs -c 0 -i 0 -C 0 -r 0 /boot/ramdisk/ramd512
mount /boot/ramdisk/ramd512 /boot/ramdisk/ram/ -o loop
rm /boot/ramdisk/ram/lost+found
umount /boot/ramdisk/ramd512
gzip -9 /boot/ramdisk/ramd512
mv /boot/ramdisk/ramd512.gz  /boot/ramdisk/ramd512
gzip -9 /boot/ramdisk/ramd512
mv /boot/ramdisk/ramd512.gz  /boot/ramd512

You now have a 512MB ramdisk that will require the ramdisk_size=524288 and the manual ramdisk setting to be set to 512. Be aware that a 512MB ramdisk is not actually 512MB because the disk formatting takes up a certain amount of space. But it is still over 500MB and to see exactly what size it really is once in place just use the "df" command.

If you don't feel like taking the time and having the learning experience of making a ramdisk yourself. Then you can just download the one that I made during the reply on this post so that I did not miss any steps.
Code: Select all
 cd /boot/
snarf http://lewys-spot.dyndns.org/test/ramd512

I will leave it there for a while.

With regards to your firewall rules to get around squid. I would try a commands more like these.
Code: Select all
ipfwadm -I -i accept -P tcp -W eth1 -D 209.123.25.182 80
ipfwadm -I -i accept -P udp -W eth1 -D 209.123.25.182 80
ipfwadm -F -i masquerad -P tcp -W eth1 -D 209.123.25.182 80
ipfwadm -F -i masquerad -P udp -W eth1 -D 209.123.25.182 80

The example assumes that eth1 is your internal interface and NOT the one connected to the Internet and that these commands are the LAST ones in your list of commands. Also be aware that if this site uses SHTML then it might need port 443 instead of port 80
If you are afraid that you might make a mistake. The chances are high that you will never learn anything.
User avatar
Lightning
FREESCO GOD !!
 
Posts: 12079
Joined: Wed Nov 14, 2001 6:50 am
Location: Oregon, USA


Return to 3rd Party Package Support for FREESCO v0.4.x

Who is online

Users browsing this forum: No registered users and 2 guests

cron