Comments on: Chronicles of iplist in Debian NSLU2 part 2

By: johnny

johnny — Wed, 31 Dec 2008 20:45:15 +0000

Thanks for good post

By: chewearn

chewearn — Wed, 10 Dec 2008 04:53:52 +0000

hi Alan,
Thanks for the update. I have been using Debian Etch. Apart from the memory overload, I have been seeing some problems on the UI.

I guess upgrade to Lenny or Sid should solve most of these problems.

By: Alan

Alan — Tue, 09 Dec 2008 22:32:11 +0000

Hi, I have been running it for a couple of days now. using debian sid on my NSLU2, so there is no need for the netlink compiling. I run rtorrent, mpd, mt-daapd, afp, and occasionally mplayer for BBC radio. I have 256MB of swap of which 28MB is used - but not heavily swapped, except when starting ipblock. CPU[|||||||||||||||||||||||||| 51.5%] Tasks: 37 total, 1 running Mem[||||||||||||||||||||||||||||||||||||||||||18/28MB] Load average: 2.68 2.22 1.40 Swp[|||||| 27/243MB] Uptime: 20:25:33 PID USER PRI NI VIRT RES SHR S CPU% MEM% TIME+ Command 1700 mt-daapd 20 0 54332 6500 1148 S 0.0 22.0 0:00.10 /usr/sbin/mt-daapd 3131 mt-daapd 20 0 54332 6500 1148 S 0.0 22.0 0:38.20 /usr/sbin/mt-daapd 1625 mt-daapd 20 0 54332 6500 1148 S 0.0 22.0 1:25.03 /usr/sbin/mt-daapd 3129 alan 20 0 28452 6076 4304 D 27.5 20.6 1:28.75 rtorrent 3113 mpd 20 0 44832 2544 1232 S 0.0 8.6 0:00.17 /usr/bin/mpd /etc/mpd.conf 3114 mpd 20 0 44832 2544 1232 S 0.0 8.6 0:00.00 /usr/bin/mpd /etc/mpd.conf 3117 mpd 20 0 44832 2544 1232 S 0.0 8.6 0:00.01 /usr/bin/mpd /etc/mpd.conf 3107 mpd 20 0 44832 2544 1232 S 0.0 8.6 0:00.63 /usr/bin/mpd /etc/mpd.conf 3000 root 15 -5 22436 2248 792 S 0.0 7.6 0:00.00 iplist --daemon -q -f /tmp/ipblock.log -l match 3001 root 15 -5 22436 2248 792 S 0.0 7.6 0:00.06 iplist --daemon -q -f /tmp/ipblock.log -l match 3064 root 15 -5 22436 2248 792 S 0.0 7.6 0:00.40 iplist --daemon -q -f /tmp/ipblock.log -l match 2998 root 15 -5 22436 2248 792 S 0.0 7.6 1:01.16 iplist --daemon -q -f /tmp/ipblock.log -l match 3118 alan 20 0 16344 1244 1052 S 0.0 4.2 0:14.98 /usr/sbin/afpd -U uams_dhx.so,uams_clrtxt.so,uams_randnum 3149 alan 20 0 2896 1168 924 R 5.4 4.0 0:05.71 htop 2133 alan 20 0 3504 908 684 S 0.0 3.1 0:00.32 /bin/bash 2829 alan 20 0 3488 872 640 S 0.0 3.0 0:00.30 -bash 3119 root 20 0 9020 808 724 S 0.0 2.7 0:02.39 /usr/sbin/cnid_dbd /lib/afpd/alan/.AppleDB 4 2 2853 root 20 0 2984 792 576 S 0.0 2.7 0:00.52 -bash as you can see mpd & mt-daapd take more resources. I can simutaneously have Itunes playing the mt-daapd, and AFP serving an avi to mplayer on a Mac. and Downloading torrents, without any drop-outs. Armel rocks - I compiled everything with -mcpu=xscale Ipblock -l Queue Policy (mark) Ranges Target (mark) File 255 REPEAT (65534) 200776 REPEAT (65535) /var/cache/iplist/pipfilter.dat.gz REPEAT (65535) /var/cache/iplist/level1.gz REPEAT (65535) /var/cache/iplist/level2.gz REPEAT (65535) /var/cache/iplist/level3.gz REPEAT (65535) /var/cache/iplist/tbg_primarythreats.gz REPEAT (65535) /var/cache/iplist/nipfilter.dat.gz REPEAT (65535) /var/cache/iplist/bogon.gz REPEAT (65535) /var/cache/iplist/pipfilter.dat.gz REPEAT (65535) /var/cache/iplist/level1.gz REPEAT (65535) /var/cache/iplist/level2.gz REPEAT (65535) /var/cache/iplist/level3.gz REPEAT (65535) /var/cache/iplist/tbg_primarythreats.gz REPEAT (65535) /var/cache/iplist/nipfilter.dat.gz REPEAT (65535) /var/cache/iplist/bogon.gz Chain BLOCK_MATCH (2 references) pkts bytes target prot opt in out source destination 57 2948 REJECT tcp -- * * 0.0.0.0/0 0.0.0.0/0 reject-with tcp-reset 0 0 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable Last Updated Tue Dec 9 01:47:02 CET 2008 Last log messages (/tmp/ipblock.log): 23:06:03 OUTPUT: Match=Giga-H Hits=1 Target=REPEAT SRC=192.168.1.8:55568 DST=193.37.152.226:65535 Proto=TCP 23:06:22 INPUT: Match=UNITED Hits=1 Target=REPEAT SRC=130.37.30.156:64000 DST=192.168.1.8:0 Proto=TCP 23:06:22 INPUT: Match=UNITED Hits=2 Target=REPEAT SRC=130.37.30.156:64000 DST=192.168.1.8:0 Proto=TCP 23:08:11 INPUT: Match=IPpoo Hits=1 Target=REPEAT SRC=86.142.17.27:54784 DST=192.168.1.8:0 Proto=TCP 23:08:12 INPUT: Match=IPpoo Hits=2 Target=REPEAT SRC=86.142.17.27:54784 DST=192.168.1.8:0 Proto=TCP 23:08:12 INPUT: Match=IPpoo Hits=3 Target=REPEAT SRC=86.142.17.27:54784 DST=192.168.1.8:0 Proto=TCP 23:08:36 INPUT: Match=TPNET, Hits=16 Target=REPEAT SRC=83.18.100.98:52736 DST=192.168.1.8:0 Proto=TCP 23:08:37 INPUT: Match=TPNET, Hits=17 Target=REPEAT SRC=83.18.100.98:52736 DST=192.168.1.8:0 Proto=TCP 23:08:43 INPUT: Match=TPNET, Hits=18 Target=REPEAT SRC=83.18.100.98:52736 DST=192.168.1.8:0 Proto=TCP 23:14:53 OUTPUT: Match=Royal Hits=1 Target=REPEAT SRC=192.168.1.8:44032 DST=130.243.189.100:0 Proto=TCP I had to change the ipblock config : IPTABLES_CHAIN_ALLOW="OUTPUT INPUT" to whitelist the input a well as the input. ipblock also loads a lot quicker if the blocklists are converted with ipblock -c before starting. Alan

By: chewearn

chewearn — Mon, 08 Dec 2008 00:20:10 +0000

hi Alan
Thanks for commenting about your test.

I was working on the earlier arm version (as linked by uljanow) and it did not work. I guess I should now work with the updated version instead.

However, a big problem remains: the lack of memory space when the largest blocklist is enabled. As soon as the memory started to swap, everything slowed to a crawl. I can’t see any alternative except to hack a fatslug.

By: Alan

Alan — Sun, 07 Dec 2008 22:03:55 +0000

I found your blog today after failing to get Moblock and nfblockd working. The latest and best source for the Slug is at http://iplist.sf.net/source/iplist-0.24~arm0.tar.gz
This version actually appears to work, while the 24-Ubuntu3 version didn’t for me.

I removed the allow http port, for testing and only used 2 block lists

# cat /tmp/ipblock.log
2008-12-07 21:34:52 URL:http://blocklist.drowning-madness.com/biss/level1.gz [3717308/3717308] -> “level1.gz” [1]
2008-12-07 21:34:57 URL:http://bluetack4.snowmanuk.net/bluetack/bogon.gz [40809/40809] -> “bogon.gz” [1]
21:56:08 OUTPUT: Match=AppleComputer,Inc Hits=1 Target=REPEAT SRC=192.168.1.8:53249 DST=17.112.152.32:2 Proto=TCP
21:57:06 OUTPUT: Match=AppleComputer,Inc Hits=2 Target=REPEAT SRC=192.168.1.8:53249 DST=17.112.152.32:2 Proto=TCP
21:59:16 OUTPUT: Match=S.N.C.F Hits=1 Target=REPEAT SRC=192.168.1.8:56833 DST=148.169.128.12:3 Proto=TCP
21:59:23 OUTPUT: Match=S.N.C.F Hits=2 Target=REPEAT SRC=192.168.1.8:56833 DST=148.169.128.12:1 Proto=TCP
22:09:04 OUTPUT: Match=AppleComputer,Inc Hits=3 Target=REPEAT SRC=192.168.1.8 DST=17.112.152.32 Proto=ICMP
22:09:05 OUTPUT: Match=AppleComputer,Inc Hits=4 Target=REPEAT SRC=192.168.1.8 DST=17.112.152.32 Proto=ICMP
22:09:06 OUTPUT: Match=AppleComputer,Inc Hits=5 Target=REPEAT SRC=192.168.1.8 DST=17.112.152.32 Proto=ICMP

# ping http://www.google.com
PING google.navigation.opendns.com (208.69.34.231) 56(84) bytes of data.
64 bytes from google.navigation.opendns.com (208.69.34.231): icmp_seq=1 ttl=49 time=15.6 ms
64 bytes from google.navigation.opendns.com (208.69.34.231): icmp_seq=2 ttl=49 time=15.1 ms
64 bytes from google.navigation.opendns.com (208.69.34.231): icmp_seq=3 ttl=49 time=14.8 ms
64 bytes from google.navigation.opendns.com (208.69.34.231): icmp_seq=4 ttl=49 time=14.9 ms

# ping http://www.apple.com
PING http://www.apple.com.akadns.net (17.112.152.32) 56(84) bytes of data.
From Slug.local (192.168.1.8) icmp_seq=1 Destination Port Unreachable
From Slug.local (192.168.1.8) icmp_seq=2 Destination Port Unreachable
^C

Swap space ???
# free
total used free shared buffers cached
Mem: 29516 28228 1288 0 1076 10188
-/+ buffers/cache: 16964 12552
Swap: 248968 28704 220264

I haven’t found any problems yet.

By: chewearn

chewearn — Tue, 25 Nov 2008 04:51:27 +0000

hi Serkan
Thanks for the assistance, I will test the new binary.

By: uljanow

uljanow — Mon, 24 Nov 2008 19:03:16 +0000

I’ve build a binary debian package for armel (Lenny) and tested it successfully on Qemu without any modifications of libs. I used the versatile kernel.

http://iplist.sourceforge.net/source/iplist_0.24-0cli1_armel.deb

By: chewearn

chewearn — Mon, 24 Nov 2008 11:11:50 +0000

hi Serkan,
Thanks for the help! I will try it out.

I should mention I have two other blog posts already in the pipeline.

First, patching the source code, which turned out did not solve the problem. This post will be published on Thursday.

Sometime next week, another blog post will describe building iplist with the latest netfilter.org source code. The packet_mode error was solved using that. However, I now see a new problem, which might be related to missing GUI (but I haven’t got the chance to debug).

Your new code release (with GUI removed) will certainly save me a lot of time. Thank you!

p.s. Your first comment got wrongly flagged as spam by Askimet, sorry about that.

By: uljanow

uljanow — Mon, 24 Nov 2008 10:42:49 +0000

I’ve uploaded a modified version that might work on ARM. It fixed the packet_mode error , uses less RAM than the latest version and doesn’t have GUI support. However iplist is not optimized to run on embedded devices.

http://iplist.sf.net/source/iplist-0.2~arm.tar.gz

Regards,
Serkan

p.s. this might be a double post

By: uljanow

uljanow — Mon, 24 Nov 2008 09:07:06 +0000

Nice Chronicles! I’ve uploaded a modified version for arm that might work. It fixes the packet_copy error. The version 0.2 uses less RAM than the latest version but iplist isn’t optimized to run on devices with 32MB of RAM.

http://iplist.sourceforge.net/source/iplist-0.2~arm.tar.gz

Regards,
Serkan