Software Release - fwsnort-1.6.2
28 April, 2012
The 1.6.2 release of fwsnort is available for
download. The most impactful
change in this release is a switch to how fwsnort loads translated rules into the
running iptables policy. Instead of attempting to parse the local policy and only
add those rules in that appear to match protocols that the policy allows, fwsnort
now loads all translated rules by default. The reasoning for this change is in
the ChangeLog below. There are a few bug fixes and updates to get fwsnort working
without warnings on recent versions of perl as well as an ICMP type fix for recent
versions of iptables. As usual, please let me know if there are any issues.
Here is the complete fwsnort-1.6.2 ChangeLog:
- Switched --no-ipt-sync to default to not syncing with the iptables policy. By default fwsnort attempts to match translated Snort rules to the running iptables policy, but this is tough to do well because iptables policies can be complex. And, before fwsnort switched to the iptables-save format for instantiating the policy, a large set of translated rules could take a really long time to make active within the kernel. Finally, many Snort rules restrict themselves to established TCP connections anyway, and if a restrictive policy doesn't allow connections to get into the established state for some port let's say, then there is little harm in having translated Snort rules for this port. Some kernel memory would be wasted (small), but no performance would be lost since packets won't be processed against these rules anyway. The end result is that the default behavior is now to not sync with the local iptables policy in favor of translating and instantiating as many rules as possible.
- Replaced Net::IPv4Addr with the excellent NetAddr::IP module which has comprehensive support for IPv6 address network parsing and comparisons.
- Moved the fwsnort.sh script and associated files into the /var/lib/fwsnort/ directory. This was suggested by Peter Vrabec.
- Bug fix for recent versions of iptables (such as 1.4.12) where the icmp match requires --icmp-type to be set - some Snort rules look for a string to match in icmp traffic, but don't also specify an icmp type.
- Bug fix for 'qw(...) usage as parenthesis' warnings for perl > 5.14
- Removed the ExtUtils::MakeMaker RPM build requirement from the fwsnort.spec file. This is a compromise which will allow the fwsnort RPM to be built even if RPM doesn't or can't see that ExtUtils::MakeMaker is installed - most likely it will build anyway. If it doesn't, there are bigger problems since fwsnort is written in perl. If you want to build the fwsnort RPM with a .spec file that requires ExtUtils::MakeMaker, then use the "fwsnort-require-makemaker.spec" file that is bundled in the fwsnort sources.
As an aside, here is
what ent reports against the local /dev/urandom entropy source on my Linux system, and it is
the "Entropy =" line that spa-entropy.pl parses for each SPA byte slice:
Well, that still looks pretty good. Revisiting the ECB encrypted image of Tux above
for a moment - the reason that the Tux outline can be seen in the encrypted version
is that in the JPG image file there must be identical blocks in multiple locations
to represent the solid black regions. These blocks are all encrypted in the same
way by AES in ECB mode, so the outline persists. But, this is one instance of ECB
encryption against a file that has multiple identical blocks. For the encrypted SPA
packets, we're dealing with 1,000 separate instances of encrypted data (all with the
same key). Across this data set there are certainly lots of identical plaintext
blocks (all of the SPA packets request access for source IP 127.0.0.2 to destination
port tcp/22 for example), but the encrypted data still shows a high level of entropy.
This source of entropy is provided by the random salt values that are used to
generate the initialization vector and final encryption key for each encrypted SPA
packet. As proof, if we apply the following patch to force the salt to zero for all
SPA packets (of course, one would not want to use this patch in practice):
Now we can easily see where there are identical blocks across the SPA packet data set. The first
eight bytes contains the salt, so these are all zero (note that fwknop strips the
usual "Salted__" prefix before transmitting an SPA packet on the wire). The next
16 bytes are the random bytes that fwknop includes in every SPA packet so these bytes
have high entropy. Next up are the username and timestamp - the later changes with
each second, so there is some entropy there since it takes a few seconds to create the
1,000 SPA packet data set. Then the entropy goes back to zero with the next fields
and there isn't any decent entropy until the final message digest.
In CBC mode, the random data included by the fwknop client now results in decent entropy
even though the salt is zero. This is because every ciphertext block in CBC mode
depends on all previous plaintext blocks, so randomness in one plaintext block implies
that every subsequent encrypted block will look different from one SPA packet to the
next. This graphically shows that CBC mode is a better choice for strong security.
Now, if the pseudo random number generator on the local operating system is poorly
implemented, this will negatively impact ciphertext entropy regardless of the encryption
mode, but still CBC mode is a better alternative than ECB mode.
Clearly, that is not a very uniform distribution from day to day. It looks like Bing has been
hitting the gitweb interface at a rate of over 17,000 hits per day for a significant portion
of late December and early January. The other search engines hardly even show up in the graph -
you know there are big spikes when everything looks better on a logarithmic scale:
With some additional work, it looks like the gitweb.cgi links that Bing is indexing are not all
unique. That is, one might expect that Bing would hit a link, grab the content, and then not return
to the same link for a while. Some gitweb.cgi links were hit more than 10 times and more than 100,000
links were hit more than once during this time period.
Given that 1) all of the information gitweb displays is derived from the underlying git repositories,
and 2) the git repositories are directly accessible via HTTP anyway, it would seem that a better way for
search engines to behave would be to just ignore gitweb altogether and pull directly from git. That
would certainly cut down on the server-side resources necessary to service search engine requests.
Perhaps though the general strategy of search engines is not to be too smart about such things - they
probably just want access to data, and when they see a link they go after it.
Either way, the kind of dedicated and repetitive indexing the Bing is doing against gitweb is a bit much,
and it certainly seems as though they could implement a less intensive crawler. I'm curious if other
server admins are seeing similar behavior.
