Aczid's blog

Today I was looking for an automated way to find any security related server mis-configurations on my website, and found a really nice tool called Nikto that does just that. In fact it was so helpful it showed me I was doing directory indexing through Apache where I didn’t want to.
Here is an example of its use.

aczid@aczid:~$ nikto -host blog.aczid.nl
---------------------------------------------------------------------------
- Nikto 2.02/2.03     -     cirt.net
+ Target IP:       127.0.1.1
+ Target Hostname: blog.aczid.nl
+ Target Port:     80
+ Start Time:      2009-01-25 0:16:00
---------------------------------------------------------------------------
+ Server: nginx/0.6.32
+ OSVDB-0: Retrieved X-Powered-By header: Phusion Passenger (mod_rails/mod_rack) 2.0.6
- /robots.txt - contains 1 'disallow' entry which should be manually viewed. (GET)
+ OSVDB-0: GET /?mod=some_thing&op=browse : Sage 1.0b3 reveals system paths with invalid module names.
+ OSVDB-3092: GET /sitemap.xml : This gives a nice listing of the site content.
+ OSVDB-3092: GET /archives/ : This might be interesting...
+ OSVDB-3092: GET /stats/ : This might be interesting...
+ 2967 items checked: 6 item(s) reported on remote host
+ End Time:        2009-01-25 0:16:00 (23 seconds)
---------------------------------------------------------------------------
+ 1 host(s) tested

And remember, Gort! Klaatu barada nikto!

After moving most of my Ruby apps onto Apache I got back to the idea of wanting to do my own log analysis. Although the Google analytics urchin is nice, I prefer something more unobtrusive. I have used AWstats in the past, and was always quite impressed with its feature set. After installing Apache again, I figured I would give it a go. I prefer using the packaged AWstats because that way it will (hopefully) automatically update through apt. This post on AWstats by Sami Dalouche was really helpful in getting my head around the packaged AWstats configuration on Debian. This will show you how to make awstats virtualhosts for every domain on your site, show you how to set up a cronjob and prevent logrotate from rotating the logs before AWstats has run. So after you have copied a new awstats.<yourvhost>.conf file in /etc/awstats/, set up the following variables: (This of course implies you have set the access_log to the same location, /var/log/nginx/<yourvhost>-access.log)

LogFile=”/var/log/nginx/<yourvhost>-access.log”
SiteDomain=”<yourvhost>”
HostAliases=”<yourvhost>”
DirData=”/var/lib/awstats/<yourvhost>”

And set the logformat for Nginx:

LogFormat="%host %other %logname %time1 %methodurl %code %bytesd %refererquot %uaquot"

Now for the Apache Virtualhost, which is also largely boilerplate.

<VirtualHost>
        ServerName <yourvhost>
        Alias /awstatsclasses "/usr/share/awstats/lib/"
        Alias /awstats-icon/ "/usr/share/awstats/icon/"
        Alias /awstatscss "/usr/share/doc/awstats/examples/css"
        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
        ScriptAlias /stats /usr/lib/cgi-bin/awstats.pl
        # This is the important bit. It tells AWstats to use your defined vhost config from the .conf file, rather than try to parse the Apache log.
        SetEnv AWSTATS_FORCE_CONFIG <yourvhost>
</VirtualHost>

If the website you want to monitor is already being served from Apache, and Nginx proxies requests for nonexistent files to Apache you are probably done. Go check <yourvhost>/stats/! If the site is served using only Nginx (static assets) you will have to add a proxy_pass in your Nginx virtualhost for the 4 paths in the VirtualHost definition needed by awstats. Hope this helps people out who are struggling to get this set up! It took me about a day in total, I think. Now I have fancy separate stats on all my public Nginx websites!

This week I have moved my Ruby websites (which were previously running on Mongrel) to the Phusion Passenger Apache2 module. I have lived without apache for about a year, but I am really happy I switched back to it again. I am still using Nginx as a front-end proxy to serve static assets.
I am very pleased with Passenger because it makes deployment a lot easier! Basically, all Capistrano needs to do now for a deployment is move your app into the DocumentRoot and touch a "restart.txt" file. It supposedly works with any Rack-based web framework. I am using it with Merb and Rails.

I have more available memory and CPU cycles because there are no idle mongrels running, and availability is increased because new instances of the apps are spawned as needed (where memory is shared between multiple instances of an app).

Life is good with passenger!

This is what it should look like if you are in Japan.

Since late last summer me and my friends have been experimenting with Nginx. It’s a neat little server, sublimely fast and it’s config format doesn’t make you wince. I use it especially for proxying dynamic content from other servers (like Mongrel) but it handles static content equally well.
I’ve played around with geolocation in most of my Rails apps and it made me think this sort of environmental information might just as well be managed by the server. The folks at MaxMind felt the same way, it seems. There are server-side modules available for Apache and Lighty. For Nginx it takes a little more effort, but it’s very doable. Let’s make it so!
Get the GeoLite CSV:

 $ wget http://www.maxmind.com/download/geoip/database/GeoIPCountryCSV.zip

Unpack it:

 $ unzip GeoIPCountryCSV.zip

Get the Nginx tarball:

 $ wget http://sysoev.ru/nginx/nginx-0.5.35.tar.gz

Unpack it:

 $ tar zxvf nginx-0.5.35.tar.gz

You can also get Nginx through you package manager but mine (Debian) didn’t install the included Perl script to convert the MaxMind CSVfile. You can run the script on the CSV file you extracted by running it like so:

 $ perl ./nginx-0.5.35/contrib/geo2nginx.pl < GeoLiteCity_20080301/GeoLiteCity-Location.csv > countries.conf

This generates a countries.conf which Nginx can use. You have to include it in your nginx.conf. Here’s what I added to my nginx.conf in the http namespace. Before this step I moved the generated countries.conf file to my /etc/nginx directory.

geo $country {
   default no;
   include /etc/nginx/countries.conf;
   127.0.0.0/24  us;
}

This $country variable you set up only lives in Nginx so far. You have to add a proxy_set_header or fastcgi_param like so:

 fastcgi_param COUNTRY $country

Or

 proxy_set_header COUNTRY $country;

This will give you a $_SERVER[‘COUNTRY’] (in PHP) or something simmilar.
I’ve used this to automatically set the preferred language in my WebSVN site.

I’ve finally bought my own domain, aczid.nl. Besides the blog, I want to host my music, code and other creations here.
All you readers out there can visit my blog at blog.aczid.nl.
I would like to thank Lasert for providing me with a 2 GHz/256 512 MB Xen slice on a big unclogged eweka tube. It’s a fun and stimulating experience to set up your own front-end/proxy server, subversion, et cetera.
I plan to do most if not all of the site in Ruby on Rails. The blog is served using Nginx, Mongrel and Typo.