First, I had Server 2008 R2 Datacenter as my operating system on both the existing server (denoted double) and the new server (denoted ruby).  The servers could clearly communicate with each other within the same datacenter as well so that file / data transfers could be effective.  The first thing to adding any server is to give the new server a name and know its network configuration details ahead of time.  I’m not going to go into details about how to install Windows, since it’s pretty simple.  Just make sure that you have the correct drivers installed for your server after the installation completes.

Note about installing Windows: just so that you don’t run into any issues with your system, make sure that your primary hard drive is attached to the first port for your motherboard’s interface, in my case, the first hard drive was connected to SATA 0.  If you don’t do this, you’ll run into a lot of problems and waste a lot of time (like I did!).  In addition, make sure that your BIOS has AHCI enabled prior to installing Windows, this also caused problems in my scenario.  Once Windows is installed, make sure that you can disable write caching on your hard disk without the operating system freezing (in computer management, under storage, right click the disk and optimize for quick removal).  If you followed the above, this shouldn’t cause Windows to hang, and will prevent Active Directory from incapacitating your system during the restart phase of the installation.  In addition, this is a good time to name your computer, join it to some default workgroup, add remote administration features, and change the time zone / clock settings.

Before continuing, I’d also recommend disabling Internet Explorer’s advanced security features; this is done by going to the computer management main screen and scrolling down, finding IE ESC and turning it off (acknowledging all of the warnings).  If you keep them on, you’ll find yourself doing way too much work to download necessary applications, etc.  These features are only useful if you plan on doing general web surfing on the server (which I would not recommend for security reasons).  I also activated windows, ran all the necessary windows updates (several reboots and optional updates as well).  After this all has been done, I ended up rebooting the machine a final time.

At this point, I installed several applications (using IE to get Google Chrome initially).

• Google Chrome (http://chrome.google.com) for web surfing / downloading the rest of these
• Adblock plus for chrome (http://adblockplus.org/en/) to block malware / ads
• 7zip (http://www.7-zip.org/download.html) for good archive file support
• Microsoft Security Essentials (http://windows.microsoft.com/en-US/windows/products/security-essentials), Windows 7 version works fine, this is a quick, safe, free antivirus solution for all of those who hate Symantec’s ability to slow down the entire system (hey, that’s me!)   This is also good for desktop users!  I would recommend disabling the scheduled Sunday scan since real-time protection is sufficient in my case.
• Magic ISO (http://www.magiciso.com/tutorials/miso-magicdisc-overview.htm) to mount ISO images (including the Exchange 2010 ISO if you downloaded it from MSDN like I did).
• Office Filter Packs which are a prerequisite for Exchange 2010 (http://www.microsoft.com/download/en/details.aspx?id=17062)

At this point, I added 2 custom firewall rules on both servers that allowed unrestricted incoming traffic from each server.  I then started the Active Directory installer by installing the AD User Service Role, and then ran DCPromo.exe as suggested by the installer.  I did an advanced mode install, adding a new controller to an existing domain in an existing forest, installing the DNS role locally, and I let it install.  For the AD Restore Password, make sure you remember what you set it to since this will be the password to the local (inaccessible) administrator account on the server incase everything fails.  At this point, the server should reboot at least once on its own.

Next, I mounted the Exchange ISO and went through the step by step screens to install it.  For me, I installed the mailbox, client access, hub transport, and management tools roles.  I told it that the client role would be internet facing to the OWA website (matthouse.org).  Exchange takes roughly 3 hours to install at this point.  After it is done, you should enter the Exchange Management Console (EMC), enter a product key to active it, and add a send connector for your organization (for the new server specifically).  If you’re wondering why your server isn’t sending mail (and you’re new to this and installing Exchange for the first time), try adding a send connector that is internet facing and allows *, that will fix your problem.

You will want to run Windows updates again and make sure that all of the Exchange updates are installed before continuing.  This may require several reboots of the server.

At this point, Exchange should be synched with the other server mostly so it’s time to start migrating services.  I first recommend changing all of your DNS records for mail over to the new server and give them time to propagate (as per the Time to Live [TTL] value on the record).  I also did mailbox remove requests (through the EMC) to the database on the new server; this should be fairly intuitive for anyone with a background in at least some systems administration.  I also went through all the client access role options and made sure that the internal / external sites for IMAP, POP, OWA, OAB, and ECP were properly set up for my main OWA address (matthouse.org).

Since Exchange by default requires https://server/owa to gain access to Outlook Web Access [OWA], I needed to add a few files to the web root of the domain to properly forward the user onto the OWA website when they went to the main website.  To do this, simply go to the IIS manager, go to the Default Website, and right click and open the document root.  In here, add 2 files as follows:

Web.config:
<?xml version=”1.0″ encoding=”UTF-8″?>
<configuration>
<system.webServer>
<httpErrors>
<error statusCode=”403″ subStatusCode=”4″ path=”https://matthouse.org” responseMode=”Redirect” />
</httpErrors>
</system.webServer>
</configuration>

Default.aspx

<script language=”c#” runat=”server”>
private void Page_Load(object sender, System.EventArgs e)
{
Response.Status = “301 Moved Permanently”;
Response.AddHeader(“Location”,”https://matthouse.org/owa”);
}
</script>

Basically, these 2 files will forward anyone from the web root to the appropriate OWA directory in SSL (https) mode.  I figure that anyone competent can figure out what needs to be changed, it isn’t rocket science, after all.

At this point, I added a real RapidSSL certificate to the server.  To do this, I went to the Exchange Management Console, went to the server tab, found the place to generate a CSR (Certificate Signing Request) and I created one.  I pasted everything from the CA (Certificate Authority) and imported it to Exchange and set all the services to use it (IIS, SMTP, POPS, IMAPS).  I also found the remote desktop session host manager window, right clicked the configuration of the server, and right clicked on rdp-tcp and went to properties.  I selected the general tab, selected the appropriate already installed certificate and ok’d everything, after restarting my RDP session, I had the new secure connection.

For anyone who is curious about anti-spam, there is a hidden anti-spam feature on the Exchange Hub Transport role, to get this, you can run the below steps in the Exchange PowerShell environment.  Afterwards, you will see an Anti-spam option in the Hub Transport role node under the Organization Configuration node of the EMC.  As for me, I opted for Forefront Protection 2010 (formerly Forefront Security 2010) since it provides a much more sophisticated scanning engine, although it does cost more and takes a lot more memory and configuration to get running smoothly.

• cd /
• cd c:\
• cd program files
• cd microsoft
• cd exchange server
• cd v14
• cd scripts
• ./install-AntispamAgents.ps1
• Restart-Service MSExchangeTransport

After all of this, I also found an IE9 / EMC interoperability bug where you can’t close the EMC if IE9 is installed on the system, this seems to be a bug with the Microsoft Management Console (MMC), so the patch can be directly downloaded from Microsoft, I would recommend searching Google for hotfix 2624899 to get the patch.  Keep in mind that a hotfix rollup in the future will include this patch from Microsoft so I’d recommend only installing it if you have this issue.

At this point, I’d recommend securing the firewall, RDP’s port, and adding some backup scripts.

Next, it is time to remove the old server.  I ran the following in the Exchange Management Power Shell Environment: “Get-Mailbox -Arbitration -Database db1 | New-MoveRequest -TargetDatabase db2”, where db1 is on the old server, and db2 is on the new server.  In EMC, I went to the organization node > mailbox > offline address book (OAB), added a new OAB generated by the new server and removed the existing one generated by the old server.  I also went to hub transport under organization, went to send connectors and removed the old server from the send connector.

On the new server, go to Active Directory Sites and Services under Administrative Tools and find each domain controller and view the properties.  Make sure the new domain controller is a global catalog and the old domain controller is not a global catalog server (respectively), these will be under the NTDS settings properties page.  Next, we need to transfer several roles, I used (http://www.petri.co.il/seizing_fsmo_roles.htm) as a guide for this.

• open command prompt (run cmd)
• ntdisutil
• roles
• connections
• connect to server <new domain controller>
• q
• transfer naming master
• transfer infrastructure master
• transfer PDC
• transfer RID master
• transfer schema master
• q

On the old server, remove the Active Directory Certificate Services role if it exists (you can probably ignore any warnings since Exchange should be using external certificates).  You may have to reboot the server.  Finally, go to add/remove programs on the old server, and remove Exchange 2010 by deselecting all of the roles.  Exchange automatically detects if it is safe to remove everything and will transfer anything left behind over to the new server.  Do the same for Active Directory by running DCPromo.exe (under the Active Directory node of the server management console), and running through the prompt.  If you get any warnings / errors when attempting to remove Exchange / Active Directory, take the advice and don’t continue since you might end up creating a lot more work for yourself.

Once everything is removed, you can trash the old server from the network and Exchange has been successfully moved.  For me, I ended up having about 4 reboots that affected OWA for users for a total of roughly 5 minutes each while Exchange rebooted.

As usual, thanks for reading.  Disclaimer: this information is provided on an as-is basis, I do not guarantee that this will work in your scenario, but I hope that it can help someone else out that is having similar difficulties to the ones that I’ve described.

Mac, like Linux and its BSD roots, has a flaw when you have physical access to the machine.  Basically, passwords are stored as password hashes in a simple file on the server.  To change the password of any user account, you basically have to gain write access to the said password (or shadow) file and change the hash to something that matches the new desired password.  Typically, this is very easy to do for any kind of Linux or BSD operating system.  For all of you Windows fans, Windows stores the passwords encrypted in SAM files (if I recall correctly), and there are also password reset disks available that you boot from and overwrite the administrator account password with your own password.  Once again, resetting a password is very easy to do to gain access to a machine provided you have physical access.  One tiny exception to this rule on every platform, if the encryption feature is enabled on a user account (provided it exists), the files that were encrypted will never be readable again if you change the password using this method.

Since this is resetting the password of a Mac, I’m only going to cover the Mac OS.  Mac, like Linux and BSD, has something called single user mode which is basically a recovery environment built into the operating system.  When something doesn’t work properly, this environment typically boots and gives the local user (on the console) a command prompt that is running as the root user (or superuser).  Typically in single user mode, the local hard disk is read only, but because you are root, you can simply change that to write mode.  Afterwards, its a matter of changing the password.  If you aren’t a single user mode fan, Mac even puts a reset password link in the utilities menu of their installation disk that does the work manually.

So, using the single user mode shell, here is the procedure to resetting the password of your mac:

1. Shut down the computer
2. While holding the command (apple) key and S simultaneously, power the computer on
3. The command line will come up shortly
4. Type in “/sbin/mount -wu /” without the quotes, this enables writing to the root operating system directory
5. type in “passwd <user>”, where <user> is the username whose password needs to be reset
6. type in “reboot”
7. Login as the user that you reset the password for, using the new password you set.

Restrictions: Crons can be ran if your username is in the cron.allow file, the location of this file varies across different systems.  If the cron.allow file does not exist, and your username does not exist in cron.deny, you can use crons.  The format of each file is one username per line.

Commands: The most common command I use is “crontab -e” which means edit the crons that run under your user account.  This command opens up the default editor of your system to do this.  The “l” flag means display the crons that are configured under the user’s account.  The “r” flag removes the current crons under a user’s account.   Finally, the “v” tag shows the last modification time of the cron configuration for a user account, note that this command is not available on all systems.

Syntax: The cron file is set up with a certain syntax.

The first line can configure a default email address to email the completion (any script output) to or it can immediately start with cron jobs, using the default email address of the account (defined in the .forward file or a local mail queue usually).

Defining an email address on the first line:

#Don't email anything by default
MAILTO=""
#email all cron job completions to admin@example.com by default
MAILTO="admin@example.com"

Each subsequent line looks like the following

* * * * * <command>
- - - - -
| | | | |
| | | | +----- day of week (0 - 6) (Sunday=0)
| | | +------- month (1 - 12)
| | +--------- day of month (1 - 31)
| +----------- hour (0 - 23)
+------------- minute (0 - 59)

In the above, if you enter * (the wildcard character), the job will run for all possible points of the value.  For example, if you do * for the minute value, this script will run at every minute.  If you want to run a job every second (not recommended), you can specify the minute value as ” */60 ” meaning run at every 60th of a minute.  This is not always supported, but for the majority of systems I use, it is.

The section is the command you would run via command line, for example, if you want to update the spam assassin rules every night at midnight, the command is “/usr/bin/sa-update“.  The crontab entry to do this nightly would be ” 0 0 * * * /usr/bin/sa-update “.

Emails: Finally, if you want a certain command to not email you regardless of completion, you can do the following (using the above example):

0 0 * * * /usr/bin/sa-update  > /dev/null 2>&1

Adding ” > /dev/null 2>&1 ” to the end of the command redirects the output of the script running to the null device (/dev/null), which is a special file that discards all data that is written to it.  Essentially /dev/null is the trash can.  The > symbol is a unix pipe symbol that redirects the output of the left side to a file descriptor (or device) located on the right side.  the “2>&1″ at the end says pipe the standard error output (device 2) to the standard output device (device 1), so this essentially unifies all error and regular messages from the program so that they both can be sent to /dev/null.  In case you’re wondering, device 0 is standard input from the command line.  You can understand this a lot more thoroughly by looking at Linux file descriptors and pipes.

Path: Cron usually invokes commands using (/bin/sh) which is the default shell on most systems, it typically starts in the user’s home directory with a very simplified PATH.  If you believe your cron isn’t working properly, try using absolute paths in your commands from /.

Credit: I’d like to finally credit http://adminschoice.com/crontab-quick-reference in helping me make my reference more comprehensive, I didn’t want to completely write this from my own experience.

The original script I posted does not verify that the backups actually completed.  The emails I got were simply gibberish.  I was willing to accept that for a while until during monthly maintenance when I manually verify backups, I was finding that backups didn’t always complete.  I’ve tracked this down in the error logs and found that the memory in the server isn’t enough for the backups at times which has them fail at certain times.  Due to the lack of user base on my server (4 light users), I can’t justify adding more ram (I currently have 4GB) because I’d have to upgrade the entire server.  So instead, I did some RAM optimization and re-wrote the backup script to email me the actual backup names that completed successfully.

First for the tips.  I recently learned that Active Directory can be modified from the backend, so using this, I modified the Information Store service (store.exe) in Exchange to only use at most 512MB of ram.  I used the information at http://terrytlslau.blogspot.com/2011/03/limiting-exchange-server-2007-and-2010.html for doing this, I briefly repeat the procedure here in the even that this link is no longer reachable.

1. At Domain Controller, login as a Domain Administrator.
2. Click "Start", enter "adsiedit.msc" into the search box, hit enter.
3. Right-click "ADSI Edit", select "Connect to".
4. Enable the Naming Context view, click ok to connect
4. Under the "Naming Context" menu, select "Configuration".
6. Expand to "Configuration > Services > Microsoft Exchange >  > Administrative Groups > Exchnage Administrative Group > Servers >  > InformationStore".
7. Right-click "InformationStore", select "Properties".
8. Select "msExchESEParamCacheSizeMax".
9. This value is set in pages, in Exchange 2010 the size is 32KB/page; Exchange 2007 is 8KB/page.  Simply figure out the number of pages for the amount of ram you want to limit store.exe to using.

For instance, if you want to limit the Database Cache to 4 GB of an Exchange 2010 server, set msExchESEParamCacheSizeMax to 131072 (4 GB = 4.194.304 KB / 32 KB). If you want to limit the Database Cache to 2 GB of an Exchange 2007 server, set msExchESEParamCacheSizeMax to 262144 (2 GB = 2.097.152 KB / 8KB).
10. Ok everything and restart the Information Store service (possibly the server)

After limiting the Exchange Information Store service, I simply restarted the Information Store service and that seems to have fixed the gouging memory issue.

As a second optimization procedure, I started tackling the IIS Worker Processes.  Exchange has several application pools that it uses, you can think of an application pool in IIS as a separate instance of Tomcat or Apache for each website.  Application Pools isolate websites to that they can’t affect each other.  On the downside, application pools also hog a great amount of memory and for the features of Exchange that you may not use often (e.g. powershell, calendar, exchange control panel), it takes some time for these features to load initially (for me, its about 30 seconds).  My solution was to limit Exchange to two application pools.  For anything service related, I used the Exchange Service Pool (e.g. EWS, Powershell, Autodiscover), and anything client site based (e.g. OWA, Calendar, ECP, ActiveSync) in the OWA pool.  I still do not know if any update to Exchange may reverse this or break this, but I do keep it in mind during updates.  The result of doing this is that not only is memory consumption reduced significantly, but Outlook Web Access, Exchange’s Calendar display (for the public), and Exchange Control Panel all load much faster now since the overhead in IIS is already loaded.  Of course, I wouldn’t recommend doing this unless you can’t easily upgrade the amount of memory in your server.

Finally, I will leave you with an improved export script that replaces the script in my previous blog at http://famousphil.com/blog/2011/01/a-decent-backup-strategy-for-exchange-2010-sp1/.  This script verifies that all of the users were actually uploaded and emails the complete report to you (instead of some garble).  I’ve found it to be very helpful in determining if a mailbox was failing to export to a PST without having to login to the file server and check.  As always, use this script at your own risk, I am willing to provide limited support as time permits.

# Exchange 2010 SP1 Mailbox Export Script
# Originally from Steve Goodman.
# Modified by Philip Matuskiewicz for Matthouse.us / famousphil.com 1/2/11 FIXED *7/2/11*

#define information here
$server = "host.example.com" #server hostname
$users = @("Joe", "Mary", "Phil") #users to archive
$destination = "localhostpstbackups" #network share to backup to
$emailfrom = "server@yourdomain.com"
$emailto = "you@yourdomain.com"
#define some internal variables
$output = ""
$error = 0
$date = Get-Date

#check for errors
if (!(Get-ExchangeServer $server -ErrorAction SilentlyContinue)){
    $output += "Exchange Server $server not found`n";
	$error = 1
}
if (!(Get-MailboxDatabase -Server $server -ErrorAction SilentlyContinue)){
    $output += "Exchange Server $server does not have mailbox databases";
	$error = 1
}

#create a batch job if the above tests succeeded
if ($error -ne 1){

	$jobname = "Export_$($date.Year)-$($date.Month)-$($date.Day)_$($date.Hour)-$($date.Minute)-$($date.Second)"
	$output += "Job title is: '$($jobname)' `n"
	Write-Output "Job title is: '$($jobname)' "

	foreach ($mailbox in $users){
		#remove existing PST file
		if (Get-Item "$($destination)$($mailbox).PST" -ErrorAction SilentlyContinue){
			Remove-Item "$($destination)$($mailbox).PST" -Confirm:$false
			$output += "Existing PST was deleted (Normal): '$($mailbox)' `n"
			Write-Output "Existing PST was deleted (Normal): '$($mailbox)' "
		} # end if

		#request a backup of the mailbox, Exclude the recoverable items / deleted items
		$mailboxjobname = "$($mailbox)-$($jobname)"
		New-MailboxExportRequest -BatchName $jobname -Mailbox $($mailbox) -FilePath "$($destination)$($mailbox).PST" -ExcludeDumpster -Name $mailboxjobname
		$output += "Mailbox Queued: '$($mailbox)' `n"
		Write-Output "Mailbox Queued: '$($mailbox)' "

	} #end foreach
} #end $error -ne 1

#wait for the jobs to complete
$time = 0;
while ((Get-MailboxExportRequest -BatchName $jobname | Where {$_.Status -eq "Queued" -or $_.Status -eq "InProgress"})){
	Write-Output "Waiting on backup, it has been $($time) seconds"
	$output += "Waiting on backup, it has been $($time) seconds `n"
	sleep 600 #10 minutes
	$time = $time + 720;
} #end while

#check for any jobs that didn't complete
$incomplete = Get-MailboxExportRequest -BatchName $jobname | Where {$_.Status -ne "Completed"} | Get-MailboxExportRequestStatistics | Format-List
$complete = Get-MailboxExportRequest -BatchName $jobname | Where {$_.Status -eq "Completed"} | Get-MailboxExportRequestStatistics | Format-List

if($incomplete){
	Write-Output "ERROR: Something didn't complete, output is '$($incomplete)'"
	$output += "ERROR: Something didn't complete, output is '$($incomplete)' `n"
}

if($complete){
	Write-Output "Completed Successfully, output is '$($complete)'"
	$output += "Completed Successfully, output is '$($complete)' `n"
}

# Remove Requests and clean up
Write-Output "Cleaning up requests that were part of the job '$($jobname)'"
$output += "Cleaning up requests that were part of the job '$($jobname)' `n"
Get-MailboxExportRequest -BatchName $jobname | Remove-MailboxExportRequest -Confirm:$false

#verify that all the PST files were created...
foreach ($mailbox in $users){
		#remove existing PST file
		if (Get-Item "$($destination)$($mailbox).PST" -ErrorAction SilentlyContinue){
			$output += "PST FOUND!!!: '$($mailbox)' `n"
			Write-Output "PST FOUND!!!: '$($mailbox)' "
		}else{
			$output += "ERROR: PST NOT FOUND: '$($mailbox)' `n"
			Write-Output "ERROR: PST NOT FOUND: '$($mailbox)' "
		}
}

$SmtpClient = new-object system.net.mail.smtpClient("double.matthouse.org")
$msg = new-object Net.Mail.MailMessage
$msg.From = "$($emailfrom)"
$msg.To.Add("$($emailto)")
$msg.Subject = "EXCHANGE EMAIL BACKUP DETAILS"
$msg.Body = $output
$SmtpClient.Send($msg)

Write-Output "Script complete!"

The zip process went through fine and created a zip archive of the directory, but this is because the zip was done on a 64 bit build of CentOS Linux (where zip is compiled for large file support).  I transferred the archive to the destination server (32 bit build of CentOS Linux).  As any decent admin does, I verified that the checksum (openssl md5 file.zip) matched on both ends (to check for corruption).  The md5 checksums matched, so I proceeded to unzip the file (unzip file.zip).  Below is the error I received:

End-of-central-directory signature not found. Either this file is nota zipfile, or it constitutes one disk of a multi-part archive. In the latter case the central directory and zipfile comment will be found on the last disk(s) of this archive.

unzip: cannot find zipfile directory in one of file.zip or
file.zip.zip, and cannot find file.zip.ZIP, period.

It turns out that the 32 bit version of CentOS includes the zip utility that isn’t compiled with large file support.  To rectify this, we must download the zip source code and recompile with large file support.  The instructions that I used are at http://www.stevenbarre.com/blog/2007/03/04/how-to-unzip-large-files-greater-than-2-gb/

1. install zlib and zlib_devel using yum if you don’t already have them.

2. Download the source code of unzip from http://www.info-zip.org/.

wget http://sourceforge.net/projects/infozip/files/UnZip%205.x%20and%20earlier/5.52/unzip552.tar.gz/download

3. Extract the source code and move the appropriate make file to the main directory

tar xzf unzip552.tar.gz
cd unzip-5.52/
mv unix/Makefile ./

4. Modify the MakeFile to enable large file support

Find:

CF="-O3 -Wall -I. -DASM_CRC $(LOC)"

Replace With:

CF="-O3 -Wall -I. -DASM_CRC -DLARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 $(LOC)"

5. Compile unzip

make linux

6. Use the executable unzip to extract file.zip

#assume file.zip is at /root/test/file.zip
mv unzip /root/test
cd ..
#clean up the directory, remove unzip's source code
rm -Rf unzip-5.52
rm unzip552.tar.gz
#unzip the file
cd /root/test
./unzip file.zip

Here is how you share a screen session:

#User 1, initiator of the screen session
screen -R sessionName

#User 2, connecting to an existing session
screen -x -R sessionName

The -R flag means try to reattach to an existing session, if the existing session does not exist, create a new session and attach to that.  The -x flag means to connect to an already attached session, which allows for session sharing.

I ran into a problem as a non-root user (that I normally run as) when I first started using shared screen sessions.  The error is as follows:

Cannot open your terminal '/dev/pts/0' - please check.

There are several terminal devices that screen can use under the /dev/pts directory, starting with 0.  Only root can traditionally access these terminal devices, therefore, we need to modify each necessary device to allow normal users access.  I didn’t research the security implications that the following fix has, so use this at your own risk.  The server that I was using has only trusted users on it, so I was not concerned about possible risks.

#run this as root, or as sudo
#this gives others read and write access to the device
chmod o+rw /dev/pts/0

Finally, up until last week, I could never get myself out of a screen session safely without completely closing my putty terminal window.  Here is the proper key sequence to detach from a screen session.

ctrl-A  followed by  ctrl-D

To terminate a session in screen, simply type exit.

Nagios is a server monitoring solution that can monitor entire systems and services running on those systems.  For Matthouse, Nagios runs on a reliable virtual private server that has a SLA to be up all of the time, the provider I choose has complete redundancy so that this VPS will not go down due to localized hardware, network, or power failures.  Nagios also runs on a system that is completely off-site from all of my other servers, so it can see if a datacenter goes down and notify me properly.  If you’re going to host Nagios yourself, I’d strongly suggest getting a similar setup since you don’t want your monitoring system to fail with everything else!

So let’s get into the actual installation of Nagios. (THIS WAS UPDATED 6/23/11)

We first need to install some supporting packages through yum and make a user that Nagios will work under.

yum install gcc glibc glibc-common gd gd-devel
/usr/sbin/useradd -d /home/nagios nagios
#You probably don't need to set the password
passwd nagios
<CHOOSE A PASSWORD>
groupadd nagcmd
usermod -a -G nagcmd nagios
usermod -a -G nagcmd apache

Now, as the user nagios, we want to download the latest version of Nagios.  My recommendation is to visit http://www.nagios.org/download/core/thanks/ and download the latest stable release.  As of this blog, the latest release is 3.2.3, which is what I will use in the next section.

su nagios
cd ~
wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.2.3.tar.gz
tar xzf nagios-3.2.3.tar.gz
cd nagios-3.2.3
./configure --with-command-group=nagcmd
make all
make install
#initialize the Nagios configuration files for the first time
make install-init
make install-config
make install-commandmode

Congratulations, Nagios is now installed.  Now you need to change to the /usr/local/nagios/etc directory and start modifying configuration files.  I’m going to provide some sample configuration files that I use in production, your environment may require different options to be specified.  Don’t worry about the check_nrpe command for now; we will be installing that onto the remote server(s) in a little while (before starting the Nagios service).

cat /usr/local/nagios/etc/nagios.cfg

log_file=/usr/local/nagios/var/nagios.log
cfg_file=/usr/local/nagios/etc/objects/commands.cfg
cfg_file=/usr/local/nagios/etc/objects/contacts.cfg
cfg_file=/usr/local/nagios/etc/objects/startup.cfg
cfg_file=/usr/local/nagios/etc/objects/templates.cfg
cfg_dir=/usr/local/nagios/etc/servers
object_cache_file=/usr/local/nagios/var/objects.cache
precached_object_file=/usr/local/nagios/var/objects.precache
resource_file=/usr/local/nagios/etc/resource.cfg
status_file=/usr/local/nagios/var/status.dat
status_update_interval=30
nagios_user=nagios
nagios_group=nagios
check_external_commands=1
command_check_interval=-1
command_file=/usr/local/nagios/var/rw/nagios.cmd
external_command_buffer_slots=4096
lock_file=/usr/local/nagios/var/nagios.lock
temp_file=/usr/local/nagios/var/nagios.tmp
temp_path=/tmp
event_broker_options=-1
log_rotation_method=d
log_archive_path=/usr/local/nagios/var/archives
use_syslog=1
log_notifications=1
log_service_retries=1
log_host_retries=1
log_event_handlers=1
log_initial_states=0
log_external_commands=1
log_passive_checks=1
service_inter_check_delay_method=s
max_service_check_spread=30
service_interleave_factor=s
host_inter_check_delay_method=s
max_host_check_spread=30
max_concurrent_checks=0
check_result_reaper_frequency=10
max_check_result_reaper_time=30
check_result_path=/usr/local/nagios/var/spool/checkresults
max_check_result_file_age=3600
cached_host_check_horizon=15
cached_service_check_horizon=15
enable_predictive_host_dependency_checks=1
enable_predictive_service_dependency_checks=1
soft_state_dependencies=0
auto_reschedule_checks=0
auto_rescheduling_interval=30
auto_rescheduling_window=180
sleep_time=0.25
service_check_timeout=60
host_check_timeout=30
event_handler_timeout=30
notification_timeout=30
ocsp_timeout=5
perfdata_timeout=5
retain_state_information=1
state_retention_file=/usr/local/nagios/var/retention.dat
retention_update_interval=60
use_retained_program_state=1
use_retained_scheduling_info=1
retained_host_attribute_mask=0
retained_service_attribute_mask=0
retained_process_host_attribute_mask=0
retained_process_service_attribute_mask=0
retained_contact_host_attribute_mask=0
retained_contact_service_attribute_mask=0
interval_length=30
check_for_updates=1
bare_update_check=0
use_aggressive_host_checking=0
execute_service_checks=1
accept_passive_service_checks=1
execute_host_checks=1
accept_passive_host_checks=1
enable_notifications=1
enable_event_handlers=1
process_performance_data=0
obsess_over_services=0
obsess_over_hosts=0
translate_passive_host_checks=0
passive_host_checks_are_soft=0
check_for_orphaned_services=1
check_for_orphaned_hosts=1
check_service_freshness=1
service_freshness_check_interval=60
check_host_freshness=0
host_freshness_check_interval=60
additional_freshness_latency=15
enable_flap_detection=0
low_service_flap_threshold=5.0
high_service_flap_threshold=20.0
low_host_flap_threshold=5.0
high_host_flap_threshold=20.0
date_format=us
p1_file=/usr/local/nagios/bin/p1.pl
enable_embedded_perl=1
use_embedded_perl_implicitly=1
illegal_object_name_chars=`~!$%^&*|'"?,()=
illegal_macro_output_chars=`~$&|'"
use_regexp_matching=0
use_true_regexp_matching=0
admin_email=root@localhost
admin_pager=root@localhost
daemon_dumps_core=0
use_large_installation_tweaks=0
enable_environment_macros=1
debug_level=0
debug_verbosity=1
debug_file=/usr/local/nagios/var/nagios.debug
max_debug_file_size=1000000

cat /usr/local/nagios/etc/objects/command.cfg

define command{
        command_name        notify-host-by-email
        command_line        /usr/bin/printf "%b" "***** Nagios *****nnNotification Type: $NOTIFICATIONTYPE$nHost: $HOSTNAME$nState: $HOSTSTATE$nAddress: $HOSTADDRESS$nInfo: $HOSTOUTPUT$nnDate/Time: $LONGDATETIME$n" | /bin/mail -s "** $NOTIFICATIONTYPE$ Host Alert: $HOSTNAME$ is $HOSTSTATE$ **" $CONTACTEMAIL$
}

define command{
        command_name        notify-service-by-email
        command_line        /usr/bin/printf "%b" "***** Nagios *****nnNotification Type: $NOTIFICATIONTYPE$nnService: $SERVICEDESC$nHost: $HOSTALIAS$nAddress: $HOSTADDRESS$nState: $SERVICESTATE$nnDate/Time: $LONGDATETIME$nnAdditional Info:nn$SERVICEOUTPUT$" | /bin/mail -s "** $NOTIFICATIONTYPE$ Service Alert: $HOSTALIAS$/$SERVICEDESC$ is $SERVICESTATE$ **" $CONTACTEMAIL$
}

define command{
        command_name        check_nrpe
        command_line        $USER1$/check_nrpe -H $HOSTADDRESS$ -p 5666 -c $ARG1$
}

define command{
        command_name        check_ping
        command_line        $USER1$/check_ping -H $HOSTADDRESS$ -w $ARG1$ -c $ARG2$ -p 5
}

define  command {
        command_name        check_bl
        command_line        $USER1$/check_bl -H $HOSTADDRESS$ -B zen.spamhaus.org bl.spamcop.net dnsbl.ahbl.org dnsbl.njabl.org dnsbl.sorbs.net virbl.dnsbl.bit.nl rbl.efnet.org phishing.rbl.msrbl.net 0spam.fusionzero.com list.dsbl.org multihop.dsbl.org unconfirmed.dsbl.org will-spam-for-food.eu.org blacklist.spambag.org blackholes.brainerd.net blackholes.uceb.org spamsources.dnsbl.info map.spam-rbl.com ns1.unsubscore.com psbl.surriel.com l2.spews.dnsbl.sorbs.net bl.csma.biz sbl.csma.biz dynablock.njabl.org no-more-funn.moensted.dk  ubl.unsubscore.com dnsbl-1.uceprotect.net dnsbl-2.uceprotect.net dnsbl-3.uceprotect.net spamguard.leadmon.net opm.blitzed.org bl.spamcannibal.org rbl.schulte.org dnsbl.ahbl.org virbl.dnsbl.bit.nl combined.rbl.msrbl.net
}

define command{
        command_name        check_nt
        command_line        $USER1$/check_nt -H $HOSTADDRESS$ -p 12489 -s nID83kjIUHShsi -v $ARG1$ $ARG2$
}

cat /usr/local/nagios/etc/objects/contacts.cfg
#You can define many contacts here

define contact{
        contact_name           nagiosadmin
        use                               generic-contact
        alias                             Nagios Admin
        email                           admin@matthouse.us
}
define contactgroup{
        contactgroup_name      admins
        alias                                  Nagios Administrators
        members                         nagiosadmin, nagiosadminpage
}</pre>

<pre>cat /usr/local/nagios/etc/objects/startup.cfg

define hostgroup{
        hostgroup_name   generic-servers
        alias            Production Server
}

define timeperiod{
        timeperiod_name 24x7
        alias           24 Hours A Day, 7 Days A Week
        sunday          00:00-24:00
        monday          00:00-24:00
        tuesday         00:00-24:00
        wednesday       00:00-24:00
        thursday        00:00-24:00
        friday          00:00-24:00
        saturday        00:00-24:00
}</pre>

Finally, we need to add servers.  Below are two sample configuration files, one that monitors the CPanel server (bit.matthouse.us), another that monitors our Exchange server(double.matthouse.us), and one that monitors the local monitoring server (short.matthouse.us).  We only want to configure servers after we have installed NRPE on the remote server(s) (as below).  For now, it is probably best to only configure the local server.

<pre>cat /usr/local/nagios/etc/servers/bit.cfg

define host{
        use                             generic-server
        host_name                       bit.matthouse.us
        alias                           Matthouse CPanel Web Server
        address                         64.85.172.184
}
define service{
        use                             generic-service
        host_name                       bit.matthouse.us
        service_description             Not on Any Spam Blacklists
        check_command                   check_bl
}
define service{
        use                             generic-service
        host_name                       bit.matthouse.us
        service_description             Disk: SMART STATUS System Drive
        check_command                   check_nrpe!check_smarta
}
define service{
        use                             generic-service
        host_name                       bit.matthouse.us
        service_description             PING
        check_command                   check_ping!150.0,20%!500.0,60%
}
define service{
        use                             generic-service
        host_name                       bit.matthouse.us
        service_description             Uptime
        check_command                   check_nrpe!check_uptime
}

define service{
        use                             generic-service
        host_name                       bit.matthouse.us
        service_description             Total Processes
        check_command                   check_nrpe!check_total_procs
}

define service{
        use                             generic-service
        host_name                       bit.matthouse.us
        service_description             SWAP
        check_command                   check_nrpe!check_swap
}

define service{
        use                             generic-service
        host_name                       bit.matthouse.us
        service_description             Current Load
        check_command                   check_nrpe!check_load
}

define service{
        use                             generic-service
        host_name                       bit.matthouse.us
        service_description             Disk: Root Partition
        check_command                   check_nrpe!check_root
}

define service{
        use                             generic-service
        host_name                       bit.matthouse.us
        service_description             Disk: Tmp Partition
        check_command                   check_nrpe!check_tmp
}

define service{
        use                             generic-service
        host_name                       bit.matthouse.us
        service_description             Disk: Home Partition
        check_command                   check_nrpe!check_home
}</pre>

<pre>cat /usr/local/nagios/etc/servers/double.cfg

define host{
        use                          generic-server
        host_name                    double.matthouse.us
        alias                        Double - Exchange 2010 Server
        address                      64.85.167.224
}
define service{
        use                          generic-service
        host_name                    double.matthouse.us
        service_description          PING
        check_command                check_ping!150.0,20%!500.0,60%
}
define service{
        use                          generic-service
        host_name                    double.matthouse.us
        service_description          NSClient++ Version
        check_command                check_nt!CLIENTVERSION
}
define service{
        use                          generic-service
        host_name                    double.matthouse.us
        service_description          Uptime
        check_command                check_nt!UPTIME
}
define service{
        use                          generic-service
        host_name                    double.matthouse.us
        service_description          CPU Load
        check_command                check_nt!CPULOAD!-l 60,99,100
}
define service{
        use                          generic-service
        host_name                    double.matthouse.us
        service_description          C: Drive Space
        check_command                check_nt!USEDDISKSPACE!-l c -w 80 -c 90
}
define service{
        use                          generic-service
        host_name                    double.matthouse.us
        service_description          D: Drive Space
        check_command                check_nt!USEDDISKSPACE!-l d -w 97 -c 99
}
define service{
        use                          generic-service
        host_name                    double.matthouse.us
        service_description          Not on Any Spam Blacklists
        check_command                check_bl
}</pre>

<pre>cat /usr/local/nagios/etc/servers/short.cfg

define host{
        use                     generic-server
        host_name               short.matthouse.us
        alias                   Short - Main Site, Backup, SVN, Monitoring Server
        address                 127.0.0.1
}

define command{
        command_name    check_disk
        command_line    $USER1$/check_disk -w $ARG1$ -c $ARG2$ -p $ARG3$
}
define command{
        command_name    check_localram
        command_line    $USER1$/check_mem -w 1 -c .1 -f
}
define  command {
        command_name    check_local_uptime
        command_line    $USER1$/check_uptime
}
define command{
        command_name    check_procs
        command_line    $USER1$/check_procs -w $ARG1$ -c $ARG2$ -s $ARG3$
}
define command{
        command_name    check_load
        command_line    $USER1$/check_load -w $ARG1$ -c $ARG2$
}

define service{
        use                             generic-service
        host_name                       short.matthouse.us
        service_description             Root Partition
        check_command                   check_disk!10%!5%!/
}
define service{
        use                             generic-service
        host_name                       short.matthouse.us
        service_description             Memory
        check_command                   check_localram
}
define service{
        use                             generic-service
        host_name                       short.matthouse.us
        service_description             Uptime
        check_command                   check_local_uptime
}
define service{
        use                             generic-service
        host_name                       short.matthouse.us
        service_description             Total Processes
            check_command               check_procs!250!400!RSZDT
}
define service{
        use                             generic-service
        host_name                       short.matthouse.us
        service_description             Current Load
        check_command                   check_load!5.0,4.0,3.0!10.0,6.0,4.0
}
define service{
        use                             generic-service
        host_name                       short.matthouse.us
        service_description             Not on Any Spam Blacklists
        check_command                   check_bl
}</pre>

Up next, we need to create a username and password for the Nagios configuration web page (to protect it).

<pre>cd ~nagios/nagios-3.2.3
make install-webconf
htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
#enter a password, the user nagiosadmin will be the username to the Nagios website interface
#exit to root to execute the following command
service httpd restart</pre>

There is a small chance that I modified the Apache configuration to make http://www.matthouse.us/nagios the website interface.  Therefore, I am including the configuration file that works on my server.  You will want to remove the suPHP directive if your server doesn’t run suPHP (unlike mine).  If you are using suPHP, you will need to modify your suphp.conf file (under /etc), I had to set “docroot=/” since my web files were under /home and nagios is under /usr.  This isn’t a huge security risk provided none of your PHP scripts are owned by root, but you may want to install nagios under /home to tighten this up slightly.

As an aside, suPHP is a module for Apache that runs all PHP scripts as the user who owns those scripts (instead of apache or nobody).  This helps isolate PHP attacks to only the files that the user owns.  suPHP is fairly easy to install so I will leave that up to the user.  If someone requests the installation procedure, I might blog about it in the future.

<pre>cat /etc/httpd/conf.d/nagios.conf

ScriptAlias /nagios/cgi-bin "/usr/local/nagios/sbin"
<Directory "/usr/local/nagios/sbin">
# SSLRequireSSL
Options ExecCGI
AllowOverride None
Order allow,deny
Allow from all
# Order deny,allow
# Deny from all
# Allow from 127.0.0.1
AuthName "Nagios Access"
AuthType Basic
AuthUserFile /usr/local/nagios/etc/htpasswd.users
Require valid-user
suPHP_Engine on
suPHP_UserGroup nagios nagios
</Directory>
Alias /nagios "/usr/local/nagios/share"
<Directory "/usr/local/nagios/share">
# SSLRequireSSL
Options None
AllowOverride None
Order allow,deny
Allow from all
# Order deny,allow
# Deny from all
# Allow from 127.0.0.1
AuthName "Nagios Access"
AuthType Basic
AuthUserFile /usr/local/nagios/etc/htpasswd.users
Require valid-user
suPHP_Engine on
suPHP_UserGroup nagios nagios
</Directory></pre>

Next, we need to install nagios-plugins to enable the actual server check commands.  These will also be installed on each server that we monitor using NRPE.

<pre>su nagios
cd ~
#get the latest plugins from http://www.nagios.org/download/plugins/
wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.15.tar.gz
tar xzf nagios-plugins-1.4.15.tar.gz
cd nagios-plugins-1.4.15
export LDFLAGS=-ldl
./configure --with-nagios-user=nagios --with-nagios-group=nagios --enable-redhat-pthread-workaround
make
make install</pre>

We will now install NRPE to enable monitoring of remote Linux servers.

<pre>#download the latest NRPE from http://sourceforge.net/projects/nagios/files/nrpe-2.x/
#as nagios user
cd ~
wget http://sourceforge.net/projects/nagios/files/nrpe-2.x/nrpe-2.12/nrpe-2.12.tar.gz/download
tar xvfz nrpe-2.12.tar.gz
cd nrpe-2.1.2
./configure
make all
make install-plugin</pre>

We will need to enable the check_bl plugin to check spam blacklists.  This isn’t included by the nagios-plugins, so we create the script manually.

<pre>touch /usr/local/nagios/libexec/check_bl
chmod 755 /usr/local/nagios/libexec/check_bl
vim /usr/local/nagios/libexec/check_bl

#!/usr/bin/perl -w
#
# check_bl plugin for nagios
# $Revision: 1.0 $
#
# Nagios plugin designed to warn you if you mail servers appear in one of the
# many anti-spam 'blacklists'
#
# By Sam Bashton, Bashton Ltd
# bashton.com/content/nagios-plugins
#
#    This program is free software; you can redistribute it and/or modify
#    it under the terms of the GNU General Public License as published by
#    the Free Software Foundation; either version 2 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program; if not, write to the Free Software
#    Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA  02111-1307  USA

use strict;
use lib "/usr/local/nagios/libexec/";
use utils qw($TIMEOUT %ERRORS &amp;print_revision &amp;support);
use Net::DNS;
use vars qw($PROGNAME);
my ($verbose,$host),;
my ($opt_V,$opt_h,$opt_B,$opt_H,$opt_c);
$opt_V = $opt_h = $opt_B = $opt_H = $opt_c = '';
my $state = 'UNKNOWN';
sub print_help();
sub print_usage();

$PROGNAME = "check_bl";

$ENV{'BASH_ENV'}='';
$ENV{'ENV'}='';
$ENV{'PATH'}='';
$ENV{'LC_ALL'}='C';

use Getopt::Long;
Getopt::Long::Configure('bundling');
GetOptions(
  "V"   => $opt_V,   "version"       => $opt_V,
  "h"   => $opt_h,   "help"          => $opt_h,
  "H=s" => $opt_H,   "hostname=s"    => $opt_H,
  "B=s" => $opt_B,   "blacklists=s"  => $opt_B,
  "c=s" => $opt_c,   "critical=s"    => $opt_c
);

# -h means display verbose help screen
if ($opt_h) { print_help(); exit $ERRORS{'OK'}; }

# -V means display version number
if ($opt_V) {
  print_revision($PROGNAME,'$Revision: 1.0 $ ');
  exit $ERRORS{'OK'};
}

# First check the hostname is OK..
unless ($opt_H) { print_usage(); exit $ERRORS{'UNKNOWN'}; }

if (! utils::is_hostname($opt_H)){
  print "$opt_H is not a valid host namen";
  print_usage();
  exit $ERRORS{"UNKNOWN"};
}else{
  if ($opt_H =~ /[a-zA-Z]/ )
  # If the host contains letters we assume it's a hostname, not an IP
  {
    $host = lookup($opt_H);
  }
  else { $host = $opt_H }
}

# $opt_c is a count of the blacklists a mail server is in,
# after which state will be CRITICAL rather than WARNING
# By default any listing is CRITICAL
my $critcount = 0;
if ($opt_c) { $critcount = $opt_c };

# $opt_B is a comma seperated list of blacklists
$opt_B = shift unless ($opt_B);
unless ($opt_B) { print_usage(); exit -1 }
my @bls = split(/,/, $opt_B);

# Just in case of problems, let's not hang Nagios
$SIG{'ALRM'} = sub {
  print ("ERROR: No response from BL server (alarm)n");
  exit $ERRORS{"UNKNOWN"};
};
alarm($TIMEOUT);

my %listed; # Hash of blacklists we're listed in.
foreach(@bls)
{
  if (blcheck($host,$_)) { $listed{$_} = 1 }
}

if (scalar(keys(%listed)) == 0) { $state = 'OK' }
elsif (scalar(keys(%listed)) < $critcount) { $state = 'WARNING' }
else { $state = 'CRITICAL' }

if (%listed)
{
  print "Listed at";
  foreach (keys(%listed)) { print " $_" }
  print "n";
}
else { print "Not black-listedn" }

exit $ERRORS{$state};

########  Subroutines ==========================

sub print_help() {
  print_revision($PROGNAME,'$Revision: 1.0 $ ');
  print "n";
  support();
}

sub print_usage () {
  print "Usage: n";
  print " $PROGNAME -H host -B [blacklist1],[blacklist2] [-c critnum]n";
  print " $PROGNAME [-h | --help]n";
  print " $PROGNAME [-V | --version]n";
}

sub blcheck
{
  my ($ip, $bl) = @_;
  my $lookupip = $ip;
  $lookupip =~
  s/([0-9]{1,3}).([0-9]{1,3}).([0-9]{1,3}).([0-9]{1,3})/$4.$3.$2.$1.$bl/;
  if (lookup($lookupip)) { return 1 }
  else { return 0 }
}

sub lookup
{
  my $tolookup = shift;
  my $res = Net::DNS::Resolver->new;
  my $query = $res->search($tolookup);
  if ($query)
  {
    foreach my $rr ($query->answer)
    {
      next unless $rr->type eq "A"; # We're not interested in TXT records
      return $rr->address;
    }
  }
}</pre>

It’s now time to enable Nagios on the monitoring server. We will execute the following command to verify that there are no errors, this might have to be done as root, I can’t remember.  Finally, as root, you will need to add Nagios to the automatic startup list and start it.

<pre>#exit to root
exit
/usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
#if no errors, continue
chkconfig --add nagios
chkconfig nagios on
service nagios start</pre>

Finally, to check that Nagios is working, login using http:///nagios using the username and password that you defined above with the htpasswd command.  The username is most likely nagiosadmin.

I will now cover how to remotely monitor a remote machine.

Let’s first cover a Linux box.

<pre>yum install xinetd
chkconfig xinetd on
service xinetd start

useradd -d /home/nagios nagios
passwd nagios
<CHOOSE A PASSWORD>
groupadd nagcmd
usermod -a -G nagcmd nagios</pre>
<pre>cd ~nagios
#get the latest plugins from http://www.nagios.org/download/plugins/
wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.15.tar.gz
tar xzf nagios-plugins-1.4.15.tar.gz
cd nagios-plugins-1.4.15
export LDFLAGS=-ldl
./configure --with-nagios-user=nagios --with-nagios-group=nagios --enable-redhat-pthread-workaround
make
make install

cd ~
wget http://sourceforge.net/projects/nagios/files/nrpe-2.x/nrpe-2.12/nrpe-2.12.tar.gz/download
tar xvfz nrpe-2.12.tar.gz
cd nrpe-2.12
./configure
make all
make install-plugin
make install-daemon
make install-daemon-config
make install-xinetd</pre>
<pre>chown -R nagios:nagcmd /usr/local/nagios</pre>

We now need to modify some files to enable NRPE’s service.

<pre>#Add to /etc/xinetd.d/nrpe:
only_from       = 127.0.0.1 98.142.216.195

#Add to /etc/services
nrpe 5666/tcp # NRPE

service xinetd restart

#finally test NRPE

netstat -at | grep nrpe
#you will see something like:
#       tcp 0      0 *:nrpe *:*                         LISTEN

#yet another test
/usr/local/nagios/libexec/check_nrpe -H localhost
#you should see:
#   NRPE v2.12</pre>

Finally, we need to modify a file to enable the remote check_nrpe command from the Nagios monitoring server.  At this point, you will also want to unblock port 5656 in both directions on both the monitoring server and remote server so that NRPE can freely communicate.  In my case, my backup server ignores / allows all traffic from internal network servers, so this wasn’t an issue for me.

<pre>cat /usr/local/nagios/etc/nrpe.cfg
#from my CPanel server, I only included the COMMAND DEFINITIONS section at the end of the file

command[check_smarta]=/usr/local/nagios/libexec/check_ide_smart -d /dev/sda
command[check_uptime]=/usr/local/nagios/libexec/check_uptime
command[check_total_procs]=/usr/local/nagios/libexec/check_procs -w 250 -c 300
command[check_swap]=/usr/local/nagios/libexec/check_swap -w 10% -c 5%
command[check_load]=/usr/local/nagios/libexec/check_load -w 25,20,15 -c 30,25,20
command[check_sda1]=/usr/local/nagios/libexec/check_disk -w 20% -c 10% -p /dev/sda1
command[check_mem]=/usr/local/nagios/libexec/check_mem -w .2 -c .1 -f
command[check_users]=/usr/local/nagios/libexec/check_users -w 5 -c 10</pre>

Smart Status checking (a way to determine if a hard disk is going to die before it actually dies) in Nagios has some special permissions requirements.  Below is a list of commands you can execute to make the check_ide_smart script work correctly.

<pre>chown root:root /usr/local/nagios/libexec/check_ide_smart
chmod u+s /usr/local/nagios/libexec/check_ide_smart
chmod o+x /usr/local/nagios/libexec/check_ide_smart
ls -l /usr/local/nagios/libexec/check_ide_smart
#should look like
#-rwsr-x--x 1 root root 18052 Jun 11 21:56 /usr/local/nagios/libexec/check_ide_smart</pre>

To enable uptime monitoring, we need to make a script for this:

<pre>cat /usr/local/nagios/libexec/check_uptime

#!/bin/sh
echo "OK `uptime`"</pre>

To enable memory checks, we have to make the check_mem executable in the libexec directory.

<pre>touch /usr/local/nagios/libexe/check_mem
chmod 755 /usr/local/nagios/libexe/check_mem
vim /usr/local/nagios/libexe/check_mem

#!/usr/bin/perl -w
# $Id: check_mem.pl 2 2002-02-28 06:42:51Z egalstad $
# Original script stolen from:
# check_mem.pl Copyright (C) 2000 Dan Larsson <dl@tyfon.net>
# hacked by
# Justin Ellison <justin@techadvise.com>
#
# This program is free software; you can redistribute it and/or
# modify it under the terms of the GNU General Public License
# as published by the Free Software Foundation; either version 2
# of the License, or (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty
# of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# you should have received a copy of the GNU General Public License
# along with this program (or with Nagios);  if not, write to the
# Free Software Foundation, Inc., 59 Temple Place - Suite 330,
# Boston, MA 02111-1307, USA

# Tell Perl what we need to use
use strict;
use Getopt::Std;

#TODO - Convert to Nagios::Plugin
#TODO - Use an alarm

# Predefined exit codes for Nagios
use vars qw($opt_c $opt_f $opt_u $opt_w $opt_C $opt_v %exit_codes);
%exit_codes   = ('UNKNOWN' ,-1,
                         'OK'      , 0,
                 'WARNING' , 1,
                 'CRITICAL', 2,
                 );

# Get our variables, do our checking:
init();

# Get the numbers:
my ($free_memory_kb,$used_memory_kb,$caches_kb) = get_memory_info();
print "$free_memory_kb Freen$used_memory_kb Usedn$caches_kb Cachen" if ($opt_v);

if ($opt_C) { #Do we count caches as free?
    $used_memory_kb -= $caches_kb;
    $free_memory_kb += $caches_kb;
}

# Round to the nearest KB
$free_memory_kb = sprintf('%d',$free_memory_kb);
$used_memory_kb = sprintf('%d',$used_memory_kb);
$caches_kb = sprintf('%d',$caches_kb);

# Tell Nagios what we came up with
tell_nagios($used_memory_kb,$free_memory_kb,$caches_kb);

sub tell_nagios {
    my ($used,$free,$caches) = @_;

    # Calculate Total Memory
    my $total = $free + $used;
    print "$total Totaln" if ($opt_v);

    my $perfdata = "|TOTAL=${total}KB;;;; USED=${used}KB;;;; FREE=${free}KB;;;; CACHES=${caches}KB;;;;";

    if ($opt_f) {
      my $percent    = sprintf "%.1f", ($free / $total * 100);
      if ($percent <= $opt_c) {
          finish("CRITICAL - $percent% ($free kB) free!$perfdata",$exit_codes{'CRITICAL'});
      }
      elsif ($percent <= $opt_w) {
          finish("WARNING - $percent% ($free kB) free!$perfdata",$exit_codes{'WARNING'});
      }
      else {
          finish("OK - $percent% ($free kB) free.$perfdata",$exit_codes{'OK'});
      }
    }
    elsif ($opt_u) {
      my $percent    = sprintf "%.1f", ($used / $total * 100);
      if ($percent >= $opt_c) {
          finish("CRITICAL - $percent% ($used kB) used!|$perfdata",$exit_codes{'CRITICAL'});
      }
      elsif ($percent >= $opt_w) {
          finish("WARNING - $percent% ($used kB) used!|$perfdata",$exit_codes{'WARNING'});
      }
      else {
          finish("OK - $percent% ($used kB) used.|$perfdata",$exit_codes{'OK'});
      }
    }
}

# Show usage
sub usage() {
  print "ncheck_mem.pl v1.0 - Nagios Pluginnn";
  print "usage:n";
  print " check_mem.pl -<f|u> -w <warnlevel> -c <critlevel>nn";
  print "options:n";
  print " -f           Check FREE memoryn";
  print " -u           Check USED memoryn";
  print " -C           Count OS caches as FREE memoryn";
  print " -w PERCENT   Percent free/used when to warnn";
  print " -c PERCENT   Percent free/used when criticaln";
  print "nCopyright (C) 2000 Dan Larsson <dl@tyfon.net>n";
  print "check_mem.pl comes with absolutely NO WARRANTY either implied or explicitn";
  print "This program is licensed under the terms of then";
  print "GNU General Public License (check source code for details)n";
  exit $exit_codes{'UNKNOWN'};
}

sub get_memory_info {
    my $used_memory_kb  = 0;
    my $free_memory_kb  = 0;
    my $total_memory_kb = 0;
    my $caches_kb       = 0;

    my $uname;
    if ( -e '/usr/bin/uname') {
        $uname = `/usr/bin/uname -a`;
    }
    elsif ( -e '/bin/uname') {
        $uname = `/bin/uname -a`;
    }
    else {
        die "Unable to find uname in /usr/bin or /bin!n";
    }
    print "uname returns $uname" if ($opt_v);
    if ( $uname =~ /Linux/ ) {
        my @meminfo = `/bin/cat /proc/meminfo`;
        foreach (@meminfo) {
            chomp;
            if (/^Mem(Total|Free):s+(d+) kB/) {
                my $counter_name = $1;
                if ($counter_name eq 'Free') {
                    $free_memory_kb = $2;
                }
                elsif ($counter_name eq 'Total') {
                    $total_memory_kb = $2;
                }
            }
            elsif (/^(Buffers|Cached):s+(d+) kB/) {
                $caches_kb += $2;
            }
        }
        $used_memory_kb = $total_memory_kb - $free_memory_kb;
    }
    elsif ( $uname =~ /SunOS/ ) {
        eval "use Sun::Solaris::Kstat";
        if ($@) { #Kstat not available
            if ($opt_C) {
                print "You can't report on Solaris caches without Sun::Solaris::Kstat available!n";
                exit $exit_codes{UNKNOWN};
            }
            my @vmstat = `/usr/bin/vmstat 1 2`;
            my $line;
            foreach (@vmstat) {
              chomp;
              $line = $_;
            }
            $free_memory_kb = (split(/ /,$line))[5] / 1024;
            my @prtconf = `/usr/sbin/prtconf`;
            foreach (@prtconf) {
                if (/^Memory size: (d+) Megabytes/) {
                    $total_memory_kb = $1 * 1024;
                }
            }
            $used_memory_kb = $total_memory_kb - $free_memory_kb;

        }
        else { # We have kstat
            my $kstat = Sun::Solaris::Kstat->new();
            my $phys_pages = ${kstat}->{unix}->{0}->{system_pages}->{physmem};
            my $free_pages = ${kstat}->{unix}->{0}->{system_pages}->{freemem};
            # We probably should account for UFS caching here, but it's unclear
            # to me how to determine UFS's cache size.  There's inode_cache,
            # and maybe the physmem variable in the system_pages module??
            # In the real world, it looks to be so small as not to really matter,
            # so we don't grab it.  If someone can give me code that does this,
            # I'd be glad to put it in.
            my $arc_size = (exists ${kstat}->{zfs} &amp;&amp; ${kstat}->{zfs}->{0}->{arcstats}->{size}) ?
                 ${kstat}->{zfs}->{0}->{arcstats}->{size} / 1024
                 : 0;
            $caches_kb += $arc_size;
            my $pagesize = `pagesize`;

            $total_memory_kb = $phys_pages * $pagesize / 1024;
            $free_memory_kb = $free_pages * $pagesize / 1024;
            $used_memory_kb = $total_memory_kb - $free_memory_kb;
        }
    }
    else {
        if ($opt_C) {
            print "You can't report on $uname caches!n";
            exit $exit_codes{UNKNOWN};
        }
        my $command_line = `vmstat | tail -1 | awk '{print $4,$5}'`;
        chomp $command_line;
        my @memlist      = split(/ /, $command_line);

        # Define the calculating scalars
        $used_memory_kb  = $memlist[0]/1024;
        $free_memory_kb = $memlist[1]/1024;
        $total_memory_kb = $used_memory_kb + $free_memory_kb;
    }
    return ($free_memory_kb,$used_memory_kb,$caches_kb);
}

sub init {
    # Get the options
    if ($#ARGV le 0) {
      &amp;usage;
    }
    else {
      getopts('c:fuCvw:');
    }

    # Shortcircuit the switches
    if (!$opt_w or $opt_w == 0 or !$opt_c or $opt_c == 0) {
      print "*** You must define WARN and CRITICAL levels!n";
      &amp;usage;
    }
    elsif (!$opt_f and !$opt_u) {
      print "*** You must select to monitor either USED or FREE memory!n";
      &amp;usage;
    }

    # Check if levels are sane
    if ($opt_w <= $opt_c and $opt_f) {
      print "*** WARN level must not be less than CRITICAL when checking FREE memory!n";
      &amp;usage;
    }
    elsif ($opt_w >= $opt_c and $opt_u) {
      print "*** WARN level must not be greater than CRITICAL when checking USED memory!n";
      &amp;usage;
    }
}

sub finish {
    my ($msg,$state) = @_;
    print "$msgn";
    exit $state;
}</pre>

Finally modify permissions of these scripts.

<pre>chown -R nagios:nagcmd /usr/local/nagios/libexec/check_mem
chown -R nagios:nagcmd /usr/local/nagios/libexec/check_uptime
chmod 755 /usr/local/nagios/libexec/check_uptime
chmod 755 /usr/local/nagios/libexec/check_mem</pre>

On Windows, we will use nsclient++.  At the time of this writing, the latest stable version is 0.3.8 and we want the msi installer file for it (either 32 bit or 64 bit).  Once its downloaded, install the msi with all the default options.

Then modify the file at C:Program FilesNSClient++NSC.ini.  Here is what I did:

• Uncomment all the modules listed in the [modules] section, except for CheckWMI.dll and RemoteConfiguration.dll
• Optionally require a password for clients by changing the ‘password’ option in the [Settings] section.
• Uncomment the ‘allowed_hosts’ option in the [Settings] section. Add the IP address of the Nagios server to this line, or leave it blank to allow all hosts to connect.
• Make sure the ‘port’ option in the [NSClient] section is uncommented and set to ’12489′ (the default port).

Then open up an elevated command prompt (right click on command prompt and run as administrator).

<pre>cd c:program filesnsclient++
nsclient++ /install</pre>

Finally, reboot the system (or start the service from the services control panel) and Nagios should be able to monitor your Windows server!

My final bit is on how to upgrade Nagios in the future. To upgrade Nagios, you will want to download the latest archives of nagios-core, nagios-plugins, and nrpe/nsclient++.  You will install all of them as above, except for, once you’re done with the make install step, you are finished with the upgrade.  Nagios doesn’t touch the existing configuration files, so it’s very easy to upgrade.

With all of the above said, Nagios is actually very easy to configure and install once you understand it.  I’m hopeful that this blog will help you understand Nagios better and have more confidence in deploying it on your systems!  As always, thanks for reading!

A webmaster has many things to think about from choosing the right host to making sure everything is running smoothly on the backend servers. Fortunately, different tools, applications and developments exist that can help us simplify the process of running a website. Subtext skins can help designers more effectively and efficiently create the styles used for personal blogs on their website.

While you can always decrease your workload by choosing options like managed hosting, if you are responsible for web design then you should look into the many advantages of Subtext skins for blogging.

What is a Subtext skin?
Subtext is a blogging platform offered as open source software under the BSD license. The whole concept behind Subtext was to create a very simplified blogging engine that allows bloggers to concentrate on creating actual blog content rather than trying to figure out how to use the blogging software.

Almost all websites these days have their own blogs. In addition, many sites offer blogging as a service to their users who can sign up to create their own personal blogs. You can use Subtext both for the site’s own blog and to offer blogs to site members.

A Subtext skin refers to the styling and layout of the blog page. Skins are versatile as they allow you to create page formats and designs without coding from scratch. A Subtext skin is actually made up of five separate style sheets that can be used to format different elements on the page like divs, spans, boxes, headers, footers, sidebars and forms.

The five style sheets used for each Subtext skin are:

• style.css – the default style sheet that determines the basic underlying skin layout.
• secondary style sheet – this file handles the specific Stylesheet attribute of the skin. The secondary style sheet is generally used to create styling contrasts over the style.css file. For example, the web designer may choose to give the header a different border than the sidebars where in the style.css file they use the same border.
• custom.css – this style sheet is used for custom design features set by the blog author. Primarily used to set styles and layout for personal badges or for the actual post content.
• non-attribute css files – generally used for CSS frameworks or for system styles.
• css files with limited attributes – these stylesheets have a title and media attribute, and cannot be merged with other css files. The attributes specify IE version compatibility and that they are applicable only in screen mode or in printing mode.

Packaged and custom Subtext skins
Web designers can choose from a large library of pre-designed skins or they can create their own custom skins. Creating a custom Subtext skin requires some basic knowledge of CSS scripting and also learning the basic Subtext parsing rules.

Fortunately, Subtext is highly simplified to allow users to quickly master the underlying script.

Skin templates are folders that can be used to render different Subtext skins. Each folder actually contains a number of skins that are related in certain attributes and themes. The folders or templates have their own series of controls that are used to render a skin in that template along with associated style sheets.

Once you have a library of packaged and/or custom skins, you will be able to style blogs quickly after learning how to tweak the skins and templates.

Instead of racking your brain each time you need to come up with a new design, you can simply browse through your selection of Subtext skins. Each skin can be modified according to the primary and secondary CSS files associated with the skin. You can easily find the types of skins you are looking for because they will be arranged in skin templates or families according to related attributes. For example, you can have a Rainbow template. a Geometric template and an Origami template – each having multiple skins that express the same theme. The number of templates is only limited by your imagination and willingness to create custom skins.

When using the Subtext blogging software, the skin templates are arranged in folders in an easy to manage directory system. The setup makes it simple to find the skin you are looking for and simply click on the selection to implement it on the Subtext blog.

By using Subtext skins, you can save time and energy creating page designs and layouts for multiple blogs. No need to code style sheets from scratch when you can access an easy-to-use Subtext skin library and quickly find what you need.

Each skin will handle the styling and layout of all page elements including headers, footers, sidebars, text format, boxes and forms.

You can find out more about Subtext and skins at the Subtext Project Site: http://www.subtextproject.com/.

#get latest pdflib lite 7 from http://www.pdflib.com/download/
#http://www.pdflib.com/download/pdflib-family/pdflib-lite-7/

cd ~
wget http://www.pdflib.com/binaries/PDFlib/705/PDFlib-Lite-7.0.5.tar.gz
tar xzf PDFlib-Lite-7.0.5.tar.gz
cd PDFlib-Lite-7.0.5
#java isn't installed on the server, compile without java
./configure --prefix=/usr/local/pdflib --without-java
make
make install
pecl install pdflib 
#when asked for a path, enter "/usr/local/pdflib" then hit <enter>

I opted to install the lite version of PDFLib since its free and my client didn’t need all of the features that the program includes.  I installed pdflib to its own directory that isn’t included in path since I don’t anticipate anyone actually using this via other languages on the server (besides PHP).  Once PDFLib is installed, we will need to use the command line (as root) to install the pecl extension and tell it where to find the compiled version of pdflib.  Note that CPanel’s PECL installer will not be able to install this extension since there is no way to enter the path (and I’d prefer to not add this to the global PATH variable on my server).  The extension is now installed, in order to test the extension, make a PHP file for the below code and execute it in a web browser, you should see “Hello World” as a PDF document.

<?php

try {
$p = new PDFlib();

/*  open new PDF file; insert a file name to create the PDF on disk */
if ($p->begin_document("", "") == 0) {
die("Error: " . $p->get_errmsg());
}

$p->set_info("Creator", "hello.php");
$p->set_info("Author", "AUTHOR");
$p->set_info("Title", "Hello world (PHP)!");

$p->begin_page_ext(595, 842, "");

$font = $p->load_font("Helvetica-Bold", "winansi", "");

$p->setfont($font, 24.0);
$p->set_text_pos(50, 700);
$p->show("Hello world!");
//$p->continue_text("continue text");
$p->end_page_ext("");

$p->end_document("");

$buf = $p->get_buffer();
$len = strlen($buf);

header("Content-type: application/pdf");
header("Content-Length: $len");
header("Content-Disposition: inline; filename=hello.pdf");
print $buf;
}
catch (PDFlibException $e) {
die("PDFlib exception occurred in hello sample:n". $e->get_errnum() ." " . $e->get_apiname() . " : " .$e->get_errmsg() . "nn");
}
catch (Exception $e) {
die($e);
}
$p = 0;
?>

Source: I used the guide at http://www.supportfacility.com/blog/cpanel/install-pdflib-php-on-cpanel-dedicated-server/ to write an updated version for my blog.  I’d like to thank them for blogging about this topic!

Gitorious is an open source repository management system similar to Trac on SVN.  The software looks really nice (gitorious.org), and it has a lot of built in features for managing git repositories.  For those of you who are not familiar with coding, code repositories are often set up to enable many coders to work on the same project simultaneously (different portions of course).  Repositories also enable versioning so that you can easily revert to an older copy of your code base if something breaks along the way.  Repositories are often centralized so that only a single location has to be backed up, in this case Gitorious centralizes git repositories.

Truthfully, there are three memorable software programs that have been notoriously difficult to install and configure in my experience, these are as follows:

• Shibboleth- I never did successfully install this myself, this was because I was asked to help a friend install it over the phone.  I don’t work well when I can’t see the problem and play with stuff, I test hunches, and doing it with this method would have taken way too long.  From what I’ve seen of Shibboleth, I would rate this as worst of the worst programs to install.
• Gitorious – This is probably the second worst application I’ve ever installed.  It required a lot of research and resources since it is poorly documented and requires a lot of Ruby Gems to operate successfully.  Of course, I will hopefully document it a little more today!
• Exchange – I wish I would have documented the install of my Exchange server internally so I could have shared it here.  Exchange isn’t awful to install, but the configuration of Exchange is very tricky and there are many gotchas.  This is why I place it as the best of the worst software programs to install.

With all of the above said, the installation process of gitorious isn’t for the leisure system administrator to install.  This took me approximately 3 days of research to successfully install and understand.  I installed it on a separate system only because I didn’t want it hurting any of my existing production systems.  In the end, I’m glad I did this because I learned that Ruby isn’t very memory efficient and this application easily eats up most of the 1GB of ram allocated to it in a VPS (Virtual Private Server).  And with this said, I invite you to continue reading if you really want to know how to install this software.

Update 6/14: After a few requests, I’ve decided that I will provide a Gitorious installation service on your CentOS server.  I will be happy to install Gitorious and make it work (as described below) for a one time paypal payment of $100.  For details, contact me directly (see my contact page).

LAST UPDATE: 6/8/2011 – Suggestions from Gitorious Discussion Group

Before I begin, I’d like to note that although I used OpenVZ for my deployment of Gitorious, it should work on Xen and other platforms  (even dedicated servers).  I say should because I originally deployed this on Amazon Web Services, and the installation process had a few changes here and there (due to errors with package dependencies, etc) when I finally moved it over to OpenVZ.  The procedure below was used for an OpenVZ machine as detailed below.  Finally, my last note is that you can expect this installation process to take roughly an hour or two to complete as a lower bound (provided everything goes well).

I decided to use a VPS from chicagovps to host this software program because of a good past experience with them.  Although I normally bash OpenVZ virtualization (with really good reasons because Xen is better), I chose to use OpenVZ since it is typically faster and I wanted performance.  OpenVZ doesn’t have swap space, and its memory management isn’t that great, so in the end, applications can use almost 3x the amount of memory on OpenVZ VPS containers compared to running on equivalent Xen containers.  I can easily see Xen requiring less than 384MB of dedicated Ram for Gitorious.  On OpenVZ, I strongly suggest not getting less than 1GB of dedicated memory.  With ChicagoVPS, I got the professional package, and my memory usage hovers around 800MB for this install.  For the Operating System, I chose the CentOS 5.5 x64 template and using yum update, the operating system became CentOS 5.6.  Everything below is performed as root unless otherwise noted.

The first thing that I did was set my DNS to have 2 A-records pointing to the hostname of my server (string.matthouse.us), and another that points to the future Gitorious website URL (git.matthouse.us).  After that was done, I logged into the VPS and set up public keys for the root account to make logging into the machine easier in the future.  I’ve written two blogs about public key authentication, so I won’t detail this portion of the process (that’s optional anyways).  I’d also recommend setting up a secure root password (also mentioned sometime in the past).

Next, I’d recommend disabling SELinux if it is enabled, thankfully, on VPS nodes, its typically disabled out of the box.  If you’re on Xen, you might want to make and enable a swap file (which I detailed on my AWS install of Tomcat blog).  The next step is to execute the following commands, which remove common packages that you won’t need and turn off common services which you won’t use.  In this stack of commands, you will also need to edit the SSH configuration to disable DNS lookups, which in my experience speeds up the login process to SSH by a lot!

yum remove -y samba-common bind-libs dnsmasq portmap postgresql-libs nscd
service atd stop
chkconfig atd off
service nfslock stop
chkconfig nfslock off
service rpcidmapd stop
chkconfig rpcidmapd off
service bluetooth stop
chkconfig bluetooth off
service anacron stop
chkconfig anacron off
service gpm stop
chkconfig gpm off
service hidd stop
chkconfig hidd off
service pcscd stop
chkconfig pcscd off
service portmap stop
chkconfig portmap off
service avahi-daemon stop
chkconfig avahi-daemon off
service pcscd stop
chkconfig pcscd off
service sendmail start
chkconfig sendmail on
vim /etc/ssh/sshd_config
#uncomment UseDNS and change to no
UseDNS no

service sshd restart

Next, we need to enable the RPMForge repository.  I enable the EPEL and REMI repositories later on, but at this point, if those are enabled, you will hit a certain known bug with MySQL that has yet to be fixed by CentOS.

http://rpmrepo.org/RPMforge/Using

wget http://packages.sw.be/rpmforge-release/rpmforge-release-0.5.2-2.el5.rf.i386.rpm
rpm -ivh rpmforge-release-0.5.2-2.el5.rf.i386.rpm

Now lets update the system and install some required packages.  This should take roughly 10 or 15 minutes.

yum update -y
yum groupinstall -y "Development tools" "Development Libraries"
yum install -y git-core git-svn java-1.6.0-openjdk vim-* apg pcre pcre-devel zlib zlib-devel libyaml-devel GeoIP-devel sphinx mysql-devel mysql-server mysql

At this point, I configure MySQL before adding other software that triggers bugs with its initial setup phase.

service mysqld start
/usr/bin/mysql_secure_installation

In the secure installation, I set the root password of MySQL to something tricky and I answer yes to all of the prompts.

The next step is to add some configuration to the my.cnf file to minimize the footprint of MySQL (which will still use a lot of memory on OpenVZ).

[mysqld]
datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
user=mysql
old_passwords=1
max_connections = 5
key_buffer = 1K
read_buffer_size = 1K
max_allowed_packet = 512K
thread_stack = 16K
table_cache = 32
sort_buffer = 16K
net_buffer_length = 1K
thread_stack = 4K
query_cache_type = 1
query_cache_limit = 1K
query_cache_size = 1K
innodb_buffer_pool_size = 1K
innodb_additional_mem_pool = 1K
# Disabling symbolic-links is recommended to prevent assorted security risks;
# to do so, uncomment this line:
# symbolic-links=0
[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

At this point, lets restart mysql.

service mysqld restart

Now let’s install Ruby Enterprise Edition.  The enterprise version of Ruby is still free, but it uses older, more stable components of Ruby, it’s also focused towards memory conservation (although it still compares to Java’s memory appetite).  I would strongly recommend not using any other version/distribution of Ruby since I found many compatibility errors that caused me to have to start all over again many times!  Remember that # is a comment and the command line will NOT process the command.

#- get latest stable ruby enterprise (the rubylang 1.9 branch will not work properly)
#-http://www.rubyenterpriseedition.com/download.html
wget http://rubyenterpriseedition.googlecode.com/files/ruby-enterprise-1.8.7-2011.03.tar.gz
tar xzf ruby-enterprise-1.8.7-2011.03.tar.gz
cd ruby-enterprise-1.8.7-2011.03
./installer
#(accept the defaults, 2 prompts will come up)

Next, we need to configure ruby and path variables.  Follow the comments in my notes.

cd /opt
ln -s ruby-enterprise-1.8.7-2011.03 ruby
vim /etc/profile

#add the following to /etc/profile [ "$EUID" = "0" ]
pathmunge /opt/ruby/bin
export RAILS_ENV=production
export PATH=/usr/local/sphinx/bin:/usr/local/bin:$PATH
export _JAVA_OPTIONS=-Xmx64m
export MAGICK_HOME=/usr/local
export DYLD_LIBRARY_PATH=/usr/local/lib

Strange enough, later on, another bug will surface where the profile isn’t read to setup these paths and options.  Therefore, I also added the following to my internal documentation install script.

vim /etc/bashrc
#add to the very bottom

pathmungea () {
                if ! echo $PATH | /bin/egrep -q "(^|:)$1($|:)" ; then
                        if [ "$2" = "after" ] ; then
                                PATH=$PATH:$1
                        else
                                PATH=$1:$PATH
                        fi
                fi
        }
pathmungea /opt/ruby/bin
export RAILS_ENV=production
export PATH=/usr/local/sphinx/bin:/usr/local/bin:$PATH
export _JAVA_OPTIONS=-Xmx64m
export MAGICK_HOME=/usr/local
export DYLD_LIBRARY_PATH=/usr/local/lib
unset pathmungea

At this point, restart the shell that you’re in to pick up the new path information.  You might want to reboot, but that isn’t necessary at this point.  After the restart, we need to install ImageMagick.

#Imagemagick on yum is outdated, so a bug will show up if we don’t do this for now.
cd ~
yum install -y tcl-devel libpng-devel libjpeg-devel ghostscript-devel bzip2-devel freetype-devel libtiff-devel
#wget url to ImageMagick download
wget ftp://ftp.imagemagick.org/pub/ImageMagick/ImageMagick-6.7.0-2.tar.gz
tar xzf ImageMagick-6.7.0-2.tar.gz
cd ImageMagick-6.7.0-2
./configure --prefix=/usr/local --with-bzlib=yes --with-fontconfig=yes --with-freetype=yes --with-gslib=yes --with-gvc=yes --with-jpeg=yes --with-jp2=yes --with-png=yes --with-tiff=yes
make
make install

Up next, we need to install some ruby gems.  This list isn’t comprehensive (there will be more to come later), but it will get us started.  I omitted mongrel as a webserver (which would come at this step according to other guides) because I intend on using Apache.  I also know that echoe, textpow, and oniguruma are either included or incompatible with this version of Ruby, so they were omitted.  I didn’t find any problems at the end without them, so I’m assuming they were extras.

UPDATE 6/8: You probably don’t need to do this since bundle exec (below) will cover this for you.  You will need to run “gem install bundle” instead at this step.

gem install sphinx rmagick ultrasphinx mime-types chronic ruby-hmac daemons mime-types BlueCloth ruby-yadis ruby-openid rspec rspec-rails RedCloth stompserver --no-ri --no-rdoc

Now let’s add the extra repositories and fix some centos bugs.  Don’t worry if the packages aren’t found.

#fix some centos bugs:
yum remove perl-Net-SSLeay perl-IO-Socket-SSL

#install some repositories
wget http://download.fedora.redhat.com/pub/epel/5/i386/epel-release-5-4.noarch.rpm
wget http://rpms.famillecollet.com/enterprise/remi-release-5.rpm
rpm -Uvh remi-release-5*.rpm epel-release-5*.rpm

Finally, its time to move onto the installation of gitorious.  The su command will drop you into a shell owned by git (cool huh?).

useradd -d /home/git git

su git
cd ~
mkdir log
mkdir conf
git clone git://gitorious.org/gitorious/mainline.git gitorious
cd gitorious/
mkdir -p tmp/pids
exit
#(exit here will return to root)

ln -s /home/git/gitorious/script/gitorious /usr/local/bin/gitorious
cd /home/git/gitorious/
chmod ug+x script/*
chmod -R g+w config/ log/ public/ tmp/

Next, we need to modify several files and begin preparing Gitorious to run.

Update 6/8/11: When getting to the gem install portions, you probably only need to run the bundle commands, skipping the gem commands all together.

su git

#modify each vim'd file appropriately

vim /home/git/gitorious/doc/templates/centos/git-daemon
RUBY_HOME="/opt/ruby"
GITORIOUS_HOME="/home/git/gitorious"

vim /home/git/gitorious/doc/templates/centos/git-ultrasphinx
GITORIOUS_HOME="/home/git/gitorious"

vim /home/git/gitorious/doc/templates/centos/git-poller
RUBY_HOME="/opt/ruby"
GITORIOUS_HOME="/home/git/gitorious"
export _JAVA_OPTIONS=-Xmx64m

vim /home/git/gitorious/doc/templates/centos/stomp
RUBY_HOME="/opt/ruby"
GEMS_HOME="/opt/ruby"
GITORIOUS_HOME="/home/git/gitorious"

exit  #return to root

#next as root:

gem install --no-ri --no-rdoc -v 1.5.0 json
gem install --no-ri --no-rdoc -v 1.3.1.1 rdiscount

cd /home/git/gitorious
bundle install
bundle exec rake gems:install

gem install --no-ri --no-rdoc -v 0.8.7 rake
gem install --no-ri --no-rdoc -v 1.1.0 daemons

gem uninstall rake -v 0.9.1
gem uninstall daemons -v 1.1.3

gem uninstall geoip
#select all

gem install --no-ri --no-rdoc -v 0.8.9 geoip
gem install --no-ri --no-rdoc -v 1.0 raspell

cp config/database.sample.yml config/database.yml
cp config/gitorious.sample.yml config/gitorious.yml
cp config/broker.yml.example config/broker.yml

su git
mkdir /home/git/data
mkdir /home/git/data/repositories
mkdir /home/git/data/tarballs
mkdir /home/git/data/tarball-work
chown -R git:git /home/git/data
#enable key management
mkdir /home/git/.ssh
chmod -R 700 /home/git/.ssh
touch /home/git/.ssh/authorized_keys
chmod 600 /home/git/.ssh/authorized_keys
chmod -R 700 /home/git/data
exit #back to root

#permissions fix:
chown -R git:git /home/git
chmod 711 /home/git

At this point, we need to make a secret cookie for a configuration file, make note of the output from this command.

apg -m 64
#sample output below, DON’T use it!
[root@string ~]# apg -m 64
SwouncievNivJucivrevnisfookEcnasiarHamgogdipmopyicbyctyikBagArim
gronugAmIsUkDifpoyftEggobviuzIpArgecHewElsOcubnuejEtDecerlyooHib
smivyecivfalakMarfAvikokip^ojyorwahonkIrEdeibZepbivsaftIdmapmic)
frivetcetEceivepJuxNeipnidzaroyffAgVevfuIvDesemAfyacAppAtdeavays
wenEjUcoofnafryefGewboshkyersufcawJontIavZenFeifWoitGejEajnoiWry
sawidHewofOkheTwiOjFigWigbobinCushBurnErnodedLuenAtTeyznoynoryop

There are several ruby environments for different modes of Ruby, these are Test, Development, and Production.  Although we have to set up the Development and Test environments (mostly to avoid errors and warnings), I will only use the Production Environment in the end.  Also, don’t use the key above, or below, I just inserted it to give you an idea of how it should look.

vim /home/git/gitorious/config/gitorious.yml

#For production, development, and test sections:
#copy paste exactly (new lines count), if you get an error below, this file will be where to look.
development:
  cookie_secret: SwouncievNivJucivrevnisfookEcnasiarHamgogdipmopyicbyctyikBagArimgronugAmIsUkDifpoyftEggobviuzIpArgecHewElsOcubnuejEtDecerlyooHibsmivyecivfalakMarfAvikokip^ojyorwahonkIrEdeibZepbivsaftIdmapmic)frivetcetEceivepJuxNeipnidzaroyffAgVevfuIvDesemAfyacAppAtdeavayswenEjUcoofnafryefGewboshkyersufcawJontIavZenFeifWoitGejEajnoiWrysawidHewofOkheTwiOjFigWigbobinCushBurnErnodedLuenAtTeyznoynoryop
  repository_base_path: "/home/git/data/repositories"
  extra_html_head_data:
  system_message:
  gitorious_client_port: 80
  gitorious_client_host: git.matthouse.us
  gitorious_host: git.matthouse.us
  gitorious_user: git
  exception_notification_emails: admin@matthouse.us
  mangle_email_addresses: true
  public_mode: true
  locale: en
  archive_cache_dir: "/home/git/data/tarballs"
  archive_work_dir: "/home/git/data/tarball-work"
  only_site_admins_can_create_projects: true
  hide_http_clone_urls: false
  is_gitorious_dot_org: false

test:
  cookie_secret: SwouncievNivJucivrevnisfookEcnasiarHamgogdipmopyicbyctyikBagArimgronugAmIsUkDifpoyftEggobviuzIpArgecHewElsOcubnuejEtDecerlyooHibsmivyecivfalakMarfAvikokip^ojyorwahonkIrEdeibZepbivsaftIdmapmic)frivetcetEceivepJuxNeipnidzaroyffAgVevfuIvDesemAfyacAppAtdeavayswenEjUcoofnafryefGewboshkyersufcawJontIavZenFeifWoitGejEajnoiWrysawidHewofOkheTwiOjFigWigbobinCushBurnErnodedLuenAtTeyznoynoryop
  repository_base_path: "/home/git/data/repositories"
  extra_html_head_data:
  system_message:
  gitorious_client_port: 80
  gitorious_client_host: git.matthouse.us
  gitorious_host: git.matthouse.us
  gitorious_user: git
  exception_notification_emails: admin@matthouse.us
  mangle_email_addresses: true
  public_mode: true
  locale: en
  archive_cache_dir: "/home/git/data/tarballs"
  archive_work_dir: "/home/git/data/tarball-work"
  only_site_admins_can_create_projects: true
  hide_http_clone_urls: false
  is_gitorious_dot_org: false

production:
  cookie_secret: SwouncievNivJucivrevnisfookEcnasiarHamgogdipmopyicbyctyikBagArimgronugAmIsUkDifpoyftEggobviuzIpArgecHewElsOcubnuejEtDecerlyooHibsmivyecivfalakMarfAvikokip^ojyorwahonkIrEdeibZepbivsaftIdmapmic)frivetcetEceivepJuxNeipnidzaroyffAgVevfuIvDesemAfyacAppAtdeavayswenEjUcoofnafryefGewboshkyersufcawJontIavZenFeifWoitGejEajnoiWrysawidHewofOkheTwiOjFigWigbobinCushBurnErnodedLuenAtTeyznoynoryop
  repository_base_path: "/home/git/data/repositories"
  extra_html_head_data:
  system_message:
  gitorious_client_port: 80
  gitorious_client_host: git.matthouse.us
  gitorious_host: git.matthouse.us
  gitorious_user: git
  exception_notification_emails: admin@matthouse.us
  mangle_email_addresses: true
  public_mode: true
  locale: en
  archive_cache_dir: "/home/git/data/tarballs"
  archive_work_dir: "/home/git/data/tarball-work"
  only_site_admins_can_create_projects: true
  hide_http_clone_urls: false
  is_gitorious_dot_org: false

Now let’s configure the broker.  For a while, I was confused about why other guides had me install both stomp and activemq.  It turns out that you only need one or the other.  For me, stomp was more memory friendly, so I choose that.  Towards the end of this blow, I also provide the procedure for installing ActiveMQ (which doesn’t pertain to what I’m installing).  If you want ActiveMQ, you simply have to replace “stomp” below with “activemq” to get this to work I believe (although I didn’t test that) and install ActiveMQ around this point (as detailed at the end).

vim /home/git/gitorious/config/broker.yml

production:
    adapter: stomp
development:
    adapter: stomp
test:
    adapter: stomp

Now let’s go through some more commands, including database configuration.  Replace the prompts with your own.

f

#permissions fix:
chown -R git:git /home/git

mysql -uroot –p’<ROOT’S MYSQL PASSWORD>'

create database gitorious;
create database gitorious_test;
create database gitorious_dev;
CREATE user 'git'@'localhost' IDENTIFIED BY '<PASWORD>';
GRANT ALL ON gitorious.* TO 'git'@'localhost';
GRANT ALL ON gitorious_test.* TO 'git'@'localhost';
GRANT ALL ON gitorious_dev.* TO 'git'@'localhost';
FLUSH privileges;
exit;

One more major edit to a file to go!  Once again, replace with the one you used above.

vim /home/git/gitorious/config/database.yml

development:
  adapter: mysql
  database: gitorious_dev
  username: git
  password: <PASWORD>
  host: localhost
  encoding: utf8

test:
  adapter: mysql
  database: gitorious_test
  username: git
  password: <PASWORD>
  host: localhost
  encoding: utf8

production:
  adapter: mysql
  database: gitorious
  username: git
  password: <PASWORD>
  host: localhost
  encoding: utf8

Now some more commands to configure gitorious.  I was a little obsessive with fixing permissions (since I did a lot as root), but I wanted to make sure that I wouldn’t hit unknown errors and bugs.

#fix permissions
cd /home
chown -R git:git /home/git
cd /home/git
chmod -R 755 data
chmod 755 gitorious

cd /home/git/gitorious
bundle exec rake db:create:all
bundle exec rake db:setup
bundle exec rake db:migrate

#fix permissions
cd /home
chown -R git:git /home/git
cd /home/git
chmod -R 755 data
chmod 755 gitorious

It’s now time to start some services related to Gitorious.  Make sure that there are no errors in this section.  I recommend executing line by line.

ln -s /home/git/gitorious/doc/templates/centos/git-daemon /etc/init.d/git-daemon
chmod +x /etc/init.d/git-daemon
chkconfig --add git-daemon
service git-daemon start

ln -s /home/git/gitorious/doc/templates/centos/stomp /etc/init.d/stomp
chmod +x /etc/init.d/stomp
chkconfig --add stomp
service stomp start

ln -s /home/git/gitorious/doc/templates/centos/git-poller /etc/init.d/git-poller
chmod +x /etc/init.d/git-poller
chkconfig --add git-poller
service git-poller start

At this point, we can get into the Apache configuration.  To download from gitorious, we need mod_xsendfile.  To the best of my knowledge, Gitorious drops the executable bit off from the tar archives it creates, so the files cannot be accessed traditionally.  This presents a problem and this modification makes the files downloadable without the executable bit being set.

yum install -y httpd httpd-devel mod_xsendfile
#mod qos for sloworis attack control and DOS attack control
cd ~
mkdir apachemod
cd apachemod
wget http://sourceforge.net/projects/mod-qos/files/mod_qos-9.57.tar.gz/download
tar xzf mod_qos-9.57.tar.gz
cd mod_qos-9.57/apache2
apxs -i -c mod_qos.c
chmod 755 /usr/lib64/httpd/modules/mod_qos.so
#configure xsendfile

vim /etc/httpd/conf.d/xsendfile.conf
#add below LoadModule line
XSendFile on
XSendFilePath /home/git/data/tarballs
#UPDATE 6/8/11: Add a path to the repositories folder for git to work over http.
XSendFilePath /home/git/data/repositories

#install ruby Passenger to make Ruby applications work on Apache.

/opt/ruby-enterprise-1.8.7-2011.03/bin/passenger-install-apache2-module
#accept the defaults

At this point, we now need to configure Apache.  I do this with a two file approach separating the website configuration from the server configuration.  If you copy paste my httpd.conf file, it will serve you well, provided you change the “ServerAdmin” and “ServerName” directives (options).  The same isn’t true of my host.conf file, you’re going to have to modify that for your particular installation.

cd /etc/httpd/conf
vim httpd.conf

### Section 1: Global Environment
ServerTokens OS
ServerRoot "/etc/httpd"
PidFile run/httpd.pid
Timeout 120
KeepAlive Off
MaxKeepAliveRequests 100
KeepAliveTimeout 15
Listen 80

LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule auth_digest_module modules/mod_auth_digest.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authn_alias_module modules/mod_authn_alias.so
LoadModule authn_anon_module modules/mod_authn_anon.so
LoadModule authn_dbm_module modules/mod_authn_dbm.so
LoadModule authn_default_module modules/mod_authn_default.so
LoadModule authz_host_module modules/mod_authz_host.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_owner_module modules/mod_authz_owner.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so
LoadModule authz_dbm_module modules/mod_authz_dbm.so
LoadModule authz_default_module modules/mod_authz_default.so
LoadModule ldap_module modules/mod_ldap.so
LoadModule authnz_ldap_module modules/mod_authnz_ldap.so
LoadModule include_module modules/mod_include.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule logio_module modules/mod_logio.so
LoadModule env_module modules/mod_env.so
LoadModule ext_filter_module modules/mod_ext_filter.so
LoadModule mime_magic_module modules/mod_mime_magic.so
LoadModule expires_module modules/mod_expires.so
LoadModule deflate_module modules/mod_deflate.so
LoadModule headers_module modules/mod_headers.so
LoadModule usertrack_module modules/mod_usertrack.so
LoadModule setenvif_module modules/mod_setenvif.so
LoadModule mime_module modules/mod_mime.so
LoadModule dav_module modules/mod_dav.so
LoadModule status_module modules/mod_status.so
LoadModule autoindex_module modules/mod_autoindex.so
LoadModule info_module modules/mod_info.so
LoadModule dav_fs_module modules/mod_dav_fs.so
LoadModule vhost_alias_module modules/mod_vhost_alias.so
LoadModule negotiation_module modules/mod_negotiation.so
LoadModule dir_module modules/mod_dir.so
LoadModule actions_module modules/mod_actions.so
LoadModule speling_module modules/mod_speling.so
LoadModule userdir_module modules/mod_userdir.so
LoadModule alias_module modules/mod_alias.so
LoadModule rewrite_module modules/mod_rewrite.so
LoadModule proxy_module modules/mod_proxy.so
LoadModule proxy_balancer_module modules/mod_proxy_balancer.so
LoadModule proxy_ftp_module modules/mod_proxy_ftp.so
LoadModule proxy_http_module modules/mod_proxy_http.so
LoadModule proxy_connect_module modules/mod_proxy_connect.so
LoadModule cache_module modules/mod_cache.so
LoadModule suexec_module modules/mod_suexec.so
LoadModule disk_cache_module modules/mod_disk_cache.so
LoadModule file_cache_module modules/mod_file_cache.so
#LoadModule mem_cache_module modules/mod_mem_cache.so
LoadModule cgi_module modules/mod_cgi.so
LoadModule version_module modules/mod_version.so
#LoadModule cern_meta_module modules/mod_cern_meta.so
#LoadModule asis_module modules/mod_asis.so

LoadModule unique_id_module modules/mod_unique_id.so
LoadModule qos_module /usr/lib64/httpd/modules/mod_qos.so

LoadModule passenger_module /opt/ruby/lib/ruby/gems/1.8/gems/passenger-3.0.7/ext/apache2/mod_passenger.so
   PassengerRoot /opt/ruby/lib/ruby/gems/1.8/gems/passenger-3.0.7
   PassengerRuby /opt/ruby/bin/ruby

Include conf.d/*.conf
#ExtendedStatus On
User apache
Group apache

### Section 2: 'Main' server configuration

ServerAdmin admin@matthouse.us
ServerName string.matthouse.us:80
UseCanonicalName Off
DocumentRoot "/var/www/html"

<Directory />
    Options FollowSymLinks
    AllowOverride All
</Directory>

<IfModule mod_userdir.c>
    UserDir public_html
</IfModule>

DirectoryIndex index.html index.html.var index.htm
AccessFileName .htaccess

<Files ~ "^.ht"> #deny serving ht files
    Order allow,deny
    Deny from all
</Files>

TypesConfig /etc/mime.types
DefaultType text/plain
<IfModule mod_mime_magic.c>
#   MIMEMagicFile /usr/share/magic.mime
    MIMEMagicFile conf/magic
</IfModule>

HostnameLookups Off
#EnableMMAP off
#EnableSendfile off
ErrorLog logs/error_log
LogLevel warn

LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"" combined
LogFormat "%h %l %u %t "%r" %>s %b" common
LogFormat "%{Referer}i -> %U" referer
LogFormat "%{User-agent}i" agent
#LogFormat "%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i" %I %O" combinedio

#CustomLog logs/access_log common
#CustomLog logs/referer_log referer
#CustomLog logs/agent_log agent
CustomLog logs/access_log combined

ServerSignature On

Alias /icons/ "/var/www/icons/"
<Directory "/var/www/icons">
    Options Indexes MultiViews
    AllowOverride None
    Order allow,deny
    Allow from all
</Directory>

#
# WebDAV module configuration section.
#
<IfModule mod_dav_fs.c>
    # Location of the WebDAV lock database.
    DAVLockDB /var/lib/dav/lockdb
</IfModule>
ScriptAlias /cgi-bin/ "/var/www/cgi-bin/"
<Directory "/var/www/cgi-bin">
    AllowOverride All
    Options None
    Order allow,deny
    Allow from all
</Directory>

# Redirect permanent /foo http://www.example.com/bar

# Directives controlling the display of server-generated directory listings.
IndexOptions FancyIndexing VersionSort NameWidth=* HTMLTable
AddIconByEncoding (CMP,/icons/compressed.gif) x-compress x-gzip
AddIconByType (TXT,/icons/text.gif) text/*
AddIconByType (IMG,/icons/image2.gif) image/*
AddIconByType (SND,/icons/sound2.gif) audio/*
AddIconByType (VID,/icons/movie.gif) video/*
AddIcon /icons/binary.gif .bin .exe
AddIcon /icons/binhex.gif .hqx
AddIcon /icons/tar.gif .tar
AddIcon /icons/world2.gif .wrl .wrl.gz .vrml .vrm .iv
AddIcon /icons/compressed.gif .Z .z .tgz .gz .zip
AddIcon /icons/a.gif .ps .ai .eps
AddIcon /icons/layout.gif .html .shtml .htm .pdf
AddIcon /icons/text.gif .txt
AddIcon /icons/c.gif .c
AddIcon /icons/p.gif .pl .py
AddIcon /icons/f.gif .for
AddIcon /icons/dvi.gif .dvi
AddIcon /icons/uuencoded.gif .uu
AddIcon /icons/script.gif .conf .sh .shar .csh .ksh .tcl
AddIcon /icons/tex.gif .tex
AddIcon /icons/bomb.gif core
AddIcon /icons/back.gif ..
AddIcon /icons/hand.right.gif README
AddIcon /icons/folder.gif ^^DIRECTORY^^
AddIcon /icons/blank.gif ^^BLANKICON^^
DefaultIcon /icons/unknown.gif
#AddDescription "GZIP compressed document" .gz
#AddDescription "tar archive" .tar
#AddDescription "GZIP compressed tar archive" .tgz
ReadmeName README.html
HeaderName HEADER.html
IndexIgnore .??* *~ *# HEADER* README* RCS CVS *,v *,t

AddLanguage ca .ca
AddLanguage cs .cz .cs
AddLanguage da .dk
AddLanguage de .de
AddLanguage el .el
AddLanguage en .en
AddLanguage eo .eo
AddLanguage es .es
AddLanguage et .et
AddLanguage fr .fr
AddLanguage he .he
AddLanguage hr .hr
AddLanguage it .it
AddLanguage ja .ja
AddLanguage ko .ko
AddLanguage ltz .ltz
AddLanguage nl .nl
AddLanguage nn .nn
AddLanguage no .no
AddLanguage pl .po
AddLanguage pt .pt
AddLanguage pt-BR .pt-br
AddLanguage ru .ru
AddLanguage sv .sv
AddLanguage zh-CN .zh-cn
AddLanguage zh-TW .zh-tw
LanguagePriority en ca cs da de el eo es et fr he hr it ja ko ltz nl nn no pl pt pt-BR ru sv zh-CN zh-TW
ForceLanguagePriority Prefer Fallback
AddDefaultCharset UTF-8
#AddType application/x-tar .tgz
#AddEncoding x-compress .Z
#AddEncoding x-gzip .gz .tgz
AddType application/x-compress .Z
AddType application/x-gzip .gz .tgz

AddHandler cgi-script .cgi .pl
#AddHandler send-as-is asis
AddHandler type-map var
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml

# 1) plain text 2) local redirects 3) external redirects
#ErrorDocument 500 "The server made a boo boo."
#ErrorDocument 404 /missing.html
#ErrorDocument 404 "/cgi-bin/missing_handler.pl"
#ErrorDocument 402 http://www.example.com/subscription_info.html

Alias /error/ "/var/www/error/"

<IfModule mod_negotiation.c>
<IfModule mod_include.c>
    <Directory "/var/www/error">
        AllowOverride None
        Options IncludesNoExec
        AddOutputFilter Includes html
        AddHandler type-map var
        Order allow,deny
        Allow from all
        LanguagePriority en es de fr
        ForceLanguagePriority Prefer Fallback
    </Directory>

#    ErrorDocument 400 /error/HTTP_BAD_REQUEST.html.var
#    ErrorDocument 401 /error/HTTP_UNAUTHORIZED.html.var
#    ErrorDocument 403 /error/HTTP_FORBIDDEN.html.var
#    ErrorDocument 404 /error/HTTP_NOT_FOUND.html.var
#    ErrorDocument 405 /error/HTTP_METHOD_NOT_ALLOWED.html.var
#    ErrorDocument 408 /error/HTTP_REQUEST_TIME_OUT.html.var
#    ErrorDocument 410 /error/HTTP_GONE.html.var
#    ErrorDocument 411 /error/HTTP_LENGTH_REQUIRED.html.var
#    ErrorDocument 412 /error/HTTP_PRECONDITION_FAILED.html.var
#    ErrorDocument 413 /error/HTTP_REQUEST_ENTITY_TOO_LARGE.html.var
#    ErrorDocument 414 /error/HTTP_REQUEST_URI_TOO_LARGE.html.var
#    ErrorDocument 415 /error/HTTP_UNSUPPORTED_MEDIA_TYPE.html.var
#    ErrorDocument 500 /error/HTTP_INTERNAL_SERVER_ERROR.html.var
#    ErrorDocument 501 /error/HTTP_NOT_IMPLEMENTED.html.var
#    ErrorDocument 502 /error/HTTP_BAD_GATEWAY.html.var
#    ErrorDocument 503 /error/HTTP_SERVICE_UNAVAILABLE.html.var
#    ErrorDocument 506 /error/HTTP_VARIANT_ALSO_VARIES.html.var

</IfModule>
</IfModule>

BrowserMatch "Mozilla/2" nokeepalive
BrowserMatch "MSIE 4.0b2;" nokeepalive downgrade-1.0 force-response-1.0
BrowserMatch "RealPlayer 4.0" force-response-1.0
BrowserMatch "Java/1.0" force-response-1.0
BrowserMatch "JDK/1.0" force-response-1.0
BrowserMatch "Microsoft Data Access Internet Publishing Provider" redirect-carefully
BrowserMatch "MS FrontPage" redirect-carefully
BrowserMatch "^WebDrive" redirect-carefully
BrowserMatch "^WebDAVFS/1.[0123]" redirect-carefully
BrowserMatch "^gnome-vfs/1.0" redirect-carefully
BrowserMatch "^XML Spy" redirect-carefully
BrowserMatch "^Dreamweaver-WebDAV-SCM1" redirect-carefully

<IfModule prefork.c>
StartServers       1
MinSpareServers    1
MaxSpareServers  1
ServerLimit      128
MaxClients       128
MaxRequestsPerChild  1000
</IfModule>

### Section 3: CONF STUFF
Include conf/host.conf

Next is the virtual host configuration.  I chose to use https / ssl with a self-signed certificate, so that procedure is below.  The following probably won’t be a perfect cut / paste for you, but it will be close enough provided you fill in the blanks.  If you wish to not use https, simply copy the basic host.conf file and remove the SSL elements (including the port 443 stuff).

cd ~
yum install –y mod_ssl openssl
openssl genrsa -out ca.key 2048
openssl req -new -key ca.key -out ca.csr
#Enter in the information

#sample information
#Country Name (2 letter code) [GB]:US
#State or Province Name (full name) [Berkshire]:NEW YORK
#Locality Name (eg, city) [Newbury]:GREAT VALLEY
#Organization Name (eg, company) [My Company Ltd]:MATTHOUSE
#Organizational Unit Name (eg, section) []:
#Common Name (eg, your name or your server's hostname) []:string.matthouse.us
#Email Address []:admin@matthouse.us
#
#Please enter the following 'extra' attributes to be sent with your certificate request
#A challenge password []:
#An optional company name []:

openssl x509 -req -days 365 -in ca.csr -signkey ca.key -out ca.crt
cp ca.crt /etc/pki/tls/certs
cp ca.key /etc/pki/tls/private/ca.key
cp ca.csr /etc/pki/tls/private/ca.csr
mv /etc/httpd/conf.d/ssl.conf /etc/httpd/conf.d/ssl.conf.disabled

Now we make the virtualhost include file.

vim /etc/httpd/conf/host.conf

#enable ~user home directories.
<Directory /home/*/public_html>
    Options MultiViews Indexes SymLinksIfOwnerMatch Includes ExecCGI
    AllowOverride All
    Order allow,deny
    Allow from all
</Directory>

NameVirtualHost *:80

<VirtualHost *:80>
        ServerName  string.matthouse.us
        ServerAlias www.string.matthouse.us
        ServerAdmin admin@matthouse.us
        DocumentRoot /var/www
        ErrorLog  /var/www/error.log
        CustomLog /var/www/access.log combined
</VirtualHost>

<VirtualHost *:80>
        ServerName  git.matthouse.us
        ServerAdmin admin@matthouse.us
        DocumentRoot /home/git/gitorious/public
        ErrorLog  /var/www/errorgit.log
        CustomLog /var/www/accessgit.log combined
#customlog is good for awstats software
        <Directory /home/git/gitorious/public>
          AllowOverride all
          Options -MultiViews
        </Directory>
</VirtualHost>

LoadModule ssl_module modules/mod_ssl.so
Listen 443
AddType application/x-x509-ca-cert .crt
AddType application/x-pkcs7-crl    .crl
SSLPassPhraseDialog  builtin
SSLSessionCache         shmcb:/var/cache/mod_ssl/scache(512000)
SSLSessionCacheTimeout  300
SSLMutex default
SSLRandomSeed startup file:/dev/urandom  256
SSLRandomSeed connect builtin
SSLCryptoDevice builtin

NameVirtualHost *:443

<VirtualHost *:443>
        SSLEngine on
        SSLCertificateFile /etc/pki/tls/certs/ca.crt
        SSLCertificateKeyFile /etc/pki/tls/private/ca.key
        SSLProtocol all -SSLv2
        SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM:+LOW
        ServerName  git.matthouse.us
        ServerAdmin admin@matthouse.us
        DocumentRoot /home/git/gitorious/public
        ErrorLog  /var/www/errorgit.log
        CustomLog /var/www/accessgit.log combined
        <Directory /home/git/gitorious/public>
          AllowOverride all
          Options -MultiViews
        </Directory>
</VirtualHost>

At this point, test the Apache server configuration and start it.  Then we will fix permissions once more.

service httpd configtest
service httpd start
#fix permissions again
cd /home
chmod 711 git
chown -R git:git git

It’s now time to add a Gitorious user!

su git
cd ~/gitorious
env RAILS_ENV=production script/create_admin

#sample
Type in Administrator's e-mail:
admin@matthouse.us
Type in Administrator's password:
<APASSWORD>
Admin user created successfully.

exit
#exit back to root

Next, if you wish to disable most of the SSL links on your Gitorious install, do the following (I did this).  In the install that I did, I enabled SSL so that for the few links that went to https sites, Gitorious would continue to work (with a few warnings of course because self-signed certificates aren’t trusted and I’m not paying for a trusted rapidssl certificate for how little I am going to use Gitorious myself).

UPDATE 6/8/11: The SSLRequirement directive is no longer required in production.rb (below).  Instead, add “disable_ssl” OR “enable_ssl” as a line to the file /home/git/gitorious/config/gitorious.yml (as the git user).

#disable SSL links (for the most part
su git
vim ~/gitorious/config/environments/production.rb
#add the following line somewhere in the file at the top:
SslRequirement.disable_ssl_check = true
exit
#exit back to root

It’s now time to configure ultrasphinx.  I believe that this is used mostly for indexing and searching the Gitorious site, but it could be used for other functionality too.

cd /home/git/gitorious 

export RAILS_ENV=production
export PATH=/usr/local/sphinx/bin:$PATH

bundle exec rake ultrasphinx:configure RAILS_ENV=production
bundle exec rake ultrasphinx:index RAILS_ENV=production
bundle exec rake ultrasphinx:daemon:start RAILS_ENV=production

#the following command builds the sphinx dictionary.  For me it seg faults, but gets rid of a pesky dictionary error.
cd /home/git/gitorious
aspell config dict-dir
   /usr/lib64/aspell-0.60
cp vendor/plugins/ultrasphinx/examples/ap.multi /usr/lib64/aspell-0.60/
#the next command segfaults, but it makes an annoying error go away in a log
bundle exec rake ultrasphinx:spelling:build

#due to some deprecation in code, the following two changes need to happen:
vim /home/git/gitorious/config/ultrasphinx/production.conf

#change "address" to "listen" due to deprecation
  listen = 0.0.0.0
#change memlimit from 256 to 64 to conserve memory
indexer {
  mem_limit = 64M

#Finally, we need to make git-ultrasphinx a daemon.
ln -s /home/git/gitorious/doc/templates/centos/git-ultrasphinx /etc/init.d/git-ultrasphinx
chmod +x /etc/init.d/git-ultrasphinx

#make mysqld and httpd start automatically
chkconfig mysqld on
chkconfig httpd on

#a few bug fixes
ln -s /usr/local/bin/gitorious /usr/bin

#install imagemagick via yum
#yes, we compiled from source to get rid of an error… but gitorious will have other errors if we don’t
yum –y install ImageMagick

#One last time, I will fix the permissions
cd /home
chown -R git:git /home/git
cd /home/git
chmod -R 755 data
chmod 755 gitorious

We now need to install memcached for Gitorious.  I originally thought this was strictly for performance, but Gitorious expects it to be installed.  I use the remi repository for this since the other repositories have broken dependencies.

yum install -y --enablerepo=remi memcached
service memcached start
chkconfig memcached on

At this point, everything is almost configured fully.  I don’t start the git daemons / services automatically with chkconfig because they won’t run at system startup (I don’t know why).  Therefore, I created a special script and made that start automatically upon reboot using the cron daemon (using the @reboot option).

vim /root/startup.sh

#!/bin/sh
/etc/init.d/stomp start
/etc/init.d/git-daemon start
/etc/init.d/git-poller start
/etc/init.d/git-ultrasphinx start

crontab –e
#add the following to root’s crontab
MAILTO=""
@reboot /root/startup.sh

Next, we need to index the Gitorious site every hour.

su git
crontab –e
#add the following to git’s cron, which indexes the site every hour
MAILTO=""
* */1 * * * cd /home/git/gitorious && /opt/ruby/bin/rake ultrasphinx:index RAILS_ENV=production
exit
#exit to root

Finally, reboot the VPS.  With any luck, everything will work once its rebooted.  I went to the site and tested the following functionality:

• Adding an ssh key
• Adding a new repository
• Adding a wiki page
• Deleting the above stuff
• Committing to the repository from a local repository

The above is all I required of Gitorious.  Unfortunately, the included tests did not successfully complete for me due to the memory restrictions, but I saw several errors.  I don’t think the tests would fully pass, but the basic functionality that I need is present and that is what matters.  Below is what you can do (as root) to run the tests for yourself.

cd /home/git/gitorious
bundle exec rake test

With the above said, I then went ahead and configured a simple firewall to protect the Gitorious machine.  Below was the procedure for that.

vim /etc/firewall.sh

#!/bin/bash
# My system IP/set ip address of server
SERVER_IP="205.234.203.115"
# Flushing all rules
iptables -F
iptables -X

# Setting default filter policy
service iptables restart
iptables -F
iptables -P INPUT DROP
iptables -P OUTPUT ACCEPT
iptables -P FORWARD ACCEPT

#allowed inbound
#ssh
iptables -A INPUT -p tcp -s 0/0 -d $SERVER_IP --dport 22 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp -s 0/0 -d $SERVER_IP --dport 80 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp -s 0/0 -d $SERVER_IP --dport 443 -m state --state NEW,ESTABLISHED -j ACCEPT
iptables -A INPUT -p tcp -s 0/0 -d $SERVER_IP --dport 9418 -m state --state NEW,ESTABLISHED -j ACCEPT

iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
#allow ping
iptables -A INPUT -p icmp -m icmp --icmp-type 8 -j ACCEPT
iptables  -A INPUT -j DROP

Next, we need to add the firewall to system startup.

chmod +x /etc/firewall.sh

crontab –e
#add the following at the bottom
@reboot /etc/firewall.sh

The (relatively simple) firewall is now configured to allow incoming pings to the server, SSH access, HTTP access, and GIT access using the git protocol.  If you have issues like I did with an unknown error, your VPS provider will have to enable the connection tracking feature of IPTables for you, otherwise committing to a gitorious repository will not work.

I also configured sendmail to send mail from this machine appropriately.  Below is a simple procedure for this.  You could get a lot more in depth, but it isn’t necessary.  Note that using this method, all mail will be sent from user@string.matthouse.us in my case (since this is the machine hostname).

vim /etc/mail/local-host-names
#add local hosts as appropriate
localhost
string.matthouse.us
git.matthouse.us

vim /root/.forward
#add your email address to the first line
admin@matthouse.us

service sendmail restart

With that, the Gitorious install is officially Installed.  I will not continue onto the ActiveMQ installation as mentioned above, this is only required if you chose to use ActiveMQ as a broker instead of Stomp.

Below is the ActiveMQ installation procedure (as written in my internal documentation) for your enjoyment.  I will not explain this, but I thought it would be appropriate to include it since I originally followed it to a dead end!  This is strictly not required for the above installation and will do nothing to help you if you’re getting errors above (while using stomp as I did).

wget http://apache.spd.co.il/activemq/apache-activemq/5.5.0/apache-activemq-5.5.0-bin.tar.gz
tar xzvf apache-activemq-5.5.0-bin.tar.gz
mv apache-activemq-5.5.0 /usr/local/apache-activemq5.50
cd /usr/local
ln -s apache-activemq5.50 apache-activemq
cd ~
adduser activemq
chown -R activemq /usr/local/apache-activemq/data

cd /usr/local/apache-activemq5.50/

vim bin/activemq

#change comments to:

 ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.ssl=false"
#ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote"

vim /usr/local/apache-activemq/bin/activemq-admin
find: ACTIVEMQ_OPTS="-Xmx512M -Dorg.apache.activemq.UseDedicatedTaskRunner=true -Djava.util.logging.config.file=logging.properties"
replace: ACTIVEMQ_OPTS="-Xmx64M -Dorg.apache.activemq.UseDedicatedTaskRunner=true -Djava.util.logging.config.file=logging.properties"

vim activemqstart.sh

#!/bin/bash
export JAVA_HOME=/usr/lib/jvm/jre
export _JAVA_OPTIONS=-Xmx64m
/usr/local/apache-activemq/bin/activemq-admin start &

vim activemqstop.sh

#!/bin/bash
export JAVA_HOME=/usr/lib/jvm/jre
export _JAVA_OPTIONS=-Xmx64m
/usr/local/apache-activemq/bin/activemq-admin stop

chmod +x activemqstart.sh
chmod +x activemqstop.sh

cd /etc/init.d
vim activemq

#!/bin/bash
#
# activemq       Starts ActiveMQ.
#
#
# chkconfig: 345 88 12
# description: ActiveMQ is a JMS Messaging Queue Server.
### BEGIN INIT INFO
# Provides: $activemq
### END INIT INFO

# Source function library.
. /etc/init.d/functions

export _JAVA_OPTIONS=-Xmx64m

[ -f /usr/local/apache-activemq/activemqstart.sh ] || exit 0
[ -f /usr/local/apache-activemq/activemqstop.sh ] || exit 0

RETVAL=0

umask 077

start() {
       echo -n $"Starting ActiveMQ: "
       daemon /usr/local/apache-activemq/activemqstart.sh
       echo
       return $RETVAL
}
stop() {
       echo -n $"Shutting down ActiveMQ: "
       daemon su -c /usr/local/apache-activemq/activemqstop.sh activemq
       echo
       return $RETVAL
}
restart() {
       stop
       start
}
case "$1" in
 start)
       start
       ;;
 stop)
       stop
       ;;
 restart|reload)
       restart
       ;;
 *)
       echo $"Usage: $0 {start|stop|restart}"
       exit 1
esac

exit $?

#run these commands
chmod +x activemq
chkconfig activemq on

Whew, this is the longest blog I’ve ever written!  That just goes to show the complexity of Gitorious, not to mention the administrative overhead of maintaining Ruby and Imagemagick, possible even ActiveMQ (since they were installed from source).  This blog post is in no way shape or form a complete guide to getting Gitorious installed and functional, but it will help get you most of the way there.  Because of the complexity of this software, I will likely not be of much help for any problems that I might be asked to help resolve.  As a disclaimer, use the information in this blog post at your own risk, I’m not responsible for the loss of any data, time or profits that you might incur from following this guide.

As always, thanks for reading!

Special thanks to Marius Mathiesen for comments about inaccuracies in this tutorial.  Information is up to date as of 6/8/2011.