Category Archives: linux

Running a PHP Script on a QNAP TS-431

I needed to setup a persistent cron job on my QNAP TS-431 that would run a PHP script (actually the script that let’s me always know what the IP address of my home router is from anywhere in the world). However, the QNAP has the php executable buried and the directory is not in the $PATH environment variable. It’s not too hard to find though, if you need to run a PHP script just do this:

mnt/ext/opt/apache/bin/php /path/to/some-php-script.php

Specify a SSH Key for rsync

If you use rsync with the -rsh=ssh option it looks for a private key called id_rsa in the ~/.ssh folder. If (for whatever reason) you want to specify a different private key from the default then you can do it with:

-e ‘ssh -i /some/path/some_key’

This can be useful if want a key per host. Or, if like me, you idiotically created the keys with a different name than the default.

Persist crontab Entries on QNAP TS-431

QNAP helpfully explains how to add items to crontab on one of the NAS machines on their wiki here. Less helpfully they don’t explain that adding jobs with crontab -e and then restarting the cron daemon won’t persist through a reboot. Yay. However, you get around this by adding jobs directly to /etc/config/crontab. Thanks fellas.

PERL Search and Replace in Multiple Files – Easy as PIE

I help to administer a blog with about 2000 entries.  Previously the site was managed in MovableType but earlier this year we moved it over to WordPress.  At the time we migrated about 30% of the posts to WordPress (the top trafficked 30%).  It was a slow process and as time has gone by we’ve migrated more and more posts  but it’s a time consuming process.  We need to create the new WordPress post, put a 301 redirect in place and then manually delete the old MovableType post.  And by manual deletion I mean deleting the actual HTML file that MoveableType created for that post.  You see, MovableType produces static files for each and every post, archive page, and index page.  This is a blessing and a curse, it means that the load on the web server isn’t large on heavily trafficked sites, but it also means maintaining a legacy site is a PAIN IN THE A**.  It’s actually such a pain that I have given up with MT itself and now I am working with the static HTML files directly.

This week I needed an easy way of doing a search and replace in all the legacy HTML files, all 2000 or so of them.  It needed to be recursive and ideally it needed to happen without me needing to FTP all the files to a local computer and then FTP them back.  I have command line access to the server the blog is on so I checked to see if PERL offered a way to do what I wanted working on the files in situ.  As it turns out it’s pretty simple.  I wanted to find and replace all internal links that used the old domain name (did I mention this site just changed domain names) with the new domain name.  The PERL command to do this with all the files in a single folder is like this:

perl -p -i -e 's/oldstring/newstring/g;' *.html

The -p means the script we’re running will be put through the C pre-processor before PERL compilation. The -i option means that PERL will edit the files in place. The -e option allows the running of PERL commands from the command line (it doesn’t look for a script file). The actual PERL operator s is the substitution operator while the he oldstring and newstring are regular expressions so special characters need to be escaped appropriately. And finally the operator /g means the command will do a global match. So to replace my URLs I needed something like this:

perl -p -i -e 's/www\.old\-domain\.com/www\.new\-domain\.com/g' *.html

The issue with that is that I needed the script to process sub-directories recursively to search and replace in all the HTML files. That could be done in a few different ways, the immediately obvious were chaining with FIND or GREP. I chose FIND and ended up with a command that looks like this:

perl -p -i -e 's/www\.old\-domain\.com/www\.new\-domain\.com/g' ' `find ./ -name "*.html"`

Ran that from the ubuntu commmand line and thousand or more files were processed in under a second. Very cool.

Renewing my Namecheap SSL Certificate on Linode Apache2 Web Server

This post is something of a reminder to myself. How to renew a NameCheap Comodo SSL certificate on one of my Linode Apache2 web-servers. It’s something I only have to do every few years so I essentially have to re-learn the process every time which is rather tiring. This post will be here for my future self to make the process easier.

1. Renew the SSL Certificate on NameCheap

I use NameCheap for all my domain name registrations and SSL certificates. They have many different types of SSL certificates available from several different providers ranging from simple domain certificates right up to full wildcard organization based certificates. The great thing about any of them is that NameCheap will email you well in advance of them expiring. When they do so it’s simply a matter of logging into your NameCheap control panel and renewing the certificate. A new entry will be created with the status of “Waiting Activation”.

2. Create the CSR File

My host runs apache2 on Ubuntu and OpenSSL. Apache looks for certificates in /etc/apache2/ssl and I usually create a sub-folder for new CSR and key files when the old ones expire. I feel its a bit more secure to generate a new Certificate Signing Request (CSR) whenever a certificate expires, which is why I do this. So I create a folder for the files:

cd /etc/apache2/ssl
mkdir december-2014
cd december-2014

Then create a CSR file using:

openssl req -newkey rsa:2048 -days 1095 -nodes -keyout www.my-domain.com.key -out www.my-domain.com.csr

The openssl req command will then ask for some information to encode with the request. That looks something like this:

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:AU
State or Province Name (full name) [Some-State]:South Australia
Locality Name (eg, city) []:Adelaide
Organization Name (eg, company) [Internet Widgits Pty Ltd]:My Company Name
Organizational Unit Name (eg, section) []:
Common Name (eg, YOUR name) []:www.mydomain.com
Email Address []:admin@mydomain.com

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:My Company Name

This will create the CSR file (www.mydomain.com.csr) and a SSL certificate key file (www.mydomain.com.key). The key needs to be protected with:

chmod 400 www.mydomain.com.key

3. Issue the SSL Certificate via NameCheap

The contents of the CSR file are needed to issue the SSL certificate via the NameCheap control panel. View the contents with:

more www.mydomain.com.csr

The contents will look something like this:

-----BEGIN CERTIFICATE REQUEST-----
123DQDCCAigCAQAwgbAxCzAJBgNVBAYTAkFVMRgwFgYDVQQIEw9Tb3V0aCBBdXN0
cmFsaWExETAPBgNVBAcTCEFkZWxhaWRlMSAwHgYDVQQKExdUaW1lc2hlZXRzIE1U
UyBTb2Z0d2FyZTEkMCIGA1UEAxMbd3d3LnRpbWVjbG9ja210cy1vbmxpbmUuY29t
MSwwKgYJKoZIhvcNAQkBFh1hZG1pbkB0aW1lY2xvY2ttdHMtb25saW5lLmNvbTCC
ASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAOeLw456FYT+reZ8VH2PFD6B
pxk4NGHczMeAajXVabF/O+KBTqveBeqCEoXQzPO7Z4rovVdNJp5z06t54mOmILue
pENONp7mmfLMaPrvPrunOzdCo1ltoegNhQh+SYK0gY2SoMA69hEm2jgDL9umBueY
7kSCa/p0p7znZWCxt1yFf/5VhKVgXgh0C+6PD9VJp4nEBYcVSwOlbuHi1gsCFmwb
DXOpU6vAK61Ot43AaOdWGN8Aao9GqvMPDyV1hW0GcH/WCyAD19u5YUSz2jZP9qnc
MP1IhDD2smz24qkdZD8/LwPusEnAILL8HVp3z5v+4FMAVHWWNGj6X+ZgNQYPlFUC
AwEAAaBKMCAGCSqGSIb3DQEJBzETExFzb2dneWhvcnNlYmlzY3VpdDAmBgkqhkiG
9w0BCQIxGRMXVGltZXNoZWV0cyB789MgU29mdHdhcmUwDQYJKoZIhvcNAQEFBQAD
ggEBAIj4cddghuRyEGcJvSzsdarfaIHIUHUUN9kc9udsqS0pNcnk0s4xBMz2E72R
0ArwP43QBAiA8x8aganbqcj5WB+bDhasTDCBlOomeplQJ4LsYwdd2geMTxrxsoaq
VXc5TaS9Ct+vGqDIg+tQmd8PbGk2m3zzGFeacGn42xnRGBxe21eSmkMo4yPF7KZl
h/M5l6kZFhejSSM+jyvXqMaRjX4gtQxloXisUxiyxO2mSA5gc0p4/tJ+oC1Dj3mw
rE3EYcoOi1XkAcuflLhp/YN4COd/TkH5DwwKCMJOvmuWdqurxjXIqlxlBchqb9Ml
SXCA5yk12345678wOayT2/kQJG0=
-----END CERTIFICATE REQUEST-----

Copy the contents to the clipboard and go to your NameCheap control panel and select the SSL certificate to activate. Select the correct server configuration (in this case Apache2 + OpenSSL) and paste the contents of the CSR file into the CSR Text field and submit the activate request. NameCheap will then ask for an email address to confirm the issuing of the SSL certificate for the domain. Confirm the request and you’re done with NameCheap for this process.

4. Confirm the SSL Certificate Issue

The SSL certificate I renewed was a Comodo certificate and after a few seconds I got an email from Comodo asking me to confirm the issuing of a certificate. The email looked like this:

Comodo Confirm Issue SSL Certificate Email

Comodo Confirm Issue SSL Certificate Email

It’s simply a matter of clicking on the confirmation link in that email and pasting the “validation code” into a text box on the resultant web page. Once you’ve done that another email will arrive in a few minutes with a ZIP file attached. The zip file contains the following files:

Comodo SSL Files

Comodo SSL Files

The AddTrustExternalCARoot.crt is the CA (certificate authority) root certificate which identifies the company we’ve paid to issue the SSL certificate. In this case, Comodo via NameCheap. You’ll need that file. You’ll also need the www_mydomain_com.crt file which is the signed CSR.

5. Install and Protect Certificate File

Both files mentioned above need to be copied to the folder we created in step 1, namely:

/etc/apache2/ssl/december-2014/

Once there protect the signed crt file with:

chmod 400 www_mydomain_com.crt

6. Create the Certificate Bundle

To complete the SSL chain for Apache you must create a bundle with the files send by the certificate authority. This is done easily with:

cat AddTrustExternalCARoot.crt COMODORSAAddTrustCA.crt  COMODORSADomainValidationSecureServerCA.crt www_mydomain_com.crt > www.mydomain.com.ca-bundle

7. Configure Apache to use New SSL Certificate

The last step in the process is to configure Apache to use the new SSL Certificate. Open up the site configuration file with:

nano /etc/apache2/sites-available/site-name.conf

In my case I looked for the SSL VirtualHost section of the file which looked like this:

<VirtualHost  222.222.222.222:443>
    SSLEngine On
    SSLCertificateFile /etc/apache2/ssl/december-2011/www.mydomain_com.crt
    SSLCertificateKeyFile /etc/apache2/ssl/december-2011/www.mydomain.com.key
    SSLCACertificateFile /etc/apache2/ssl/december-2011/PositiveSSLCA.crt
    SSLCertificateChainFile /etc/apache2/ssl/december-2011/www.mydomain.com.ca-bundle
    ServerName www.mydomain.com
    ServerAdmin general@mydomain.com
    ServerAlias mydomain.com
    DocumentRoot /srv/www/public_html/
    ErrorLog /srv/www/logs/error.log
    CustomLog /srv/www/logs/access.log combined
    AddHandler cgi-script .cgi .pl
</VirtualHost>

You can see that the SSLCertificateFile SSLCertificateKeyFile and SSLCACertificateFile directives need to be changed to use the new certificate files. I modified them to look like this:

<VirtualHost  222.222.222.222:443>
    SSLCertificateFile /etc/apache2/ssl/december-2014/www_mydomain_com.crt
    SSLCertificateKeyFile /etc/apache2/ssl/december-2014/www.mydomain.com.key
    SSLCACertificateFile /etc/apache2/ssl/december-2014/AddTrustExternalCARoot.crt
    SSLCertificateChainFile /etc/apache2/ssl/december-2014/www.mydomain.com.ca-bundle

Saved the file, exited Nano and then restarted Apache with:

service apache2 restart

8. Verify New Certificate is Installed

If you’ve done everything correctly it’s a simple matter of checking if the certificate is installed correctly. I did it by navigating to the website in Google Chrome and clicking on the https padlock icon and viewing the certificate information. It looked like the image below and you should see that the expiry date has updated correctly. Job done.

Success - SSL Certificate Installed

Success – SSL Certificate Installed

Bash Shell Vulnerability Found – Patching Ubuntu

The interwebz is abuzz with the latest vulnerability for *nix based systems that us the BASH shell. The so-called Shell Shock exploit allows devious users to run commands against your system using a bug in the way BASH handles environment variables. Ouch. To check if your system is vulnerable run the following in a shell:

env x='() { :;}; echo vulnerable’ bash -c “echo this is a test”

If your system is vulnerable you’re going to see this:

vulnerable
 this is a test

If your system is not vulnerable you’re going to see something like this:

bash: warning: x: ignoring function definition attempt
bash: error importing function definition for `x'
this is a test

There’s a patch available for Ubuntu systems already. All I did was this:

sudo apt-get update
sudo apt-get install bash

And hey presto the system is no longer vulnerable.

How I Reload my iptables Rules on Ubuntu Reboot

There’s a few ways that custom iptables rules can be re-loaded when your Ubuntu server reboots. I’ve chosen to reload mine using the /etc/network/interfaces file. Here’s what I’ve included in that file:

auto lo
iface lo inet loopback
pre-up iptables-restore < /etc/iptables.firewall.rules
auto eth0
iface eth0 inet dhcp

The key line here is the line starting with pre-up. This directs the iptables-restore to reload my rules from the /etc/iptables.firewall.rules file.

Another way of accomplishing the same thing is to create a script file in the /etc/network/if-pre-up.d/ directory and put the following in it:

#!/bin/sh
/sbin/iptables-restore < /etc/iptables.firewall.rules

Then set the permissions on the script file with:

sudo chmod +x /etc/network/if-pre-up.d/your-filename

Setting up Apache Permissions for www-data and an FTP User

A problem I have when setting up a website on a new server (or a new site on an existing server) is sorting out permissions for the Apache user (www-data) and an FTP user. I can never quite remember how to set things up so that I am not continually needing to log into a console to adjust permissions so I can FTP some files to the server. This post is the definitive reminder to myself showing how it should be done. This example applies to Ubuntu but I guess it is equally applicable to other flavours of Linux.

Add the FTP User to the www-data Group

First thing we want to do is add the FTP user (you have created an FTP user haven’t you?) to the www-data usergroup. www-data is the user/group used by Apache.

sudo adduser ftp-username www-data

Change Ownership of Files

The next step is to set the www-data group as the owner of all files and directories in the HTML source directory.

sudo chown -R www-data:www-data /var/www

Grant Group Permissions

Now we want to add write permission for the www-data group for all the files and directories in the HTML source directory.

sudo chmod -R g+w /var/www

Add umask for New Files

The final step is to make a change to an Apache configuration file so that the umask for new files created by Apache is such that the www-data group has write permissions on them. Open /etc/apache2/envvars in your text editor of choice and add this to the bottom of the file:

umask 007

The three octal digits for umask are for the Owner/Group/Others. The 0 leaves permissions unmasked (ie left at read/write/execute) and 7 gives no permissions at all. This would be equivalent to chmod 770. There’s a useful chart here showing the relationship between the binary rwx permissions and the octal numbers used by chmod and umask.

Credit for this must go to the top voted answer to this question on askubuntu.com.

Installing Munin on a Linode Node

I’ve followed this guide for installing Munin on my nodes at Linode a few times now. Munin is a remote server monitoring tool that allows you to monitor all sort of things about the health of your servers. Servers you want to monitor are “munin-nodes” and the server where you view all of the reports from is the “munin-master”. It all makes sense but the Linode installation guide fails horrifically if your apache webroot is not the default /var/www that Munin expects it to be. So as a reminder to myself, here’s how I got Munin working on a server where the web root was /srv/www.

First, follow the install guide exactly as instructed by the good folk at Linode. Then modify /etc/munin/apache.conf to remove references to /var/www and change them to your web root of choice. My file looks something like this:

Alias /munin /srv/www/munin
<Directory /srv/www/munin>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride all
                Order allow,deny
                allow from all
    <IfModule mod_expires.c>
        ExpiresActive On
        ExpiresDefault M310
    </IfModule>
</Directory>

Then I restarted Munin with:

sudo /etc/init.d/munin-node restart

I happen to use my munin-master as a dumb web host so the pretty Munin web pages are just in a folder off of the web root. You could be tricky and put them on their own sub-domain but I’ve chosen not to do that. So they live in a sub-folder and I can access them using this address:

http://www.your-site.com/munin/

And when I did I was seeing a 403 forbidden error. And when I took at look at the apache error log saw an entry like this:

[Mon Feb 03 04:09:23 2014] [error] [client 123.123.123.123] client denied by server configuration: /var/cache/munin/

Clearly Munin is still trying to access a folder outside of my webroot. So then it was a matter of adding this new section to my /etc/apache2/sites-available/default file:

        <Directory /var/cache/munin/www/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>

Then restarting apache with

sudo /etc/init.d/apache2  restart

Now if I visited the Munin pages using the same address as above I can see pretty graphs telling me all about my servers! w00t!

Installing the PHP MCRYPT Extension on Ubuntu

I use the PHP mycrypt extension to generate some encrypted URLs for one of my online applications. I had reason to test some of this functionality on my local development web server recently and the following error was thrown by PHP:

Call to undefined function mcrypt_create_iv()

Turns out that I didn’t actually have the MCRYPT extension installed on my little Ubuntu web server. Doing so was simply a matter of typing this at a command prompt:

sudo apt-get install php5-mcrypt

And that’s it. No need to re-start Apache or anything else. Groovy.