Ubuntu 12.04 Server Developer VM in VirtualBox

Posted on November 28, 2012 by Chris

Over the past few months, I’ve grown increasingly interested in simplifying deployment of my applications. I intend to create a one click deployment process for my latest Ruby on Rails application. My approach is to build a VM running the application then write a script to clone and configure the VM. I want the script to require as little user input as possible to create a brand new production instance of the application. Once the process completes, it will be replicated for future applications.

Today I’m going to discuss the process of building an Ubuntu 12.04 LTS server on VirtualBox. I’ll focus on:

Creating the virtual machine.
Installing Ubuntu 12.04.1 LTS Server Edition with OpenSSH
Network configuration
Managing the server

My host operating system is Ubuntu 12.10 Desktop Edition. I will assume you are using the same or know how to translate my instructions for use in your environment of choice.

Creating a virtual machine in VirtualBox is pretty trivial. Launch Oracle’s VM VirtualBox Manager, and click the “new” button. A wizard will appear with pretty simple instructions. Give the machine a meaningful name. I use scripts and aliases in most instances where I need to reference my virtual machine name so I favor verbosity over brevity here. I used the title ubuntu-12.04.1-LTS.

Use your judgement in allocating resources. I’m pretty stingy, but I give my machines 1024 MB (1 GB) RAM and 8 GB disk space on a fixed size VDI (VirtualBox Disk Image). The fixed size disk ensures that there’s never confusion about whether the VM can actually use more disk space. With dynamic allocation, the host may run out of room when the VM still thinks it should be able to get more.

I consider the addition of network adapters to be part of building the machine. I suggest the use of a bridged adapter on adapter 1 and a host only adapter on adapter 2. I’ll run through the process, but first, a little information about what we’re doing and why:

The bridged connection on adapter 1 allows your VM to access the internet via your hosts web connection without any additional configuration.

The host only connection on adapter 2 allows your host to connect directly to the VM. This allows you to connect via SSH and to send HTTP requests through your browser. The host only connection requires a host only network which we build from the VirtualBox VM Manager.

Before we start adding network adapters, lets make sure there’s a host only network ready for use. Access the VM Manager preferences from File -> Preferences or Ctrl+G then click “Network” on the left side of the dialog box. Your existing host only networks will be listed. If there is already a network that is not being used by a VM that will run at the same time as the new Ubuntu 12.04 server, take note of the name and close the box. Otherwise, click the icon with the + and a new network will appear on the list (vboxnet0 by default). Take note of the network name. Click the name to highlight it then click the edit button and take note of the IPv4 Address (In my case, 192.168.56.55).

Click the name of your new VM in the list of VMs on the right of the VM Manager window to select it. A list of settings and options appears on the right side of the window. Click the heading Network. With the Adapter 1 tab open, ensure that Enable Network Adapter and select Bridged Adapter from the Attached to select box. For Name choose your preferred network connection. (I use wlan0 which is my WIFI).

Switch to Adapter 2 and ensure that Enable Network Adapter is checked here too. Select Host-only Adapter from the Attached to select box, and select the host only network that we set up in the VM Manager preferences earlier for Name.

Just like that the VM’s hardware is configured. The next step is to download the Ubuntu 12.04.1 LTS server edition ISO and attach it to the VM’s CD/DVD drive. Download the ISO then click Storage in VB’s settings dialog. Right click IDE Controller and select Add CD/DVD Device. When prompted, click Choose disk and select the Ubuntu server ISO. Click OK to close the settings dialog.

You’re ready to boot the server. Click the name of the server then click Start. The server will begin to prompt for installation options. Make sure to enable networking and OpenSSH. I use mostly default settings and avoid installing packages until I’ve booted the system. If you have trouble, here’s the server guide.

You can interface with the server with the window provided by VirtualBox, but I prefer to use my native applications. First I need to get the server to recognize it’s host only connection which will appear at eth1. Run:

$ ifconfig

Notice there is an entry for eth0 which is the bridged connection to the internet. This means that you can run your package manager to begin installing software, but you don’t have a connection for running SSH. To fix this, we need to edit the network interface. I prefer to use vim when editing server files, so install it with:

$ sudo apt-get install vim

Or don’t, whatever. Open the /etc/network/interfaces file and add the following code:

auto eth1
iface eth1 inet static
address 192.168.56.55
netmask 255.255.255.0
network 192.168.56.0
broadcast 192.168.56.255

In each section of italics, the first three sets of numbers are the same for each italicized IP address. These come from the first three sets of numbers you noted from the vboxnet0 host only network. Only the address line has the fourth set of numbers highlighted. You can specify whatever you want as long as it’s between 0 and 255 (exclusively). I use 55 for sentimental reasons and because it’s easy to remember.

After saving the changes, restart networking:

$ sudo /etc/init.d/networking restart

Then run ifconfig to ensure that the eth1 network is present and the IP is the one specified in the address line of the interface file.

Shut it down again and open the terminal for the host machinge. Enter the following command:

$ VBoxManage startvm ‘ubuntu-12.04.1-LTS‘ –type headless

Replace the italics with whatever you named your server. The server will start, but without the VirtualBox manager’s window. Wait a few seconds to let it boot then SSH into the server using the IP address specified for eth1.

$ ssh chris@192.168.56.55

Once you are logged on, it’s probably a good idea to install updates, then install all the packages you need to run your server.

Using sshfs you can mount a directory on your VM to a directory on the local machine. This is convenient if you want to access VM files with local applications (such as gedit).

$ mkdir ~/dev/app-remote

$ sshfs chris@192.168.56.55:/home/chris/dev/app /home/chris/dev/app-remote

You can then access files from the server by browsing to the local directory /home/chris/dev/app-remote.

In future articles, I’ll discuss setting up application environments and clone them for quick server setup.

Great Python Articles

Posted on July 18, 2012 by Chris

I’ve been reading some really awesome Python articles lately, and I thought I might as well start sharing them. I’m going to start a few series that consist of blog and tutorial collactions. Security and PHP are two other topics I’d like to post about regularly. I’m also planning on adding a few of my own tutorials over the next few weeks so stay tuned.

The Beginners guide from Python.org
OK, so it’s not something I read over the last few weeks, but I did read it about 3 times before and I won’t be surprised if I turn to it again. It covers the basics of the language fast and gets you ready to develop sophisticated programs.

Code like a Pythonista: Idiomatic Python
This is an excellent tutorial that covers dozens of Python tricks that encourage really good craft. It’s basically a cookbook of some of the best short cuts in the language. Learn them, know them, use them.

This is the best tutorial on Python decorators I’ve seen so far. Check out the rest of the site too, it’s pretty enlightening.

virtualenv
I’ll probably do another article that covers a list of useful libraries before too long, but for now I wanted to mention this because it makes your life as a Python programmer much easier and more productive. Keeping a clean sandbox prevents you from forgetting about common dependencies.

Python Tips for PHP Developers Part 1: Existence of Variables

Posted on July 7, 2012 by Chris

I’ve begun to work on several small projects to build up my Python chops lately. After considerable research I’ve landed on the Python side of the Python vs. PHP debate. I don’t intend to join the flame wars between the languages as I enjoy working with both. As I move forward in my exploration of the Python programming language, I intend to write several of these quick and simple posts that will help PHP developers work with Python.

Python has several important differences from PHP. You can’t use the same tricks in Python that you would use in PHP. One basic difference is the method for determining whether a variable has been declared. The reason that this is so different is because Python doesn’t have variables in the same way as PHP. Instead, Python has names. In PHP, you can always use the empty() function to determine whether a variable has been created and populated:

/* I want to use the variable $var, but I don't know if it is initiated/has a value.
* I also don't care if it is initiated if it doesn't have a value
*/
if ( ! empty( $var ) ) {
// Do something with $var
}

That code will work with any type of variable including objects, arrays, and scalars.

To my knowledge there is no analogous function in Python. The type of the variable determines the technique necessary to check it’s existence. If you simply want to know whether the name exists in the current scope:

try:
    var
except NameError:
    # var does not exist, deal with it

What if I want to figure out whether or not an array has a specific key? In PHP, we could do this a couple ways:

$arr = array(1);
if ( ! empty($arr[0]) ) {
    echo $arr[0];
}

if ( ! empty($arr[1]) ) {
    echo $arr[1];
} else {
    echo "No $arr[1]";
}

The output of the above will be:

1
No $arr[1]

We can also use the array_key_exists() function to check whether or no the key has been used. I showed the empty() method first to show how PHP will allow you to use the same technique for any variable. The nice thing about using empty() is that you don’t even need to know that $arr exists (or that it is an array).

In Python we don’t have arrays, we have lists and dictionaries. They behave quite similarly, but we can’t test for the existence of an index the same way that we can in PHP. For a list, we’ll just check the length and see if it is sufficient to contain the index we want to use:

import sys

if len(sys.argv) < 1:
    # We can use sys.argv[1]
else:
    # We need to use the default value or report back that a necessary argument is missing.

We have one more type to check. The dictionaries in Python allow a syntax that is reminiscent of the PHP associative array. In PHP we can test for the existence of an index to an associative array in exactly the same way that we check for the index of a numeric array. Python requires something like the following:

dict = &quot;a&quot;: 1}

if dict.has_key(&quot;a&quot;):
    print dict["a"]

if dict.has_key("b"):
    print dict["b"]

The efficiency of the above example can be improved by using “in” syntax as follows:

dict = {"a": 1}

if "a" in dict:
    print dict["a"]

if "b" in dict:
    print dict["b"]

I think that covers all of the relevant checks. Use the comments for tips, corrections, questions, and suggestions!

Migrating WordPress Sites

Posted on April 18, 2012 by Chris

I’ve spent extra time on the WordPress support forums this week, and I’ve seen several questions about WordPress migration. Having gone through the ringer on WordPress migration myself, I decided to write up a quick tutorial on the subject.

First, it is important to use WordPress responsibly. The only files you should edit are those within ~/wp-content and ~/wp-config.php. I recommend setting up the wp-content directory as a project in a version control system (VIS) such as git or svn. Alternatively, you can manage these files with FTP or SFTP, but in so doing you sacrifice some excellent features and stability.

The first step in migrating your WordPress site is to download the latest version and upload it to your new host.

Delete the wp-content directory and replace it with your own. You can do so using a git clone, svn checkout or FTP/SFTP transfer depending on your preference.

Next you need to create a new MySQL database on your new host. If you have access to the MySQL command shell, you can create your database with this command:

mysql> CREATE DATABASE wordpress;

If you don’t have shell access, you’ll need to use whatever tool your host provides for MySQL management (CPanel and PHPMyAdmin or some combination of the two is common). Also, make sure to create a MySQL user with a strong password and full rights on the WordPress database:

mysql> CREATE USER ‘wp_user’@'localhost’ IDENTIFIED BY ‘reallystrongpassword’;
mysql> GRANT ALL PRIVILEGES ON *.* TO ‘wp_user’@'localhost;

Now you need to transfer your database contents from your old host to the new one. Again, this is pretty host specific. If you have shell access, you can log into your site with SSH and dump the database then download it with SFTP:

Local $ ssh user@host
Host $ mysqldump -u’mysqlusername’ -p’mysqlpassword’ databasename > databasefile.sql
Host $ exit
Local $ sftp user@host
Host $ get databasefile.sql
Host $ exit

Then import the database on your new host. Again, assuming shell access:

Local $ sftp user@host
Host $ put databasefile.sql
Host $ exit
Local $ ssh user@host
Host $ mysql -u’mysqlusername’ -p’mysqlpassword’ databasename < databasefile.sql
Host $ exit

Now we reach the tricky part which I wrote about recently. Upload the tool described in this post to upgrade your database then DELETE THE SCRIPT.

I hope this helps a few of you. I’m going to add some details for doing some of the things I mention here in PHPMyAdmin and CPanel. If you have trouble please leave a comment. Also, if you would like to see other file transfer or database tools explored, let me know. I’d be happy to add them.

WordPress Database Migration

Posted on April 14, 2012 by Chris

Today I launched my new portfolio site and was reminded of the hassle associated with moving a Word Press web site from one environment to another. The development version of my portfolio uses the domain portfolio.chris.com, but I needed to publish the site to www.twincitieswebdeveloper.com.

For previous Word Press migrations I’ve used msyqldump and the linux utility sed piped through grep to find and replace all URLs in the sql file before importing the resulting file into the new location. This works, but it’s a bit tedious and technical so it’s not ideal for the point and click crowd to which Word Press really caters.

A quick Google search directed me to http://interconnectit.com/124/search-and-replace-for-wordpress-databases/. David Coveney et al. have built a utility that will perform the search and replace on the database with virtually no effort from the user.

Once I downloaded the file, I placed it in the root directory of my Word Press installation because then it gracefully grabs database information from my wp-config.php file. I followed the instructions in the excellent user interface provided by James Whitehead, and the process was complete in less than five minutes.

I subsequently deleted the file from my site since it is an incredible security risk if it sits there unattended.

Thanks to David Coveney, James Whitehead, and Robert O’Rourke for this tool!

Introducing GameSpace

Posted on March 27, 2012 by Chris

GameSpace is a soon to be open source gaming engine built purely in JavaScript. The engine includes:

A 2D tile based graphics engine
Tile based level editor
Multiple keystroke detection
Gravity
Collision Detection
Layering

Features that are ready to integrate:

Online multiplayer over Web sockets with node.js

Features that will be coded soon:

Camera
Improved Collision Detection
Widget Based Editor

Someday features:

OpenGL
3D

WordPress Automated Testing Suite

Posted on March 27, 2012 by Chris

Today I checked out the WordPress Automated Testing Suite for the first time. The setup and installation was relatively simple, but I did hit a couple of road blocks. Here’s a brief recap of what I did to install the suite.

First I needed to checkout the test suite from svn:

svn co http://svn.automattic.com/wordpress-tests/ wordpress-tests

Next I needed to copy the wp-config-sample.php file to wp-config.php like I’m installing WordPress:

cd wordpress-tests/
cp wp-config-sample.php wp-config.php

Then we need a database:

mysql -u'dummy' -p

CREATE DATABASE wptest

Once the database has been created, just edit the wp-config.php file and update the database name, username, and password. I copied the wp-config.php file into the wordpress directory, but I’m not sure that’s necessary:

cp wp-config.php wordpress/

According to the WordPress testing documentation, you should now be able to run the testing suite:

php wp-test.php

but it wasn’t so simple. First, you need to install PHPUnit which is a bit more hassle than it ought to be:

sudo apt-get install phpunit
php wp-test.php

gave me an error:

PHP Fatal error: require_once(): Failed opening required ‘PHP/CodeCoverage/Filter.php’ (include_path=’.:/usr/share/php:/usr/share/pear’) in /usr/bin/phpunit on line 38

Luckily, I discovered a post by Gio Carlo Cielo Borje that explained the added steps to setting up PHPUnit:

sudo pear upgrade pear
sudo pear channel-discover pear.phpunit.de
sudo pear channel-discover components.ez.no
sudo pear channel-discover pear.symfony-project.com
sudo pear install --alldeps phpunit/PHPUnit

With PHP Unit finally installed I could run the test:

php wp-test.php

and found 72 errors and several warnings. Tracing the errors, I found that the majority trace back to a call to:

known_bug(int $var);

You can move these errors to the messages category by specifying the -s flag for “skip known bugs” (see wordpress-tests/wp-test.php for all options):

php wp-test.php -s

Development Portfolio

For Software Engineer Chris Hanson