So, I thought I had Apache2 running, more or less, only to find php didn't work. You mean apache doesn't come with php as standard?

apt-get install libapache2-mod-php4

That got php installed, but then wordpress started spewing forth this error: Your PHP installation appears to be missing the MySQL which is required for WordPress..

But mysql is installed and so are the mysql php files. Fortunately, this page had the answer:

If you do, one solution is to place this bit of code in wp-settings.php before the check for MySQL which will load the mysql module if it's available:

    if (!extension_loaded('mysql'))
    {
    if (!dl('mysql.so')) {
    exit;
    }
    } 

However, next time you upgrade WordPress you will need to remember to re-hack this code into wp-settings.php (unless it gets added to an official release someday).

The better solution is to edit your php.ini (generally in /etc/php4/apache or similar under linux) and add the mysql module to the list of extensions at the very end of the file with:

extension=mysql.so