rootblog

Configuring Personal Gallerys on Radix

Due to our customized Apache installation, there are a couple of notes to make about installing the popular Gallery program for yourself on Radix. Specifically, PHP scripts are run in CGI mode, and as the user that owns them. So when following the Installation Instructions for Unix with Shell Access on the Gallery page, please do the following things:

1. Do not do any of the "chmod 0777" or "chmod 0666" commands that they tell you to. These are only needed if gallery will be run as the web user.
2. After you have untarred the gallery package, cd into the gallery directory and run the command "find ./ -name \*.php -exec chmod a+x {} \;" to make all the .php files exectuable by the web server.
3. Follow their instructions to execute a "touch config.php .htaccess", but do not execute the associated chmod commands.
4. Go to http://url/gallery/setup/index.php
5. Make sure there are no extra warnings other than "Apache is not obeying the 'php_value' lines in your .htaccess file." (If you are running PHP in CGI mode, this message is unavoidable)
6. In the setup wizard's Location and URL page set the Temporary Directory to /tmp

After completing these steps gallery should be ready to go! If you have problems, please let us know so we can update the instructions.

Apache Configuration

We have customized the web server configuration on Radix in an attempt to standardize our various web site configurations and make it easier for each end-user to self-manage their site. Most of these settings will be transparent, but there are a couple of special things to note for people who use special configurations, especially PHP scripts.

SuEXEC for CGIs and PHP scripts

Since one of the most common uses for PHP scripts on Radix is personal web galleries that people have set up, and due to the general inconvenience of running all PHP and CGI scripts as a single "www" user, we're going to configure Apache's SuEXEC module so that any CGIs and PHP scripts run from your home or web directories will be executed under your user id. This means that you will no longer need us to chown album directories to the web server and that you will be able to directly manage the files under your gallery directories.

Changes to make: We will be configuring the web server to automatically run PHP scripts in CGI-mode. This means that existing scripts that are interpreted directly by the web server will need to be made executable so they are runnable as a CGI. We've configured the OS to automatically recognize .php files and run them with the proper interpreter, all you need to do is chmod them so the Apache server can execute them. This can be done for all .php files under your web directory by running the following command from your public_html directory:

find ./ -name \*.php -exec chmod a+x {} \;

Making .php files executable is safe to do even on library files.

Changes to make: Since PHP and CGIs will be running as specific users, old installations of Gallery or other tools that write to the local filesystem will now be running as your UID. This means that the old album/ directories will be owned by the wrong user. We're *not* planning on doing a mass chown of files in web directories back to the original owners simply because we're not sure what people have changed and what they haven't. This will fall in to the "you'll need to look at your system and test it to make sure its working" category. Expect further recommendations on setting up Gallery itself.

Implementing VirtualDocumentRoot

To remove the need for multiple special-case configurations in Apache's httpd.conf we're going to standardize on one configuration that applies to all domains Radix hosts. This means essentially that a request for http://domain.tld/ will automatically look in /www/domain.tld/doc for HTML files and /www/domain.tld/cgi-bin for scripts to run (more on CGIs below). This is an incredibly simple way to set up the system, but it doesn't automatically handle server aliases like http://www.domain.tld/ (which some people use and some don't).

Changes to make: If your domain is regularly accessed using something other than the standard http://domain.tld/ configuration you will need to let us know what ServerAliases you want to configure to support. Please only consider doing this if you have www.domain.tld published far and wide, as its simpler for everyone (us, you, Google, statistics generators, etc) to standardize on one domain.

.htaccess AllowOverrides

To get rid of special-case cgi-bin and directory access configurations in the global httpd.conf, we're going to open up the use of .htaccess files for pretty much everything you can configure, from authentication to CGI control to error pages. This means that essentially every configuration option you might want to tweak for your site will be available to you by modifying these files in your web directories. An excellent overview of what kind of flexibility you will have can be found here:

http://httpd.apache.org/docs/howto/htaccess.html

When you read this page you'll notice that Apache actually recommends against people using .htaccess files for a couple of reasons, one of which follows:

The second consideration is one of security. You are permitting users to modify server configuration, which may result in changes over hich you have no control. Carefully consider whether you want to give your users this privilege.

We're doing this to make everyone's life easier, so we're again reminding people to follow our general terms of service: "Don't be a hozer, ehh."

If you're curious as to what specific changes we made to the configuration file, they are documented here.

Per-user Quotas Expanded to 5GB

Per-user Quotas Expanded to 5GB. This should be plenty for any sane person. The system will send you a warning mail when you exceed 4.5GB. We will soon be implementing per-user quotas on /var/mail as well.

You can use the "quota" command from a shell prompt to easily see how much disk space you are currently using.

Ancient Backups

We mentioned before that there were some really old backups (six months) of Radix's data that we were going to restore at some point. Turns out that these files are actually about a year old, and include a complete image of /home as of 26-04-2004 and an snapshot of files that had changed from that point until 22-05-2004.

But the news keeps getting better. The full image snapshot has been corrupted. Right now I've only been able to pull about 5 home directories off. I don't know if we'll get any more unless we can find a TAR recovery tool. The best bets to look for recovered data are in:

/srv/RECOVERED/public/old-backups/partial-full-2004-04-26/home
/srv/RECOVERED/public/old-backups/incr-2004-05-22/home

We'll let you know if we pull anything else useful out of here.

/crypt/users Recovery

The copy of old data proceeds. As we mentioned before the /home directories were restored for all but about 10 people, and it looks like we completely lost the /www tree (a skeleton is back there now which you can begin to reconstruct). /crypt/users was unaffected by the crash and is about 90% copied back. Only the people with the largest directories are still coming over. You can find everything that has made it back in /src/RECOVERED/public/crypt-users. THIS IS A TEMPORARY LOCATION ONLY. /srv/RECOVERED is only where we are copying recovered data too. Even though you can write to it, consider this a read-only location. We will be removing all files from there eventually, so use it only for recovering your old data, do not put new content there.

We are moving to a monolithic /home disk for all user data. This means that what you used to put in /crypt/users, you should now put in your home directory. We are keeping local snapshots of the home directories, but there's too much data in there to perform off-site backups. We have currently deployed a 2 Gigabyte quota on /home for all users, and a shared 10 Gigabyte quota for anything owned by the Apache user (this means all "gallery" installations will be subject to a shared 10 Gigabyte quota). More news to follow.

So to sum up:

• Copy data from /src/RECOVERED/public/crypt-users to your home directory (this directory will eventually go away)
• There is a 2 Gigabyte quota for all users on /home now

SMTP-AUTH Enabled for Outgoing Mail Relay

For all you Thunderbird and Eudora loving folks out there, you can once again relay mail through Radix's MTA, but you have to properly authenticate (before we had a hack in place so you didn't have to configure this). Basically, you have to configure your mail client's Outgoing SMTP Server to both use TLS (may be called SSL) as well as a Username/Password (possibly called SMTP-AUTH).

Thunderbird

Go to Tools :: Account Settings, then select "Outgoing Server (SMTP)" on the left-hand side. Set the following:

• Server Name: radix.cryptio.net
• Port: 587
• Check the box next to Use name and password
• User Name: (Enter your regular user name here)
• Use secure connection: Select TLS

Now you should be able to send mail through Radix. When you first try to do so it will ask you to accept the TLS certificate presented (we'll have something to quiet that down later), and then ask for your password. This is the same as your usual Radix password. You should be all set!

Websites (partially) back

Radix now thinks it's hosting all the sites it used to. Of course, we don't have the data to go in said websites, but the framework is back up. If you have files you want to put back in the web tree, you can find your DocumentRoot's in /var/www/[www.yourdomain.com]/doc/

Data and OS Updates

First of all, we're running Debian Linux now instead of FreeBSD, someone pointed out that would be worth mentioning.

Also, WRT to data recovery, we've copied back all the home directories we were able to recover. If your data isn't there, we couldn't recover it. We have the disk though and may consider hiring a professional data recovery service if there are enough people who want to chip in for it (our only hesitation is that these folks are usually very expensive for only a marginal rate of success).

/crypt/users was unaffected and is currently copying over to the new box at a very slow pace. I wouldn't expect all data to make it over for the next week or so (we're talking nearly 100 Gigs of photos and what not).

The old backups are also copying over, and should come online sometime after this weekend. We'll note when they are available.

Everything that we've been able to recover and that is safe for public consumption is availabe in /srv/RECOVERED/public/. We've already copied over everything that has completed a copy, so the only reason to look here is to see if maybe your /crypt/users directory got copied over earlier. DO NOT COPY THIS DATA TO YOUR HOME DIRECTORY YET.

Note that we are currently running without a net, there are no backups being performed and there is no disk space management happening on /var (where /home is currently located). If any single user fills up the disk it will stop machine operations for all other users.

Webmail Running

Squirrelmail is available again, and can be accessed at:

https://radix.cryptio.net/squirrelmail/

If you have trouble logging in, please email root@radix.cryptio.net from a separate account. If you have no other access to email please contact your friendly admin directly between the hours of 9am and 9pm Pacific time.

At this point the most essential services are up and running, so we are going to try and get some sleep. Some things will come back up over the weekend, and we continue to transfer data from the old disks to the new ones, which should complete some time next week (there is a lot of data).

Wednesday Night (Late)

We have successfully recovered everyone's mail spool and restarted mail and DNS services. This means that mail is flowing in to Radix from various points around the net. We have recovered nearly 80% of /home directory data and re-enabled SSH access to the box. Web mail and remote IMAP/POP access is not yet configured. ETA for those is still sometime tomorrow, err, today....ummm...later...