Doc / Linux / the Project as Website

``How I Work'' article #6

Any good software-development project has a web site that includes things like the documentation, the download directory, possibly a browsable and cross-referenced version of the source code, and so on. There is also a directory structure that you get by checking out a working copy of the project from CVS. The key observation behind this document is that these two directory trees ought to be the same, or at least can be treated the same for many purposes.

There are several advantages to this approach, and making it easy to maintain the project's web site is perhaps the least of them. A more significant one is that you can get a good overview of the entire project simply by running a web server on your local workstation and pointing its document root at the project.

Once you have a local web server pointed at your working directory, there are a couple of useful tricks you can use for linking your browser with your text editor and other development tools.


If you look at any directory in a project that I'm working on, and any directory under my home directory, you'll notice a curious thing: every directory contains a file called HEADER.html, which is what Apache displays at the top of a directory listing in a directory that doesn't have an index.html file. There's even a HEADER.html file in the documentation directories, which also contain index.html files.

If you actually look at several of these HEADER.html files, you'll see that they are all remarkably similar, and that the information in them is often quite different from what you would expect to see on a website -- for example, it usually includes information that could be of interest only to a developer. It's often a little out of date, too.

Just as an example, compare the HEADER.html file in this directory with its index.html file (which you're reading right now). Now look at ../HEADER.html and ../index.html. Get the idea?

The general idea is to make your local working directory a subdirectory of some convenient website, for example the one on localhost or your local intranet server. Then you can browse it with any web browser. What's more, there are lots of useful web-based tools for browsing source code and CVS archives -- two that I've used are lxr (the Linux (kernel) Cross Reference) and cvsweb. We'll get to those later.

The most convenient thing about this arrangement is that you can write your documentation, implementation notes, to-do lists, and so on in HTML and link directly to the appropriate places in the source code.

If your project has a website, or if your project is a website, you're going to end up with two very different views: the one the public sees, and the one you and the other developers use. We're mainly talking about the developers' view here. There are, of course, several different ways to provide both views on one server -- the one I usually use is to make a subdirectory for the source view, with a .htaccess file that supresses normal web server behavior.

If your project is a web application that's meant to be part of a web site, there are a couple of other things you can do. I'll discuss them in a later paper when my ideas on the subject have had a chance to congeal a little.

WOAD: Web-Organized Application Development

The techniques described here are part of a concept I call ``Web-Organized Application Development'' because of the cool acronym, WOAD. I wrote a subsystem with that name as part of an open-source project called PIA. With luck, it's possible that the WOAD Demo is still online. Unfortunately, though it had what I consider to be some good features, it was slow and practically impossible to customize. That's because it was written in a combination of Java and XML.

I know better now, so I'm in the process of rewriting WOAD as a stand-alone project, in Perl. (Yeah, I know: I should use it as an excuse to learn Python. But I'm lazy, and mod_perl runs really well on Apache.) The later sections of this document refer to WOAD; there are red flags on the pieces that don't work yet.

Getting Started

OK. There are two main ways to do this: in your home web directory, if you have one, in the web server's document directory. It's a little less complicated the first way if you're already set up for it. You can also do it by making an alias in the web server's configuration file, but that's even trickier.

In Your Personal Web Directory

Apache is usually set up to map a subdirectory of your home directory (the default is public_html) into a web home directory, the name of which is your login name with a ``~'' (tilde) in front of it. It's usually used for personal pages on an organizational website. You can take advantage of this.

cd ~/public_html
mkdir projects
cd projects
ln -s ~/Project .

A couple of points to keep in mind:

In the Web Server's Document Root

I should mention at this point (since this series is called How I Work) that this is the way I usually do it. At work, where I have a Linux box on my desktop that never gets turned off, and that nobody else uses, I use it and keep the working directories in my local home directory, ~lsteve, with symbolic links from the web server's document root. (My other home directory, ~steve, is on a shared file server; I don't use it for software development because other peoples' Windows boxes make the network unreliable). At home, where I'm the only programmer, I use the web server on the file server and keep the working directories in /usr/local/projects/. Your mileage may vary.

Go to the web document directory on your local machine. It will be /home/httpd/html/ on old RedHat systems, /var/www/html/ on newer ones (7.0 and later), or /var/web/ on Debian. If you don't have Apache running on localhost, install it (or ssh over to your internal web server). Make a subdirectory called projects (you'll probably have to be root for this, unless you're already in a group that can hack the web data), and make a symbolic link from there to the working directory of any project you're working on.

cd /var/web
mkdir projects
cd projects
ln -s /usr/local/projects/Project .

At home, where I already have a projects directory, I just made a link to that. Your mileage may vary; you might even want to make a link to /usr/local/src/. In my case, it looks like:

cd /var/web
ln -s /usr/local/projects .

Using It

So, there you are. If all went well you should be able to browse to /projects/ or /~yourLogin/projects/ and get a directory listing with a link to your project. Follow it, and one of two things will happen:

If you don't run into an index.html file in the top level, chances are you'll hit one in the documentation subdirectory. So you'll eventually need to suppress them, and here's how: put the following into a file called .htaccess in the projects directory:

AllowOverride none

The DirectoryIndex line tells Apache not to look for an index.html, and the AllowOverride none line tells it to ignore any .htaccess files it finds in any of your projects.

Some Fine Points of Apache Indexing

The way Apache makes index pages is to put a file called HEADER.html at the top of the listing, if there is one, and a file called README.html at the bottom, if there is one. I take advantage of this by having my Makefiles set up an appropriate HEADER.html in every new directory.

If you have a CGI script that generates specially-formatted listings the way you like them, you can set that with the DirectoryIndex directive. For example:

DirectoryIndex /cgi-bin/src-index.cgi

Note that I've called it src-index because there might be some other index CGI for ordinary web directories, and most people call that one index.cgi. Your indexer can do other things for you as well, as we'll see in the next section, because it can process query parameters.

Fun With src-index.cgi

Fun With make

Other Tools

$Id: index.html,v 1.3 2006-06-24 03:49:57 steve Exp $
Stephen R. Savitzky <steve @>