Easily develop and deploy web applications from subversion

by Sander Marechal

Updated 2008-28-10. Proper version control is a must for everyone who programs more than a few lines of code. Even if you develop your applications all by yourself it is very handy to be able to branch and merge your code, be able to roll back to previous versions or undo changes you made in the past. It works great for regular applications, but managing web applications or websites is a tad harder for two reason: You need a webserver to get your application going and you usually have to manage database revisions as well.

Keeping database revisions in sync with your code revisions is a complex subject that I will leave until another time. In this article I will show you how you can configure your own computer or development server in such a way that checking out or deploying a web application is just as easy as any other piece of code.

First I will show you how to configure Apache on your development server so that it picks up your checked out working copies as separate subdomains. Using this, you can simply make a checkout of your project and it will automagically be up and running. No need to touch the Apache configuration. After that I will show you how to use dnsmasq so you can achieve the same effect on your own development machine. That way you can develop your web applications locally and you won't need a central development server. In my examples I will be assuming you use subversion for your version control, but it works virtually the same with other version control packages, such as git or bazaar.

Creating subdomains for every working copy

Here is what we want to achieve. Suppose that you have a development server at http://dev.example.org where you develop your websites. We want that every working copy is it's own subdomain on that server. If I check out a working copy and name it “foobar” then it should be available on http://foobar.dev.example.org. In order for this to work, you need a wildcard DNS entry so that all subdomains of dev.example.org go to your development server. I won't cover how you do that. If you do not run your own DNS server, ask your network administrator about it or use dnsmasq as explained in the second part of this article.

Start by creating a directory where all your working copies will live. For this article I will assume that this is /home/checkout. Every directory under this directory will become a subdomain on your development server. Create a trunk directory under the checkouts directory (/home/checkout/trunk). This is where the main dev.example.org domain will go to. When a subdomain does cannot resolve to a directory under /home/checkouts we want to show an error page. I created /home/checkouts/error.html containing some useful information, but you could also redirect it to a 404 error if you want.

Then create a new virtual host configuration for Apache.

  1. <VirtualHost *:80>
  2.         DocumentRoot /home/checkout/trunk
  3.         ServerName dev.example.org
  4.         ServerAlias *.dev.example.org
  5.  
  6.         LogLevel warn
  7.         ErrorLog /var/log/apache2/error.log
  8.         CustomLog /var/log/apache2/access.log combined
  9.         ServerSignature On
  10.  
  11. </VirtualHost>

We will use mod_rewrite to tell Apache about all the subdomains. The following mod_rewrite rules extract the subdomain from the request, see if a directory exists in /home/checkout that matches the subdomain and then redirects the request to that directory (hat-tip to Stuart).

  1.         RewriteEngine On
  2.  
  3.         # If the request contains a subdomain and the directory exists, redirect
  4.         RewriteCond %{HTTP_HOST} ^([^\.]+)\.dev\.example\.org
  5.         RewriteCond /home/checkout/%1 -d
  6.         RewriteRule ^(.*) /home/checkout/%1/$1 [L]

Next, we add some more rules to redirect any non-existing subdomains to the error page:

  1.         # Redirect non-existing subdomains to the error page
  2.         RewriteCond %{HTTP_HOST} ^([^\.]+)\.dev\.example\.org
  3.         RewriteRule ^(.*) /home/checkout/error.html [L]

Update: Kevin points out in this comment that using the VirtualDocumentRoot directive is an even nicer solution than these rewrite rules.

Finally we want to hide the .svn directories that subversion creates when you make a new checkout. If you use a different version control system such as git, then replace .svn with .git in the configuration below:

  1.         # Hide .svn directories in checkouts
  2.         <Directory ~ "/\.svn/">
  3.                 order allow,deny
  4.                 deny from all
  5.         </Directory>

Restart Apache and you're done. Now you can simply log into the development server, make a new checkout in the /home/checkout directrory and you can immediately browse to it. After executing the example below, you can simply browse to http://my-working-copy.dev.example.org and see your website.

  1. $ ssh dev.example.org
  2. Last login: Wed Aug  6 09:36:33 2008 from 10.0.0.99
  3.  
  4. dev$ cd /home/checkout
  5. dev$ svn checkout https://svn.example.org/project/trunk my-working-copy

The full Apache configuration file now looks like this (download):

  1. <VirtualHost *:80>
  2.         DocumentRoot /home/checkout/trunk
  3.         ServerName dev.example.org
  4.         ServerAlias *.dev.example.org
  5.  
  6.         LogLevel warn
  7.         ErrorLog /var/log/apache2/error.log
  8.         CustomLog /var/log/apache2/access.log combined
  9.         ServerSignature On
  10.  
  11.         RewriteEngine On
  12.  
  13.         # If the request contains a subdomain and the directory exists, redirect
  14.         RewriteCond %{HTTP_HOST} ^([^\.]+)\.dev\.example\.org
  15.         RewriteCond /home/checkout/%1 -d
  16.         RewriteRule ^(.*) /home/checkout/%1/$1 [L]
  17.  
  18.         # Redirect non-existing subdomains to the error page
  19.         RewriteCond %{HTTP_HOST} ^([^\.]+)\.dev\.example\.org
  20.         RewriteRule ^(.*) /home/checkout/error.html [L]
  21.  
  22.         # Hide .svn directories in checkouts
  23.         <Directory ~ "/\.svn/">
  24.                 order allow,deny
  25.                 deny from all
  26.         </Directory>
  27.  
  28. </VirtualHost>

Developing locally with dnsmasq

Developing your websites on a central development server works, but most often it's easier to develop on your local machine instead. It's usually faster and there are quite a few handy tools out there that don't work transparently over an ssh connection. When you have a Linux desktop or laptop it's dead easy to do. Simply install Apache, PHP, MySQL or whatever server-side tools you need on your desktop, copy the configuration files from the server and modify them.

The only problem with using the above Apache configuration locally is the DNS wildcard. Unless your desktop is assigned a hostname by your network's DNS server and you can set the wildcard there, you will have to make do with your localhost address. You can install dnsmasq to act as a local caching DNS server and put the wildcard on your own machine. As a bonus, the DNS caching can also speed up your web browsing (hat-tip to Carthik). If you want, you can replace localhost with another name (like your hostname) in the commands below. The commands below are for Debian Lenny, but should be similar on other distributions. Let's start my installing dnsmasq.

  1. # apt-get install dnsmasq

Now we edit /etc/dnsmasq.conf So that it listens on 127.0.0.1 for our DNS requests and tell it to point all subdomains of localhost to 127.0.0.1. Near line 92 you will find the listen-address configuration, which tells dnsmasq to listen for DNS requests. Add the following line:

  1. listen-address=127.0.0.1

You can force domains to resolve to a certain IP with the address directive, which you find near line 63. Add the following line to make localhost have all it's subdomains pointing to 127.0.0.1:

  1. address=/localhost/127.0.0.1

Most likely your machine connects to the network via DHCP and gets it's nameservers from the DHCP server. We need to configure DHCP so that it always asks the local dnsmasq server first before trying the other DNS servers. Open up /etc/dhcp3/dhclient.conf and uncomment the following line:

  1. prepend domain-name-servers 127.0.0.1;

The next time you connect to the network, the DHCP client will add 127.0.0.1 as a nameserver to /etc/resolv.conf, before the other nameservers. If you do not want to reconnect to the network to get this change, you can add it manually for now. Open up /etc/resolv.conf and add the following line before the nameserver directives, but after the search directive:

  1. nameserver 127.0.0.1

Restart dnsmasq for the changes to take effect:

  1. # /etc/init.d/dnsmasq restart
  2. Restarting DNS forwarder and DHCP server: dnsmasq.

Now you can configure Apache according to the first part of this article, replacing dev.example.org with localhost. Do not forget to remove Apache's default virtual host, which also listens on localhost.

  1. # rm /etc/apache2/sites-enabled/001-default

Reload Apache and now you can simply make a checkout of your website to your local machine and immediately view it in your browser by going to my-working-copy.localhost. Happy coding!

References

  1. http://en.wikipedia.org/wiki/Wildcard_DNS_record
  2. http://httpd.apache.org/docs/2.2/mod/mod_rewrite.html
  3. http://muffinresearch.co.uk/archives/2006/08/20/redirecting-subdomains-to-directories-in-apache/
  4. http://ubuntu.wordpress.com/2006/08/02/local-dns-cache-for-faster-browsing/
Creative Commons Attribution-ShareAlike

Comments

#1 Anonymous Coward

We would definitely like to hear more on sync your database with SVN. It is interesting to not that we may not sync the data or records but need to maintain master data with the structure in the SVN.

#2 xyz

i have configured virtual hostsin weblogic server9.2 as follows:
1. one standalone managed server with channel name
2. virtual host name with network access point name as same as channel name of server and targetted to managed server.
3. in c:/windows/system32/etc/hosts i have given my host name like www.son.com
4.deployed application on virtual hosts,the error i am getting is---comments:None of the targets for this Deployment are running. No test points possible

plz find a solution for this, whether i have to configure apache for this to work so that i can deploy application on virtual host.

by,
xyz

#3 Sander Marechal (http://www.jejik.com)

I have absolutely no idea xyz. I haven't used Windows seriously since Windows 98. The only reason I ever touch it is to test a webdesign I made (and I prefer to use IE on Wine for that). I case you didn't notice, this website is about Free and Open Source Software, not Windows :-)

#4 Who

"I case you didn't notice, this website is about Free and Open Source Software, not Windows" Seriously! This website is about open source software... That much I understand but your comment makes the assumption that open source software does not or should not work on windows which is far from the case. If you don't know the answer it is just as simple as saying so and leave it at that don't you think? I don't mean to sound harsh but it is getting very old hearing the windows bashing everywhere and it all boils down to a lame excuse from someone who just does not have the answer.

#5 Anon

Other than that it was a great article though. Also I will add that this was helpful when combined with http://svn.spears.at/ in getting SVN 1.4.6 working with apache 2.2.4 on windows XP SP2

#6 Sander Marechal (http://www.jejik.com)

@who: I'm not bashing Windows. I'm merely stating that I haven't used it since Windows 98 was all hip 'n cool. And I do think that a FOSS blog is a strange place to as for help with deploying Weblogic Windows hosts.

@Anon: If you're interested in running Subversion under Apache, you might like my older article: Apache and Subversion authentication with Microsoft Active Directory. It not only explains how to make Subversion go under Apache, but also explains how you can authenticate Subversion users through Active Directory.

#7 dan (http://cssgallery.info)

hi
I would like to know an "easy" solution for hosted development environments, where i cannot alter the apache configuration. Thank you.

#8 Lo c Hoguin (http://wee.extend.ws)

That's an interesting approach. I'll save it for later. Thank you.

#9 Sander Marechal (http://www.jejik.com)

I would like to know an "easy" solution for hosted development environments, where i cannot alter the apache configuration.


Well, you could ask the hosting company to change the Apache configuration for you. All they would need to do is change the ServerAlias line and put the "*." in front of the domain name. You could put all the other stuff in a .htaccess file in the DocumentRoot directory.

But if you're serious about web development then you should really get your own development server to play with. It doesn't have to cost any money. You don't even need an internet connection. Get any old Pentium III or better (you can find those for free. Lots of people throw these old machines out) then install Linux and hook it into your home network.

You can make the checkouts and do all your development on your home server. No internet required! Then once in a while you make a checkout and (s)ftp it to your shared hosting account to update your production website.

#10 Anonymous Coward

I dont understand the need for dnsmasq, just another service to have running. You can do it all via /etc/hosts (or the Windows equiv c:/windows/system32/etc/hosts)

So lets say I am working on a site for acme, I will use the standard form of "acme.local" for the domain, the /etc/hosts syntax is

127.0.0.1 acme.local

Then I configure my Apache/nginx/lighttpd vhost to listen for "acme.local" and boom, you're done.

#11 Sander Marechal (http://www.jejik.com)

I dont understand the need for dnsmasq


That won't work with the above setup. Your /etc/hosts file does not support wildcards. If you put "127.0.0.1 acme.local" in your hosts file then "trunk.acme.local" or "my-branch.acme.local" will not resolve to 127.0.0.1. That's why you need dnsmasq. You need to be able to resolve "*.acme.local".

#12 Kevin

This is exactly what I need - Thanks!

Question - I would like a request to: project1.example.org not to go to /home/checkout/project1 but to /home/checkout/project1/web

I am having trouble getting this to work - I don't have a lot of experience with mod_rewrite

Any Help?

#13 Kevin

I think I figured it out - I changed the line:

RewriteRule ^(.*) /home/checkout/%1/$1 [L]

to

RewriteRule ^(.*) /home/checkout/%1/web/$1 [L]

This seems to work. What I am trying to do is get a symfony framework project working... Since symfony does its own Rewrite I think I am putting it into an infinite rewrite loop or something. I get a "500 Internal Server Error" when I try to test the project (sf_sandbox.example.com which redirects to /home/checkout/sf_sandbox/web) - When I look in apache's error log I see this error:

[Mon Oct 27 10:03:02 2008] [error] [client 10.0.2.15] Request exceeded the limit of 20 internal redirects due to probable configuration error. Use 'LimitInternalRecursion' to increase the limit if necessary. Use 'LogLevel debug' to get a backtrace.

Any idea what I can do? Do you have any experience with getting a symfony project to work with this configuration?

#14 Sander Marechal (http://www.jejik.com)

Hi Kevin,

I haven't used Symphony before. I donwloaded their source code and looked at the .htaccess file in the /web/ directory. There is a commented out "RewriteBase /" directive there. Did you try uncommenting that?

If that doesn't work then you can also modify Symphony's .htaccess file to point to the correct location. However, when you do that then it will only work on your development server. When you deploy to some different server somewhere else then you need to modify the .htaccess again. If you want to try it, replace this:

  # no, so we redirect to our front web controller
  RewriteRule ^(.*)$ index.php [QSA,L]


With this:

  # no, so we redirect to our front web controller
  RewriteCond %{HTTP_HOST} ^([^\.]+)\.dev\.example\.org
  RewriteRule ^(.*)$ /home/checkout/%1/index.php [QSA,L]


Note: I haven't tested this, but I *think* it should do the trick. Let me know if you still have problems with it.

#15 Kevin

Hi Sander,

Thanks for the quick reply!

Uncommenting the "RewriteBase /" seems to have done the trick!

What exactly did this do? Was I correct in assuming it was in an endless rewrite loop?

I really appreciate your help!

-Kevin

#16 Kevin

One other thing... I need an alias /sf available to all projects - ie project1.example.com/sf and project2.example.com/sf should both point to: /usr/share/php/data/symfony/web/sf

I placed "Alias /sf /usr/share/php/data/symfony/web/sf" in the VHost file but it doesn't seem to work - any ideas?

This is almost coming together.

Kevin

#17 Sander Marechal (http://www.jejik.com)

What exactly did this do? Was I correct in assuming it was in an endless rewrite loop


Yes. Remember that when you rewrite an URL to a different directory with .htaccess, all the rewrite rules are applied *again*. So when the request got rewritten to index.php then the rewrite deployment script would kick in again, and then the .htaccess again, etcetera, etcetera.

One other thing... I need an alias /sf available to all projects. I placed "Alias /sf /usr/share/php/data/symfony/web/sf" in the VHost file but it doesn't seem to work - any ideas?


I am guessing that the alias happens before the rewrite rules are applied. So, /usr/share/php/data/symfony/web/sf gets rewritten to /home/checkout/your-checkout/sf/ again. There are multiple ways to solve this. You could make /sf a symlink on your filesystem to the correct directory. Alternatively, you could add a RewriteRule that catches the /sf before any of the other rules so. Try adding something like this after the "RewriteEngine" on in the virtual host configuration (leave the Alias in place):

RewriteRule ^/sf$ - [L,QSA]


That should catch any request to /sf do nothing with it and stop processing the other rules.

#18 Kevin

Hi Sander,

I am not able to get this to work without a symlink...

I have the Alias right before the "RewriteEngine On" and "RewriteRule ^/sf$ - [L,QSA]" directly after.

I did find another way of doing it with VirtualDocumentRoot. Does it in one line "VirtualDocumentRoot /home/checkout/%1/web/" and the Alias works.

Any reason I shouldn't be using this?

Thanks,
Kevin

#19 Sander Marechal (http://www.jejik.com)

None that I can think off. I dare say that's an even nicer solution than the RewriteRules in the apache configuration. Go for it!

#20 Kevin

Only problem I really see if I don't have the error checking from your solution and the default 'trunk'. When I go to dev.example.org it gives me an error - I had to make a second VirtualHost that handles it.

#21 Sander Marechal (http://www.jejik.com)

You can still use the RewriteRule to catch non-existing subdomains if you want. Something like this:

        # Redirect non-existing subdomains to the error page
        RewriteEngine on
        RewriteCond %{HTTP_HOST} ^([^\.]+)\.dev\.example\.org
        RewriteCond /home/checkout/%1 !-d
        RewriteRule ^(.*) /home/checkout/error.html [L]


That should only catch non-existing subdomains, leaving the existing subdomains to be handled by VirtualDocumentRoot.

#22 Sean Gravener (http://sean.gravener.net)

Beautiful! Worked perfectly the first go on RHEL 5.2

#23 Anonymous Coward

Hello Sander,

thanks for the article! I'm stuck at the following point "Then create a new virtual host configuration for Apache." Where do all those changes need to be applied? Which Files?

Kind regards,
one who is ashamed of not knowing APACHE and his new Ubuntu 10.10. that well....

#24 Sander Marechal (http://www.jejik.com)

@Anonymous: Under Ubuntu/Debian, virtual hosts go into separate files placed under /etc/apache2/sites-available. Then you activate your new virtual host using the a2ensite command or by symlinking your file from /etc/apache2/sites-enabled. Don't forget to reload Apache (using apache2ctl graceful)!
Post a new comment

Registration is not required to post comments, but cookies must be enabled. One of the advantages of registration is that you can edit your comments later on (editing not yet implemented). You can register or login here.




Your e-mail address will not be published, but your website URL will. All links that you post will tagged rel="nofollow" to throw off spammers. You are allowed to use the following XHTML tags in your comment: <em> <strong> <u> <b> <i> <strike> <blockquote> <big> <small> <ul> <ol> <li> <a href=""> <pre> <code> <tt> <br>. Please allow up to 60 second processing time after you post a comment. Our spam filters may take some time.