CVSviaFTP: Automagic Web Site Mirroring via FTP

CVSviaFTP is a life-saver for Web site designers and administrators who need to update their site remotely using FTP protocol only.

Introduction

CVS via FTP is used to maintain multi-file multi-level WWW directories if one has only FTP access to the tree (happens in shared hosting).

Usually maintainer has a test site on his local computer and a production site on the remote computer accessible only via FTP. Maintainer would have to manually FTP over all the changes from the test site to the production site, which is, to put it mildly, painful.

So we decided to modify revision control system named CVS to make it update a mirror of the directory tree via FTP protocol. The strong points of this design are:

• Communication is minimal: only updated files are sent over FTP and nothing else.
• New directories are created automatically at the remote site (by mkdir). Unneeded directories are automatically removed (by rmdir).
• In addition to Update you can do a complete Checkout of the local directory tree to the remote site.

What is CVS

First, you need to know what the CVS is. If you do not know yet, visit http://www.cvshome.org.

CVS is a revision control system that is built on top of RCS. The good thing about it is that it works, it is free, and we like it. RCS is a standard low-level Revision Control System most widely used on UNIX and everywhere else.

How to Install CVSviaFTP

First, download CVSviaFTP here, ungzip and untar it.

Then follow the instructions in the INSTALL file that is a part of distribution. CVS is installed just like any other GNU software package.

How CVSviaFTP Works

We modified CVS so that it can do cvs checkout and cvs update at the remote site using FTP protocol.

To use the CVSviaFTP feature, download and install CVSviaFTP (described above), and thentype in your checked out directory:

      cvs update   -F ftp_commands_file		OR
      cvs checkout -F ftp_commands_file
    

After updating your local directory in the file ftp_commands_file you will get a sequence of FTP commands that upgrades the FTP-accessible mirror of this local directory.

That is, CVSviaFTP will not start an FTP session for you, it will just write out a file that you can later feed to your favorite FTP client. It was done on purpose:
(a) Different people like different FTP clients;
(b) You need to prepend login commands and append quit or other finishing commands to the sequence produced by CVSviaFTP.
(c) For real sensitive sites you may want to manually check FTP command sequence just to be sure everything is OK.

How to Integrate It

This is an example of a C shell script that does the whole "local commit / remote update" thing:

      if ( "$1" != "" ) then
      cd ~/html
      cvs commit -m "$1"
      endif
      cd ~/html-dist/html
      cvs update -F ftp.out -P -d
      cat ~/etc/ftp-start $ftpfile ~/etc/ftp-finish >ftp-comms
      ncftp -u gunky.junky.com <ftp-comms
    

In this script the test tree is kept at ~/html, and the tree that is to be mirrored at the remote site is kept at ~/html-dist/html.

The script commits changes in the test tree to the repository, checks out changes to ~/html-dist/html and writes FTP commands to ftp.out. Then header and finisher lines are added to the ftp.out and FTP is called that executes the update at the remote site.

Notes On Use

Read them if you want to know more:

• CVS does not like symbolic links. CVSviaFTP does not like them either. It expects that cd xxx/..always gets to the original directory.
• You cannot do "cvs commit" via FTP, that is, repository should be accessible to your machine in a "regular" CVS way:
  (a) Repository is mounted on local file system;
  (b) Repository is on NFS-mounted file system;
  (c) Repository is accessible via CVS client-server relationship.
• Our approach only works if initially the checked-out local directory and the remote mirror directory trees are exactly the same, because CVSviaFTP does not fstat remote files.
  
  Please note that both directories can initially be empty. The reason it does not do it is that it slows things down considerably (FTP protocol is slow), it is not portable (there is no fstat command in FTP and format of ls output is non-standard).


• There is a bug in CVS: when you do "cvs update" with no arguments, CVS does "cd ." and after processing matches it with "cd .." which gets you to the level above from where you started. Since current directory is not kept upon exit, you do not notice it, but my ftp_cd() function does. However it keeps quiet about it most time. Still fixing this bug is a good idea.
• Before doing actual FTPing, it would not hurt to check file with FTP commands.

Patch

Patch from CVS-1.8.1 to CVS-via-FTP is included in the distribution.
It is also available here.

Legal

This software is distributed according to the GNU Public License, that is for free. You can use it for free, even to develop commercial products, but if you modify it, the result is subject to the GNU Public License.

There is no any explicit or implied warranty on this product, or any promise thereof, that the product will work. By downloading the tarred-n-zipped distribution file you assume all the liability that may arise from the use of this product.