README for filechkd : File Check Daemon --------------------------------------- Contents -------- Overview Getting the Software Setup/Configuring filechkd Running filechkd Filespec Configuration TODO Overview -------- The file check daemon monitors files according to rules defined in configuration files. When a file is considered stable (due to its age, presence of a flag file, etc.) then it gets copied to a new location. Rotating backups of the destination file can be made and owner, group and permissions can be specified for the destination. Some examples of where this utility has been found to be useful are: Moving files out of an incoming FTP directory in a timely manner. Moving files uploaded to a web server into directories with different user/group. This lets the administrator run the web server as a non-root user and accept uploads using web server based authentication and then move the files to a more secure area after the transfer. The details of how to determine whether a file is stable and what to do with it once it is are defined in a "Filespec" configuration file. There is a separate filespec for each file that will be monitored which means that each file can have unique behavior associated with it. Getting the Software -------------------- The filechkd home page is the first place you should look for updates and news: http://filechkd.sourceforge.net The latest filechkd software is hosted by sourceforge and its sources can be retrieved from: http://download.sourceforge.net/filechkd and also from ftp://filechkd.sourceforge.net/pub/filechkd Please refer to the file named INSTALL for directions on building and installing filechkd. Setup/Configuring filechkd -------------------------- Once filechkd has been installed it is easily configured either via command line switches or a configuration file. At startup the program will read all of the files in the configuration directory that end in ".fs" (the filespecs). The details of formatting the filespecs are in the "Filespec Configuration" section. If a configuration file is used it should be placed in the same directory as the filespecs - /usr/local/etc/filespecd and should be named filespecd.conf. Values in the file are specified as attribute/value pairs - space delimited. The following options can be specified via the command line, the configuration file attribute and values are shown at the bottom of each option description: -c <config_dir> This specifies an alternate configuration directory from which to read both the filespec and daemon configuration files, this defaults to /usr/local/etc/filespecd -F Run in the foreground, filechkd will run as a daemon by default - this will not work if you are starting the program from the inittab. To start the program under a watchdog such as init it must be run in the foreground. config file: FOREGROUND true -l <log_file> Write log messages to the specified file. If this option is not specified then no logging will take place. You should also specify a minimum severity for log messages (see the next option). config file: LOGFILE log_file -L debug | info | notice | warning | error | serious | criticaldnl Set the minimum severity required for a message to be written to the log file. Log messages equal to or greater in severity will be written, the default minimum severity is "warning". config file: LOGSEVERITY debug | info | notice | warning | error | serious | critical -p <pid_file> Write the process ID of filechkd to this file- the default is /var/run/filechkd.pid (if the directory exists). config file: PIDFILE pid_file Command line options always take precedence over options specified in the configuration file. Running filechkd ---------------- Filechkd can be invoked via init.d to keep it running (although filechkd NEVER crashes :), or can be invoked from the system startup scripts. Filechkd can also be invoked via the command line or cron if you only need to have it run occasionally. An entry in /etc/inittab might look like this: fc:3:respawn:/usr/local/bin/filechkd -F -l /var/log/filechkd.log -L info You could put it in a system startup script, I recommend copying one of the shorter ones in /etc/init.d (Solaris) or /etc/rc.d/init.d (Linux) such as the one for cron. If you are starting filechkd from the command line, cron or an init script then the only difference from starting via inittab would be that you do not specify the -F option to run in the foreground (let it run as a daemon). Filespec Configuration ---------------------- Each file that is monitored must have a filespec configuration which is a text file consisting of attribute/value pairs (1 per line). These files are named with an ".fs" suffix e.g. something.fs and are placed in the configuration directory specified on the command line. The default configuration directory is /usr/local/etc/filechkd, change this via the -c command line option. Blank lines and lines beginning with '#' are ignored as comments. Only the first four characters of the attribute name is considered (case is ignored). The following attributes can be assigned for a file: NAME name If no name is specified then the name of the filespec file is used. This name is written to log records, useful since all logging is done to a single file. SOURCEFILE filename The name of the file to monitor - this should be the absolute path to the file since filechkd changes its running directory to '/'. Wild-cards/globbing is currently NOT supported. This attribute is mandatory. DESTFILE filename The name of the file to copy the source file to once it is determined stable. Wild-cards/globbing is currently NOT supported. This attribute is mandatory. LOCKFILE filename If this file exists then the source file is always considered unstable. Once the lock file no longer exists the source file must still meet other stability tests before being copied. STABLEFLAG filename If this file exists then the source file is considered stable, it does not need to meet any other stability tests. This can be useful when another process knows that it is stable and wants it to be picked up next time tests are done. USER username Change the owner of the destination file to the specified user name. If this is not given then the owner of the source file will also be the owner of the destination file and any backups that might be created. GROUP groupname Change the group of the destination file to the specified group name. If this is not given then the group of the source file will also be the group of the destination file and any backups that might be created. MODE accessmode Change the access mode on the destination file to the specified mode. This must be using numeric, octal notation such as 660 for -rw-rw---. If this is not given then the mode on the source file will be used for the destination and any backup files. NBACKUPS num Make "num" backups of the destination file, as new files are copied the backups are "rolled" using integer suffixes. For example a destination file /archive/foo might have backups named /archive/foo.0 and /archive/foo.1. If a new foo needs to be created and NBACKUPS is set to 2 then foo.1 would be unlinked, foo.0 would be renamed to foo.1, foo would be renamed to foo.0 and the new foo destination file would then be written. If "num" is 0 then no backups are made (the default). MINAGE nseconds A file is not considered stable until it has not been observed to change for nseconds seconds. Once the minimum age has been satisfied the file will be considered stable (unless a lock file exists) and it will be copied. The default minimum age is 120 seconds. RMSOURCE true If this attribute is valued with "true" then the source file will be unlinked after the destination file has been successfully copied. This defaults to "false". RMSTABLEFLAG true If this attribute is valued with "true" then the file used to flag stability will be removed once the destination file is successfully written. This defaults to "false". A simple filespec might contain the following lines: SOURCEFILE /public/ftp/incoming/myproj.tar DESTFILE /public/ftp/myproj/myproj.tar USER theman GROUP wheel MODE 644 RMSOURCE true TODO ---- The following items are on the radar, if any of them are particularly important to you then drop me a note and I will see what I can do to reprioritize them. 1 - check for memory leaks 2 - hpux binaries 3 - INSTALL configure options 4 - explain the alogorithm used to execute the copy 5 - upload to metalab and add loc to README 6 - register with freshmeat 7 - regex/glob support in source filespec 8 - configure check for bool type (Sun CC 4.2) 9 - SIGHUP support for reset 10 - accept parameters for wrapping log 11 - syslog option for logging 12 - lockf()/fcntl() locking on source file 13 - track stats on stab/lock files to detect stale locks 14 - accept uid/gid in place of names 15 - translate symbolic constants for MODE 16 - mail support 17 - parse mail into multiple mail addresses 18 - check using fuser or proc filesystem for open writers 19 - fork to do copy in case it takes longer than expected ---------- end README $Id: README.m4,v 1.2 2000/03/12 14:01:32 gwiley Exp $