@(#) $Header: INSTALL,v 1.26 98/05/12 17:24:45 leres Exp $ (LBL)

We are a InterNetNews site and so these instructions are biased towards
this news system. This package is known to compile and nominally run
under SunOS 4.1.4 and Solaris 2.5; it configures and compiles under
FreeBSD 2, BSD/OS 2 and OSF 3. If you port it to a new system, please
forward any changes to dexpire@ee.lbl.gov.

You will need an ANSI C compiler to build dexpire. If your compiler is
not ANSI compliant, use the GNU C compiler, available via anonymous
ftp:

	ftp://prep.ai.mit.edu/pub/gnu/gcc-*.tar.gz

Dexpire can use either statvfs() or statfs() to determine disk usage.
If your system doesn't have one of these (this is unlikely), you'll
have to add some code to disk.c; the supplied program diskfree is
useful for testing changed to disk.c. Since the routine disk usage is
only invoked once, it would be acceptable to fork() and parse the
output of /bin/df.

Dexpire uses autoconf. So the first thing to do after unpacking the
files is to run configure:

	./configure

The dexpire specific configure options are as follows:

	--exec-prefix=EPREFIX   install architecture-dependent files in EPREFIX
	--with-owner=USER       use netnews owner USER
	--with-group=GROUP      use netnews group GROUP
	--with-inndir=DIR	use INN source directory DIR
	--without-gcc           don't use gcc

The default EPREFIX is /usr/news. The script checks for netnews
owner news or usenet and usenet group news or usenet. You can view
all supported configure options using:

	./configure --help

The configure scripts builds and runs the test program blocktest
to make sure that dexpire's assumptions about the filesystem block
size are correct. It tests to the st_blocks field of the stat
structure. Dexpire assumes that the units are in 512 byte blocks
(perhaps rounded up to the next even block because the filesystem
fragment size is 1024 bytes). If blocktest doesn't successfully
build and report:

	blocktest: Success: This machine appears to use 512 byte blocks

then your os is somewhat unusual and running dexpire might be
dangerous (e.g. it might free twice as much space as requested).

Various files need to be updated with INN specific information. If
your dexpire source tree has the same parent directory as your INN
source tree, configure will autodetect the location of subst(1)
and the INN config.data file used to build your news software. Then
you can update the necessary dexpire files using:

	make subst

If you are building dexpire somewhere else, you can use the
--with-inndir flag to configure to specify the location of the top
level INN source directory.

Note that the substupdate script refuses to write read/only files.
Depending on how you unpacked the dexpire package, most files will
be read/only. So before running "make subst" you might want to
enter the files into some source control system such as rcs or just
make the writable (e.g. chmod +w ...)

Next build the programs:

	make

Now configure a dexplist file. If you currently use the C News expire,
it's pretty easy to convert a explist file to the dexplist format;
basically, edit it down so that you only have the first 3 fields.
Otherwise, make a copy of the sample dexplist and start hacking.

Now test your new dexplist file. If you have more than one spool
filesystem, read the dexpire man page to see how to use the -f flag.
It's strongly recommended that you use -n until you're sure everything
works ok. Try something like:

	dexpire -vn -f 100M

Put the output on a file so you can examine it at your leisure. Make
sure the reported disk statistics match what /bin/df says. (Be careful
since some df's report 512 byte blocks instead of 1 Kbyte blocks.) The
first pass should always find at least one article that could be
deleted. The end of the run should report a reasonable number of
articles and the correct number of bytes to be deleted.

Included are sample scripts for InterNetNews. (There are also some C
News scripts in the cnews directory.) They are good starting places as
wrapper scripts for running dexpire and histtrim. In particular,
dohisttrim knows how to lock a the news system and how to build the
dbm/dbz history file.

You will also want to edit dexpire.conf to configure dexpire
for your system. You need to list the spool filesystems (relative
to the spool). For example:

	DEXPIRE_FREE_LIST=". alt"

says that you have a top level spool filesystem and an alt
sub-filesystem. If your spool directory is /var/spool/news, then you
are telling dexpire you have these two filesystems:

	/var/spool/news
	/var/spool/news/alt

For each filesystem listed in DEXPIRE_FREE_LIST, you need to specify
the amount of free space. Using the previous example:

	DEXPIRE_FREE="250M"
	DEXPIRE_FREE_alt="450M"

says to free 250 Mbytes in /var/spool/news and 450 Mbytes in
/var/spool/news/alt. Finally, you might want to tune the
minimum number of days history entries are kept:

	HISTTRIM_MINDAYS="16"

Next install the binaries, config file, man pages and scripts:

	make install
	make install-man
	make install-conf
	make install-scripts

Note that the install-conf and install-scripts targets will not
clobber already installed configuration files or scripts.

The next step is to add hooks to run the various components of the
dexpire package at the appropriate times. Dexpire and runexp should
be run once (or twice) a day. Histtrim needs to run once a week.

Here are sample usenet crontab entries:

	# Emergency dexpire check
	0 * * * * /usr/news/bin/news.hourly

	# Flag weekly histtrim (on Saturday)
	42 6 * * 6 date > /var/news/log/.dohisttrim

	# Morning logfile rotation, dexpire and daily news scripts
	52 6 * * * (/usr/news/bin/newlog ;
	    /usr/news/bin/news.dexpire ;
	    /usr/news/bin/news.daily noexpire norenumber) 2>&1 |
	    egrep -v "^Can't read .* No such file or directory\$" |
	    Mail -s "Usenet daily report" newsreport

	# Nightly dexpire script
	52 18 * * * /usr/news/bin/news.nightly

The sample dodexpire script (run by the news.dexpire and news.nightly
scripts) uses a script called makedexplist to create a dexplist
dynamically. The idea is to default most groups to a very low priority
but raise the priority of newsgroups recently read.

This means you need to install the dexplist file you created earlier
with the name name dexplist.base:

	install -m 444 dexplist /usr/news/etc/dexplist.base

Note that the first few times you use dexpire, it isn't unusual for
there to be hundreds of consecutive unproductive passes. But since
dexpire caches article timestamps, these extra passes use very little
cpu time (and perform no I/O); and after the news system catches up
with the new policy (usually in a few days), things will settle down to
10 to 20 passes.

If you are running FreeBSD 2.2.*, you may need to make some login
class changes. The main symptom is that the weekly history file
rebuild never completes.  This is due to the new login class code.
The most straight forward way to fix this is to put the news user
in the news login class (see vipw(8) and passwd(5)) and change all
the of the news cron jobs to specify the login class (see crontab(5)).
(It's not clear cron doesn't get the login class from the password
file!) Finally, you will more than likely need to edit /etc/login.conf
and upgrade the silly defaults for news. I use something like:

    default:\
	    :cputime=infinity:\
	    :datasize-cur=16M:\
	    :stacksize-cur=8M:\
	    :memorylocked-cur=10M:\
	    :memoryuse-cur=30M:\
	    :filesize=infinity:\
	    :coredumpsize=infinity:\
	    :maxproc-cur=64:\
	    :openfiles-cur=64:\
	    :priority=0:\
	    :umask=022:\
	    :tc=auth-defaults:
    #
    # Settings used by news subsystem
    #
    news:\  
	    :cputime=infinity:\
	    :datasize=512M:\
	    :datasize-cur=512M:\
	    :datasize-max=512M:\
	    :stacksize=32M:\
	    :stacksize-cur=32M:\
	    :stacksize-max=32M:\
	    :coredumpsize=infinity:\
	    :maxmemorysize=512M:\
	    :maxmemorysize-cur=512M:\
	    :maxmemorysize-max=512M:\
	    :memorylocked=32M:\
	    :maxproc=128:\
	    :maxproc-cur=128:\
	    :maxproc-max=128:\
	    :openfiles=256:\
	    :openfiles-cur=256:\
	    :openfiles-max=256:\
	    :umask=002:\
	    :tc=default:

Make sure that that "datasize-cur" is spelled correctly (in at
least one version it is spelled "datasize-curr" and this error is
silently ignored). Also, don't make the data or stack sizes too
small. Finally, you won't want a maximum file size and also probably
don't want to limit the size of core dumps (unless you plan on
never debugging problems!) See login.conf(5) and passwd(5) for more
details.

I know there's a lot to configure here but once you get things setup
and do a little fine tuning, you can almost forget about netnews
expiring.

One more thing you might want to do if you are running INN 1.5.1 or
lower is install the unofficial "lowmark" patch. It's included in the
dexpire distribution in:

	patches/ctlinnd

Basically it adds support for a new ctlinnd(8) command that passes a
file to innd that is used to update the low article numbers in the
active file. Dexpire supports this and the scripts included in this
distribution attempt to use the feature; if your innd doesn't support
it, the scripts automatically fall back on the old method (which is
renumber groups one at a time using the old, slow method). The first
version of INN to official support this is 1.6.

Also, there is a patch to teach expireover how to read the lowmark file:

	patches/expireover

This adds support for a -l flag to expireover(8). Updating the overview
files from the lowmark file is really fast. As with the other patch,
the supplied wrapper scripts automatically fall back on the old method
(expireover -s) if you don't install this.

For more information about INN, check out the FAQ:

	ftp://rtfm.mit.edu/pub/usenet-by-hierarchy/news/software/nntp
	ftp://ftp.xlink.net/pub/news/docs/

(Just be sure to ignore the nasty things it says about dexpire!)

Our news node is a 233 MHz Pentium II running FreeBSD. Our spool
filesystem consists of two Seagate Barracuda disks.

FILES
-----
CHANGES		- description of differences between releases
FILES		- list of files exported as part of the distribution
INSTALL		- this file
Makefile.in	- compilation rules (autoconf prototype)
README		- this file
VERSION		- version of this release
aclocal.m4	- autoconf macros
blocktest.c	- block size test program
cnews		- C News scripts (obsolete)
config.guess	- autoconf support
config.sub	- autoconf support
configure	- configure script (run this first)
configure.in	- configure script (autoconf prototype)
dexpire.8	- manual entry
dexpire.c	- main program
dexpire.conf	- dexpire configuration script
dexpire.conf.5	- manual entry
dexpire.h	- configuration
dexplist	- sample control file
disk.c		- disk usage routines
disk.h		- forward declarations
diskfree.5	- manual entry
diskfree.c	- report free disk space and inodes
dodexpire	- dexpire wrapper script
dodexpire.8	- manual entry
doexpireover	- expireover wrapper script
doexpireover.8	- manual entry
dohisttrim	- histtrim wrapper script
dohisttrim.8	- manual entry
err.c		- 4.4 BSD err() and friends
err.h		- forward declarations
file.c		- active and dexplist parsers
file.h		- forward declarations
gdbrc		- gdb debugging macros
hash.c		- hash table utilities
hash.h		- forward declarations
histtrim.8	- manual entry
histtrim.c	- history file trimmer
install-sh	- BSD style install script
lbl/gnuc.h	- gcc macros and defines
lbl/os-*.h	- os dependent defines and prototypes
makedexplist	- script to dynamically create a dexplist file
makedexplist.8	- manual entry
mkdep		- BSD style dependency script
myerr.h		- forward declarations
news.dexpire	- primary dexpire wrapper script
news.hourly	- hourly script to do emergency dexpire check
news.nightly	- night time dexpire script
patches/ctlinnd	- INN 1.5.1 ctlinnd lowmark patch
patches/expireover - INN 1.5.1 expierover lowmark patch
patchlevel.h	- patchlevel (required by some weenies)
rotate_files	- script to rotate log files
runexp.8	- manual entry
runexp.c	- process explicit expires
setproctitle.c	- set process title routine
setproctitle.h	- forward declarations
strerror.c	- 4.4 BSD strerror()
substupdate.in	- subst wrapper script (autoconf prototype)
topgroups	- script to watch which newsgroups are read
util.c		- random utility routines
util.h		- forward declarations
