Introduction to
BYTE's Unix Benchmarks

Ben Smith

  Conceptually, BYTE's Unix Benchmarks are the same as
BYTE's benchmarks for MS-DOS: we combine low-level
evaluation of the different operations of the computer, with
high-level applications programs that show the combined
performance of the entire system as it would appear to the
user; performance in the real world.
  But Unix is considerably different from MS-DOS. In the
first place, it is a !ITAL!multi!ENDITAL!-tasking,
!ITAL!multi!ENDITAL!-user operating system. But is also a
portable operating system; that is to say, it can run on
many different kinds of computers.  MS-DOS is a single-
tasking, single-user operating system, and it is intended to
run on essentially one kind of computer, a IBM-PC or PC
clone, hardware that is functionally identical to the IBM-
PC.
  In detail, the Unix benchmarks are considerably different
from the MS-DOS benchmarks; they are measuring different
things.  You will find some equivalents at the very low
level, hardware performance. But even there you will find
differences; for instance, you will get a different
Dhrystone rating from the MS-DOS Dhrystone program than you
will from the Unix Dhrystone program running on the very
same machine. This discrepency points up another difference
between benchmarking MS-DOS machines and Unix machines: the
MS-DOS operating system is the same from machine to machine;
there are different Unix operating systems for different
machines. The benchmarks are measuring the performance of
the implementation of Unix as well as the hardware. And, for
that matter, we use the compiler that is resident with the
version of UNIX.  (MS-DOS does not have a standard compiler
associated with it.) The Unix benchmarks reflect the
performance resulting from the basic optimization of the
compiler as well.
  With so many variables, what is constant? Well, we have
established a baseline, SCO Xenix 386 version 2.3.1. running
on the Everex 386/33 with 4Mbytes of RAM and an 80387
mathcoprocessor. We selected this version of Unix because it
is running on more computers in the world than any other
version. It is specifically designed for PC/AT computers
with full 32 bit memory access.  We selected this machine
because it is one of today's highest performance 386
computers that is properly configured to run a full 32 bit
operating system. (Some 386 computers cannot access memory
through single 32 bit operations; small matter if you are
just running MS-DOS, a 16 bit operating system.)  We hope
that by using this combination for a baseline, we have
numbers that are relevant for at least a few years.

!SUBHED!The Bench Program Levels!ENDSUBHED!
  The BYTE Unix benchmarks consist of seven groups of
programs: arithmetic, system calls, memory operations,
dhrystone, database operations, system loading, and
misclaneous.  These can be roughly divided into the low-
level tests (arithmetic, system calls, memory, and
dhrystone) and high-level tests (database operations, system
loading, and the C-compiler test that is part of the
misclaneous set). To qualify as a high-level application,
the test must involve operations of a real-world application
program including heavy use of the CPU and disk I/O.
  The arithmetic tests are all the same source code with
different data types substituted for the operations:
register, short, int, long, float, double, and an empty loop
for calculating the overhead required by the program. The
system calls tests are: system call overhead, pipe
throughput, pipe context switching, spawning of child
processes, execl (immediate replacement of the current
process by a new process), and file read, write, and copy.
The memory test evaluates the time it takes to access memory
and assign its value to a variable. The dhrystone test is
knowm more formally as Dhrystone 2. It has no floating point
operations in it.
  In the high level tests, the database operations consist
of random read, write, and add operations on a database
file. The operations are handled by a server process; the
requests come from client processes. The test is run with 1,
2, 4, and 8 client processes. The system loading is a shell
script that is run by 1, 2, 4, and 8 concurrent processes.
The script consists of an alphabetic sort one file to
another; taking the octal dump of the result and doing a
numeric sort to a third file; running grep on the result of
the alphabetic sort file; teeing the result to a file and to
wc (word count); writing the final result to a file; and
removing all of the resulting files. This script was used in
the orginal BYTE Unix benchmarks (1983), but the source file
is several magnitudes larger than the original. The C
compile and link is nothing more than that.
  The remaining tests in the misclaneous group are Tower of
Hanoi (a test of recursive operations), and a test of the
Unix arbitrary precision calculator calculating the square
root of two to 99 decimal places.
  No doubt, we will be adding tests to this sweet as we see
the need to test and evaluate from different perspectives.
The idea of having different perspectives, is very
important. The use of a single number index, a sort of
reduction of all of the benchmark tests to a single value,
is not something that we are doing.  Having a single index
rating for a machine may make good cocktail conversation,
but it is incredibly simplistic. It is like reducing a
complex scultural shape to a single point;  you no longer
can tell what you are looking at.

!SUBHED!How They Work!ENDSUBHED!
  The greatest part of the benchmark programs are written in
C and compiled on the test machine prior to running the
tests.  The benchmarking is managed by a 500 line Bourne
shell-script.  The statistics and reports are produced by
!MONO!awk!ENDMONO!  scripts. Most of the times are captured
using the !MONO!time!ENDMONO!  command, except for the very
hardware specific tests of memory and disk transfer rates
which have internal devices for removing system overhead.
The benchmarks have been successfully run on many different
Unix systems with varyious versions of Unix.  Differences in
methods of getting time information are taken care of by
!MONO!#ifdef!ENDMONO! statements within the code. The system
calls have been kept to those that are common to both System
V based Unix and BSD based Unix, whereever possible.




