

Marc Mertens

LISP DEBUG

Introduction

LISP DEBUG is a source level debugger, stepper and profiler for LISP
programs. The current implementation works with GCL 2.2.2,CMUCL Lisp,ACL5
and CLISP and is tested on LINUX 2.0. Although LISP DEBUG makes use
of the TCL/TK system for the GUI, no TCL/TK extensions are needed
for LISP DEBUG. LISP DEBUG has the following features :

* Source level debugger (highlight in a sourcewindow the code being
  executed).

* Lispform oriented not line oriented (function call is highlighted).

* Multisource debugging is possible (the system will switch to another
  source if needed).

* Controls :

  * Step (execute the current form and stop at the next form)

  * Step over (execute the current form and stop at the next form in
    the current list)

  * Next (execute the current form and stop at the highlighted form)

  * Continue (execute the current form and don't stop anymore)

* Breakpoints and conditional breakpoints are placed on lisp forms
  , not on lines.

* Placement of watchpoints on variables and lispforms is possible.

* Evaluation of lispexpressions during debugging.

* Timetraveling (the debugger keeps track of the lisp forms executed
  + their environment , this makes it possible to step backwards and
  forwards through the code)

* Profiling (count how many times a lisp form is executed)

* Extensible (you can extend the lisp control structures the debugger
  recognizes by use of syntax definitions)

LISP DEBUG works by instrumentation. When a lisp source is loaded in
the debugger the following is happening :

1. A temp file is created which contains the lisp source with debug
  code added to it (essentially code to call the debugger , to save
  position info and to save the environment of the function call).

2. The temp file is loaded in the LISP system (you can ask to compile
  it before the load)

3. The source code is loaded in a sourcewindow to be displayed during
  the debug process.

4. You can now use the sourcewindow to set breakpoints ,watchpoints
  ... . When you run now code in the LISP system the execution will
  stop at the breakpoints and the lispform which should be executed
  next is highlighted in the sourcewindow. Form then one you can fully
  control the execution of the code.

5. When you close a source the source will be removed from the sourcewindow
  and the original code (without instrumentation code) is loaded in
  the LISP system.

The original source code is never touched.

1.1 Warnings

* Adding instrumentation code has some risks associated with it.

  * Bugs in the debugger can cause malfunction of your code (I have
    done my best to avoid bugs but nobody is perfect :().

  * Bugs in the debugger can make that not every part of your program
    is available for debugging.

* Macros are not easy to debug , given a macrocall the system will
  try to expand it and then try to add debugging code to the expanded
  code. System macro's which represents a special controlstructure
  (like cond,if ...) are handled in a special way. If you define your
  own controlstructures you can extend the debugger to handle them
  in the correct way.

* Breakpoints can be placed on every lispform you can evaluate (with
  the exception of atomic expressions and some macros (example (defgeneric
  ...), (function ...)).

* If you can't place a breakpoint on a form it is usually possible
  to place a breakpoint on certain subforms.

* If you make changes to the source , you have to reload the source
  in the debugger (you can use copy/paste to load sources in the debugger
  if you use emacs or xemacs).

* If there are bugs in your source code which causes a halt during
  loading , the debugger will halt when he loads the modified source
  code and you will enter the native debugger. If you exit the native
  debugger the rest of your code will be loaded. Always make sure
  that you can load code before you try to debug the code.

* The debugger sits in package ``DEBUGGER'' , don't use this package.

* In the ``USER'' package the function name ``debug'' (``deb'' in ACL)
  is exported , so you may not use this name for a function.

* Allegro uses the name ``debug'' so I used ``deb'' in the case of
  Allegro.

Overview of the debugger.

2.1 Quick procedure to get you running.

1. Make sure that X Windows is running,

2. Start LISP (standalone or as a process in emacs (xemacs)

3. Type (debug) at the lisp prompt ((deb) for Allegro). A separate debug
  window should open (see Figure: Starting Debugger) . You control
  debugging using this window.([fig] Start Debugger\includegraphics{pic1.ps} )  

4. Load a lisp source in the debugger using <File><Open> .([fig] File Selection\includegraphics{pic2.ps} ) 

5. After loading the source, you can set a breakpoint by first selecting
  a lisp form and then clicking on <Breakpoint>. If a breakpoint is
  possible the breakpointed code will have another foreground color
  (default red). 

  Using the middle mousebutton you select a whole function , double
  clicking select a list.([fig] Source Window\includegraphics{pic3.ps} ) 

6. Return then to the lisp prompt in the LISP system and type an expression.
  If the execution of the expression causes the breakpointed code
  to run , the currently executed code will be highlighted in the
  source window and the debugger waits for your commands. You can
  now step through the code , set watchpoints , evaluate expressions
  ... .([fig] Executing Code\includegraphics{pic4.ps} ) 

7. For more detailed information of using the debugger go to the user
  manual.

8. To stop debugging , use <File><Exit> , the original source (without)
  debugging code will be loaded back in the LISP system. 

2.2 Using the debugger

2.2.1 Starting the debugger.

To start the debugger , first make sure that X Windows is running ,
then start lisp (in a shell , in an emacs (xemacs) process or whatever
editor you want). At the lisp prompt type (debug) ((deb) if you are
using Allegro). A debug window will then start allowing you to control
the debugging process.

2.2.2 Debugging programs.

The debug window consists of the following parts :

* A menu line ('File','Source','Edit','Options','Tools') to Open/Close
  sources , to switch between loaded sources , to paste code to configure
  the debugger and to do profiling.

* A button panel containing the most used functions of the debugger
  (Stepping , Breakpointing , Watching , Evaluating and timetraveling).

* A source pane which shows the source code , during debugging the
  debugpoints have a red foreground color , the code to be executed
  is highlighted in blue and if you do profiling , code called the
  specified amount of times is also highlighted in blue.

* A watch pane , contains the watched variables and expressions in
  addition to their values. Displays also the value of an evaluation.

* Command pane , used to type in an expression which can be used as
  a watchexpression , which can be evaluated or which can be used
  in a conditional breakpoint.

To debug a function you follow in general the following steps :

1. Load the sourcecode of the function in the debugger , this can be
  done in two ways

  (a) The code is defined in a sourcefile on the harddisc , you use then
    <File><Open> to load the whole sourcefile in the debugger. This
    is the preferred way of working because you can then debug all
    related functions in the sourcefile.

  (b) The code is defined in an editor (like emacs , xemacs ...) , select
    then the code , use the <Copy> function of the editor to put the
    code in the copybuffer. Use then <Edit><Paste> on the debugwindow
    to load the code in the sourcewindow. (You should see this happening
    otherwise Copy/Paste is not working with your editor :().

2. Set a breakpoint , this can be done in two ways :

  (a) Select a form in the sourcepane and press then 'Breakpoint' to
    set a breakpoint , the color of the text in the form changes then
    to red , to indicate a breakpoint set on this form. The color
    will only change if you can really set a breakpoint . Example
    : in (defgeneric fac (n) (:method ((n integer)) (if (zerop n)
    1 (* n (fac (1- n))))) (:method ((n string)) (concatenate n ``!'')))
    you can set a breakpoint on (:method ((n integer)) (if (zerop
    n) 1 (* n (fac (1- n))))), (if (zerop n) 1 (* n (fac (1- n))))
    , (zerop n) , (* n (fac (1- n))) , (fac (1- n)) , (1- n) , (:
    method ((n string)) (concatenate n ``!'')) , (concatenate n ``!'')
    but not on the (defgeneric ....) form. It should be clear that
    a breakpoint is not set on a line but on a form like a function
    call ((function arg1 .... argn) , ...) , a special function call
    ((cond (( ....) ...) ...) , (defun ...) , (let ...) , (do ....)
    ....) or a macro call. Breakpoints just like forms can be nested.
    The 'Breakpoint' act as toggle if you select a form which already
    has a breakpoint selected with it you remove the breakpoint. 

  (b) Type in an expression in the command pane , select a form in the
    source pane and press 'Break If' , this sets a conditional breakpoint
    , execution is halted only if the evaluation of the condition
    gives true as a result. An error during the evaluation is considered
    false (also a dialogbox explaining the error is displayed). The
    'Break If' button act like the 'Breakpoint' button as a toggle.

3. Set a watchpoint , this can be done in two ways :

  (a) Select a variable in a function and then press 'Watch' . In the
    watchpane you will see now the variable followed by an arrow to
    its value. If the variable is not defined the value will be 'Undefined'.

  (b) Select a form in the source pane, type a expression in the command
    pane and press 'Watch exp'. This will set a watchpoint on the
    expression in the command pane which is evaluated if the selected
    code is executed (otherwise 'Undefined' is used as value).

    A common mistake is to forgot to make a specific selection.

4. To unset a watchpoint. Select the whole line of the watchpoint in
  the resultpane and press then 'Unwatch'.

5. You go now to the LISP system to type in some lisp expressions. If
  a breakpoint is encountered then the execution of the function will
  be paused and the code to be executed will be highlighted in the
  source pane of the debug window (if needed the corrected source
  will be loaded or a repositioning will take place so that the executing
  code is always visible). You can react now in the following ways :

  (a) Press the 'Step' button and execute the highlighted form , pausing
    at the next form to be executed (this could be a subform of the
    highlighted form). 

  (b) Press the 'Step Over' button , execute the highlighted form , pausing
    at the next form within the parent form (so we will not step through
    code called during execution of the highlighted form , unless
    breakpoints are placed of course).

  (c) Select the code where the debugger should pause next and press
    then the 'Next' button. The debugger will then start executing
    the code until a breakpoint is encountered or the selected code
    is reached.

  (d) Press the 'Continue' button , the debugger will then start executing
    the code until a breakpoint is encountered or nothing has to be
    executed.

  (e) Press the 'Back' or 'Forward' button to do time traveling (look
    at the previous/next executed code and variable context at that
    point).

    If you have used Step or Step Over and all code is executed , then
    the system stays in a stepped state , if you execute another debugged
    function from the lisp prompted then the system pauses execution
    waiting on you to press 'Step' , 'Step Over' , 'Continue' or 'Eval'.

6. Select a source via <Source>.

7. You stop debugging of a source by selecting the source and then <File><Close>.
  The original source code is then loaded in the LISP system.

8. You stop the debugger via <File><Exit>. Before exiting , the debugger
  will load the original sources of all debugged code back in the
  LISP system.

Reference Manual

3.1 Menus

3.1.1 Overview

Following menu items exist in the debugger :

* <File><Open>

* <File><Close>

* <File><Exit>

* <Source>

* <Edit><Paste>

* <Edit><Find>

* <Options><Color Break>

* <Options><Color Breakpoint>

* <Options><Color Breakpoint If>

* <Options><Color Profiling>

* <Options><Font>

* <Options>[Compile debugged code]

* <Options><Save Options>

* <Options>[Save on exit]

* <Tools><Start profiling>

* <Tools><Stop profiling>

3.1.2 Detailed description of menu items

3.1.2.1 <File><Open>

Opens a window which allows you to select a LISP source. When pressing
Ok , the selected source is processed to add debugging code and to
save the modified code in a temp file. If the option <Options>[Compiled
debugged code] is true then the temp file will be compiled and the
compiled code will be loaded in the lisp system, in the other case
the temp file will be loaded in the lisp system.([fig] <File><Open>\includegraphics{pic2.ps} ) 

3.1.2.2 <File><Close>

The original code of the file in the sourcewindow is loaded in the
lisp system. The sourcewindow is then emptied and filled with first
loaded source (if it exist).

3.1.2.3 <File><Exit>

Exit the debugger. Before this happens the original version of all
debugged sources is loaded in the lisp system.

3.1.2.4 <Source>

Allows you to select which of the sources loaded in the debugger is
shown in the source window.

3.1.2.5 <Edit><Paste>

If in emacs (xemacs) (or maybe your editor of choice) a selected area
is copied to the copy buffer , then this selected area is used to
create a tempory source which is loaded in the debugger. This allows
you to debug functions which are in a editor but not yet in a source
file. (Every copy/paste operation supported by tk_textPaste should
work).

3.1.2.6 <Edit><Find>

Activate the search function , to search the source currently displayed
in the source window. Three types of search are possible :

* Case , case sensitive search.

* No Case , case insensitive search.

* Regular expression , allows you to use regular expressions in the
  search.([fig] Find\includegraphics{pic5.ps} ) 

3.1.2.7 <Options><Color Break>,<Options><Color Breakpoint>,<Options><Color
  Breakpoint If>,<Options><Color Profiling> 

Allows you to change the color of the current executed code , the breakpoints
, the conditional breakpoints and the profiled code.([fig] Color\includegraphics{pic6.ps} ) 

3.1.2.8 <Options><Font>

Use this menu to change the font used in the source,result and command
pane.([fig] Font\includegraphics{pic7.ps} ) 

3.1.2.9 <Options>[Compile debugged code]

When this switch is enabled , the source with debugcode added is first
compiled before it is loaded in the lisp system. This could be usefull
on lisp systems which do a precompile of a function before executing
it for the first time (because the extra debugging code , this precompile
could take a long time). 

3.1.2.10 <Options><Save Options>

Save the options in a <Options> in the file $HOME/.lispdebug.lisp.
This file is a standard text file and contains the following settings :

* (DEBUGGER::setting "COMPILE_CODE" 0)

* (DEBUGGER::setting "SAVE_ON_EXIT" 0)

* (DEBUGGER::setting "DEBUGPOINT_COLOR" "#ff0000")

* (DEBUGGER::setting "DEBUGPOINTIF_COLOR" "#808000")

* (DEBUGGER::setting "CURRENT_COLOR" "#00ffff")

* (DEBUGGER::setting "PROFILE_COLOR" "#ffff00")

* (DEBUGGER::setting "FONT" "-*-helvetica-medium-r-normal-*-120-*-*-*-*-*-*")

If you modify or create this file manual (for example because default
color or font is not supported on your system) be sure to not forget
DEBUGGER (in case of Allegro it must be in uppercase to).

3.1.2.11 <Options>[Save on Exit]

When enabled, the current options are save in $HOME/.lispdebug.lisp
when the debuggers stops.

3.1.2.12 <Tools><Start Profiling>

Enables the profiling part of the debugger. The profile counters (how
many time a piece of code is executed) is reset to zero and an extra
slider is made viable in the left part of the debugwindow. When code
is executed the system keep track of the number of times a lispform
is executed. By using the slider you highlight the code which is at
least as many times executed as the number shown by the slider.([fig] Debugger when profiling\includegraphics{pic8.ps} ) 

3.1.2.13 <Tools><Stop Profiling>

Disables the profiling part of the debugger.

3.2 Source Pane

3.2.1 Content.

The source pane contains the source of the code which is debugged.
The source which is displayed depends on the following conditions:

* If you open a new source , then this is the source displayed.

* You can change the source displayed via the <Source> menu.

* If you close a source , the first opened source is displayed.

* If the system encounters a breakpoint , the source containing the
  breakpoint is displayed , the window on the source is also repositioned
  so that the executing code is visable.

3.2.2 Selection

Selection is the base for setting breakpoints , setting conditional
breakpoints and setting watch expressions. Selection can be done in
the following ways.

* Dragging. Click with the mouse in the sourcepane (use the left button),
  drag the mouse to an other point and release the button.

* Click somewhere in a function definition with the middle mouse button.
  This selects the whole function.

* Double click with the left button , this selects a whole list.

3.2.3 Highlighting.

Pieces of the sources are highlighted as follows (we use the default
colors of the debugger).

* Red foreground , this is a breakpoint.

* Dark green foreground , this is a conditional breakpoint.

* Light blue background , this code will be executed next.

* Yellow background , this code fulfills the profile count (is at least
  as many times executed).

3.3 Result Pane

3.3.1 Content.

The result pane contains the value of watch variables , the results
of evaluating watch expressions , the result of a evaluation or the
result of previous executed code. The format used is as follows 'exp
--> value' . Here 'exp' could be a variable , a watched expression,
a evaluated expression or the result of previous executed code (this
is indicated by enclosing this code with hooks). 'value' could be
'Undefined' (the variable is not defined or the expression couldn't
be evaluated) , (values .....) the result was caused by a (values
...) statement or just the value(result) of the variable (expression).
You can remove watchpoints and watchexpressions by selecting the whole
of 'exp-->value' and then pressing <Unwatch>.

For GCL there is no support for values results , in case of the use
of (values ...) only the first value is displayed , you can try to
use 'multiple-value-list' to display every possible value.

3.4 Command Pane

3.4.1 Content.

The command pane is the only pane which can be edited. You use this
to type in expressions needed by the debugger. This will be the case
in the following situations:

* Conditional breakpoints (NIL or non NIL of the evaluated expression
  determines if the system stops or not).

* Watch Expression (the expression to watch).

* Eval (the expression to evaluate).

3.5 Buttons

3.5.1 Overview.

The following buttons are available:

* <Step>

* <Step Over>

* <Next>

* <Continue>

* <Breakpoint>

* <Break If>

* <Watch>

* <Watch Exp>

* <Unwatch>

* <Eval>

* <Back>

* <Forward>

3.5.2 Detailed description of the buttons.

3.5.2.1 <Step>

When the program has halted because of a breakpoint (conditional breakpoint),
press <Step> to execute the highlighted code and advance execution.
The debugger will halt the program at the first following debugged
lispform (the debugger can only halt in debugged code = code loaded
in the debugger).

3.5.2.2 <Step Over>

When the program has halted because of a breakpoint (conditional breakpoint),
press <Step Over> to execute the highlighted code and advance execution.
The debugger will halt the program in the following cases (whichever
first occurs).

* A breakpoint is encountered.

* A lispform in the same enclosing list as the highlighted form must
  be executed.

3.5.2.3 <Next>

When the program has halted because of a breakpoint (conditional breakpoint),
use <Next> to advance execution. To use <Next> first select another
lisp form and press then <Next> the execution with then proceed until
one of the following two cases are encountered (whichever first occurs).

* A breakpoint is encountered.

* The highlighted code must be executed.

3.5.2.4 <Continue>

Pressing <Continue> will advance a halted program , the program will
either fully execute or will halt at the next breakpoint.

3.5.2.5 <Breakpoint>

Use this to set a breakpoint. To set a breakpoint just highlight a
lispform and then press <Breakpoint>. If during executing of code
the highlighted code must be executed then the debugger halts execution
just before this code gets executed. Use <Step>,<Step Over>,<Next>
or <Continue> to continue execution.

3.5.2.6 <Break If>

Use this to set a conditional breakpoint. To set a conditional breakpoint,
highlight a lispform, type a condition in the command window and press
<Break If>. If during execution of code the highlighted code must
be executed then the debugger evaluates the condition and if the result
is non nil execution will halt. 

Errors during evaluating the condition counts as a NIL and a message
detailing the error is displayed.

3.5.2.7 <Watch>

Sets a watchpoint on a variable. To do this selects the variable and
then press <Watch> , the variable and its value is then visible in
the result pane.

3.5.2.8 <Watch Exp>

Sets a watchpoint on a expression. First select a area where the watchpoint
is valid , then type in the expression in the command pane and press
<Watch Exp>. When execution happens in the selected area the exp is
evaluated and the result is displayed in the result pane. If the evaluation
of the expression causes an error this is displayed as an error message
and the result is undefined.

3.5.2.9 <Unwatch>

Stops watching a variable or expression. To do this selects the variable(expression)
in the result pane together with the result and press <Unwatch>.

3.5.2.10 <Eval>

Evaluates a expression in the debugging context. If during debugging
(when execution has halted and you have control) you feel the need
to evaluate a expression in the context of the highlighted code do
the following. Type in an expression in the command pane and press
<Eval>, the result will then be visable in the result pane. If the
evaluation of the expression causes an error this is displayed as
an error message and the result is undefined.

3.5.2.11 <Back>

Activates time traveling , the debugger keeps track of the 100 lasts
forms executed together with their context. By pressing <Back> you
go to the previous executed form and change the context to the context
at that time. Has the same functionality as the frame concepts in
the system debuggers.

3.5.2.12 <Forward>

Activates time traveling , the debugger keeps track of the 100 lasts
forms executed together with their context. By pressing <Forward>
you go to the next executed form and change the context to the context
at that time. 

Extentending the debugger

4.1 Introduction.

LISP is a very extensible language in the way that you easily use macros
to define you own control structures. This debugger tries to behave
logical for the control structures as defined in CLTL2 but doesn't
know of course how to handle controle structures you define yourself.
To solve this you can modify the way how this debugger adds instrumentation
code to the source, so that your extensions are covered correctly.
The way I have made this possible is to provide you with a special
language which allows you to express the syntax of lispforms with
a little bit of semantics as well. In fact the core of this debugger
is generated automatically by a source of syntax diagrams defining
the syntax of most special lispforms in CLTL2.

If you want to modify the debugger keep in mind the following points :

* Function calls are handled correctly by the debugger , you don't
  have to do anything at all for this.

* Macro's are the problem . The standard lisp macros are covered in
  the debugger and he tries to make the best of the ones you define
  (first he expands the macro call and then tries to add debugging
  code to the expanded code) but the result is not always what you
  expect. It is here that you can modify the behavior of the debugger.

* Modifying the debugger can stops it working so be carefull. Nothing
  can go wrong if you make sure that you have made backup copies of
  the following files:

  * lispsyntax (this is the source file defining the working of the
    current debugger)

  * lispcode.o,lispcode.x86f , lispcode.fas or lispcode.fasl (this
    is the core of the debugger for GCL,CMUCL,CLISP or ACL5, loaded
    when you start (debug) or (deb)).

4.2 Modifying the debugger

The steps to modify the debugger are as follows:

1. Create a source file defining how the lisp debugger should transform
  sources. You can take the file delivered with the debugger as a
  base and extend it. This file is '/usr/local/lib/lispdebug/lispsyntax'
  if you have done a default installation of the debugger. Remember
  this file will redefine the way how the debugger transform sources
  to debugged sources , if you make errors or forget things the debugger
  can act strangely. The syntax and semantics of the language to use
  is described in the next chapter.

2. Use the lisp function DEBUGGER::process-definition-file to compile
  this file to lisp code containing the parse and convertion code
  of the debugger. The syntax to use is either :

  (a) (DEBUGGER::process-definition-file <Source-file>) , the source
    file is compiled to a lispfile (debugcode.lisp) which is then
    compiled by the lisp system and then loaded in the lisp system
    modifying the behavior of the debugger. 

  (b) (DEBUGGER::process-definition-file <Source-file> <Out>) , the source
    file is compiled to a lispfile (with filename <out>.lisp) which
    is then compiled by the lispsytem and then loaded in the lisp
    system modifying the debugger.

3. Once 2 is finished you can test the debugger to see if he handles
  the new definitions well (to test the normal lisp constructs look
  at the testgcl.lisp,testcmucl.lisp,testclisp.lisp and testacl5.lisp
  files). If you are convinced that the debugger is working correctly
  you can make your modifications permanent by copying the object
  file generated in (2) to /usr/local/lib/lispdebug or to the value
  of the LISPDEBUG environment variable. (If you use 2.a. the object
  files are either debugcode.o,debugcode.x86f,debugcode.fas or debugcode.fasl
  for gcl,cmucl,clisp or acl).

4.3 Extension Language Of the Debugger.

The extension language of the debugger is strongly based on the syntax
descriptions used in CLTL2 so that writing extensions is a simple
as writing a syntax diagram. Be carefull however, although the language
looks simple their are some know cavecats which should be dealt with.

4.3.1 The default file used by the debugger.

To give you an idea of how the extension language looks like I give
you here the transcript of 'lispsyntax' the source used to generate
the parser/transformer of the debugger delivered with this package.
Although the language used is not yet defined it should look familiar
to you.

;; *****************************************************************************************

;; This file contains all syntax definitions used in the lisp debugger ,

;; to automatical generate code to transform lisp code in debugged
lisp code

;; 

;; Be very carefull to edit or modify this file , if you generate code
form a modified 

;; file via (process-definition-file "lispsyntax") and load then the
generated 

;; compiled file in the lisp debugger the working of the debugger is
changed.

;; ****************************************************************************************

;;

;; Defintions used throughout this file 

;;

;; *****************************************************************************************

;; Bodies used in different definitions

;; *****************************************************************************************

 

body-1 = {{(declare {_declaration}*) {"string" @ | @} |

           "string" {(declare {_declaration}*) @ | @} |

           @}

          {#form}*}

 

body-2 = {{(declare {_declaration}*) | @} {#form}*}

 

body-3 = {{(declare {_declaration}*) | #form} {#form}*}

 

;; *****************************************************************************************

;; Lambda list definitions

;; *****************************************************************************************

 

lambda-list = ({~var}*

               [&optional {~var | (~var [#initform [~svar]])}*]

               [&rest ~var]

               [&key {~var | ({~var | (_keyword ~var)} [#initform [~svar]])}*

                     [&allow-other-keys]]

               [&aux {~var | (~var [#initform])}*])

 

specialized-lambda-list = ({~var | (~var _parameter)}*

                           [&optional {~var | (~var [#initform [~svar]])}*]

                           [&rest ~var]

                           [&key {~var | ({~var | (_keyword ~var)} [#initform [~svar]])}*

                                 [&allow-other-keys]]

                           [&aux {~var | (~var [#initform])}*])

 

;; ****************************************************************************************

;; Option used in CLOS definitions

;; ****************************************************************************************

 

option = {(:method {_method-qualifier}* specialized-lambda-list  body-1) |

          _other}

 

;;*****************************************************************************************

;; slot-entry , used in CLOS definitions

;; ****************************************************************************************

 

 

slot-entry = {~slot-name | (~var _slotname)}

 

 

 

;; *****************************************************************************************

;;

;; This file contains syntax definitions for

;;

;; and

;; assert

;; block

;; case

;; catch

;; ccase

;; compiler-let

;; cond

;; ctypecase

;; decf

;; defclass

;; defgeneric

;; define-method-combination (deactivated , from my humble knowledge
of lisp this like defmacro so it shouldn't be debugged)

;; defmethod

;; defun

;; destructuring-bind

;; do

;; do-symbols

;; do-external-symbols

;; do-all-symbols

;; do*

;; dolist

;; dotimes

;; etypecase

;; eval-when

;; flet

;; generic-flet

;; generic-function

;; generic-labels

;; if

;; incf

;; ignore-errors

;; handler-case

;; handler-bind

;; labels

;; lambda

;; let

;; let*

;; locally

;; loop

;; macrolet

;; mapping (defined but not yet tested because of no implementation
of series)

;; multiple-value-bind

;; multiple-value-call

;; multiple-value-list

;; multiple-value-prog1

;; multiple-value-setq

;; nth-value

;; or

;; pprint-logical-block (defined but not yet tested because of no knowledge
of how to use)

;; prog

;; prog*

;; prog1

;; prog2

;; progn

;; progv

;; psetf

;; psetq

;; push

;; pushnew

;; return

;; return-from

;; setf

;; setq

;; symbol-macrolet

;; tagbody

;; the

;; throw

;; unless

;; unwind-protect

;; when

;; with-accessors

;; with=added-methods (defined but not tested because not implemented
in the lisps I know)

;; with-compilation-unit

;; with-hash-table

;; with-input-from-string

;; with-open-file

;; with-open-stream

;; with-output-to-string

;; with-package-iterator

;; with-simple-restart

;; with-slots

;; **************************************************************************************************************

 

;; **************************************************************************************************************

;; and (and {form}*)

;; **************************************************************************************************************

 

(and {#form}*)

 

;; **************************************************************************************************************

;; assert ,  assert test-form [({place}*) [string {arg}*]]

;; **************************************************************************************************************

 

(assert #form [_place [#string {#arg}*]])

 

;; **************************************************************************************************************

;; block (block name {form}*)

;; **************************************************************************************************************

 

(block _name {#form}*)

 

;; **************************************************************************************************************

;; case  case keyform {({({key}*) | key} {form}*)}*

;; **************************************************************************************************************

 

 

(case #keyform {({({_key}*) | _key} {#form}*)}*)

 

;; **************************************************************************************************************

;; catch  catch tag {form}*

;; **************************************************************************************************************

 

(catch _tag {#form}*)

 

;; **************************************************************************************************************

;; ccase keyplace {({({key}*) | key} {form}*)}*

;; **************************************************************************************************************

 

(ccase _keyplace {({({_key}*) | _key} {#form}*)}*)

 

;; **************************************************************************************************************

;; compiler-let   compiler-let ({var | (var [value])}*) {form}* 

;; **************************************************************************************************************

 

(compiler-let ({_var | (_var [_value])}*) {#form}*)

 

;; **************************************************************************************************************

;; cond  cond {(test {form}*)}*

;; **************************************************************************************************************

 

(cond {(#test {#form}*)}*)

 

;; **************************************************************************************************************

;; decf place [delta]

;; **************************************************************************************************************

 

(decf _place [#delta])

 

;; **************************************************************************************************************

;; defclass

;; defclass class-name ({superclass-name}*)

;;          ({slot-specifier}*) [[?class-option]]

;;

;;

;; class-name ::= symbol

;; superclass-name ::= symbol

;; slot-specifier ::= slot-name | (slot-name [[?slot-option]])

;; slot-name ::= symbol

;; slot-option ::= {:reader reader-function-name}*

;;                 | {:writer writer-function-name}*

;;                 | {:accessor reader-function-name}*

;;                 | {:allocation allocation-type}*

;;                 | {:initarg initarg-name}*

;;                 | {:initform form}*

;;                 | {:type type-specifier}*

;;                 | {:documentation string}*

;; reader-function-name ::= symbol

;; writer-function-name ::= function-name/

;; function-name ::= {symbol | (setf symbol)}

;; initarg-name ::= symbol

;; allocation-type ::= :instance | :class

;; class-option ::= (:default-initargs initarg-list)

;;                 | (:documentation string)

;;                 | (:metaclass class-name)

;; initarg-list ::= {initarg-name default-initial-value-form}*

;; **************************************************************************************************************

 

(defclass _class-name _superclass

          ({slot-specifier}*)

          [class-option]*)

 

slot-specifier = {(_name {slot-option}*)|_name}

slot-option = {:initform #form |

               _other _other}

class-option = {(:default-initargs {_initarg #form}*) | _other}

 

;; **************************************************************************************************************

;; defgeneric 

;;

;;defgeneric function-name lambda-list 

;;              [[?option | {method-description}*]]

;;function-name ::= {symbol | (setf symbol)}

;;lambda-list ::= ({var}* 

;;                 [&optional {var | (var)}*]

;;                 [&rest var] 

;;                 [&key {keyword-parameter}* 

;;                       [&allow-other-keys]])

;;keyword-parameter ::= var | ({var | (keyword var)}) 

;;option ::= (:argument-precedence-order {parameter-name}+)

;;                 | (declare {declaration}+) 

;;                 | (:documentation string)

;;                 | (:method-combination symbol {arg}*)

;;                 | (:generic-function-class class-name)

;;                 | (:method-class class-name)

;;method-description ::= (:method {method-qualifier}*

;;                         specialized-lambda-list

;;                         [[ {declaration}* | documentation ]]

;;                         {form}*)

;;method-qualifier ::= non-nil-atom 

;;specialized-lambda-list ::=      ({var | (var parameter-specializer-name)}*

;;                                  [&optional {var | (var [initform [supplied-p-parameter]])}*]

;;                                  [&rest var]

;;                                  [&key {specialized-keyword-parameter}*

;;                                        [&allow-other-keys]]

;;                                  [&aux {var | (var [initform])}*])

;;specialized-keyword-parameter ::=   var | ({var | (keyword var)} [initform
[supplied-p-parameter]])

;; parameter-specializer-name ::= symbol | (eql eql-specializer-form)

;; **************************************************************************************************************

 

(defgeneric _name _lambda-list {option}*)

 

;; **************************************************************************************************************

;; define-method-combination

;; define-method-combination name [[?short-form-option]]

;; define-method-combination name lambda-list

;;                           ({method-group-specifier}*)

;;                           [(:arguments . lambda-list)]

;;                           [(:generic-function generic-fn-symbol)]

;;                           [[{declaration}* | doc-string]]

;;                           {form}*

;; short-form-option ::= :documentation string

;;                     | :identity-with-one-argument boolean

;;                     | :operator operator

;; method-group-specifier ::= (variable {{qualifier-pattern}+ | predicate}

;;                             [[?long-form-option]])

;; long-form-option ::= :description format-string

;;                    | :order order

;;                    | :required boolean

;; **************************************************************************************************************

;; Deactivated , could not find usefull example to test this (I am
not a specialist of CLOS

;; (define-method-combination _name {:documentation _string |

;;                                  :identify-with-one-argument _boolen |

;;                                  :operator _operator |

;;                                  lambda-list ({method-group-specifier}*)

;;                                              {argument-lambda-list (:generic-function _fn) body-1 |

;;                                               (:generic-function _fn) body-1 |

;;                                               body-1}}) 

;;

;;

;; method-group-specifier = (~var {_rest}*)

;; argument-lambda-list = (:arguments {~var}*

;;                                   [&optional {~var | (~var [#initform [~svar]])}*]

;;                                 [&rest ~var]

;;                                 [&key {~var | ~var ({~var | (_keyword ~var)} [#initform [~svar]])}*
[&allow-other-keys]]

;;                                 [&aux {~var | (~var [#initform])}*])

;;                                 

 

;; **************************************************************************************************************

;; defmethod

;; defmethod function-name {method-qualifier}*

;;           specialized-lambda-list

;;           [[ {declaration}* | doc-string]]

;;           {form}*

;; function-name ::= {symbol | (setf symbol)}

;; method-qualifier ::= non-nil-atom 

;; parameter-specializer-name ::= symbol | (eql eql-specializer-form)

;; **************************************************************************************************************

 

 

(defmethod _function-name 

           {_method-qualifier}* 

           specialized-lambda-list 

           body-1)

 

;; **************************************************************************************************************

;; defun

;;  defun name lambda-list [[ {declaration}* | doc-string ]] {form}*

;; ({var}*

;;  [&optional {var | (var [initform [svar]])}*]

;;  [&rest var]

;;  [&key {var | ({var | (keyword var)} [initform [svar]])}* [&allow-other-keys]]

;;  [&aux {var | (var [initform])}*])

;; **************************************************************************************************************

 

(defun _name lambda-list body-1)

 

;; **************************************************************************************************************

;;  destructuring-bind lambda-list expression {declaration}* {form}*

;; **************************************************************************************************************

 

var-list = {~var | ({var-list}*)}

macro-lambda-list = ({&whole ~var

                      [var-list]*

                      [&optional {~var | (~var [#initform [~svar]])}*]

                      [{&rest ~var | &body ~var}]

                      [&key {~var | ({~var | (_keyword ~var)} [#initform [~svar]])}* [&allow-other-keys]]

                      [&aux {~var | (~var [#initform])}*] 

                     |

                      [var-list]*       

                      [&optional {~var | (~var [#initform [~svar]])}*]

                      [{&rest ~var | &body ~var}]

                      [&key {~var | ({~var | (_keyword ~var)} [#initform [~svar]])}* [&allow-other-keys]]

                      [&aux {~var | (~var [#initform])}*]

                     }

                    )

 

(destructuring-bind macro-lambda-list #expression body-2)

 

;; **************************************************************************************************************

;; do 

;; do ({(var [init [step]])}*)

;;    (end-test {result}*)

;;    {declaration}* {tag | statement}*

;;

;; **************************************************************************************************************

 

(do ({(~var [#init [#step]])}*)

    (#end-test {#result}*)

    body-2)

 

;; **************************************************************************************************************

;; do*

;; do* ({(var [init [step]])}*)

;;     (end-test {result}*)

;;     {declaration}* {tag | statement}* 

;; **************************************************************************************************************

 

 

(do* ({(^var [#init [#step]])}*)

     (#end-test {#result}*)

     body-3)

 

;; **************************************************************************************************************

;; do-all-symbols 

;; do-all-symbols (var [package [result-form]])

;;                {declaration}* {tag | statement}*

;; **************************************************************************************************************

 

(do-all-symbols (~var [#package [#resultform]]) body-2)

 

;; **************************************************************************************************************

;; do-external-symbols 

;; do-external-symbols (var [package [result-form]])

;;                     {declaration}* {tag | statement}*

;; **************************************************************************************************************

 

(do-external-symbols (~var [#package [#resultform]]) body-2)

 

;; **************************************************************************************************************

;; dolist

;; dolist (var listform [resultform])

;;        {declaration}* {tag | statement}*

;; **************************************************************************************************************

 

(dolist (~var #listform [#resultform]) body-2)

 

;; **************************************************************************************************************

;; dotimes

;; dotimes (var countform [resultform])

;;         {declaration}* {tag | statement}*

;; **************************************************************************************************************

 

(dotimes (~var #countform [#resultform]) body-2)

 

;; **************************************************************************************************************

;; do-symbols 

;; do-symbols (var [package [result-form]])

;;            {declaration}* {tag | statement}*

;; **************************************************************************************************************

 

(do-symbols (~var [#package [#resultform]]) body-2)

 

 

;; **************************************************************************************************************

;; ecase keyform {({({key}*) | key} {form}*)}*

;; **************************************************************************************************************

 

(ecase #keyform {({({_key}*) | _key} {#form}*)}*)

 

;; **************************************************************************************************************

;;  etypecase keyform {(type {form}*)}*

;; **************************************************************************************************************

 

(etypecase #keyform {(_type {#form}*)}*)

 

;; **************************************************************************************************************

;; ctypecase keyplace {(type {form}*)}*

;; **************************************************************************************************************

 

(ctypecase _keyplace {(_type {#form}*)}*)

 

;; **************************************************************************************************************

;;  eval-when ({situation}*) {form}*

;; **************************************************************************************************************

 

(eval-when _situation {#form}*)

 

;; **************************************************************************************************************

;; flet

;; flet ({(name lambda-list

;;              [[ {declaration}* | doc-string ]] {form}*)}*)

;;       {form}* 

;; **************************************************************************************************************

 

(flet ({(_name lambda-list body-1)}*) {#form}*)

 

;; **************************************************************************************************************

;; if 

;;  if test then [else] 

;; **************************************************************************************************************

 

(if #test #then [#else])

 

;; **************************************************************************************************************

;; generic-flet ({(function-name lambda-list

;;                 [[?option | {method-description}* ]])}*)

;;              {form}*

;; **************************************************************************************************************

 

(generic-flet ({(_name _lambda-list $ {option}*)}*) {#form}*)

 

;; **************************************************************************************************************

;; generic-function lambda-list [[?option | {method-description}*]]

;; option ::= (:argument-precedence-order {parameter-name}+)

;;                 | (declare {declaration}+)

;;                 | (:documentation string)

;;                 | (:method-combination symbol {arg}*)

;;                 | (:generic-function-class class-name)

;;                 | (:method-class class-name)

;; method-description ::= (:method {method-qualifier}*

;;                                 specialized-lambda-list

;;                                 {declaration | documentation}*

;;                                 {form}*)

;; **************************************************************************************************************

 

(generic-function _lambda-list $ {option}*)

 

;; **************************************************************************************************************

;; generic-labels ((function-name lambda-list

;;                  [[?option | {method-description}*]])}*)

;;                {form}*

;; **************************************************************************************************************

 

(generic-labels ({(_name _lambda-list $ {option}*)}*) {#form}*)

 

;; **************************************************************************************************************

;;  handler-bind ({(typespec handler)}*) {form}*

;; **************************************************************************************************************

 

(handler-bind ({(_typespec #handler)}*) {#form}*)

 

;; **************************************************************************************************************

;;  handler-case expression {(typespec ([var]) {form}*)}*

;; **************************************************************************************************************

 

(handler-case #expression {(_typespec ([~var]) @ {#form}*)}*)

 

;; **************************************************************************************************************

;;  ignore-errors {form}*

;; **************************************************************************************************************

 

(ignore-errors {#forms}*)

 

;; **************************************************************************************************************

;; incf

;;  incf place [delta] 

;; **************************************************************************************************************

 

(incf _place [#delta])

 

;; **************************************************************************************************************

;; labels

;; labels ({(name lambda-list

;;          [[ {declaration}* | doc-string ]] {form}*)}*)

;;        {declaration}* {form}*

;; **************************************************************************************************************

 

(labels ({(_name lambda-list body-1)}*)

        body-2)

 

;; **************************************************************************************************************

;; lambda 

;; (lambda lambda-list [[{declaration}* | doc-string]] {form}*)

;; **************************************************************************************************************

 

(lambda lambda-list body-1)

 

;; **************************************************************************************************************

;; let

;; let ({var | (var [value])}*) {declaration}* {form}* 

;; **************************************************************************************************************

 

(let ({~var | (~var [#value])}*) body-2)

 

;; **************************************************************************************************************

;; let*

;;  let* ({var | (var value)}*) {declaration}* {form}* 

;; **************************************************************************************************************

 

(let* ({^var | (^var [#value])}*) body-3)

 

;; **************************************************************************************************************

;; locally

;; locally {declaration}* {form}

;; **************************************************************************************************************

 

(locally body-3)

 

;; **************************************************************************************************************

;; loop

;; loop {form}*

;; **************************************************************************************************************

 

(loop {#form}*)

 

;; **************************************************************************************************************

;; macrolet

;; macrolet ({(name varlist

;;          [[ {declaration}* | doc-string ]] {form}*)}*)

;;       {form}* 

;; **************************************************************************************************************

 

(macrolet _macrolist

          {#form}*)

 

;; **************************************************************************************************************

;;  mapping ({({var | ({var}*)} value)}*) {declaration}* {form}*

;; **************************************************************************************************************

 

(mapping ({({~var | ({~var}*)} #value)}*) body-2)

 

;; **************************************************************************************************************

;; multiple-value-bind

;; multiple-value-bind ({var}*) values-form

;;                       {declaration}* {form}*

;; ************************************************************************************************************** 

 

(multiple-value-bind ({~var}*) #values-form body-2)

 

;; **************************************************************************************************************

;; multiple-value-call

;;  multiple-value-call function {form}*

;; **************************************************************************************************************

 

(multiple-value-call #function {#form}*)

 

;; **************************************************************************************************************

;; multiple-value-list 

;; multiple-value-list form

;; **************************************************************************************************************

 

(multiple-value-list #form)

 

;; **************************************************************************************************************

;; multiple-value-prog1

;; multiple-value-prog1 form {form}*

;; **************************************************************************************************************

 

(multiple-value-prog1 {#form}*)

 

;; **************************************************************************************************************

;; multiple-value-setq 

;; multiple-value-setq variables form

;; **************************************************************************************************************

 

(multiple-value-setq _variables #form)

 

;; **************************************************************************************************************

;; nth-value

;; nth-value n form

;; **************************************************************************************************************

 

(nth-value #n #form)

 

;; **************************************************************************************************************

;; or

;; or {form}* 

;; **************************************************************************************************************

 

(or {#form}*)

 

;; **************************************************************************************************************

;; pprint-logical-block

;; pprint-logical-block (stream-symbol list [[ { :prefix | :per-line-prefix}
p | :suffix s ]])

;;                      {form}*

;; **************************************************************************************************************

 

(pprint-logical-block ~list {#form}*)

 

;; **************************************************************************************************************

;; prog

;; prog ({var | (var [init])}*) {declaration}* {tag | statement}*

;; **************************************************************************************************************

 

(prog ({~var | (~var [#init])}*) body-2)

 

;; **************************************************************************************************************

;; prog*

;; prog* ({var | (var [init])}*) {declaration}* {tag | statement}*

;; **************************************************************************************************************

 

 

(prog* ({^var | (^var [#init])}*) body-3)

 

;; **************************************************************************************************************

;; prog1

;;  prog1 first {form}*

;; **************************************************************************************************************

 

(prog1 {#form}*)

 

;; **************************************************************************************************************

;; prog2

;;  prog2 first second {form}* 

;; **************************************************************************************************************

 

(prog2 {#form}*)

 

;; **************************************************************************************************************

;; progn

;;  progn {form}*

;; **************************************************************************************************************

 

(progn {#form}*)

 

;; **************************************************************************************************************

;; progv

;;  progv symbols values {form}*

;; **************************************************************************************************************

 

(progv {#form}*)

 

;; **************************************************************************************************************

;; psetf

;; psetf {_place form}*

;; **************************************************************************************************************

 

(psetf {_place #form}*)

 

;; **************************************************************************************************************

;; psetq 

;; psetq {symbol form}*

;; **************************************************************************************************************

 

(psetq {_symbol #form}*)

 

;; **************************************************************************************************************

;; push

;;  push item place

;; **************************************************************************************************************

 

(push #form _place)

 

;; **************************************************************************************************************

;; pushnew

;; pushnew item place &key :test :test-not :key

;; **************************************************************************************************************

 

(pushnew #form _place [_key]*)

 

;; **************************************************************************************************************

;; return

;;  return [result]

;; **************************************************************************************************************

 

(return [#form])

 

;; **************************************************************************************************************

;; return-from

;;  return-from name [result]

;; **************************************************************************************************************

 

(return-from _name [#result])

 

;; **************************************************************************************************************

;; setf

;;  setf {place newvalue}*

;; **************************************************************************************************************

 

(setf {_place #value}*)

 

;; **************************************************************************************************************

;; setq

;; setq {var form}*

;; **************************************************************************************************************

 

(setq {_var #form}*)

 

;; **************************************************************************************************************

;; symbol-macrolet

;; symbol-macrolet ({(var expansion)}*)                  {declaration}* {form}*

;; **************************************************************************************************************

 

(symbol-macrolet ({(~var _expansion)}*) body-2)

 

;; **************************************************************************************************************

;; tagbody

;;  tagbody {tag | statement}*

;; **************************************************************************************************************

 

(tagbody {#form}*)

 

;; **************************************************************************************************************

;; the

;; the value-type form

;; **************************************************************************************************************

 

(the _value-type #form)

 

;; **************************************************************************************************************

;; throw

;; throw tag result

;; **************************************************************************************************************

 

(throw _tag #result)

 

;; **************************************************************************************************************

;; unless

;;  unless test {form}*

;; **************************************************************************************************************

 

(unless {#form}*)

 

;; **************************************************************************************************************

;; unwind-protect

;; unwind-protect protected-form {cleanup-form}*

;; **************************************************************************************************************

 

(unwind-protect {#form}*)

 

;; **************************************************************************************************************

;; when

;; when test {form}*

;; **************************************************************************************************************

 

(when {#form}*)

 

;; **************************************************************************************************************

;; with-accessors

;; with-accessors ({slot-entry}*) instance-form

;;                {declaration}* {form}*

;; **************************************************************************************************************

 

(with-accessors ({slot-entry}*) #instance-form body-2)

 

;; **************************************************************************************************************

;; with-added-methods (function-name lambda-list

;;                    [[?option | {method-description}*]])

;;                    {form}*

;; **************************************************************************************************************

 

(with-added-methods (_name lambda-list $ {option}*) {#form}*)

 

;; **************************************************************************************************************

;;  with-compilation-unit ({option-name option-value}*) {form}*

;; **************************************************************************************************************

 

(with-compilation-unit ({_option #option-value}*) {#form}*)

 

;; **************************************************************************************************************

;;  with-hash-table-iterator (mname hash-table) {form}*

;; **************************************************************************************************************

 

(with-hash-table-iterator (_mname #hash-table) {#form}*)

 

;; **************************************************************************************************************

;; with-input-from-string

;; with-input-from-string (var string keyword {value}*)         {declaration}*
{#form}*

;; **************************************************************************************************************

 

(with-input-from-string (~var #string {_keyword}*) body-2)

 

;; **************************************************************************************************************

;; with-open-file

;; with-open-file (stream filename {options}*)       {declaration}* {#form}*

;; **************************************************************************************************************

 

(with-open-file (~stream #filename {_options}*) body-2)

 

;; **************************************************************************************************************

;; with-open-stream 

;; with-open-stream (var stream) {declaration}* {form}*

;; **************************************************************************************************************

 

(with-open-stream (~var #stream) body-2)

 

;; **************************************************************************************************************

;; with-output-to-string

;; with-output-to-string (var [string [:element-type type]])       {declaration}*
{#form}*

;; **************************************************************************************************************

 

(with-output-to-string (~var [#string [:element-type _type]]) body-2)

 

;; **************************************************************************************************************

;; with-simple-restart (name format-string {format-argument}*)

;;            {form}*

;; **************************************************************************************************************

 

(with-simple-restart (_name _string {_arg}*) {#form}*)

 

;; **************************************************************************************************************

;; with-slots

;; with-slots ({slot-entry}*) instance-form       {declaration}* {form}*

;; **************************************************************************************************************

 

(with-slots ({slot-entry}*) #instance-form body-2)

 

 

;; End of definitions

4.3.2 Syntax of the language. 

The language is composed of the following elements :

comments Everything on a line after a ';' is considered a comment
  and is neglected.

white spaces Used as separators , the following is a white space :

* blanc

* newline

* tab

* return

definitions These are of the form : 

* symbol = expression

expressions Thes are of the form :

* symbol

* _symbol

* ^symbol

* ~symbol

* #symbol

* @

* $

* [ expression ... expression ]

* [ expression ... expression ]*

* {expression ... expression}

* {expression ...|....|...| expression... }*

* ( expression expresion ...)

* ``text''

symbols Any string of characters with the exception of white spaces
  , @ , $ , _ ,^,~,#,(,),[,]

A source in our language is a text file containing definitions and
expressions.

4.3.3 Semantics of the language.

The syntax tells us what the welformed expressions and definitions
are in our language but it says nothing about their meaning, for this
we need a little bit of semantics. The best way to understand the
semantics of the language is the consumer/producent metaphor. When
a expression in our language is applied on a lisp expression two things
can happens :

1. The expression recognizes the lisp expression consuming part of it
  and producing another lisp expression.

2. The expression does not recognize the expression an it generates
  a throw, no lisp expression is produced.

Lets now put these ideas in practice on the different type of expressions
of our language. Lets P the list produced ,E a expression in our language
and L the lisp expression on which we applies expressions in our language.
The parser/generator in the debugger will apply each expression on
a given lisp expression until it gets not a throw and the lisp expression
is fully consumed, the produced list is then the lisp expression with
debugcode added. If this sounds inefficient you are right this is
just a semantic explanation , our language is actually compiled to
become the parser/generator of the debugger which has the same effect
as our semantic explanation, but he does it in a more efficient way.

4.3.3.1 Symbol.

If we apply a 'symbol' on a lisp expression L=(e1 e2... en) or L=()
we have a throw if e1 is not equal to our 'symbol' or if L is the
empty list. If e1 is equal to our 'symbol' then we append P with the
symbol and L becomes (e2 ... en). 

Let P=() , L=(defun f (n) (princ n)) and E=defun then applying E on
L gives L=(f (n) (princ n)) and P becomes (defun).

Let P=(), L=(defun f(n) (princ n)) and E=let then applying E on L gives
a throw.

4.3.3.2 ``text''

If we apply ``...'' on a lisp expression L=(e1 e2 ... en) or L=() we
have a throw if L is empty or if e1 is not a string. In all other
cases L becomes (e2 ... en) and the first element e1 is added to P.
You can think of ``text'' as standing for any string.

4.3.3.3 _symbol

If we apply '_symbol' on a lisp expression L=(e1 e2.... en) or L=()
we have a throw if L is empty. In all other cases L becomes (e2 ...
en) and the first element e1 is added to P. You can think of _symbol
as standing for any list element which must not be changed.

Let P=() , L=(a b c d) and E=_sym then applying E on L gives L=(b c
d) , P=(a).

Let P=(),L=((a b) c d) and E=_sym then applying E on L gives L=(c d)
, P=((a b))

Let P=(),L=() and E=_sym then applying E on L gives a throw.

4.3.3.4 ^symbol

If we apply '^symbol' on a lisp expression L=(e1 e2 .. en) or L=()
we have a throw if L is empty or e1 is a list. In all other cases
L becomes (e2 ... en) and the first element e1 is added to P. Also
e1 is added to the lexical environment of the debugger. The lexical
scope is the parent parent list of L and e1 becomes active just after
L in the parent list. You can use ^ to represent variables in do*
and let* constructs. When you use ^ the debugger will try to save
the value of this variable when it becomes active and this value becomes
available in the lexical scope.

Let P=() , L=(a b c d) and E=^sym then applying E on L gives L=(b c
d) , P=(a).

Let P=(),L=((a b) c d) and E=^sym then applying E on L gives a throw.

Let P=(),L=() and E=^sym then applying E on L gives a throw.

4.3.3.5 ~symbol

If we apply '~symbol' on a lisp expression L=(e1 e2 .. en) or L=()
we have a throw if L is empty or e1 is a list. In all other cases
L becomes (e2 ... en) and the first element e1 is added to P. Also
e1 is added to the lexical environment of the debugger. The lexical
scope is the parent parent list of L and e1 becomes active by using
$ or a @. You can use ~ to represent variables in do and let constructs.
When you use ~ the debugger will try to save the value of this variable
when it becomes active and this value becomes available in the lexical
scope.

Let P=() , L=(a b c d) and E=^sym then applying E on L gives L=(b c
d) , P=(a).

Let P=(),L=((a b) c d) and E=^sym then applying E on L gives a throw.

Let P=(),L=() and E=^sym then applying E on L gives a throw.

4.3.3.6 #symbol

If we apply '#symbol' on a lisp expression L=(e1 e2 ... en) or L=()
we have a throw if L is empty. In all other cases L becomes (e2 ....
en) and the system tries to add debugging code to e1 before it is
added to P. The debugging code added makes that you can place a breakpoint
on e1 , see where e1 is located in the source and allows you to look
at the lexical environment of e1. In some cases no debugging code
is added (if e1 is not a list or if e1 represents a macro not recognized
by the debugger). You can use # to indicate that you should be able
to set breakpoints.

Let P=() , L=(a b c d) and E=^sym then applying E on L gives L=(b c
d) , P=((add-debug-code a)).

Let P=(),L=() and E=^sym then applying E on L gives a throw.

4.3.3.7 @

We can always apply @ to L . It consumes nothing and it produces nothing
, its solely purpose is for its side effect , it will make variables
defined with ~ active and it allows you to set a breakpoint on the
whole expression where it is part of. Use this in (defun ...) (let
...) just before the body of these functions.

4.3.3.8 $

We can always apply $ to L . It consumes nothing and it produces nothing
, its solely purpose is for its side effect , it will make variables
defined with ~ active. 

4.3.3.9 [expression1 ... expressionn]

If we apply [expression1 ... expressionn] on a lisp expression L .
It will first do nothing , the expressions after [...] are first applied
on L if this gives a throw the system will first trying to apply expression1
then expression2 ... on L followed by the expressions after [...].
Consider this as a kind of optional syntax.

E=[a b] c , L=(a b c d) and P=() then applying E on L gives L=(d) ,
P=(a b c)

E=[a b] c , L=(c d) and P=() then applying E on L gives L=(d) , P=(c)

E=[a b] d , L= (c d) and P=() then applying E on L gives a throw

4.3.3.10 {e11...e1n|...| em1....emk}

If we apply {e11...e1n|...| em1....emk} on a lisp expression L . It
will first try to apply e11...e1n on L followed by all expressions
after { ... } if this causes a throw, it will try e21...e2l followed
by the rest ... . You can consider { ... } as a kind of or where you
want to try different alternatives.

E={a b} c , L=(a c d) and P=() then applying E on L gives L=(d),P=(a
c)

E={a b} c, L=(b c d) and P=() then applying E on L gives L=(d),P=(a
b)

E={a b} c, L=(c d) and P=() then applying E on L gives a throw

4.3.3.11 [expression1 | ... | expressionn]* , {expression1 ... expressionn}

If we apply {...}* or [...]* on a lisp expression. We first try to
apply the expressions after { ...}* or [...]* , if this causes a throw
we try first {...} or [...] followed by the rest, if this fails then
we try {...}{...} or [...][...] and so on. To avoid infinite looping
we stop if one {..} or [...] gives a throw. Use this if you have more
then occurrence of the same elements.

E=[a b]* c, L=(a b a b c) and P=() then applying E on L gives L=(c)
, P=(a b a b)

4.3.3.12 (expression1 ... expressionn)

If we apply (expression1 ... expressionn) on L=(e1 ... en) or L=()
we get a throw if L is empty or if e1 is not a list . The result in
the other cases is the result of applying expression1 .... expressionn
on e1 where we start with a empty result list, this result list is
then added to the original list.If e1 is not consumed fully we have
also a throw.

E=(a b c) , L=((a b c) d) , P=() then applying E on L gives L=(d) ,
P=((a b c))

E=(a b c),L=((a b c d) d), P=() then applying E on L gives a throw.

4.3.3.13 definitions (name = expression)

A definition gives a name to a expression , applying this name has
the same effect as applying the expression. You can use definitions
to to define a expression ones and then use it many times. Definitions
can be used recursively , one can refer in a definition to itself.

4.3.4 Cavecats.

* When using recursive definitions make sure that you do not introduce
  infinite looping.

* Be carefull when using #,_,^ or ~and [],{}* and []* , the effect
  is not always what you want. Look at the following expression 

  * [{``string'' | (declare [_declaration])}] {#exp}* 

* This will when applied to (``Description'' (declare (string a)) (princ
  x)) apply with success {#exp}* to ``Description'' and (declare ..)
  and add debugcode to ``Description'' and (declare ...) which gives
  in the case of (declare ..) a syntax error in lisp. The way out
  of this problem is using some extra {}, our code becomes then something
  like :

    * {``string'' {#exp}* | ``string'' (declare [_declaration]) {#exp}*
      | 
       (declare [_declaration]) {#exp}* | (declare [_declaration]) ``string''
      {#exp}* | {#exp}*}

Porting of the debugger

The debugger is coded in such a way that it should be easy to port
it to another LISP implementation or even another Unix. This chapter
describes how such a port can be done.

5.0.1 Conditions.

The port will be the easiest if certain conditions are fullfilled:

1. The lisp adheres to CLTL2.

2. TCL/TK version 8.0 (previous versions could work also but I have
  not tested them) is installed in the OS together with its libraries.
  In contrast to the previous version of the debugger , LISP must
  not support TCL/TK.

3. The OS and LISP supports sockets.

If (1) is not fullfilled then the best strategy to follow is to write
the missing CLTL2 functions used in the code of the debugger. In case
of failing of TCL/TK the whole interface must be rewritten using another
graphical library. If (3) is not fullfilled you should either extend
LISP with some C coding or choose another connection method to the
interface program. To help porting I will explain how the lisp system
talks with the interface.

5.0.2 Overview of the debugger.

To easy porting, the debugger is splitted in two components.

* The user interface , this is the GUI of the debugger and it is written
  in C using the TCL/TK system as a graphical library.

* A extension of lisp containing the code to parse and change sources
  , generate a new parser/generator and code called by the debugged
  functions.

The interaction between the the interface and the lisp extensions is
done using sockets because this is in my opinion the most supported
IPC in LISP and the OS. 

5.0.2.1 The GUI interface.

The C code is using the following C source files :

* interface.c , this contain the main body of the interface.

* hash.c , hash.h , this contains some extensions for hashing and list
  manipulation used in interface.c.

* tclinvoke.c , code to call TCL/TK in a more efficient way.

These three file must be compiled and linked together with the libraries
of TCL/TK to produce the executable 'interface'. This program can
be started in three ways , modifying the way how it talks to the lisp
system. For socket communication, the interface is always the server
and it is listening for connections of the lisp system to it. The
program 'interface' is started via the lisp system, who decides how
to start the interface.There are current three ways of dooing this.

1. 'interface', the communication takes place via a Unix socket with
  address '/tmp/lispdebugger' (this is the method used by CMUCL)

2. 'interface pid' , pid is the process-id of the parent of this progam
  (the lisp system). This pid is then used by 'interface' to send
  a signal SIGUSR1(=10) to the lisp system each time it has output
  for the lisp system and thus let the lisp system act on the input.
  The communication still takes place via Unix sockets with address
  '/tmp/lispdebugger' (this is the method used for GCL).

3. 'interface -port' , port is the port number to use , communication
  takes place via stream oriented tcp/INET sockets on the port indicated
  (this is used ACL and CLISP).

The communication from 'interface' to lisp is done by sending a lispcommand
as a string followed by a return to the lispsystem (in case of 2 after
the send a signal is send to wakeup the lisp , in all other cases
the lisp system wakes up if there is input on their socket connection).
After sending a command, the 'interface' will not wait for a return.
The functions called from the 'interface' to the lispsystem are:

* (debug-open-file <filename> <compile-t-nil>) to add debugcode to
  the program (if needed compile it) and load the changed code in
  the lisp system.

* (step-back-in-time) causes the debugger to step back in time

* (step-forward-in-time) causes the debugger to step forwards in time

* (display-result-exp exp) causes the lisp system to evaluate the expression
  in the current context of the debugger.

* (stop-interface) stops and cleans up the lisp system socket part
  of the interface.

Communation form the lisp system to the 'interface' to the lisp system
takes place in the following way :

* First a number followed by a blanco indicating the function to call
  in the 'interface' is send , this number implicit says also how
  to handle the arguments. The following functions must be implemented
  in the interface (function together with function-code and description).
  Positions are expressed in number of bytes from the beginning of
  the source.

  * 0=hightlight-source , arguments begin(integer),end(integer),type(string),color(string)
    , should highlight the source text in the indicated color from
    position=begin to position=end. type is used to make a distinction
    between selection.

  * 1=display-message, arguments message(string) , displays the message
    string.

  * 2=highlight-error, arguments begin(integer) highlights the line
    containing the position begin.

  * 3=set-possible-breakpoints , arguments source(string),begin(integer),end(integer)
    gives the debugger information about the begin and endpoints of
    breakpoints in the source.

  * 4=give-control-to-interface , arguments source(string),begin(integer),end(iteger)
    called from the debugged code to allow the debugger to intervine.
    In the 'interface' give-control-interface blocks until the user
    steps or press continue ... .

  * 5-display-result-in-interface, arguments nr-value-arg-pairs(integer),value(string),arg(string)
    .... used to display the calculated results in the resultpane.

  * 6=display-exp-in-interface, arguments exp(string),result(string)
    used to display the result of evaluating a expression.

  * 7=if-breakpoint, arguments none , check for conditional breakpoints.

  * 8=display-time-en, arguments begin(integer),end(integer),source(string)
    equivalent as give-control-to-interface but used in time traveling.

  * 9=setting, arguments var(string),value(string) used to set the
    options settings in the debugger.

* The arguments of the functions are send as follows :

  * integers , in printed format followed by a blanc character so they
    can be read with scanf.

  * strings , first the length of the string is send , then a blanc
    character followed by the string followed by a blanc character.

5.0.2.2 The lisp code.

The lispcode is splitted up in two parts :

* The main body of the debugger (debugger.lisp)

* The interface to the 'GUI interface' (cmucl.lisp,gcl.lisp,acl5.lisp,clisp.lisp)

If you want to port the debugger to another lisp , you can leave debugger.lisp
and write a new <lisp name>.lisp file to interface with the 'GUI interface'
If you use the current 'GUI interface' the following functions should
be implemented in this file:

* (start-interface) , to start 'interface' and make a socket connection.

* (stop-interface) , to clean up everything after 'interface' is stoppped.

* (send-command command &arg-list) , to send the command function code
  followed by the arguments to the interface.

* (process-incoming) see if there is input waiting on the socket read
  this in and evaluate it.

* A way of detecting input on the socket and then call process-incoming.

For examples I refere to cmucl.lisp,gcl.lisp,acl5.lisp and clisp.lisp.
