Tibor's Musings

Common Lisp Runtime Redefinition

I think Lisp is the ideal enviroment for rapid prototyping: even better than Python. One example is that you can modify your code on-the-fly while running it; there is no better debugging tool.

Dynamic redefinition of class methods

Consider the following rectangle class [you can type "lisp" on pcdh91 and then enter my examples]:

(defclass rectangle ()
  ((width :accessor rectangle-width :initarg :width)
   (height :accessor rectangle-height :initarg :height)))

and the following method to calculate rectangle's area:

(defmethod rectangle-area ((r rectangle))
  (* (rectangle-height r) ; typo intended!
     (rectangle-height r)))

now if you test it:

(setf my-test-rect (make-instance 'rectangle :width 80 :height 20))
(rectangle-area my-test-rect)

Lisp prints "400" for you... wait, that's not good... oops and you spot a typo in the rectangle-area function, so you rewrite it without quitting Lisp runtime environment thusly:

(defmethod rectangle-area ((r rectangle))
  (* (rectangle-width r)
     (rectangle-height r)))

and now you can call again the test function, without recreating new instance:

(rectangle-area my-test-rect)

you will receive the correct answer, i.e. 1600.

This small example demonstrates that you can modify a class during runtime and all the existing instances of that class are corrected accordingly! You do not need to quit, recompile, restart, recreate instances and set the same program testing state. This saves a lot of developer's time, and I think that all the other debugging models of Python/OCaml/C/C++/Foo/Bar do not come even close.

Dynamic redefinition with winding/unwinding stack

It's even more impressive if we join dynamic redefinition with winding/unwinding of stack. Unlike other languages that in case of runtime error just stop and unwind the stack, CL offers you full inspecting, patching and recompiling capabilities on the current stack. IOW, you can debug and fix any problem you have and simply go on running. I'll try a small example on this theme.

Consider the classical school drill:

I won't speak loudly during a lesson.
I won't speak loudly during a lesson.
I won't speak loudly during a lesson.
[...]

Let us write a little drill function that calls some phrase generator function f n times and that prints out generated phrases with some noops added for demo purposes:

simko@pcdh91:~$ clisp
[1]> (defun drill (n f)
      "Repeat n times the drill phrase.  Call f to obtain i-th drill
       phrase."
      (dotimes (i n)
        (format t "~%Phrase ~d begins. " i)
        (sleep 1)
        (format t "~s" (funcall f i))
        (sleep 1)
        (format t " Phrase ~d ends." i)))

and let us write a phrase generator to praise my favourite language and to calculate some simple arithmetics:

[2]> (defun drill-phrase-generator (i)
       "Return i-th drill phrase."
       (format nil "I love Java! BTW, ~d*~d=~d." i i (* i i)))

Now let us run it 10 times:

[3]> (drill 10 'drill-phrase-generator)
Phrase 0 begins. "I love Java! BTW, 0*0=0." Phrase 0 ends.
Phrase 1 begins. "I love Java! BTW, 1*1=1." Phrase 1 ends.
Phrase 2 begins. "I love Java! BTW, 2*2=4." Phrase 2 ends.
Phrase 3 begins.

but oops after several repetitions we notice that my favourite programming language was misspelled, and the arithmetics "broken" (say), so let's stop the app with ^C here:

** - Continuable Error
SYSTEM::%SLEEP: User break
If you continue (by typing 'continue'): Continue execution
1. Break [4]>

which brought us to the "application listener" where we can correct ourselves:

1. Break [4]> (defun drill-phrase-generator (i)
                (format nil "I love Common Lisp! BTW, ~d+~d=~d." i i (+ i i)))
DRILL-PHRASE-GENERATOR

and simply continue the execution:

1. Break [4]> continue
"I love Common Lisp! BTW, 3+3=6." Phrase 3 ends.
Phrase 4 begins. "I love Common Lisp! BTW, 4+4=8." Phrase 4 ends.
Phrase 5 begins. "I love Common Lisp! BTW, 5+5=10." Phrase 5 ends.
Phrase 6 begins. "I love Common Lisp! BTW, 6+6=12." Phrase 6 ends.
Phrase 7 begins. "I love Common Lisp! BTW, 7+7=14." Phrase 7 ends.
Phrase 8 begins. "I love Common Lisp! BTW, 8+8=16." Phrase 8 ends.
Phrase 9 begins. "I love Common Lisp! BTW, 9+9=18." Phrase 9 ends.

See the CL power? The application was patched during runtime and continued its course as if nothing happened. Now imagine this example in the middle of a big, slow, and large application, and I hope it's clear how much it helps not to have to quit, recompile, relink, rerun, restore state, and redebug.

Let's redefine drill as well

To see even more CL power, let us redefine the drill function too:

simko@pcdh91:~$ clisp
[1]> (defun drill (n f)
      "Repeat n times the drill phrase.  Call f to obtain i-th drill
       phrase."
      (dotimes (i n)
        (format t "~%Phrase ~d begins. " i)
        (sleep 1)
        (format t "~s" (funcall f i))
        (sleep 1)
        (format t " Phrase ~d ends." i)))

[2]> (defun drill-phrase-generator (i)
       "Return i-th drill phrase."
       (format nil "I love Java! BTW, ~d*~d=~d." i i (* i i)))
DRILL-PHRASE-GENERATOR

[3]> (dotimes (i 10) (drill 3 'drill-phrase-generator))
Phrase 0 begins. "I love Java! BTW, 0*0=0." Phrase 0 ends.
Phrase 1 begins. "I love Java! BTW, 1*1=1." Phrase 1 ends.
Phrase 2 begins. "I love Java! BTW, 2*2=4." Phrase 2 ends.
Phrase 0 begins. "I love Java! BTW, 0*0=0." Phrase 0 ends.
Phrase 1 begins.

** - Continuable Error
SYSTEM::%SLEEP: User break
If you continue (by typing 'continue'): Continue execution

1. Break [4]> (defun drill-phrase-generator (i)
                  (format nil "I love Common Lisp! BTW, ~d+~d=~d." i i (+ i i)))
DRILL-PHRASE-GENERATOR

1. Break [4]> (defun drill (n f)
           "Repeat n times the drill phrase.  Call f to obtain i-th drill
            phrase."
           (dotimes (i n)
             (format t "~%Drill ~d begins. " i)
             (sleep 1)
             (format t "~s" (funcall f i))
             (sleep 1)
             (format t " Drill ~d ends." i)))
DRILL

1. Break [4]> continue

"I love Common Lisp! BTW, 1+1=2." Phrase 1 ends.
Phrase 2 begins. "I love Common Lisp! BTW, 2+2=4." Phrase 2 ends.
Drill 0 begins. "I love Common Lisp! BTW, 0+0=0." Drill 0 ends.
Drill 1 begins. "I love Common Lisp! BTW, 1+1=2." Drill 1 ends.
Drill 2 begins. "I love Common Lisp! BTW, 2+2=4." Drill 2 ends.

Incremental compilation

Now several languages (including Java) try to offer "incremental compilation" IDEs that are directed precisely in this dynamic redefinition area. Kent Pitman has a column on how programming terms often get "redefined" for commercial purposes, and wrote in 1993:

The term ``incremental compilation'' is another one that I've been
sad to see recycled in the marketplace. Here again is another key
feature of Lisp not duplicated in most of its competitors: The
ability to compile and load new code without exiting your running
application.  [...]  To those who have only had full file
compilation before, even this stripped down kind of service may seem
like a real step up.  But to me it's a step down from what I expect
from ``incremental compilation'' since it forces me to exit my
running application. So I think it's an abuse of long-standing
terminology, perhaps even in some cases with a deliberate intent to
mislead.

    -- Kent Pitman, "What's in a Name? Uses and Abuses of Lispy Terminology"
      <http://world.std.com/~pitman/PS/Name.html>

Now in 2002 for Java there is a good-looking candidate, judging by the FAQ of IBM VisualAge + WebSphere Test Environment.

This reminds me again of the Greenspun's tenth rule of programming. In CL the dynamic redefinition process is very natural since built-in into the language by design. In popular languages, Java and Python included, there is no such a powerful built-in capability, so they are left to simulate it, via IDEs or "dirty" language tricks such as Python's new.instancemethod() and friends. Dunno how much successful these simulations are; but why to wait for a feature X provided by IDE Y in year Z, if everything is already built-in right now in the Common Lisp? :-)

lisp