Performance Results

We performed a few simple benchmarks to investigate the performance of our extended interpreter. These results must be considered preliminary, since we have not tuned our interpreter performance. Also, since the Java interpreter we were modifying runs at only a fraction of the speed of machine code, these results cannot be definitive. However, the results suggest that parameterized code adds little overhead to Java and offers performance improvements when its use is appropriate.

The results of our benchmarks are shown in Figures 8, 10, and 11. The benchmarks were run several times, so measurement variation is smaller than the significant digits shown. All run times are in seconds.

First, we confirmed that our extended interpreter did not significantly slow the execution of ordinary unparameterized code, by running the Java compiler javac (itself a large Java program) on both interpreters. As shown in Figure 8, the extended interpreter runs only 2% slower than the original interpreter on non-parameterized code.

We also ran some small benchmarks to compare the speed of parameterized and non-parameterized code, using a simple parameterized collection class and some non-parameterized equivalents. In these comparisons, the code was almost identical, except for changes in type declarations and a dynamic cast. The code used repeatedly extracts an object from a parameterized collection and invokes a method on that object, as shown in Figure 9.

    Cell[Element] c = new Cell[Element]();
    c.add(new Element());
    for (i = 0; i < 1000000; i++) {
	Element t = c.get();
	t.do_method();
    }

In the first micro-benchmark, we compared the performance of the parameterized class Cell[T]; to a hard-wired ElementCell class, which can only contain objects of type Element. Both of these classes represent a simple collection that contains only one element. Using this simple collection isolates the performance effects that we want to measure. The hard-wired class, which is less generally useful, will obviously run faster. But as Figure 10 shows, there is only a 4% penalty for running parameterized code.

Figure 10: Collection Class Results

In the second micro-benchmark, we compared the parameterized class to the same collection class where the types of the components are all Object - the option for old-style Java code that the existing Java utility classes mostly follow. This version of the collection class gives up some static type checking, and also requires that the programmer write an explicit type cast when extracting values from the collection (in order to call do_method). Figure 10 shows that for our benchmark, this approach is 17% slower than using parameterized types.

Figure 11: Invocation Speed

In the third micro-benchmark, we compared the speed of invoking where-routines (with the invokewhere bytecode), ordinary methods (with invokevirtual), and interface methods (with invokeinterface). Our test program intensively called these respective bytecodes. The results are shown in Figure 11. In our results, all three method invocation paths are within 6% of each other. This experiment represented a best case for invokeinterface, since its inline caching worked perfectly.

Although these performance results are produced by modifying an interpreter, we believe that the advantages of the extended bytecodes and of parameterized code will only increase with compiled machine code, since the interpreter tends to add a constant overhead to all operations. The slight overhead of Figure 8 is not intrinsic to parameterized code and can be eliminated with more careful implementation. The advantage of parameterized code shown in Figure 10 will be more dramatic, particularly since the cost of the runtime cast from Object will be relatively more expensive than method invocation.