Performance of Distributed Object Systems:

Issues in the Educational Information Architecture

James E. Ries, MS

Department of Health Management and Informatics

University of Missouri

324 Clark Hall

Columbia, MO 65201

Abstract

The Educational Information Architecture (EIA) is a distributed environment designed to support student-centered self-paced learning for computer science education and is being developed at the University of Missouri. The distributed nature of the EIA means that many different sorts of communications are required among its various components. Communication usage scenarios are developed herein to characterize the requirements of the various EIA components.

Each of the EIA communication scenarios has its own needs and tolerances for latency and bandwidth. In addition, EIA implementation can be greatly facilitated by employing object-oriented techniques, including distributed object techniques. These factors can be addressed to varying degrees by Java Remote Method Invocation (RMI), Remote Procedure Calls (RPC), the Distributed Component Object Model (DCOM), the Common Object Request Broker Architecture (CORBA), and raw Java TCP/IP socket calls.

Strengths and weaknesses of these communication techniques are explored, and solutions are proposed for given EIA scenarios. Methodologies for benchmarking and comparing these communications mechanisms are also examined.

1. Introduction

I was fortunate to attend the ACM’s Java Grande conference in San Francisco in the summer of 1999. There, I heard a paper [[1]] presented on replacing Java RMI with a higher performance system. This intrigued me, and it seemed natural to study RMI (and remote procedure mechanisms in general) as applied to the Educational Information Architecture (EIA) [[2]].

The EIA must employ several different sorts of communication among its components and each of these may have their own tolerances for performance. For example, the throughput requirements of EIA’s proposed “student chat room” are minimal, but this subsystem has very little tolerance for latency. Conversely, the transfer of large multimedia files among EIA’s course agents can tolerate large latencies, but requires high throughput (bandwidth) to perform adequately.

Thus, the EIA is an excellent case study for comparing the strengths and weaknesses of various remote procedure call / remote object schemes. This paper provides an overview of several papers which study RMI and other remote procedure call mechanisms with regard to performance, and applies these concepts to the EIA. In addition, a companion study [[3]] based on this framework is being conducted to empirically study the performance of various remote procedure mechanisms in the context of EIA. Candidate communication transports include Java Remote Method Invocation (RMI) [[4]], Distributed Computing Environment (DCE) Remote Procedure Calls (RPC) [[5]], the Distributed Component Object Model (DCOM) [[6]], the Common Object Request Broker Architecture (CORBA) [[7]], and raw Java TCP/IP socket calls.

2. Why EIA Needs Distributed Object Computing

The EIA blueprint [2] indicates that the architecture will be implemented using a distributed object-oriented software environment. This makes a great deal of sense because distributed object computing hides many of the details of distributed applications programming, making implementation much easier for developers. For example, a developer wishing to implement the student-to-student chat facility in EIA need not learn the details of socket programming. Instead, he or she may use one of the popular distributed object computing frameworks to simply make a method call on a remote object which represents a remote student engaged in a given conversation. Because developers are familiar with calling methods on objects, distributed object-oriented computing is a natural abstraction.

The EIA contains many such scenarios including student-to-student chats, content delivery from teaching assistants to students, and content transfer and synchronization among course agents to name a few. Hand-rolling the communication code for each of these systems would be a prohibitive burden for implementers of the system. In addition, custom communication code would tend to interfere with system extensibility. Distributed object computing, on the other hand, is naturally extensible through polymorphism based on interface reference. That is, remote objects supporting the same interface can replace each other with no knowledge or changes on the part of the caller.

3. Java RMI

Clearly, the obvious candidate for initial implementation of EIA is Java, including Java RMI. Java is rapidly becoming the object-oriented general-purpose language of choice, and is doubly popular for network-oriented applications. EIA’s concern with the World Wide Web (WWW), and heterogeneous deployment platforms make Java a natural choice. Of course, the native distributed computing mechanism for Java is Java RMI.

3.1 Strengths

RMI provides a truly object-oriented, as opposed to object-based, distributed computing system [[8]]. That is, Java RMI allows distributed object computing to take place in both the pass by reference sense and the pass by value sense [[9]]. This means that, for Java RMI, systems can be designed which take advantage of polymorphism without regard to deployment of a given class or subclass.

For example, imagine that the EIA makes use of a Java class called CourseContent, and that CourseContent can be serialized and thus passed by value to remote objects. The implementers of EIA might wish to create subclasses of CourseContent such as CourseImageContent or CourseVideoContent. Polymorphism allows such subclasses to be passed to methods that take CourseContent as a parameter. However, in traditional object-based distributed computing (e.g., CORBA or DCOM), this sort of polymorphism is not allowed. This is because CORBA and DCOM (or RPC’s for that matter) require that the stubs generated by their Interface Definition Language (IDL) compiler be able to marshal the binary image of a class (or structure) to be transmitted. Java RMI utilizes this same technique, but improves on it in that the remote (receiving) object need not have the marshalling/unmarshalling code statically linked, but can dynamically retrieve this code (the stub) during the RMI call.

Similarly, true pass by value of objects is not allowed in object-based computing (non-Java RMI object computing). That is, as mentioned, the receiving side of an object must have the implementation of the object a-priori (it must be statically linked in). Only the data contained in the object is actually marshalled and passed on the wire. So, in general, one cannot pass an object whose implementation is not already available to the receiving object. Java RMI, conversely, can marshal the code as well as the data for an object since Java byte codes are not platform dependent.

3.2 Weaknesses

Java RMI is notoriously slow [1] [[10]] in a number of ways. Java itself is often criticized for its performance due to its interpreted nature. Since Java runs on a Java Virtual Machine (VM), there is a significant overhead in interpreting the byte codes as opposed to native compilation. Recently, however, Just in Time (JIT) compilers have become available for Java, which have markedly improved the situation for many applications.

Another issue that can adversely affect Java performance in general is garbage collection. In any Java program, and certainly in a program using Java RMI, garbage collection can take a large slice of processing time if many objects are be frequently created. The Java garbage collection algorithm will tend to run when low memory situations are encountered. In the case of always-on servers (e.g., those in the EIA), it is a certainty that garbage collection will eventually run. Thus, for servers that create and discard large numbers of objects, the overhead of Java garbage collection may be significant.

RMI specifically can encounter performance issues due to its serialization mechanism, its interaction with TCP/IP (on which it rests), and also due to its lack of any object-caching mechanism. We will explore these issues later in the paper.

4. Alternatives to RMI

We have seen that Java RMI is easy to use for developers and supports all of the common object-oriented paradigms with which developers are familiar. However, Java RMI can be unacceptably slow in performance for some applications. This section suggests alternatives to “native” Java RMI (the version of RMI that is standard with Java 1.2), and explores the trade-offs inherent in these alternatives.

4.1 RMI Enhanced Systems

Since RMI has several clear advantages as a distributed computing system, it seems appropriate to try to address RMI's performance issues rather than replace RMI altogether. Two papers discuss modifications to RMI that can enhance its performance [1] [10].

One method for improving RMI's performance is to change the underlying protocol on which it currently rests (TCP). By using UDP as the base protocol, and implementing a custom mechanism to guarantee reliability on top if it, it may be possible to enhance RMI performance. Since RMI uses a "request-response" model, a datagram-based protocol can be developed which waits for the response to piggyback acknowledgement of the original message. This avoids the overhead of the additional acknowledgements that TCP would insert into the communication stream. Such a scheme ("R-UDP") is discussed in [10] and is based on similar work with regard to RPC's By Birrell and Nelson [[11]]. KaRMI [1] also utilizes this technique. However, KaRMI removes RMI's exposed socket layer so that other protocols can be utilized. This makes KaRMI more general in a sense, but may break backward compatibility for some applications.

Another method for improving performance also discussed in [10] is object caching. The idea here is to keep a cached copy of objects that would otherwise have to be transmitted across the wire. Thus, the first reference to a cacheable object would create an instance of it locally. Future references could be directed to the local object rather than going through the network. Of course, some objects would be "un-cacheable", such as objects that must run for one reason or another on a specific machine.

Some of RMI's overhead is actually due to Java's serialization mechanisms. An improvement on Java serialization is discussed in [1]. The replacement serialization method discussed here includes what the authors call "slim encoding of type information". This means that types (classes) transmitted across the wire need not include their ByteCode. Unfortunately, this idea rests on the assumption that a common file system is available to all nodes, which may not be the case for EIA (though it is the case in the Distributed Computing Lab test environment). Additional proposed enhancements to Java serialization includes improved buffering. Just as a custom protocol can be used to take advantage of RMI characteristics, a custom stream-buffering technique can be employed to take advantage of the way RMI works as well. Of course, such a custom approach trades modularity for speed.

4.2 Using a Custom Protocol In Place of RMI

It may be worthwhile to write a custom TCP/IP socket implementation (or other protocol such as RPC) for some situations in the EIA. For example, if one is primarily transmitting base data types (e.g., int, boolean, etc.) it may be advantages to avoid the Java serialization layer and transmit this data by hand. Of course, this sort of performance solution would require EIA developers to deal with all of the messy socket issues that RMI could abstract away. The efficacy of such an approach will be tested in the companion work [3].

4.3 Other Distributed OO Systems

Other distributed OO systems besides Java RMI exist and may have different strengths and weaknesses than RMI. CORBA is directly supported in the JDK and can easily be evaluated as an alternative to RMI for many EIA scenarios. Other OO protocols would have to be called from Java using the Java Native Interface (JNI), and this would certainly add a degree of complexity. It is planned that both of these approaches will be tested in the companion work [3].

5. Measuring Performance

It is very difficult to measure and compare performance of network applications due to their very dynamic nature. A change in a given protocol, for example, might show improvement due to low network traffic or CPU load at the time of the test as compared to conditions at the time of other tests.

In order to overcome such situations, there are essentially 4 alternatives. One can do many repetitions of all tests in the hope that a large sample size will normalize outside factors over the course of the trials. Of course, a large number of trials may greatly lengthen the time needed to conduct the study and even then may not guarantee that outside factors are not influencing performance. One can build an isolated testing environment which is dedicated to the testing scenarios at hand and thus unaffected by external jobs, network traffic, etc. This alternative is expensive in the sense that one is dedicated a large number of resources (network adapters, cable, machines, etc.) to a task and may not be feasible in many environments. One may use analytical techniques. However, this can be very difficult for real systems and my not even be possible.

Finally, one can create a simulation environment [[12]][[13]]. A simulation environment can be very advantageous in that it is unaffected by external forces or can carefully control the desired external force interaction in the study. In addition, a simulation environment does not burden other users of a computer system to the degree that empirical tests can. Of course, accurate simulations are very difficult to create and it is very difficult to prove that a given simulation is, indeed, accurate.

For the EIA, it seems best to perform simple empirical benchmarking tests to evaluate the competing transport mechanisms. Since throughput and latency are the two primary areas of concern, one can wrap the transports with timing and measurement mechanisms and use these to evaluate performance. As suggested previously, however, it will likely be necessary to run multiple evaluations of each prototype in order to factor out external factors.

7. Conclusion

The EIA is a fertile testing ground for comparing performance of various distributed computing transport mechanisms. While EIA naturally lends itself to implementation in Java, there are many techniques that can be employed to plug-in alternative transports to RMI such as CORBA, DCE RPC's, DCOM, or direct TCP/IP sockets. For some scenarios in the EIA, alternative transports may be important in order to achieve acceptable system performance. An empirical study [3] of several transports is being conducted in the context of EIA, and it is hoped that techniques developed in this paper can be used to clearly evaluate this issue.

Papers studied in detail

Wollrath, A., Waldo, J., Riggs, R., “Java-Centric Distributed Computing” [8].

Waldo, J., “Remote procedure calls and Java Remote Method Invocation” [9].

Rolia, J.A., Starkey, M., Boersma, G., “Modeling RPC performance” [12].

Nester, C., Philippsen, M., Haumacher, B., “A more efficient RMI for Java” [1], Proceedings of the ACM 1999 conference on Java Grande, June 12-14, 1999, Palo Alto, CA, USA.

Krishnaswamy, V., Walther, D., Bhola, S., Bommaiah, E. Riley, G., Topol, B., Ahamad, M., “Efficient Implementations of Java Remote Method Invocation (RMI)” [10].

Dumas, S., Gardarin, G., “A workbench for predicting the performances of distributed object architectures” [13].