A Study of RMI Performance and Alternative RPC mechanisms in the Java Environment for a High Performance Educational Information Architecture
Final Project for CECS 486, Fall Semester 1999
By
James E. Ries, MS
Department of Health Management and Informatics
University of Missouri
324 Clark Hall
Columbia, MO 65201
Introduction
The Educational Information Architecture (EIA) relies on multiple distributed agents to provide services to students and course facilitators. The EIA specification states that implementation will be done using Java, XML, and CORBA. However, it seems probable that the ease of use of Java RMI will make this transport an early foundation for the development of EIA. RMI abstracts away the details of low-level communication, allowing the programmer to focus on more high-level issues. However, Java RMI is notoriously deficient in regard to performance. While performance may not be critical to many aspects of EIA, it will probably be important to the transfer of large multimedia data (e.g., video) and also to agent interactions which involve large numbers of small messages (e.g., student chats). These issues may be especially relevant as EIA scales up to be an infrastructure for national or even global education. In addition, EIA is targeted to serve underrepresented groups and institutions that may not have high speed networking resources. This also points to the importance of optimizing the performance of EIA.
This project studies the performance of Java RMI and alternative transports under a variety of conditions that may be encountered in the EIA. I examine 3 scenarios that characterize common communications within the EIA: 1) Large file data transfers such as is commonly seen in Course Agent to Course Agent synchronization, 2) Mixed size data transfers as is commonly seen in Course Agent to Teaching Agent transfers of content, and 3) Small and numerous data transfers as might be seen in student chats. In each of these scenarios Java RMI is compared to alternative transports. Alternative transports considered are KaRMI (see Nester, Philipsen, and Haumacher, "A More Efficient RMI for Java", Proceedings of the ACM 1999 conference on Java Grande, pp. 152-159) and hand-rolled TCP/IP socket transport. Each transport is compared under varying availability of bandwidth in an attempt to mirror the varying conditions that exist in EIA usage (i.e., DSL network and across the campus backbone to my office).
Testing Model
As mentioned, I designed 3 scenarios to characterize various sorts of communication present in the EIA. To simulate these 3 scenarios, I created directories containing a small number of image files (CA_To_CA), a large number of small text files (Chat), and mixed file sizes and types (TA_To_PA). I then tested each of the candidate transports (Java RMI, Sockets, and KaRMI) on each of these sets of files and recorded the results. I repeated these tests in all of the combinations of my NT machine, and two DSL machines.
Demonstration
A live demonstration is planned as part of the submission of this project. However, I also include a screen-shot here of a sample run to give the flavor of the project. The screen-shot shown below depicts an interaction between a server running on an NT 4.0 workstation (riesj.hmi.missouri.edu) and a client running on a Sun Solaris box (amberjack.cecs.missouri.edu).
Notice that the results of this particular run show that the Socket transport performs better on the small file sample (which is bounded by latency), while the RMI transport is equal or better on large and mixed size files (which are bounded by through-put). Notice also that the apparent error message ("java.io.EOFException: Expecting code") is expected since our method of determining the end of a particular run is by getting a null object.
Performance Metrics and Analysis
The following table shows the performance of the various tests. All times are in milliseconds.
The following graph depicts the same data from the above table. One can clearly see that Socket is typically the fastest transport, KaRMI comes in second, and native Java RMI third.
Although it is clear from the data that a custom-rolled socket transport outperforms both RMI and the KaRMI "drop-in" replacement for RMI, the performance differences for typical EIA scenarios is really not relevant. For example, can users really tell a noticeable difference between 3 and 25 millisecond response times in a chat? I sincerely doubt it. Even in the case of transferring large files, a few hundred milliseconds improvement is not likely to be worth the development overhead of employing custom socket programming over RMI. Likewise, the difficulty in deploying KaRMI on all users' machines probably negates the minor performance advantages that could be gained.
One caveat to consider, however, is that these tests were all conducted using local area network and fast wide area network connections. If time had permitted, I had hoped to conduct some additional testing using a 56 Kbit dial-up connection connecting to the commodity Internet. At such low speeds, some of these differences may become meaningful. For example, if it takes 10 minutes to transfer a large file using RMI and only 5 minutes using Sockets, this would be quite significant even though the same percentage difference at millisecond speeds would be negligible to a user!
Future Work
As mentioned, it would be very instructive to consider the effect of slower network connections such as 56 Kbit dial-up connections. Such connections could not be tested here due to time constraints. However, it is worth noting that the recent growth in consumer popularity of Digital Subscriber Line and cable modems (e.g., "AT&T @ Home") may make even home network connections sufficiently fast as to be comparable with the network topologies studied here.
The initial proposal for this study recommended considering Distributed Computing Environment (DCE) Remote Procedure Calls (RPC). The University of Missouri Distributed Computing Laboratory (DSL) is not currently equipped with the necessary DCE runtime to evaluate RPC. DCE RPC is available in the base installations of most popular UNIX implementations, with the notable exception of SUN Solaris (which is deployed in the DSL). SUN instead supports ONC RPC, which is not cross-platform and with which the author has no experience. Thus, this transport is currently left for future consideration. It is expected that this transport would perform well for large-sized transfers due to its highly optimized DCE pipe technology. However, the overhead of calling RPC from Java through the Java Native Interface (JNI) may be a mitigating factor.
Likewise, it was initially hoped that this study could include a look at alternative distributed object technologies such as CORBA and DCOM. These too are left to future work. In some circumstances, these technologies may provide performance benefits over native Java RMI.
Lastly, the role of compression is also left to future studies in this area. It would be very interesting to see whether large file transfers, in particular, could be improved through the use of compression. For the EIA in particular, this is probably not the case, as most large files will tend to be graphic or video files. Since files of these types are typically already in a compressed format, additional compression will not likely have a great effect.
Conclusion
While performance considerations for network transports can be very important to given applications and to the overall performance of a network as a whole, differences in the performance of particular transports become insignificant as technology advances. Differences that may have been relevant on very slow, older networks become immaterial when newer, faster networks are deployed.
In the case of the EIA, it is certainly important to investigate transport performance up front in order to determine its relevance. The results of this study do show that transport performance varies significantly as a percentage. However, the real time required to do any of the tested transactions is sufficiently low that I recommend implementation of EIA should be done using the slowest (but most convenient) tested transport, Java RMI.
Of course, these findings could change if slower networks (e.g., dial-up connections) are important in a real world deployment of EIA. Additionally, if EIA is to include whole file transfers of huge video files, the conclusions herein could be affected.
Appendix: Code
The Java source code for this project is attached on the following pages.