Hostname: page-component-8448b6f56d-t5pn6 Total loading time: 0 Render date: 2024-04-19T14:17:33.980Z Has data issue: false hasContentIssue false
  • English
  • Français

Benchmarking implementations of functional languages with ‘Pseudoknot’, a float-intensive benchmark

Part of: JFP Research Articles

Published online by Cambridge University Press:  07 November 2008

Pieter H. Hartel ,
Marc Feeley ,
Martin Alt ,
Lennart Augustsson ,
Peter Baumann ,
Marcel Beemster ,
Emmanuel Chailloux ,
Christine H. Flood ,
Wolfgang Grieskamp  and
John H. G. Van Groningen
...Show all authors
Pieter H. Hartel
Affiliation:
Department of Computer Systems, University of Amsterdam, Kruislaan 403, 1098 SJ Amsterdam, The Netherlands (e-mail: pieter@fwi.uva.nl)
Marc Feeley
Affiliation:
Département d'informatique et r.o., Université de Montréal, succursale centre-ville, Montréal H3C 3J7, Canada (e-mail: feeley@iro.umontreal.ca)
Martin Alt
Affiliation:
Informatik, Universität des Saarlandes, 66041 Saarbrücken 11, Germany (e-mail: alt@cs.uni-sb.de)
Lennart Augustsson
Affiliation:
Department of Computer Systems, Chalmers University of Technology, 412 96 Göteborg, Sweden (e-mail: augustss@cs.chalmers.se)
Peter Baumann
Affiliation:
Department of Computer Science, University of Zurich, Winterthurerstr. 190, 8057 Zurich, Switzerland (e-mail: baumann@ifi.unizh.ch)
Marcel Beemster
Affiliation:
Department of Computer Systems, University of Amsterdam, Kruislaan 403, 1098 SJ Amsterdam, The Netherlands (e-mail: beemster@fwi.uva.nl)
Emmanuel Chailloux
Affiliation:
LIENS, URA 1327 du CNRS, École Normale Supérieure, 45 rue d'Ulm, 75230 PARIS Cédex 05, France (e-mail: Emmanuel.Chailloux@ens.fr)
Christine H. Flood
Affiliation:
Laboratory for Computer Science, MIT, 545 Technology Square, Cambridge, MA 02139, USA (e-mail: chf@lcs.mit.edu)
Wolfgang Grieskamp
Affiliation:
Berlin University of Technology, Franklinstr. 28-29, 10587 Berlin, Germany, (e-mail: wg@cs.tu-berlin.de)
John H. G. Van Groningen
Affiliation:
Faculty of Mathematics and Computer Science, Univ. of Nijmegen, Toernooiveld 1, 6525 ED Nijmegen, The Netherlands (e-mail: johnvg@cs.kun.nl)
Kevin Hammond
Affiliation:
Department of Computing Science, Glasgow University, 17 Lilybank Gardens, Glasgow, GI2 8QQ, UK (e-mail: kh@dcs.glasgow.ac.uk)
Bogumil Hausman
Affiliation:
Computer Science Lab, Ellemtel Telecom Systems Labs, Box 1505, S-125 25 Älvsjö, Sweden (e-mail: bogdan@erix.ericsson.se)
Melody Y. Ivory
Affiliation:
Computer Research Group, Institute for Scientific Computer Research, Lawrence Livermore National Laboratory, P.O. Box 808 L-419, Livermore, CA 94550, USA (e-mail: ivoryl@llnl.gov)
Richard E. Jones
Affiliation:
Department of Computer Science, University of Kent at Canterbury, Canterbury, Kent, CT2 7NF, UK (e-mail: R.E.Jones@ukc.ac.uk)
Jasper Kamperman
Affiliation:
CWI, Kruislaan 413, 1098 SJ Amsterdam, The Netherlands (e-mail: jasper@cwi.nl)
Peter Lee
Affiliation:
Department of Computer Science, Carnegie Mellon University, 5000 Forbes Avenue Pittsburgh, Pennsylvania 15213, USA (e-mail: petel@cs.cmu.edu)
Xavier Leroy
Affiliation:
INRIA Rocquencourt, projet Cristal, B.P. 105, 78153 Le Chesnay, France (e-mail: Xavier.Leroy@inria.fr)
Rafael D. Lins
Affiliation:
Departamento de lnformática, Universidade Federal de Pernambuco, Recife, PE, Brazil (e-mail: rdl@di.ufpe.br)
Sandra Loosemore
Affiliation:
Department of Computer Science, Yale University, New Haven, CT, USA (e-mail: loosemore-sandra@cs.yale.edu)
Niklas Röjemo
Affiliation:
Department of Computer Systems, Chalmers University of Technology, 412 96 Göteborg, Sweden (e-mail: rojemo@cs.chalmers.se)
Manuel Serrano
Affiliation:
INRIA Rocquencourt, projet Icsla, B.P. 105, 78153 Le Chesnay, France (e-mail: Manuel.Serrano@inria.fr)
Jean-Pierre Talpin
Affiliation:
European Computer-Industry Research Centre, Arabella Straβe 17, D-81925 Munich, Germany (e-mail: jp@ecrc.de)
Jon Thackray
Affiliation:
Harlequin Ltd, Barrington Hall, Barrington, Cambridge CB2 5RG, UK (e-mail: jont@harlequin.co.uk)
Stephen Thomas
Affiliation:
Department of Computer Science, University of Nottingham, Nottingham, NG7 2RD, UK (e-mail: spt@cs.nott.ac.uk)
Pum Walters
Affiliation:
CWI, Kruislaan 413, 1098 SJ Amsterdam, The Netherlands (e-mail: pum@cwi.nl)
Pierre Weis
Affiliation:
INRIA Rocquencourt, projet Cristal, B.P. 105, 78153 Le Chesnay, France (e-mail: Pierre.Weis@inria.fr)
Peter Wentworth
Affiliation:
Department of Computer Science, Rhodes University, Grahamstown 6140, South Africa (e-mail: cspw@cs.ru.ac.za)
Rights & Permissions [Opens in a new window]

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Over 25 implementations of different functional languages are benchmarked using the same program, a floating-point intensive application taken from molecular biology. The principal aspects studied are compile time and execution time for the various implementations that were benchmarked. An important consideration is how the program can be modified and tuned to obtain maximal performance on each language implementation. With few exceptions, the compilers take a significant amount of time to compile this program, though most compilers were faster than the then current GNU C compiler (GCC version 2.5.8). Compilers that generate C or Lisp are often slower than those that generate native code directly: the cost of compiling the intermediate form is normally a large fraction of the total compilation time. There is no clear distinction between the runtime performance of eager and lazy implementations when appropriate annotations are used: lazy implementations have clearly come of age when it comes to implementing largely strict applications, such as the Pseudoknot program. The speed of C can be approached by some implementations, but to achieve this performance, special measures such as strictness annotations are required by non-strict implementations. The benchmark results have to be interpreted with care. Firstly, a benchmark based on a single program cannot cover a wide spectrum of ‘typical’ applications. Secondly, the compilers vary in the kind and level of optimisations offered, so the effort required to obtain an optimal version of the program is similarly varied.

Type
Articles
Information
Journal of Functional Programming , Volume 6 , Issue 4 , July 1996 , pp. 621 - 655
Copyright
Copyright © Cambridge University Press 1996

References

Alt, M., Fecht, C, Ferdinand, C. and Wilhelm, R. (1993) The Trafola-S subsystem. In Hoffmann, B. and Krieg-Brückner, B., editors, Program development by specification and transformation, LNCS 680, pp. 539–576. Berlin: Springer-Verlag.Google Scholar
Appel, A. W. (1992) Compiling with Continuations. Cambridge: CUP.Google Scholar
Armstrong, J., Williams, M. and Virding, R.(1993) Concurrent programming in Erlang. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
Augustsson, L. (1993) HBC user's manual. Programming Methodology Group Distributed with the HBC compiler, Department of Computer Science, Chalmers University of Technology, Sweden.Google Scholar
Augustsson, L. and Johnsson, T. (1989) The Chalmers Lazy-ML compiler. The Computer Journal, 32(2),127–141.CrossRefGoogle Scholar
Augustsson, L. and Johnsson, T. (1990) Lazy ML user's manual. Programming methodology group report, Department of Computer Science, Chalmers University of Technology, Sweden.Google Scholar
Beemster, M. (1992) The lazy functional intermediate language Stoffel. Technical report CS-92-16, Department of Computer Systems, University of Amsterdam.Google Scholar
Beemster, M. (1993) Optimizing transformations for a lazy functional language. In Withagen, W.-J., editor, 7th Computer systems, pp. 17–40, Eindhoven University of Technology, The Netherlands.Google Scholar
Bergstra, J. A., Heering, J. and Klint, P. (1989) Algebraic Specification,. New York: ACM Press (in co-operation with Addison-Wesley).Google Scholar
Cann, D. C. (1992a) The optimizing SISAL compiler: version 12.0. Manual UCRL-MA-110080, Lawrence Livermore National Laboratory, Livermore, CA.Google Scholar
Cann, D. C. (1992b) Retire FORTRAN? a debate rekindled. Comm. ACM, 35(8), 81–89.CrossRefGoogle Scholar
Chailloux, E. (1992) An efficient way of compiling ML to C. In Lee, P., editor, ACM SIGPLAN Workshop on ML and its Applications, pp. 37–51, San Francisco, CA. (School of Computer Science, Carnegie Mellon University, Technical report CMU-CS-93-105.)Google Scholar
Didrich, K., Fett, A., Gerke, C, Grieskamp, W. and Pepper, P. (1994) OPAL: Design and implementation of an algebraic programming language. In Gutknecht, J., editor, Programming Languages and System Architectures, LNCS 782, pp. 228–244. Berlin: Springer-Verlag.Google Scholar
Diwan, A., Tarditi, D. and Moss, E. (1994) Memory subsystem performance of programs with copying garbage collection. In 21st Principles of Programming Languages, pp. 1–14, Portland, OR. New York:ACM.Google Scholar
Feeley, M. and Miller, J. S. (1990) A parallel virtual machine for efficient Scheme compilation. In Lisp and Functional Programming, pp. 119–130, Nice, France. New York: ACM.Google Scholar
Feeley, M., Turcotte, M. and Lapalme, G. (1994) Using Multilisp for solving constraint satisfaction problems: an application to nucleic acid 3D structure determination. Lisp and Symbolic Computation, 7(2/3), 231–246.Google Scholar
Giegerich, R. and Hughes, R. J. M. (1994) Functional programming in the real world. Dagstuhl seminar report 89, IBFI GmbH, Schloss Dagstuhl, D-66687 Wadern, Germany.Google Scholar
Gill, A. J. and Peyton Jones, S. L. (1994) Cheap deforestation in practice: An optimiser for Haskell. In Proc. IFIP, Vol. 1, pp. 581–586, Hamburg, Germany.Google Scholar
Gopinath, K. and Hennesy, J. L. (1989) Copy elimination in functional languages. In 16th Principles of Programming Languages, pp. 303–314, Austin, TX. New York: ACM.Google Scholar
The Yale Haskell Group (1994) The Yale Haskell Users Manual (version Y2.3b). Department of Computer Science, Yale University (Distributed with the Yale Haskell compiler), July.Google Scholar
Halstead, R. H. Jr (1985) Multilisp: A language for concurrent symbolic computation. ACM Trans. on Programming Languages and Systems, 7(4), 501–538.Google Scholar
Harlequin, (1994) MLWorks draft documentation. Harlequin Ltd, Cambridge, UK.Google Scholar
Hartel, P. H., Glaser, H. W. and Wild, J. M. (1994) Compilation of functional languages using flow graph analysis. Software—Practice and Experience, 24(2), 127–173.CrossRefGoogle Scholar
Hartel, P. H. and Langendoen, K. G. (1992) Benchmarking implementations of lazy functional languages. In 6th Functional Programming Languages and Computer Architecture, pp. 341–349, Copenhagen, Denmark. New York: ACM.Google Scholar
Hausman, B. (1994) Turbo Erlang: Approaching the speed of C. In Tick, E. and Succi, G., editors, Implementations of Logic Programming Systems, pp. 119–135. Dordrecht: Kluwer.CrossRefGoogle Scholar
Hudak, P., Peyton Jones, S. L. and Wadler, P. L. (editors). (1992) Report on the programming language Haskell – a non-strict purely functional language, version 1.2. ACM SIGPLAN Notices, 27(5), R1–R162, 05.Google Scholar
Jones, M. P. (1994) The implementation of the Gofer functional programming system. Research Report YALEU/DCS/RR-1030, Department of Computer Science, Yale University.Google Scholar
Kernighan, B. W. and Ritchie, D. W. (1988) The C Programming Language – ANSI C. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
Leroy, X. (1992) Unboxed objects and polymorphic typing. In 19th Principles of Programming Languages, pp. 177–188, Albuquerque, NM. New York: ACM.Google Scholar
Leroy, X. (1993) The Caml Light system, release 0.61. Software and documentation distributed by anonymous FTP on ftp.inria.fr.Google Scholar
Lins, R. D. (1987) Categorical Multi-Combinators. In Kahn, G., editor, 3rd Functional Programming Languages and Computer Architecture, LNCS 274, pp. 60–79, Portland, OR. Berlin: Springer-Verlag.Google Scholar
Lins, R. D. and Lira, B. O. (1993) ΓCMC: A novel way of implementing functional languages. J. Programming Languages, 1(1), 19–39.Google Scholar
MacLachlan, R. A. (1992) CMU common Lisp user's manual. Technical report CMU-CS-92-161, School of Computer Science, Carnegie Mellon University.CrossRefGoogle Scholar
McGraw, J. R., Skedzielewski, S. K., Allan, S., Oldehoeft, R., Glauert, J. R. W., Kirkham, C, Noyce, B. and Thomas, R. (1985) Sisal: Streams and iteration in a single assignment language. Language reference manual version 1.2 M-146, Rev. 1, Lawrence Livermore National Laboratory, Livermore, CA.Google Scholar
Milner, R., Tofte, M. and Harper, R. (1990) The Definition of Standard ML. Cambridge, MA: MIT Press.Google Scholar
Nikhil, R. S. (1991) ID version 90.1 reference manual. Computation Structures Group Memo 284–2, Laboratory for Computer Science, MIT, Cambridge, MA.Google Scholar
Peyton Jones, S. L., Hall, C. V., Hammond, K., Partain, W. D. and Wadler, P. L. (1993) The Glasgow Haskell compiler: a technical overview. In Proc. Joint Framework for Information Technology (JF1T) Technical Conference, pp. 249–257, Keele, UK. London:DTI/SERC.Google Scholar
Peyton Jones, S. L. and Launchbury, J. (1991) Unboxed values as first class citizens in a non-strict functional language. In Hughes, R. J. M., editor, 5th Functional Programming Languages and Computer Architecture, LNCS 523, pp. 636–666, Cambridge, MA. Berlin: Springer-Verlag.Google Scholar
Plasmeijer, M. J. and van Eekelen, M. C. J. D. (1994) Concurrent Clean – version 1.0 – Language Reference Manual, draft version. Department of Computer Science, University of Nijmegen, The Netherlands.Google Scholar
Ranelletti, J. E. (1987) Graph transformation algorithms for array memory memory optimization in applicative languages. PhD thesis, Computer Science Deptartment, University of California at Davis, CA.Google Scholar
Rees, J. A. and Clinger, W. (1991) Revised4 Report on the Algorithmic Language Scheme. MIT, Cambridge, MA.Google Scholar
Röjemo, N. (1995) Highlights from nhc – a space-efficient Haskell compiler. In 6th Functional Programming Languages and Computer Architecture, pp. 282–292, La Jolla, CA. New York: ACM.Google Scholar
Schulte, W. and Grieskamp, W. (1991) Generating efficient portable code for a strict applicative language. In Darlington, J. and Dietrich, R., editors, Phoenix Seminar and Workshop on Declarative Programming, pp. 239–252, Sasbachwalden, West Germany. Berlin: Springer-Verlag.Google Scholar
Serrano, M. (1994) Bigloo user's manual. Technical report 0169, INRIA-Rocquencourt, France.Google Scholar
Serrano, M. and Weis, P. (1994) 1 + 1 = 1: an optimizing Caml compiler. In ACM-SIGPLAN Workshop on ML and its Applications, pp. 101–111. (Research report 2265, INRIA Rocquencourt, France, June.)Google Scholar
Shao, Z. (1994) Compiling Standard ML for Efficient Execution on Modern Machines. PhD thesis, Princeton Univ, Princeton, NJ.Google Scholar
Smetsers, S., Nöcker, E. G. J. M. H., van Groningen, J. and Plasmeijer, M. J. (1991) Generating efficient code for lazy functional languages. In Hughes, R. J. M., editor, 5th Functional Programming Languages and Computer Architecture, LNCS 523, pp. 592–617, Cambridge, MA. Berlin: Springer-Verlag.CrossRefGoogle Scholar
Steele, G. L. Jr. (1990) Common Lisp the Language. Bedford: Digital Press.Google Scholar
Thomas, S. (1993) The Pragmatics of Closure Reduction. PhD thesis, University of Kent at Canterbury, UK.Google Scholar
Thomas, S. (1995) The OP-TIM – a better PG-TIM. Technical report NOTTCS-TR-95-7, Department of Computer Science, University of Nottingham, UK.Google Scholar
Thompson, S. (1986) Laws in Miranda. In Lisp and Functional Programming, pp. 1–12, Cambridge, MA. New York: ACM.Google Scholar
Thomsen, B., Leth, L., Prasad, S., Kuo, T.-S., Kramer, A., Knabe, F. and Giacalone, A. (1993) Facile antigua release – programming guide. Technical report ECRC-93-20, European Computer-Industry Research Centre, Munich, Germany. (The reference manual and license agreement are available by anonymous ftp from ftp.ecrc.de.)Google Scholar
Turner, D. A. (1985) Miranda: A non-strict functional language with polymorphic types. In Jouannaud, J.-P., editor, 2nd Functional Programming Languages and Computer Architecture, LNCS 201, pp. 1–16, Nancy, France. Berlin: Springer-Verlag.Google Scholar
Turner, D. A. (1990) Miranda system manual. Research Software Ltd, 23 St Augustines Road, Canterbury, Kent CT1 1XP, UK, April.Google Scholar
Wadler, P. L. (1990) Deforestation: transforming programs to eliminate trees. Theoretical Computer Science, 73(2), 231–248.Google Scholar
Walters, H. R. and Kamperman, J. F. Th. (1995) Epic: Implementing language processors by equational programs. Technical report in preparation, Centrum voor Wiskunde en Informatica, Amsterdam, The Netherlands.Google Scholar
Weis, P. and Leroy, X. (1993) Le langage Caml. InterÉditions.Google Scholar
Wentworth, E. P. (1991) Code generation for a lazy functional language. Technical report 91/19, Department of Computer Science, Rhodes University.Google Scholar
Wentworth, E. P. (1992) RUFL reference manual. Technical report 92/1, Department of Computer Science, Rhodes University.Google Scholar
Submit a response

Discussions

No Discussions have been published for this article.