Sunday, 27 March 2011

Pros and cons of OCaml

The advantages and disadvantages of the OCaml programming language.


  • More powerful type inference than any other language: OCaml even infers union and class types!

  • Powerful module system lets you parameterize modules over other modules easily and safely with full compile-time checking and minimal run-time overhead.

  • Structural typing of modules, polymorphic variants and classes improves brevity, closing the gap with dynamic languages, but can obfuscate error messages.

  • Powerful integrated macro system lets you alter the language's syntax on-the-fly and write parsers quickly and easily.

  • Good serial performance from the x64 code gen.

  • Easy-to-use REPL.

  • Lots of high-quality tools and libraries.


  • No overloading can make numerical code over many different types (e.g. complex numbers, vectors and matrices) tedious.

  • Poor multicore support means poor performance on today's computers.

  • Some basic functionality missing (e.g. no 32-bit floats, no try..finally construct).

  • Some basic functionality is very inefficient (e.g. 32-bit ints).

  • Poor floating point performance from the x86 code gen.

  • 16MB limit on strings and arrays on 32-bit systems.

  • No way to override equality, comparison and hashing for new types and no way to catch errors created by this when the default structural versions are not applicable. Type classes are a solution to the former problem and equality types are a solution to the latter.

  • No value types means more memory use and worse performance in some important situations (e.g. filling a hash table with float keys and values in OCaml is 17× slower than F#).

Sunday, 13 March 2011

Alternatives to Numerical Recipes

The Jet Propulsion Laboratory at Nasa once hosted an interesting web page listing better alternatives to the infamous book Numerical Recipes. Here is a copy courtesy of The Wayback Machine:

There is no single alternative to Numerical Recipes. The authors of Numerical Recipes provide a superficial overview of a large amount of material in a small volume. In order to do so, they made many unfortunate compromises.

It is naïve to hope that every computational problem can be solved by a simple procedure that can be described in a few pages of chatty prose, and using a page or two of Fortran or C code. Today's ambitions for correctness, accuracy, precision, stability, "robustness", efficiency, etc. demand sophisticated codes developed by experts with deep understanding of their disciplines. We have long ago outgrown the capabilities of the simplistic approaches of 30 years ago.

Steve Sullivan has constructed a FAQ (Frequently Asked Questions) list on numerical analysis. The size of the list will give you some idea of the scope of the field. Some books are reviewed in section q165.

It would be unproductive to try to list all the excellent textbooks on numerical analysis. A few of them are (in alphabetical order of primary author):

  • Kendall Atkinson, Numerical Analysis.
  • Ward Cheney and David Kincaid, Numerical Mathematics and Computing, Brooks-Cole (Third edition, 1994). Aharon Naiman has prepared slides for a series of lectures he teaches at Jerusalem College of Technology using this text.
  • George Forsythe, Michael Malcolm and Cleve Moler, Computer Methods for Mathematical Computations, Prentice-Hall (1977). This book is probably out of print. It is a predecessor to the book by Kahaner et. al., and written in a similar style. It is less comprehensive than the newer work.
  • Francis B. Hildebrand, Introduction to Numerical Analysis, McGraw-Hill (1956, 1974), Dover (1987 0-486-65363-3). This is one of the best books ever written on numerical analysis. It's out-of-date in some areas, most notably in Least Squares computation. Hildebrand has an engaging and transparent style of exposition, similar to Press et. al. This book is, however, a mathematically sound reference to material of the same era as presented in much of Numerical Recipes.
  • David Kahaner, Cleve Moler and John Nash, Numerical Methods and Software, Prentice-Hall (1989). ISBN 0-13-627258-4. This book is probably closest in style to Numerical Recipes, but was written by practitioners in the field, rather then by experts in a different field.
    Diskette included.
  • John H. Mathews, Numerical Methods: for Mathematics, Science & Engineering, 2nd Ed., ISBN 0-13-624990-6 and 0-13-625047-5, Prentice Hall Inc. (1992). With purchase of this book free software is available from the following links:
  • John H. Mathews and Russell W. Howell,
    COMPLEX ANALYSIS: for Mathematics and Engineering, Third Edition, 1997, ISBN 0-7637-0270-6.
    With purchase of this book, free software is available.
  • Zdzislaw Meglicki has posted the text for a course on advanced scientific computing at Indiana University.
  • G. W. "Pete" Stewart posted the following in
    "I have recently published a book entitled `Afternotes on Numerical Analysis'... It is a series of 22 lectures on elementary numerical analysis. The notes themselves were prepared after the lectures were given and are an accurate snapshot of what went on in class. Although they are no substitute for a full-blown numerical analysis textbook, many people have found them a useful supplement to a first course. The book is published by SIAM. For further information contact" and on 2 Jan 1997:

    I have just completed a new set of afternotes and have posted them on the web. The original afternotes were based on an advanced undergraduate course taught at the University of Maryland. The present notes are based on the follow-up graduate course. The topics treated are approximation\,---\,discrete and continuous\,---\,linear and quadratic splines, eigensystems, and Krylov sequence methods. The notes conclude with two little lectures on classical iterative methods and nonlinear equations

    The notes may be obtained by anonymous ftp at in /pub/afternotes or by browsing my homepage
    I will be grateful for any comments, corrections, or suggestions.

There are excellent texts and reference works that focus on narrow portions of
the discipline of numerical analysis. Consider, for example:
  • Gene H. Golub and Charles F. Van Loan, Matrix Computations, Johns Hopkins (first edition 1983, second edition 1989, third edition 1996). ISBN 0-8018-5413-X (0-8018-5414-8 paper).
  • Ernst Hairer, Syvert Paul Nørsett, Gerhard Wanner, Solving Ordinary Differential Equations I: Nonstiff Problems, Springer-Verlag (1987 3-540-17145-2 0-387-17145-2). A second edition has appeared but the ISBN's here are for the first.
  • Ernst Hairer, Gerhard Wanner, Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems, Springer-Verlag (1991: 3-540-53775-9 and 0-387-53775-9; 1996: 3-540-60452-9). This book and the previous one are highly regarded.
  • Charles L. Lawson and Richard J. Hanson, Solving Least Squares Problems, Prentice-Hall (first edition 1974), SIAM Press (second edition 1995) ISBN 0-89871-356-0.
  • Philip J. Davis and Philip Rabinowitz, Methods of Numerical Integration, Academic Press (second edition 1984) ISBN 0-12-206360-0.
  • Ingrid Daubechies, Ten Lectures on Wavelets, SIAM Press.
  • Jorge J. Moré and Stephen J. Wright, Optimization Software Guide, Frontiers in Applied Mathematics 14, Society for Industrial and Applied Mathematics (1993). About evenly divided between algorithms and software, both public-domain and commercial. (This book actually covers a fair amount of the content of Numerical Recipes, especially those parts that the authors of NR deemed too complex to do well.)
  • Spaeth, Mathematical Algorithms for Linear Regression (1987).
If you have been using Numerical Recipes for software, we recommend that
you contact the computing professionals in your organization. For JPL users, you can contact the Computational Mathematics Subgroup, or obtain the Math77 and mathc90 libraries of mathematical software directly. There is also a substantial amount of software and information about software on-line.

For one-of-a-kind computations, we recommend MATLAB (The MathWorks, Inc.).