Don Syme's WebLog on F# and Other Research Projects

Updated Specification for the 1.9.4 "Spring Refresh" Release

I've just updated the links on the F# manual pages to point to the updated draft language specification (one big HTML page, or PDF) for the 1.9.4 release.

This is a major revision of the old spec to cover much of the language work done in the 1.9.4 release. It's from us at Microsoft Research, so isn't an "official" Microsoft spec.

This includes some comments from the F# team and some working notes.

Enjoy!

Don

High Performance Computing with F# and MPI

Matthew has written a great guide getting going with MPI on .NET, using F#. Here's an excerpt:

In my previous post, I looked at some of the options we have for concurrency programming in .NET applications.  One of the interesting ones, yet specialized is the Message Passing Interface (MPI).  Microsoft made the initiative to get into the high performance computing space with the Windows Server 2003 Compute Cluster Server SKU.  This allowed developers to run their given algorithms using MPI on a massive parallelized scale.  And now with the Windows Server 2008 HPC SKU, it is a bit improved with WCF support for scheduling and such.  If you're not part of the beta and are interested, I'd urge you to go through Microsoft Connect.  ...

"F# for Numerics" released

Flying Flog Consulting have recently published F# for Numerics. Here's how they describe the library:

Our new F# for Numerics library is a suite of numerical methods that leverage functional programming with F#...

This library implements numerical methods from a variety of different disciplines in a uniform way ...:

• Local and global function minimization and maximization.
• Mean, median, mode, variance, standard deviation, skew, kurtosis, Shannon entropy and other statistical quantities.
• Interpolation, curve fitting and regression.
• Matrix factorizations including eigenvalue computation.
• Numerical integration and differentiation.
• Spectral methods including the Fast Fourier Transform.

The first update has reportedly added:

Some F# Jobs

After a post on Planet F#, I looked around for 5 minutes and here are a few high-paying jobs where F# skills are a "major plus"

• C++ Developer Algorithmic Trading NY $200-250k
• CT based hedge fund - OO developer build trading system platform
• C# Experts 2-4 Years Top Tech / Finance Firm

• Senior Software developer to ... lead a medium-size team of experienced software engineers to design and develop scientific software for ... image processing, digital signal processing, image recognition, speech recognition, biometric identification, graph-based, multi-dimensional feature representation, and statistical classification

• 3-5 years Top .NET C# ServerSide

From Script to .NET Component: Huffman Coding with F#

Luke Hoban has a very nice blog entry that shows an implementation of Huffman Coding with F#. This not only shows of F# as an algorithmic problem-solving language, but, just as importantly, shows just how elegant and smooth it is to take this code and produce a .NET object-oriented component that encapsulates a Huffman encoder for use from other .NET languages.

Beautiful. Just beautiful.

F# Talks at TechDays in Paris: Slides and Videos

It's taken me a while, but I've finally posted the slides from the two talks I gave at TechDays in Paris

• Introduction to F# (beginner)
• F# and Parallel Programming (joint talk with Robert Pickering, slides here - Robert's slides cover web services and using Parallel Extensions from F#)

Videos are also available

Enjoy!

Don

Learning F#: Some great blog posts from the F# team

Brian and Chris have been writing some great blog posts of late on learning F#. I can highly recommend these ones in particular:

• F# in 20 Minutes - Part I, by Chris
  
• New F# users: some tips to help you become productive more quickly, by Brian
  
• Discriminated Unions in F#, also by Brian
  
• Pipelining in F#, by Brian

Enjoy!

Don

F# 1.9.4 Now Available: Making F# Simpler and More Consistent

 [ Note: download links updated to point to the MSI correctly ]

We're very glad to announce the release of F# 1.9.4, the Spring Refresh of F#, uploaded to the Microsoft Research download site yesterday! (MSI download, ZIP, detailed release notes) (Ok, ok, apologies to our antipodean users where it’s actually autumn... In any case, on with the show!)

This release of F# has much the same shape and form as earlier releases of F# from MSR. As I hinted in April, our focus for this release has been on polishing and simplifying the language and libraries, particularly with regard to making certain details more regular and consistent. Much of this has been driven by user feedback, and we'd like to say a big thank you to for all the feedback we've got from hubFS users, the F# list, F# users at Microsoft and our growing number of commercial users, particularly our friends at the financial institutions using F#. Also a thank you to those community members who helped us by installing early releases of this version.

The detailed release notes can be found here. This release doesn't include many of the investments we've been making in Visual Studio tools - we'll be rolling those out over the rest of the year as we move into CTP and beynd.

There have been numerous changes and enhancements in this release, documented below. These may cause you to see additional deprecation warnings or errors. We’d be particularly grateful if you could report any compilation failures, and if possible reduce them to a minimal repro for us.

Many thanks & enjoy your F# work!

Don, James, Luke, Brian, Jomo, Chris, Matteo, Santosh and Laurent

F# at TechEd Israel

Luke Hoban will be presenting F# at TechEd Israel in a couple of days time. Enjoy!

Tackling the F# Productization

Over the last few months the F# team has been diving deep into the technical work needed to productize F#. In this post I'd like to give a bit of a picture for what that means and how the team has been ramping up on our work.

Firstly, the team! In Cambridge we have myself and James Margetson, and in Redmond we have Luke Hoban, Jomo Fisher, Brian McNamara, Chris Smith, Matteo Taveggia and Santosh Zachariah, with help from Raj Pai, Yasir Alvi, Shri Borde, Dmitry Robsman and Mike Hopcroft who lead up various disciplines (QA, Dev, Program Management) within teams dealing with several .NET languages.

So, what does "productizing F#" really mean? As a language, F# is certainly staying much like it is now: succinct, efficient, expressive, simple-to-use and with good tool support. Fundamentally, productization means more of the same, only better :-) 

At a more concrete level, over the next few weeks we plan to roll out the April Update to the Microsoft Research Release of F#. This release will :

• contain over 100 bug fixes and feature completions to the core F# language
• address some language design issues that we'd been putting off for some time
• be matched by a draft of the F# Language Specification

I've greatly enjoyed working on this release: we've addressed many of the niggling issues that researchers tend to ignore ("Conditional compilation?" Yes! "assert calls System.Diagnostics.Assert?" Naturally! "Why doesn't typeof<_> work in an attribute?" Well, it will now!) . I'll follow up with more details as we make the release. At this point I want to also say a huge thank you to all those on www.hubFS.net and the F# List who've been so active in reporting issues and helping us really improve the quality of what we have. The F# user community are consistently helpful, friendly and well-informed, and are one of the main reasons why I find working on this language so enjoyable.

Looking beyond that release, Brian McNamara has written up a short summary of our plans for a CTP (Community Technology Preview) this summer and what that is likely to contain. The main features he lists are:

• Runtime components:
  • fsc.exe, the compiler that knows the F# language
  • fsi.exe, the interactive command environment
  • the F# libraries (usual stuff, ranging from immutable lists to asynchronous computations library)
  • an msbuild task which lets you build an F# ".fsproj" project from the command line
• Design-time components (Visual Studio 2008 integration):
  • F# language service, which provides syntax highlighting, intellisense, tooltips, etc.
  • F# project system, which enables you to add/remove .fs files, reference dlls, etc.
  • F# interactive window, where fsi is hosted inside Visual Studio

You'll be able to use all this in conjunction by composing these components with versions of Visual Studio such as the freely-available Visual Studio 2008 Shell or with Visual Studio 2008 itself. One of the things we've been investing in heavily is to have Jomo and Brian tackle the job of putting the F# design-time components on a sound engineering footing. This means we're no longer basing the Visual Studio components on the old MyC C++ sample, which was useful originally but had many problems too. Instead we're using VSX (Visual Studio extensibility) available in the Visual Studio SDK.  

So, it's shaping up to be a wonderful year for F#. There's lots of work for us to do, but I'm trust you'll agree it's worth it!

Kean Walmsley on using F# Asynchronous Workflows to simplify concurrent programming in AutoCAD

On Friday Kean Walmsley posted an excellent article on Using F# Asynchronous Workflows to simplify concurrent programming in AutoCAD.I've quoted some it below.

There are two things I especially like about this post. First, Kean's code is very clean and some of the best F# app-extension scripting I've seen.

Second, Kean took the time to highlight the minor differences between the synchronous and parallel versions of his algorithm. Of the 235 lines, only 13 lines of code and 7 lines of comments changed, and even for those there is a very close correspondence between the versions. Allowing programmers to achieve this correspondence was one of the key design goals of asynchronous workflows.

Here are some snippets from Kean:

As promised, in today's post we take that code and enable it to query the same data in parallel by using Asynchronous Workflows in F#. Asynchronous Workflows are an easy-to-use yet powerful mechanism for enabling concurrent programming in F#.

Firstly, a little background as to why this type of technique is important. ... Barring some disruptive technological development, the future gains in computing performance are to be found in the use of parallel processing, whether via multiple cores, processors or distributed clouds of computing resources. ... Additionally, with an increasing focus on distributed computing and information resources, managing tasks asynchronously becomes more important...

The big problem is that concurrent programming is - for the most-part - extremely difficult to do, and even harder to retro-fit into existing applications...

 Today's post looks at a relatively simple scenario, in the sense that we want to perform a set of discrete tasks in parallel, harnessing those fancy multi-core systems for those of you lucky enough to have them (I'm hoping to get one when I next replace my notebook, sometime in March)...

We are also going to be very careful only to run parallel tasks unrelated to AutoCAD. Any access made into AutoCAD's database, for instance, needs to be performed in series: AutoCAD is not thread-safe when it comes to the vast majority of its programmatically-accessible functionality. So we're going to run a set of asynchronous, parallel tasks to query our various RSS feeds, and combine the results before creating the corresponding geometry in AutoCAD. This all sounds very complex, but the good (actually great) news is that Asynchronous Workflows [do] all the heavy lifting....

In an earlier post I linked to Kean's entries where he had written his first AutoCAD application using F# and has just followed up with A mathematical F# application integrating with AutoCAD via .NET which embeds one of the F# DirectX demos directly into AutoCAD.

Update to the F# 1.9.3 Release

Hi all,

An update to the F# 1.9.3 release has been posted to the Microsoft Research Downloads site. Additional changes between1.9.3.7 and 1.9.3.14 are:

• Constructed classes may now be mutually recursive with other types, fixing an incompletness in the language implementation
  
• The Microsoft.FSharp.Core.Func module is deprecated (this contained a few rarely used functions related to function values themselves)
  
• The prefix operators "%" and "%%" have been added as ASCII-equivalent for Unicode section marker operators used for splicing into quoted expressions (i.e. for data binding into quotation expressions).
  
• Some additional identifiers in Pervasives have now been marked with the 'OCaml compatibility' attribute. We recommend people cross compiling with OCaml use --no-warn 62. [ Note: this may be a bit verbose for those cross-compiling F# code with OCaml and using these APIs heavily. In the next release we expect this warning will be off for all files ending in extension ".ml". ]
  
• Bug fixes, including some installation and documentation bugs

Don

Greg Neverov: Software Transactional Memory for F#

Greg Neverov (of active patterns fame) has placed an implementation of Software Transactional Memory for F# up on hubFS. Here's the description:

I have written a library for using software transactional memory in F#. The library exposes memory transactions as a monad using F#’s new computation expression feature. It can be downloaded here.

Software transactional memory (STM) is an approach to concurrent programming that avoids the use of traditional, error-prone locks to control access to shared memory. Instead of using locks a programmer specifies sections of code that will make atomic changes to shared memory.

The design and implementation of this F# library is blatantly copied from the paper Composable Memory Transactions. I am enormously grateful to the authors of this paper whose clear and precise explanation made the implementation of this complicated piece of software almost trivial.

The low-level STM machinery is implemented in the file stm.cs. It is written in C# mainly because I was working off existing code I’d written some time ago, but there’s no reason why it could not be written in F#. It is all managed code and internally uses monitors for synchronization. It currently uses a very coarse grained lock meaning that only one transaction can commit at once and every waiting thread is resumed when a transaction completes, but this can be improved with more work. The implementation follows the description in the paper.

The F# monadic interface is in the file stm.fs. It imports definitions from the C# assembly and defines the monad type Stm<'a> with corresponding builder class to enable computation expression syntax. An issue with the implementation of STM systems is how to access the transaction log to make reads and writes during a transaction. Two common approachs are to use thread-local storage which is slow, or implicitly pass the transaction log as an extra parameter to every function, which requires code generation. The advantage of using computation expressions (or monadic do notation) is that the transaction log is explicitly passed between functions by the programmer’s code, thereby avoiding compiler modification, but also avoiding the need for the programmer to manually manage the transaction log parameter passing scheme as well.

Fantastic stuff Greg!

Dustin Campbell and F# on Hansel Minutes

Come and work with us!

The F# team continue to have a small number of positions open for compiler and language tools experts. Positions can be based in Cambridge, UK or Redmond.

If you're interested in helping to bring a typed functional programming language into Visual Studio and think you have skills relevant to the achieving this, then please contact me or another member of the F# team to see if there might be a fit. If there's no immediate fit, then there may be one later, so don't be shy! Also consider submitting directly to Microsoft. We're looking for

• compiler experts, including optimization, type checking, performance and correctness
• language tools experts, e.g. experienced with language services and Visual Studio
• language tools QA/test experts

An MS or PhD degree in Computer Science or a related technical field is preferred.