Sharp Things

Use FsLex and FsYacc to make a parser in F#

Jomo Fisher MSFT — Tue, 15 Jun 2010 22:18:00 GMT

Update: The template I posted yesterday was for PowerPack 1.9.9.9 which is not the most recent version available. I have since updated the template to target PowerPack 2.0.0.0. If you installed the old template you may need to uninstall at Tools\Extension Manager...

-----------------------------

Jomo Fisher—F# is excellent for parsing text and creating new languages. The F# PowerPack contains two tools—FsLex.exe and FsYacc.exe—to create powerful lexers and parsers. However, it isn’t so easy to configure a Visual Studio project correctly to use these tools. To help with this, I made a project template that you can use to get started quickly. Just follow these quick steps:

1) Download the F# PowerPack from here.

2) In VS2010, go to File\New Project. Select ‘Online Templates’ and search for ‘F# Parsed’ (or my name).

The starter project implements a simple calculator language that you can grow into a more sophisticated Domain Specific Language or text processing application.

Credit goes to Matt Valerio and Brian McNamara for figuring out the details and to Don Syme for making the calculator language that this is based on. Bugs and omissions are probably my fault. Please let me know what you think, and whether you have any issues or spot any bugs that I should address.

This posting is provided "AS IS" with no warranties, and confers no rights.

Creating an OData Web Service with F#

Jomo Fisher MSFT — Sat, 29 May 2010 00:17:52 GMT

Jomo Fisher—I’ve spent a little time looking at the OData Web Service Protocol. There is quite a lot to like about it. It is an open, REST-ful format for communicating with a web service as if it were a database. You can query an OData service with LINQ.

To help me learn about it I set out to write a simple OData web service. I expected to have to do quite a lot of work because OData is very rich and just parsing the urls, which can contain SQL-like queries, looked to be hard work.

It turned out, however, to be pretty straight-forward. You basically just need to implement IDataServiceQueryProvider and IDataServiceMetadataProvider. All the heavy-lifting in terms of parsing is done for you. You get a nice System.Linq Expression which is easy to deal with (mainly because my service uses LINQ to Objects under the covers).

If you’d like to take a look at the service I wrote, I published it on Visual Studio Gallery. You just need to go to File\New Project in Visual Studio 2010:

Please let me know what you think, and whether you have any issues or spot any bugs.

This posting is provided "AS IS" with no warranties, and confers no rights.

F# Object Oriented Programming

Jomo Fisher MSFT — Wed, 26 May 2010 22:44:50 GMT

Jomo Fisher—F# has a functional heritage but it is actually very good at OO as well. Here’s an example of its nice, clean syntax for declaring a class:

Code Snippet
 type Vector(x : float, y : float) =
    member this.X = x
    member this.Y = y
 

Tim Ng posted a nice comparison of F# to C# for OO language features. If you’re a C# programmer interested in F#, this is a great place to start.

This posting is provided "AS IS" with no warranties, and confers no rights.

How to Enumerate all WMI Classes

Jomo Fisher MSFT — Tue, 25 May 2010 17:08:44 GMT

Jomo Fisher—Quick sample today. I’m mainly posting this because it took me a while to figure it out and I hope it might save someone some time. This is the F# code you would useto enumerate all WMI classes on your system and show the properties of each. of each.

Code Snippet
// WalkWmiTypes.fsx 
// Enumerate all WMI types on the local machine and show their property names and types. 
#r "System.Management" 
open System.Management 
  
let scope = ManagementScope("\\\\localhost\\root\\cimv2") 
let path = ManagementPath("") 
let mclass = new ManagementClass(scope, path, ObjectGetOptions()) 
let options = EnumerationOptions() 
options.EnumerateDeep <- true 
  
for obj in mclass.GetSubclasses(options) do 
    printfn "%s" (obj.ToString().Split(':').[1]) 
    for prop in obj.Properties do 
        printfn "    %s:%A" prop.Name prop.Type 

This is pure .NET, so it should be easily translatable to C# and VB. Let me know if you run into problems with that. The result looks like:

Output
Msft_WmiProvider_DeleteClassAsyncEvent_Post 
    ClassName:String 
    Flags:UInt32 
    HostingGroup:String 
    HostingSpecification:UInt32 
    Locale:String 
    Namespace:String 
    ObjectParameter:Object 
    provider:String 
    ResultCode:UInt32 
    SECURITY_DESCRIPTOR:UInt8 
    StringParameter:String 
    TIME_CREATED:UInt64 
    TransactionIdentifer:String 
    User:String 

This posting is provided "AS IS" with no warranties, and confers no rights.

F# Destroys Invading Aliens

Jomo Fisher MSFT — Thu, 18 Mar 2010 15:50:16 GMT

Wow, check out Chris’s StarCraft bot in F#: http://blogs.msdn.com/chrsmith/archive/2010/03/18/writing-a-starcraft-bot-in-f.aspx

F# Team is Hiring a Cross-Platform Developer

Jomo Fisher MSFT — Mon, 15 Mar 2010 20:29:10 GMT

We’re hiring a developer to help advance our cross-platform presence. Please do send and application if this looks like a good match for you.

From Don Syme’s blog:

Contract Position in the F# Team: Compiler and Visual Tools Software Engineer for Cross-Platform F#

We are now seeking applications for a contract position with the F# team!

Contract length: 9 months – 1 year
Hiring Group: Microsoft Research, Cambridge
Location: Cambridge UK. Remote working possible.

F# is a cross-platform language executing on any CLI implementation, including those found on Windows, Mac OS/X, Linux, Silverlight, XBox 360 and mobile phones. We are seeking a talented and highly motivated software engineer with experience in compilers and/or visual tools to make targeted improvements to the support the execution, development and tools experience across these platforms. Other tasks will include working with implementors of these CLI implementations, preparing web hosted material, preparing components for shared and open source release, working directly with communities interested in using F# across multiple platforms, including universities, and preparing demonstration applications for the use of F# across platforms.

The ideal candidate would have a passion for the F# language, strong .NET programming skills, knowledge of web design and programming, and knowledge of several of the above platforms. Experience in authoring parts of a compiler and/or interactive development environment would also be helpful, as would experience in open source development. The candidate will have the opportunity to work as part of the F# team in Cambridge UK, and a portion of remote working would also be considered.

Please send applications to Don Syme (dsyme@microsoft.com), or Ruth Lenton (ruth.lenton@microsoft.com)

http://blogs.msdn.com/dsyme/archive/2010/03/10/contract-position-in-the-f-team-compiler-and-visual-tools-software-engineer-for-cross-platform-f.aspx

This posting is provided "AS IS" with no warranties, and confers no rights.

Neat Samples: MEF in F# Scripts

Jomo Fisher MSFT — Wed, 10 Mar 2010 18:34:00 GMT

Jomo Fisher—I posted yesterday about using MEF in F# programs. In the comments Oldrich asked if it was possible to do the same thing in F# scripts. It is indeed possible. It actually looks like a really powerful combination because you can naturally extend your scripts just by adding new #load directives at the top.

There are three pieces to this sample. You have to break your interfaces out into a separate .fsx. Realistically, you’d probably do this in a real compiled F# program too so this isn’t a huge burden on simplicity.

Code Snippet

// MefInterfaces.fsx
// Expose an interface
type ITastyTreat =
    abstract Description : string

Code Snippet

// ChocolateCookie.fsx
#r "System.ComponentModel.Composition"
#load "MefInterfaces.fsx"
namespace ChocolateCookie
open MefInterfaces
open System.ComponentModel.Composition
[<Export(typeof<ITastyTreat>)>]
type ChocolateCookie() = 
    interface ITastyTreat with 
        member __.Description = "a delicious chocolate cookie"

Code Snippet

// MefHost.fsx
#load "ChocolateCookie.fsx"
open System.Reflection
open System.IO
open System.ComponentModel.Composition
open System.ComponentModel.Composition.Hosting
open MefInterfaces
// Set up MEF
let catalog = new AggregateCatalog()
let assemblyCatalog = new AssemblyCatalog(Assembly.GetExecutingAssembly())
let container = new CompositionContainer(catalog)
catalog.Catalogs.Add(assemblyCatalog)
// Jar that will contain tasty treats
type TreatJar() = 
    [<ImportMany(typeof<ITastyTreat>)>] 
    let cookies : seq<ITastyTreat> = null
    member __.EatTreats() =
        cookies |> Seq.iter(fun tt->printfn "Yum, it was a %s" tt.Description)
let jar = TreatJar()
container.ComposeParts(jar)
jar.EatTreats()

Thanks Oldrich, for giving me an excuse to try this out.

This posting is provided "AS IS" with no warranties, and confers no rights.

Neat Samples: Extend your F# program with MEF

Jomo Fisher MSFT — Wed, 10 Mar 2010 00:32:39 GMT

Jomo Fisher—The Managed Extensibility Framework is an interesting new technology in .NET 4.0. It lets you set up a plug in system for your application so that your program can acquire new functionality just by, for example, adding a .dll into a particular directory.

This is a simple example in F#. This code sets up MEF hosting and asks for all extensions in the c:\extensions folder.

Code Snippet
 // Host.fs
// A MEF host that eats tasty treats
open System.Reflection
open System.IO
open System.ComponentModel.Composition
open System.ComponentModel.Composition.Hosting
 
// Expose an interface
type ITastyTreat =
    abstract Description : string
 
// Set up MEF
let catalog = new AggregateCatalog()
let directoryCatalog = new DirectoryCatalog(@"c:\Extensions","*.dll")
let container = new CompositionContainer(catalog)
catalog.Catalogs.Add(directoryCatalog)
 
// Jar that will contain tasty treats
type TreatJar() = 
    [<ImportMany(typeof<ITastyTreat>)>] 
    let cookies : seq<ITastyTreat> = null
 
    member __.EatTreats() =
        cookies |> Seq.iter(fun tt->printfn "Yum, it was a %s" tt.Description)
 
let jar = TreatJar()
container.ComposeParts(jar)
jar.EatTreats()
 

This is how you implement an extension. Just build this into a .dll and drop it in the c:\extensions directory.

Code Snippet
 namespace ChocolateCookie
open Host
open System.ComponentModel.Composition
 
[<Export(typeof<ITastyTreat>)>]
type ChocolateCookie() = 
    interface ITastyTreat with 
        member __.Description = "a delicious chocolate cookie"
 

Now, if you run the host program you should see this:

Yum, it was a a delicious chocolate cookie
Press any key to continue . . .

This posting is provided "AS IS" with no warranties, and confers no rights.

Neat Samples: F#, Freebase and DGML

Jomo Fisher MSFT — Sat, 06 Mar 2010 21:37:26 GMT

Jomo Fisher – I recently posted about the freebase web service here. This sample reads biological classifications and renders them in DGML. The result is a huge graph, here’s a little piece of it:

Code Snippet
 // FreebaseDgml.fsx 
// Example of reading from freebase.com in F# using DataContract and JSON serializer and write the result
// in .dgml format.
// by Jomo Fisher 
#r "System.Runtime.Serialization" 
#r "System.ServiceModel.Web" 
#r "System.Web" 
#r "System.Xml" 
  
open System 
open System.IO 
open System.Net 
open System.Text 
open System.Web 
open System.Collections.Generic
open System.Security.Authentication 
open System.Runtime.Serialization 
  
[<DataContract>] 
type Result<'TResult> = { 
    [<field: DataMember(Name="code") >] 
    Code:string 
    [<field: DataMember(Name="result") >] 
    Result:'TResult 
    [<field: DataMember(Name="message") >] 
    Message:string 
    } 
 
[<DataContract>] 
type Classification = { 
    [<field: DataMember(Name="name") >] 
    Name:string 
    [<field: DataMember(Name="higher_classification") >] 
    HigherClassification:string 
    [<field: DataMember(Name="lower_classifications") >] 
    LowerClassifications:string array
    } 
  
let Query<'T>(query:string) : 'T = 
    let query = query.Replace("'","\"") 
    let queryUrl = sprintf "http://api.freebase.com/api/service/mqlread?query=%s" "{\"query\":"+query+"}" 
  
    let request : HttpWebRequest = downcast WebRequest.Create(queryUrl) 
    request.Method <- "GET" 
    request.ContentType <- "application/x-www-form-urlencoded" 
  
    let response = request.GetResponse() 
  
    let result = 
        try 
            use reader = new StreamReader(response.GetResponseStream()) 
            reader.ReadToEnd(); 
        finally 
            response.Close() 
  
    let data = Encoding.Unicode.GetBytes(result); 
    let stream = new MemoryStream() 
    stream.Write(data, 0, data.Length); 
    stream.Position <- 0L 
     
    let ser = Json.DataContractJsonSerializer(typeof<Result<'T>>) 
    let result = ser.ReadObject(stream) :?> Result<'T> 
    if result.Code<>"/api/status/ok" then 
        raise (InvalidOperationException(result.Message)) 
    else 
        result.Result 
  
let classifications = Query<Classification array>("[{'type':'/biology/organism_classification','name':null,'higher_classification':null,'lower_classifications':[]}]") 
 
 
let sb = StringBuilder()
  
sb.Append("<DirectedGraph xmlns='http://schemas.microsoft.com/vs/2009/dgml'>\n") |> ignore
 
sb.Append(" <Links>\n") |> ignore
for classification in classifications do 
    if classification.LowerClassifications<>null then
        for lower in classification.LowerClassifications do
            sb.AppendFormat("  <Link Source=\"{0}\" Target=\"{1}\"/>\n",classification.Name,lower) |> ignore
sb.Append(" </Links>\n") |> ignore
sb.Append("</DirectedGraph>\n") |> ignore
 
printfn "%s" (sb.ToString()) // Paste the results of this into a .dgml file.           
 

This posting is provided "AS IS" with no warranties, and confers no rights.

Neat Sample: F# and Bing API

Jomo Fisher MSFT — Sat, 06 Mar 2010 20:32:17 GMT

Jomo Fisher – I recently posted an sample of calling the Freebase web service with F#. Here’s another F# web service sample. This one uses the Bing Phone API to do a query. This time the code uses Xml instead of JSON and XmlDocument instead of a DataContract deserializer. This is pretty much a straight transliteration of one of the Bing SDK samples.

Code Snippet
 // BingPhone.fsx
// An example of using the a Bing web service API from F#
// by Jomo Fisher
open System.Net
open System.Xml
 
let appId = "your app id goes here";
 
let BuildRequest() : HttpWebRequest  = 
    let requestString = 
        "http://api.bing.net/xml.aspx?"
 
            // Common request fields (required)
            + "AppId=" + appId
            + "&Query=microsoft offices"
            + "&Sources=Phonebook"
 
            // Common request fields (optional)
            + "&Version=2.0"
            + "&Market=en-us"
            + "&UILanguage=en"
            + "&Latitude=47.603450"
            + "&Longitude=-122.329696"
            + "&Radius=10.0"
 
            // Phonebook-specific request fields (optional)
            + "&Phonebook.Count=10"
            + "&Phonebook.Offset=0"
            + "&Phonebook.FileType=YP"
            + "&Phonebook.SortBy=Distance";
 
    // Create and initialize the request.
    downcast HttpWebRequest.Create(requestString)
 
let DisplayErrors(errors:XmlNodeList) = 
    // Iterate over the list of errors and display error details.
    printfn "Errors:"
    for error in errors do
        for detail in error.ChildNodes do 
            printfn "%s: %s" detail.Name detail.InnerText
 
let DisplayResults( root:XmlNode, nsmgr:XmlNamespaceManager) = 
    // Add the Phonebook SourceType namespace to the namespace manager.
    nsmgr.AddNamespace("pho", "http://schemas.microsoft.com/LiveSearch/2008/04/XML/phonebook")
 
    let phonebook = root.SelectSingleNode("./pho:Phonebook", nsmgr)
    let results = phonebook.SelectNodes("./pho:Results/pho:PhonebookResult",nsmgr)
 
    let version = root.SelectSingleNode("./@Version", nsmgr).InnerText
    let searchTerms = root.SelectSingleNode("./api:Query/api:SearchTerms",nsmgr).InnerText
    let _,offset = System.Int32.TryParse(phonebook.SelectSingleNode("./pho:Offset", nsmgr).InnerText)
    let _,total =  System.Int32.TryParse(phonebook.SelectSingleNode("./pho:Total", nsmgr).InnerText)
 
    // Display the results header.
    printfn "Bing API Version %s" version
    printfn "Phonebook results for %s" searchTerms
    printfn "Displaying %d to %d of %d results" (offset + 1) (offset + results.Count) total
 
    // Display the Phonebook results.
    let builder = System.Text.StringBuilder();
    for result in results do
        builder.Length <- 0
        builder
            .AppendLine(result.SelectSingleNode("./pho:Business", nsmgr).InnerText)
            .AppendLine(result.SelectSingleNode("./pho:Address", nsmgr).InnerText)
            .Append(result.SelectSingleNode("./pho:City", nsmgr).InnerText)
            .Append(", ")|>ignore
 
        let stateOrProvince = result.SelectSingleNode("./pho:StateOrProvince", nsmgr).InnerText
        builder
            .AppendLine(stateOrProvince)
            .AppendLine(result.SelectSingleNode("./pho:PhoneNumber", nsmgr).InnerText)
            .Append("Average Rating: ")
            .AppendLine(result.SelectSingleNode("./pho:UserRating", nsmgr).InnerText)|>ignore
 
        printfn "%s" (builder.ToString())
 
let DisplayResponse(response:HttpWebResponse) = 
    // Load the response into an XmlDocument.
    let document = XmlDocument()
    document.Load(response.GetResponseStream());
 
    // Add the default namespace to the namespace manager.
    let nsmgr = XmlNamespaceManager(document.NameTable);
    nsmgr.AddNamespace("api","http://schemas.microsoft.com/LiveSearch/2008/04/XML/element");
 
    let errors = document.DocumentElement.SelectNodes("./api:Errors/api:Error", nsmgr)
 
    if errors.Count > 0 then
        // There are errors in the response. Display error details.
        DisplayErrors(errors);
    else
        // There were no errors in the response. Display the
        // Phonebook results.
        DisplayResults(document.DocumentElement, nsmgr);
 
 
let request = BuildRequest()
 
try
    // Send the request; display the response.
    let response = request.GetResponse()
    DisplayResponse(downcast response)
with ex ->
    // An exception occurred while accessing the network.
    printfn "%s" ex.Message
 

As before, let me know if there are other F# samples you’d like to see.

This posting is provided "AS IS" with no warranties, and confers no rights.