The Activity Designer

Note to self - how to stop VS 2012 menus being all caps

2013-05-21T20:49:14Z

reg ADD "HKCU\Software\Microsoft\VisualStudio\11.0\General" /v "SuppressUpperCaseConversion" /t REG_DWORD /d 1

I just thought I'd put this here in case I ever lose the info again. :)

Isn't setting up a new machine fun.

The mystery of the missing URL validation

2013-05-08T22:22:00Z

Further to my adventures with unobtrusive validation of yesterday, I found myself very stuck while implementing a new form for this new editable metadata feature for the NuGet Gallery. And Yet again the cause of all my pain appears to be this evil combination of UnobtrusiveValidation (2.0.30116.0) and JQuery.Validation (1.8.1). (I know it's not meant to be an evil combination, but believe me, it really starts to feel that way.))

For the sake of discussion, a feature I'm trying to add is the ability to edit your package's project url, and logo icon url on nuget.org. Since I'm trying to follow the established patterns of the codebase, I decided to use a RequestModel object to represent the editing form. So naturally I do this inside my request model:

[StringLength(256)]
[Display(Name = "Icon URL")]
[DataType(DataType.ImageUrl)]
public string IconUrl { get; set; }

[StringLength(256)]
[Display(Name = "Project Homepage URL")]
[DataType(DataType.Url)]
public string ProjectUrl { get; set; }

Unfortunately, when it comes to validation of this model, woe is me. If you enter a string which is not a valid URL and doesn't even start with https, you get no indication you've done something stupid.

Perhaps naturally, perhaps not, I had assumed that adding these DataType attributes is going to do something good for me and do URL validation. Why would it be a natural assumption? Well, I say that because UnobtrusiveValidation appears to understand DataType.Url in the following ways:

[Updated: Gee thanks a lot blogs.msdn.com for corrupting my post. Now don't do it again!]

1) It adds a type="Url" attribute to your form INPUT element
2) It has code inside that tries to add an adapter which understands URL typed input elements and apply the jquery.validate URL validation to them. At least that's what I think it's meant to do. Because it doesn't actually work in practice.

I tried to work around this by adding regex validation to my model, thinking that could replace the non-functioning URL validation. Sadly that doesn't work either!

[RegularExpression(Constants.UrlValidationRegEx, ErrorMessage = "This doesn't appear to be a valid URL")]

Some hours of debugging eventually reveals that the reason neither regexes nor URLs are working is because I chose to do [DataType.Url] prevents the focusout delegate from being registered as a validation hook upon the input element

   function delegate(event) {
    var validator = $.data(this[0].form, "validator"),
     eventType = "on" + event.type.replace(/^validate/, "");
    validator.settings[eventType] && validator.settings[eventType].call(validator, this[0] );
   }
   $(this.currentForm)
    .validateDelegate(":text, :password, :file, select, textarea", "focusin focusout keyup", delegate)
    .validateDelegate(":radio, :checkbox, select, option", "click", delegate);

yes... jquery.validate will only add focusout handlers for stuff it knows about.
And it gives you no way of customizing this. Grrr. Yuck.

Best option for working around this seems to be don't use DataType.Url. But [update #2] unfortunately that will require me to modify my css styling which was based on detecting the url datatype.
So second good option is to write some (obtrusive!) script to add the missing focusout event handlers. This was yet another good 'learning exercise'.

        $(function () {
            var form = $('form[action="/packages/PoliteCaptcha/Edit/"]').toArray()[0];
            $("input[type=url]").bind("focusin focusout keyup", function (event) {
                var target = $(event.target);
                var validator = $.data(form, "validator"),
                    eventType = "on" + event.type.replace(/^validate/, "");
                validator.settings[eventType] && validator.settings[eventType].call(validator, event.target);
            });
        });

Things that were annoying to get right here:
-Figuring out that for whatever reason I can't just do $.bind() like jquery.validate does. I have to bind to a proper jquery element (collection).
-For the longest time $.data was returning me null and I couldn't figure out why until with the javascript debugger I realized there were two different definitions of the function at the different points in time because I was including jquery.min.js after jquery.js and jquery.validate.js. Owch!
-figuring out that the object to pass to the validator.settings[eventType] func was event.target also took me a while.

Regular expressions are different in javascript! And Unintrusive Validation.

2013-05-08T02:59:29Z

Today while trying to implement some new code, and cutting my teeth on javascript validation, I disbelievingly realized that ALL of our client-side validation had stopped working. Like a month ago. And, nobody had noticed or complained! (As a brief reminder I now work on the NuGet Gallery.)

Perhaps that just goes to show how little real value client-side form validation is providing our nuget.org users, who are a smart bunch. Especially when Server side validation is there (it's always going to be there) and it gets the job done. Anyway, regardless of the true value of client side validation, I'm fully mentally and emotionally committed to doing some client side validation for our new features. So let's move ahead!

The original thing turned out to be a minor and easy thing to debug and fix, and didn't require me to understand the validation framework much at all. The script tag for jquery.validate.unobtrusive had accidentally gone missing when my compatriate made some innocent looking performance enhancements to the site. And without that script tag, you'll never see any validation. So problem solved. The way the tag went missing, honestly, was just one of those accidental things that you would think wouldn't be so easy to do. If you look at our code (yes, it's so frickin' cool that I can link to our code since it's open source)

https://github.com/NuGet/NuGetGallery/blob/master/Website/App_Start/AppActivator.cs

you'll see that we're doing script bundling.

var scriptBundle = new ScriptBundle("~/bundles/js")

.Include("~/Scripts/jquery-{version}.js")

.Include("~/Scripts/jquery.validate.js")

.Include("~/Scripts/jquery.validate.unobtrusive.js");

BundleTable.Bundles.Add(scriptBundle);

And it turned out that we were bundling a script that didn't exist (we only had jquery.validate.unobtrusive.min.js). Of course we never ever see an error even though we are bundling something that doesn't exist... Oh well. If you were thinking of playing with script bundling, you have now been warned. :)

But digressions, digressions. The real point of this post was that even after turning on unobtrusive validation, it became apparent that our email address and username validation on the user registration page were still not working. I could type in 'foo#blarg#blah' as an email address and all was well with the webby world. But why?

Well, let's see. What's supposed to happen is that we set a [RegularExpressionAttribute] on our property, like so:

    [Required]
    [StringLength(64)]
    [RegularExpression(@"(?i)[a-z0-9][a-z0-9_.-]+[a-z0-9]",
       ErrorMessage = "User names must start and end with a letter or number, and may only contain letters, numbers, underscores, periods, and hyphens in between.")]
     [Hint("Choose something unique so others will know which contributions are yours.")]
     public string Username { get; set; }

And this will cause some funky HTML to get generated, like so:

<inputclass="text-box single-line"data-val="true"data-val-length="The field Username must be a string with a maximum length of 64."data-val-length-max="64"data-val-regex="User names must start and end with a letter or number, and may only contain letters, numbers, underscores, periods, and hyphens in between."data-val-regex-insensitive="True"data-val-regex-pattern="[a-z0-9][a-z0-9_.-]+[a-z0-9]"data-val-required="The Username field is required."id="Username"name="Username"type="text"value=""/>

Which works.
And then there's a little adapter snippet in the unobtrusive validation script file which turns the data-val- attributes into a jquery.validation rule:

    $jQval.addMethod("regex", function (value, element,
params) {
        var match;
        if (this.optional(element)) {
            return true;
        }

        match = new RegExp(params).exec(value);
        return (match && (match.index === 0) &&
(match[0].length === value.length));
    });

Luckily, the first thing I tried is running this in the JS debugger, and it turns out that 'new RegExp(params)' is what fails. And it fails because (as promised in the title) regular expressions are different in javascript from .Net.

In .Net, the (?i) syntax is legal as part of the regular expression and tells the regular expression parser that you want to build a case-insensitive regular expression.

In Javascript, there are two ways of doing a case insensitive regular expression. One is to use a regular expression literal and suffix it with 'i' like this: /[a-z0-9][a-z0-9_:-]+[a-z0-9]/i

The other is to pass a second parameter "i" to the new RegExp constructor.

This leads to a GRRRRRR moment, where I privately curse the creator of the unobtrusive validation library for failing to consider this issue. And also me spending about an hour trying to understand how to solve the problem myself.

This turns out to be wasted effort, because as is often the case, some bright person has come up with a clever answer on stackoverflow.

http://stackoverflow.com/questions/4218836/regularexpressionattribute-how-to-make-it-not-case-sensitive-for-client-side-v

That said, I did spend this time coming up with another alternative approach, so might as well record it, and explain how it works. This one works not by introducing a new attribute or new annotation, but by overwriting the existing adapters in the unobtrusive validation pipeline.
The logical pipeline is this:

1. Data Model & Attributes (e.g. RegularExpressionAttribute) ->
2. DataAnnotationsModelValidatorProvider & Attribute Adapters (e.g. RegularExpressionAttributeAdapter) ->
3. HTML form + <script tags> (e.g. data-val-regex) ->
4. UnobtrusiveValidation Adapters (e.g. "regex" adapter snippet posted above) ->
5. jquery.validation rules - tada!

We can insert a hacky fix at step 2. in the pipeline:

            DataAnnotationsModelValidatorProvider.RegisterAdapter(
                typeof(RegularExpressionAttribute),
                typeof(FixedRegularExpressionAttributeAdapter));

}

        class FixedRegularExpressionAttributeAdapter : RegularExpressionAttributeAdapter
        {
            private RegularExpressionAttribute _attribute;

            public FixedRegularExpressionAttributeAdapter(
                ModelMetadata metadata, ControllerContext context, RegularExpressionAttribute attribute)
                : base(metadata, context, attribute)
            {
                _attribute = attribute;
            }

            public override System.Collections.Generic.IEnumerable<ModelClientValidationRule> GetClientValidationRules()
            {
                var rules = base.GetClientValidationRules();
                var rule = rules.First();
                var pattern = (string)rule.ValidationParameters["pattern"];
                if (pattern.StartsWith("(?i)"))
                {
                    pattern = pattern.Substring(4);
                    rule.ValidationParameters["pattern"] = pattern;
                    rule.ValidationParameters["insensitive"] = true;
                }
                return rules;
            }
        }

this will rewrite our HTML to remove the leading (?i) from the data-val-regex-pattern attribute, and create a new attribute data-val-regex-insensitive="True".
And since it's a script file, we could then modify the code of the unobtrusive validation adapter.

Well... it sort of gets the job done... but I don't actually feel like this is a very great solution to apply in practice. For one, pattern.StartsWith("(?i)") is hackiness. But for another, making changes to a script file like unobtrusive validation that we get as a nuget package is a headache because updating the nuget package in the future will lose the changes. Not to mention I'm unclear whether licensing permits editing the script files.

The right thing to do instead seems to be to contribute back to unobtrusive validation, or at least file a bug there, and hope it gets fixed, assuming it's open source.

It appears that it is open source, but it took me an awfully long time to confirm it is in the asp.net codeplex project

http://aspnetwebstack.codeplex.com/SourceControl/changeset/view/ea64fc86b54d#src/System.Web.Mvc/JavaScript/jquery.unobtrusive-ajax.js

where, for now, I've just filed a bug on the issue. As for now it makes more sense to me to workaround it using 'A-Za-z' in the regex than to incorporate a hacky non-future-proof fix, which was an interesting way to learn a bit more about these validation frameworks.

The Entity Framework Leaking

2013-04-03T04:46:45Z

'Fluent Interfaces' have been a bit of a buzz for probably long enough that I am not justified in buzzing about them. I was pretty excited when first seeing them applied to databases in form of NHibernate and also kind of excited to see all that code-first-ness come to Entity Framework, which had never really convinced me with the whole 'visual design experience' thing it had when I first looked at it way back in EF 2 or whatever. Those were the days.

Annnnyway...

It went a long time between me learning about EF, and actually getting to use it. Now I'm using it day to day. I did initially feel very excited about EF. The promise EF seems to hold me, is that it will simplify my life in pleasant ways:

1) I won't need to learn SQL syntax. I can just use code style .Where() and .Select() - all that LINQ stuff without the language integratedness which always seemed like a demoware feature to me anyway.
2) I won't have to manage database creation SQL scripts. I can use migrations to manage database creation and updating. The updating can even happen at app startup without me even thinking about it
3) Not only do I not have to learn SQL, I don't even have to know much how SQL works.

Now I've actually been using EF for 6 months, I find myself beginning to wonder if something is wrong.

See, I'm gradually learning SQL syntax. Why? Well, sometimes I might do ad-hoc queries on data. But mostly because I want to debug and optimize my EF queries, and the only way of doing that is to see the SQL queries that these EF queries get translated into, and to run them through the SQL execution plan analyzer.

Which brings me to point number two. I'm gradually learning stuff about how SQL works. Because it turns out that SQL is really good for certain things, and not so good (fast) at others. In fact, it depends a lot on your schema, how you normalize or denormalize, and what indexes you have how fast your queries are going to be. And oh yeah - as well as indexes SQL has these notions of non-nullability and uniqueness which are kind of important. And even string length is important, because it affects what you can put in indexes.

So there we are. I've realized that Entity Framework is yet another leaky abstraction.

The joy of Change

2013-02-05T23:57:33Z

Hey, it's been a long time since I updated this blog. Did anybody miss me? Didn't think so. Since the intertubes never stop piping out noise, obviously nobody will have noticed a little silence. In any case it feels nice to give everybody a warm fuzzy 'hello long time no see' post. No wait, don't yet ! There's actually news here, sort of. Well no, not the workflow news you were probably coming here to see. Because, as it turns out, I personally don't work on workflow any more. After years of trying to care deeply and hard about workflow and whether it is good or not, I've realized I needed to stop.

So what am I doing now? Well, same company, same software development game. But not much else is the same! I've switched from testing code to developing it. And I've switched from trying to care about workflows to trying to care about a completely different beast, i.e. NuGet packages! And remember, it's all open source, so you can join in the fun too!

If you've not heard of NuGet before don't worry it's not too late! Fire up Visual Studio (I suspect I'm going to be correct in assuming everyone who reads my blog has Visual Studio? :p) and go to the Extension Manager. Search online for an extension called NuGet package manager. But what am I saying, of course you've heard of NuGet package manager! But maybe you, like me, just didn't realize that it's going to be deeply relevent to your life as a developer? Still not convinced? Sigh, OK. I know I will convince you eventually!

Because, in fact, dear dot net developers - we are the last ones to catch on to this - but packages are pretty much the best way to manage your dependencies. Better than add reference, better than checking dlls into source control, better than install Foo SDK install.exes and maybe even better than shipping great big chunky .net frameworks? Yes there's a couple rough edges, but we're working on that!

Still not convinced? Until next time then.

(WF4,VS) WorkflowDesigner Extensions in Visual Studio 2012

2012-10-04T21:16:38Z

(Intro: Sometimes as part of testing I go to silly lengths to try to point out what I think is going to be a bug in a piece of code I have seen. And frequently when I do this, I am basically wrong, for any of various reasons – it’s a bug according to some world view which is not quite aligned with reality, or whatever. Overall I think I tend to frustrate some people when I go through this exercise, but an end result is often that I learn something...)

I just emerged from the testing vaults with a lost feature to share.

In VS 2010 there wasn’t a good customization story for the workflow designer in Visual Studio. What do I mean customization story? Let me try to think of some motivating examples.

Example #1: You want to customize Workflow Designer inside VS, by registering a new custom property editor for a system or custom Type. For sake of example e.g. Nullable<T>.

Example #2 You want VS to provide a new Service to your custom designers via EditingContext.Services – but the service will actually be hosted in Visual Studio’s main appdomain, and its data will persist throughout a VS session, and be shared with all WorkflowDesigner instances.

These scenarios are not supported in VS 2010. However, what I found out so far is that Example #1 and Example #2 appear to be supported in VS 2012. Now I’m calling this a lost feature because I couldn’t find any official docs for this, so these are what I picked up from talking to developer Tony, and disassembling certain assemblies.

The Tip

Here’s the basic hot tip that set me off on this hunt and summarizes the feature. Apparently there’s a directory you can place workflow designer extension DLLs in, that Visual Studio 2012 will find them somehow. On 64-bit windows OS, you’ll see it’s typically here:

C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\WorkflowDesigner

Tony also sent me an example code which is just an ordinary C# class with a MEF exported interface:

[Export(typeof(IEntityInfoService))] // note: IEntityInfoService - this is just some custom type for example, what type really doesn’t matter

Aside from this, his sample had two features which let me know the class might be hosted in the main VS app domain:

1) it imports DTE, which is the core Visual Studio programming interface. A DTE object gives you access to many things in VS such as the VS solution tree

[Import]
public DTE Dte { get; set; }

2) it inherits MarshalByRefObject – this means we can hold object references to the object from another domain, and that it will inherit stuff you need for .Net remoting lifetime management: GetLifetimeService, etc.

Another interesting find, while trying to understand the tip

In Microsoft.VisualStudio.Activities.AddIn.WorkflowDesignerAddIn, there is code which uses an internal class (AddInWorkflowDesignerExtensionManager) to discover extensions which implement IRegisterMetadata. You know, that interface for registering custom designer stuff. For each such extension found, the designer add-in calls Register() straight away.

Last find

Further in the ‘how it might work’ category – there’s another internal class that VS uses called WorkflowDesignerExtensionManager, which appears to be there for allowing Services (as in EditingContext.Services) to be discovered by the WorkflowDesigner or custom activity designers. It leverages the MEF API (and DirectoryCatalog in particular) to discover extensions from the directory Tony mentioned! Note, I heard a long time back that MEF was also planned to be key part of the extensibility story for Visual Studio 2012… looks like this sort of fits into that strategy.

Trying stuff…

Pieces of the puzzle are falling into place nicely. So it’s time to try some stuff out!

1) Create a custom dll, implement and Export (MEF) the IRegisterMetadata interface, and see if I can get called by Visual Studio. I’m guessing I don’t need MarshalByRefObject in this scenario, since the MetadataStore is generally in the WorkflowDesigner’s AppDomain.

2) Create a custom dll, implement and Export a custom interface, IFooService to a MarshalByRef object. Get and call the service from a custom activity designer, which will be loaded in one of the usual ways.

Since Visual Studio 2010 doesn’t (in my experience) do a very good job of debugging Visual Studio 2012, I’ll be creating my test projects in Visual Studio 2012.

namespace MetadataExtension

{

[Export(typeof(IRegisterMetadata))]

internal class RegisterMetadata : IRegisterMetadata

{

public RegisterMetadata()

{

}

public void Register()

{

MessageBox.Show("Hello... and goodbye sequence designer");

var builder = new AttributeTableBuilder();

builder.AddCustomAttributes(typeof(Sequence),

new DesignerAttribute(typeof(ActivityDesigner)));

MetadataStore.AddAttributeTable(builder.CreateTable());

}

I copy the debug output dll to C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\Extensions\WorkflowDesigner, and launch a new instance of Visual Studio 2012. I create a new workflow activity library project, and … success!

I have just succeeded in pranking myself – the Sequence activity is now totally unusable:

Of course while you could use this trick to prank your WF-using office mates, I don’t recommend it. We’d better think of some more constructive use for that feature.
Here’s the call stack when the metadata extension gets called.

    METADATAEXTENSION.dll!MetadataExtension.RegisterMetadata.Register() Line 24    C#
     Microsoft.VisualStudio.Activities.Addin.dll!Microsoft.VisualStudio.Activities.AddIn.WorkflowDesignerAddIn.ExecuteRegisterMetadataExtensions() + 0x82 bytes
     Microsoft.VisualStudio.Activities.AddinAdapter.dll!Microsoft.VisualStudio.Activities.AddInAdapter.IDesignerContractToViewAddInAdapter.ExecuteRegisterMetadataExtensions() + 0xc bytes
     mscorlib.dll!System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(System.Runtime.Remoting.Messaging.IMessage msg) + 0x1e7 bytes
     […more remoting stuff…]
     mscorlib.dll!System.Runtime.Remoting.Proxies.RemotingProxy.InternalInvoke(System.Runtime.Remoting.Messaging.IMethodCallMessage reqMcmMsg, bool useDispatchMessage, int callType) + 0x1cc bytes
     mscorlib.dll!System.Runtime.Remoting.Proxies.RemotingProxy.Invoke(System.Runtime.Remoting.Messaging.IMessage reqMsg) + 0x66 bytes
     mscorlib.dll!System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(ref System.Runtime.Remoting.Proxies.MessageData msgData, int type) + 0xea bytes
     Microsoft.VisualStudio.Activities.HostAdapter.dll!Microsoft.VisualStudio.Activities.HostAdapter.IDesignerViewToContractHostAdapter.ExecuteRegisterMetadataExtensions() + 0xc bytes
     Microsoft.VisualStudio.Activities.dll!Microsoft.VisualStudio.Activities.EditorPane.CreateWorkflowDesignerInIsolatedMode() + 0x2cb bytes
     Microsoft.VisualStudio.Activities.dll!Microsoft.VisualStudio.Activities.EditorPane.Microsoft.VisualStudio.Shell.Interop.IPersistFileFormat.Load(string fileName, uint formatMode, int readOnly) + 0xda bytes
     Microsoft.VisualStudio.Activities.dll!Microsoft.VisualStudio.Activities.EditorPane.Microsoft.VisualStudio.Shell.Interop.IVsPersistDocData.LoadDocData(string documentName) + 0xd bytes

From this stack we can notice that
1) our RegisterMetadata extension being called is very early indeed in the process of creating and setting up the Workflow Designer
2) it’s triggered by Visual Studio which is specifically calling for such extensions to be loaded, so it doesn’t apply for rehosted apps. Which is fine. In a rehosted app we already are able to control and hook into the workflow designer creation process.

The second example is just slightly more work.

Part #1: Define a contract

public interface ICustomContract

{

void Hello();

}

Part #2: Define an extension

[Export(typeof(ICustomContract))]

public class CustomContractExtension : MarshalByRefObject, ICustomContract

{

public CustomContractExtension()

{

}

public void Hello()

{

MessageBox.Show("Hello");

MessageBox.Show("Got DTE? " + (this.Dte != null));

}

[Import]

public DTE Dte { get; set; }

}

Now we can place our extension dll in the Extensions\WorkflowDesigner directory, fire up a new instance of VS 2012, and see if anything happens.

This time, nothing happens when we create a new class library. We need a couple more steps.

1) Add a new code activity

2) Add a new activity designer

3) Associate them (today I’ll do it the quickest way, using DesignerAttribute)

[Designer(typeof(ActivityDesigner1))]

public sealed class CodeActivity1 : CodeActivity

{

protected override void Execute(CodeActivityContext context)

{

}

4) Add a reference to the DLL where we defined ICustomContract – we can browse it from the Extensions folder

5) Modify our activity designer slightly, in the .xaml.cs file:

public partial class ActivityDesigner1

{

public ActivityDesigner1()

{

InitializeComponent();

}

protected override void OnModelItemChanged(object newItem)

{

base.OnModelItemChanged(newItem);

if (newItem is ModelItem)

{

var cc = (newItem as ModelItem).GetEditingContext().Services.GetService<ICustomContract>();

if (cc != null)

{

cc.Hello();

}

The important point here is that we will call EditingContext.Services.GetService<ICustomContract>() in order to get the custom service created above.

When we try it out, we will see our popup dialogs:

Interesting and important to note - if you set breakpoints in the constructor of CustomContractExtension, then you will see that it is being created lazily, upon demand.

Also interesting and important to note – the CustomContractExtension is indeed created in the main Visual Studio App Domain. So calls to it will work calling via .Net Remoting. Now it happens that there is no reference to that object in the main VS AppDomain, which leaves it subject to potential garbage collection. Ideally WorkflowDesigner or the EditingContext would create a remoting Sponsor in order to extend the life time of the object to match the lifetime of the EditingContext.Services object. However, in practice this does not happen. Leaving Visual Studio for idle for a few minutes, you can come back, try to use your custom activity again, and get errors such as:

Unfortunately I haven’t figured out the right way to stop Garbage collection blowing this up yet, but I think there are a couple approaches that could work: 1) sponsorship as mentioned above, 2) forced reachability to the service itself by ensuring references from a GC root in its home app domain.

(WF4) Link - Using WF4 custom activities to consume OData

2012-09-13T18:32:35Z

Today I’m just advertising another MSDN blog that got circulated my way. This post may only going to make full sense if you’ve heard of OData already. If you’re heard of OData, and have been wondering what a good way to use it from workflow might be, I think you’ll find it good food for thought.

http://blogs.msdn.com/b/derrick_vanarnams_blog/archive/2012/09/06/announcing-the-adventureworks-odata-feed-sample.aspx

http://blogs.msdn.com/b/derrick_vanarnams_blog/archive/2012/09/12/introducing-the-odata-queryfeed-activity.aspx

(Non-WF) Raw Sockets for receiving IP in C#

2012-09-06T23:27:18Z

Today, I set off in a quest to refresh my memory and hopefully also learn something new about sockets, IP, TCP protocols, and so on. Which is quite a breath of nostalgia taking me back to COSC 231 Data Communications. Isn’t it great how when working with HTTP you can forget about nearly all of the low level stuff?

Of course in my current state of C# dependency, the logical way to do this seemed to be to try to write some C# code, so (obviously) what better than to start learning about all the abstractions .Net offers for the network stack? I had already heard of a few of the beasts of the .Net network stable, such as Socket. Reading through the documentation for Socket it appears to offer some interesting capabilities, i.e. not just garden variety UDP and TCP, but also including ‘Raw’ mode where you can send and receive the raw IP packets with all their headers. Fun!

Note it appears this functionality is crippled slightly on client operating systems such as Windows 7 so that you can’t do all the stuff that hackers and penetration testers will really want to do like spoof your IP address - but my dev box is server.

So, from the .Net docs for Socket you would think receiving all the IP traffic on my box should be easy as 1, 2, right?

1) Create a raw socket for the IP addresss family
2) Receive!

I’m basing this on the fact that

“If you are using a connectionless protocol such as UDP, you do not need to listen for connections at all. Call the ReceiveFrom method to accept any incoming datagrams. Use the SendTo method to send datagrams to a remote host.”

Obviously at the IP level it’s got to be a connectionless protocol right?
And IP doesn’t have ports (those are UDP and TCP things) so we wouldn’t have to bind to a port right?

Wait, doubt surfaces, is it 1, 2, 3, would we have to bind to an IP address as step 2? Or specify somehow that we want to accept connections?
Hm, OK, 1, 2, 3 it is. Let’s try

        static void Main(string[] args)
        {
            Socket s = new Socket(AddressFamily.InterNetwork, SocketType.Raw, ProtocolType.Raw);
            IPAddress ipAddress = Dns.GetHostEntry(Dns.GetHostName()).AddressList[0];
            s.Bind(new IPEndPoint(ipAddress, 0));
            byte[] b = new byte[2000];
            int nr = s.Receive(b, SocketFlags.None);
        }

Argh!
System.Net.Sockets.SocketException was unhandled
HResult=-2147467259
Message=An address incompatible with the requested protocol was used

What went wrong? At first I thought this was a sign that .Net was going to hate me and tell me I couldn’t do this. But it turns out that the first DNS entry for my host is actually an IPV6 address.
So we modify line 4 to:

IPAddress ipAddress = Dns.GetHostEntry(Dns.GetHostName()).AddressList.Where((addr) => addr.AddressFamily == AddressFamily.InterNetwork).First();

And we have a program that runs and…. never receives anything. Hmm. What are we missing? Is it IOCTLs? Not as far as I can tell. I don’t really want to receive all IP traffic, just IP traffic that’s coming to my address would be fine. Wait, what address am I trying to listen on anyway… 127.0.0.1. Is that a valid address to listen on, or do I need to listen on my real network adapter address?

So, let’s use a real non loopback address.

And… also let’s try to connect using a client TCP app who will use that same address.

No connection could be made because the target machine actively refused it.

Say whaaaaat?

Hm. Is the problem Receive? ”Receive: Receives data from a bound Socket into a receive buffer… You can call Receive from both connection-oriented and connectionless sockets.” I did bind it, right? How could that not work? Oh wait, further down: “If you are using a connectionless protocol, you can also use the ReceiveFrom method. ReceiveFrom will allow you to receive data arriving from any host.”

Anyway, hmm… interesting.
Am I just being screwed up by a firewall? Probably not…

Aha, I just noticed an interesting line here:

“Received datagrams are copied into all SOCK_RAW sockets that satisfy the following conditions:

The protocol number specified in the protocol parameter when the socket was created should match the protocol number in the IP header of the received datagram.”

So I can’t use TCP protocol on the client side and RAW protocol on my listener. Hmm. So what if I want to reimplement TCP in the application level – can I just use SocketType.Raw with ProtocolType.Tcp? Apparently, yes. But still no dice.

It does seem like Bind is optional though: ”If a local IP address is defined for the socket, it should correspond to the destination address as specified in the IP header of the received datagram. An application may specify the local IP address by calling the bind function. If no local IP address is specified for the socket, the datagrams are copied into the socket regardless of the destination IP address in the IP header of the received datagram.”

What if I also try to bind to the specific TCP port? Hrm… nope, getting nowhere. Conclusion? Programming is sometimes really just an exercise in frustration.

To the interweb! Here is some other poor soul who has got a similar distance to me.

And… miracle of miracles. There’s an unaccepted answer which appears to be the right one: Set ProtocolType.IP!

New conclusion: Sometimes you just get lucky.

(WF4) Survey–have you customized FlowDecision?

2012-09-05T18:42:22Z

Today I was involved in a playful discussion about what would we do different about Workflow Designer if we could, and the question of ‘what do customers want?’ came up. I think there’s a lot that can change, but one of the ideas I can’t stay away from is improving Flowchart.

For instance, searching the forums for threads about FlowDecision turns up a surprising number of posts by people who want to customize FlowDecision and replace it with something a little bit different. So, here are my open questions for readers of the blog:

If you built a rehosted designer app did you in fact customize FlowDecision (by e.g. overriding the FlowDecision designer)?
Why did you do it, what did you make better?

Also interesting - maybe you have not customized FlowDecision, but you really wanted to? Did you investigate, and find it was impractical? What would you ideally have been able to do?

Replies by comment section!

(News) Workflow 1.0 Beta

2012-07-17T20:53:00Z

Dropping the stealth cloak a little, it’s time for some personal news about what I’ve been up to for the last year and a bit. And the answer is? Testing what we are calling in our docs “Workflow 1.0 Beta” (name may change) which is Microsoft’s new Workflow hosting offering that works with WF. Here’s a few quick facts about this piece of software.

What is this?

“Workflow 1.0 Beta” is a separate piece of software being released alongside the Office 2013 Beta, but also a requirement for enabling some of Office’s Sharepoint Workflow features.

It is a ‘server’ application which stores, loads and executes declarative XAML workflow definitions, the same as WF4 XAML authored custom activities.

“Workflow 1.0 Beta” is also an Azure platform service running in the cloud, which will enable you to host long-running workflow in the cloud without requiring VM role subscriptions.

Does it cost anything?

Currently nothing at all - it’s a free beta, and for the server product, it is also a separate install from Office. Disclaimer: pricing plans may change for future releases.

What are the supported platforms, prerequisites, and system requirements required to install the server product?

The answers are linked in the line above.

How is this different to hosting my own WF4 service using existing hosting technologies?

There are many differences.

One difference is that workflows now get hosted/installed on the server by publishing workflows to the server via REST-style HTTP calls. Of course there is also a user-friendly managed API for publishing workflows that wraps the HTTP if you'd rather just write simple code. As this is a multi-tenant workflow offering, you can also create multiple tenancies, and your own customers can use this same client API to upload their workflow definitions to their tenancy on your central server.

Another difference that will feel very different to existing WF4 customers is that workflows run in a sandboxed mode which disallows any potentially dangerous types and activities, sort of like running in partial trust, or as a restricted user.

This may feel like a severe limitation given how used we are to creating custom code activities. However, if you are running “Workflow 1.0 Beta” as an on-premise server, there are ways you can configure “Workflow 1.0 Beta” to allow your own custom code activities which I’ll hopefully get to posting some time soon, assuming people are interested!

Where can I get more information?

It’s all starting here! Workflow 1.0 Beta

http://msdn.microsoft.com/en-us/library/jj193528(v=azure.10)

Further questions are welcome in the comments section!