It Already Is A Scripting Language

It Already Is A Scripting Language

Rate This
  • Comments 51

My recent post about the possibility of considering maybe someday perhaps adding "top level" methods to C# in order to better enable "scripty" scenarios generated a surprising amount of immediate emphatic pushback. Check out the comments to see what I mean.

Two things immediately come to mind.

First off, the suggestion made by a significant number of the commenters is "instead of allowing top-level methods, strengthen the using directive." That is, if you said "using System.Math;" then all the static members of that class could be used without qualification.

Though that is a perfectly reasonable idea that is requested frequently, it does not actually address the problem that top-level methods solve. A better "using" makes things easier for the developer writing the call. The point of top-level methods for scripty scenarios is to make it easier on the developer writing the declaration. The point is to eliminate the "ritual" of declaring an unnecessary class solely to act as a container for code.

Second, and more generally, I am surprised by this pushback because of course C# already is a scripting language, and has had this feature for almost a decade. Does this code fragment look familiar?

<%@ Page Language="C#" %>   
<script runat="server">   
  void Page_Load(object sender, EventArgs e) 
  {
      // whatever
  }
</script>
...

Where's the class? Where are the using directives that allow "EventArgs" to be used without qualification? Where's the code that adds this method group to the event's delegate? All the ritual seems to have been eliminated somehow. That sure looks like a "top-level method" to me.

Of course we know that behind the scenes it isn't any such thing. ASP.NET does textual transformations on the page to generate a class file. And of course, we recommend that you use the "code behind" technique to make a file that actually contains the methods in the context of an explicit (partial) page class, to emphasize that yes, this is a class-based approach to server-side processing.

But you certainly do not have to use "code behind". If you're an ASP traditionalist (like me!) and would rather see C# as a "scripting language" that "scripts" the creation of content on a web server, you go right ahead. You can put your "top level" methods in "script" blocks and your "top level" statements in "<% %>" blocks and you'll thereby avoid the ritual of having to be explicit about the containers for those code elements. The ASP.NET code automatically reorganizes your "script" code into something that the C# compiler can deal with, adding all the boring boilerplate code that has to be there to keep the compiler happy.

But consider now the burden placed upon the developers of ASP.NET by the language design. Those guys cannot simply parse out the C# text and hand it to the C# compiler. They've got to have a solid understanding of what the legal code container topologies are, and jump through hoops -- not particularly difficult hoops, but hoops nevertheless -- to generate a class that can actually be compiled and executed.

This same burden is placed upon every developer who would like to expose the ability to add execution of end-user-supplied code to their application, whether that's in the form of adding extensibility via scripting, or by enabling evaluation of user-supplied expressions (such as queries). It's a burden placed on developers of productivity tools, like Jon Skeet's "snippet compiler", or LINQPad. It's a burden on developers who wish to experiment with REPL-like approaches to rapid development of prototypes, test cases, and so on.

I am not particularly excited about the convenience of the ability to save five characters by eliding the "Math." when trying to calculate a cosine. The exciting value props are that we might be able to lower the cost of building tools that extend the power of the C# language into third-party applications, and at the same time enable new ways to rapidly experiment and develop high-quality code.

Of course we do not want to wreck the existing value props of the language in doing so; as I said last time, we also believe that there is huge value in the language design naturally leading professional, large-scale application developers towards building well-factored component-based programs.

Like all design decisions, when we're faced with a number of competing, compelling, valuable and noncompossible ideas, we've got to find a workable compromise. We don't do that except by considering all the possibilites, which is what we're doing in this case.

  • I'm of the opinion that the .NET languages should not become one kitchen sink language.  Which is what appears to be happening.  If you want to have free scope functions and variables, use C++/CLI.  If you want a true OO language, stick to C#.  If you want a hybrid functional/OO language,  use F#.  They are all interoperable with C# via the .NET framework and runtime.

    There are plenty of scripty languages out there already.  The joy of the .NET framework is that interop between the various language domains is relatively painless.  To complicate C# by adding dynamic types (as opposed to inferred types) and free functions IMHO dilutes its usefulness, and its utility in its core development space.

  • C# and VB.net, as joel said, not be a catch-all for any type of language or language feature.  This will make them less and less desirable to use for production code.  

  • I'd be excited about losing the Math. since it means that I could just type cos(whatever) instead of ComplexMath.cos(whatever) or similar for cosines of non-double types.

  • "No, the defining feature of a scripting language is that it can be used to extend existing applications by programmatically controlling the functionality exposed by those applications."

    Such a definition is meaningless, especially in the managed world. The ability to extend an application is entirely dependent on what capabilities that application exposes to be extended; it has nothing to do with the language used to create the extension.

    It's very meaningful to the person who has to write the extension. -- Eric

    "We just spent a year adding features that let us more easily interoperate with Python. Is that the action of a team that considers itself the only player?"

    Yes, in my opinion, the addition of dynamic (or rather the way in which it was implemented) is further evidence that the C# sees itself as the only legitimate language in the .NET world.

    Maybe I'm just getting grumpy in my advanced old age, but I am finding myself increasing irked by people who are not me telling me what my thoughts, beliefs, opinions and principles are. Particularly irksome is when my alleged opinions are in fact the opposite of my actual opinions.

    A common criticism levelled many times in the comments to my last couple of posts is that the language has "everything including the kitchen sink", or "is trying to be all things to all people". Though I understand how one might come to that conclusion, nothing could be further from the truth.

    It certainly is the case that we seek to increase the scope of the language by "making it easier". As I pointed out in my recent blog article of that name, by making some tasks easier in the language, we run the risk of unnecessarily complicating the tool. We take that risk very seriously and carefully weigh the potential damage caused by every change; we have extensive design reviews with industry experts to make sure that the risk is worth the benefit.

    But it certainly is not the case that we seek to make the language all things to all people. We intend C# and VB to be the premiere language tools for professional developers of real-world bet-the-company-on-them business applications, whether those applications are traditional forms applications or server-side content generation, or something else entirely. That is the core unchanging principle underlying the design of the language; that cannot change because that is the business justification for the whole product. The purpose of this product is to lower the costs of professional application development and thereby make our platforms -- which we sell for money -- attractive.

    So now consider what the costs and risks are to different kinds of customers. Suppose you're running a team of professional C# coders, and you have the belief that a particular problem is better solved in Python than in C#. It is entirely reasonable to ask questions like "will it be cheap and easy to adapt our existing C# codebase to make use of the new Python functionality?" or "will our testers be able to adapt their C# test framework to test the Python code?" If the answer to those questions is "no" then that is a barrier to adopting Iron Python into the software ecosystem of the team.

    If that were the case, and if we on the C# team actually had the belief that C# should be all things to all people then we would be rubbing our hands in glee and chortling evilly at this point. Another customer trapped in C#, yay! But that is not our belief; our belief is that customers should be free to use all the powerful CLR-based tools at their disposal to solve their problem as they best see fit. Every time we on the C# team make it harder for those tools to interoperate, we create a barrier to our customers getting their problems solved. I WANT people to use Python and F# and VB and JScript and Ruby to solve their problems, and I do not want their existing investments in C# or VB to create disincentives to use new languages.

    -- Eric

    Dynamic compromises the design principles of the language in an attempt to influence developers to implement functionality in C# that would be better suited to other languages.

    You seem to believe those design principles are carved in stone. They are not. Those design principles evolve as the needs of our customers evolve. The design principles of the language are what the design team say they are, and we say that the design principle of ".NET is a multi-language paradigm; C# interoperates well with as many other languages as possible" is a more important design principle than "the C# language is strictly typesafe". (And besides, the language was never strictly typesafe in the first place. Reflection, unsafe code, runtime code spitting, array covariance, and allowing casts that might fail at runtime all break type safety in some way. We have already "compromised" that principle many, many times while always trying to remain true to its core concept.)

    We have altered many design principles of the language over the years as the industry has changed; if you were to read the 10+ year deep archive of language design notes, you would see that an original explicitly stated core design principle was "no magic" -- that is, it should be easy for the reader of the code to imagine at a pretty low level of abstraction exactly what is happening on the machine level. And now we have anonymous functions, iterator blocks, local variable type inference, query comprehensions and dynamic calls implemented by spitting new IL at runtime and caching the results in a call site object -- the language is chock full of "magic".

    Of course, when faced with a conflict between two design principles, we always attempt to find the compromise that meets our needs while conforming to the spirit of all our principles. Finding such a compromise is not always easy, and there are many disagreements both within the team and in the industry that the right compromises were reached. You can't please everyone.

    Second, no, the feature is not intended to encourage dynamic programming in C#. It is intended to facilitate interoperability with libraries and objects designed for use in dynamic environments (including legacy COM objects which use IDispatch as their primary interface to the world.)

    Had we intended to encourage "dynamic language style" development in C#, we certainly would not have stopped at enabling dynamic call sites! C# 4.0 is embarrassingly weak as a dynamic language; compare it to, say, JScript, with its prototype inheritance, ubiquitous "expando" objects and ability to redefine basic parts of the language, like the Number prototype. Even the fact that "dynamic" was made a part of the statically analyzed type system indicates the primal importance of static type analysis to the design of the C# language. We want rigidly defined areas of doubt and uncertainty, as Douglas Adams famously wrote. -- Eric

    Implementing the desired interoperability using interfaces, and remaining type safe, would have been more consistent with the design and purpose of the language;

    Every design decision entails a compromise. You are of course free to disagree that we achieved a compromise that adequately meets our goals. There are many ways to design this feature; I prototyped up several different ways to do "dynamic" before we arrived at the implemented solution. -- Eric

    but it also would have made it more likely that developers who prefer dynamic typing would remain in the dynamic world as much as possible, and only move to C# when they had to. This, in my opinion, would have been a good thing!

    Dynamic programming should be done in dynamic languages; C# doesn't have to be everything to every programmer.

    I completely agree. We seek to increase the number of programmers who can successfully work in dynamic languages in .NET by lowering the cost of modifying existing bodies of C# code to work well with code written in dynamic languages. -- Eric

    But instead, the C# team is throwing caution to the wind and encouraging dynamic programming to be done in C#, which is ill suited for it.

    I completely disagree. First, we take caution extremely seriously, and I take great offense to the utterly unfounded accusation that we do not. We spent multiple hundreds of person-hours -- and some of those persons are exceedingly well paid -- considering the impact of this design decision. Second, we are not encouraging dynamic programming to be done in C#; we are encouraging rich interoperability of a traditional statically-typed programming model with both new code written in dynamic languages, and legacy COM objects designed for use in dynamic language environments. Of course, any tool can be misused; I encourage you to not misuse it. -- Eric

  • I suppose many of those features expressed here are covered in Microsoft's own PowerShell language.

    Why would anyone want an interpreted C#

    Who said anything at all about "interpreted"? -- Eric

  • Calling methods without qualification, like your example with System.Math, is exactly what troubled me last year when I was working on a big VB.NET project. VB allows this kind of syntax and it is terrible for readability and thus maintainability. By allowing this syntax, we might gain 5% while typing code, but we loose 30% while reading code. Therefore I am against adding such a feature to C#.

  • Ok, first of all, don't take the comments on these posts personally. You know how easy it is to exaggerate or even throw unbased accusations at you or your team on the internet if you are not speaking face to face.

    Indeed, I am well aware! -- Eric

    These people don't even mean it personally. They are just concerned that their voice is not heard by the C# team, even if your blog alone is proof enough, that this is not the case.

    I know. But I find it pragmatic to occasionally firmly remind people that the C# team is made up entirely of people. Moreover, people who take their responsibilities extremely seriously and care deeply about the long-term viability of the whole developer ecology on our platforms. -- Eric

    As for extending C# with top level methods:

    When I'm developing with VS I don't mind the overhead of declaring a class for my methods, because that is largely handled by the IDE and file templates. So tooling is an important thing to consider.

    However as I said in my last comment, I can see the benefit of global methods in embedded screnarios, where you use C# to script a third-party application, but this can be done without extending the general language. You could do this with special library support (exposing the functionality from e.g. ASP.NET or T4 to "embed" C#) or by making the compiler pipeline more flexible.

    Sure. Those are all ideas that we're knocking around. But no matter how we package the functionality, the fact remains that if we do the feature in any form, we're going to need to fully design the semantics of all interactions between new and existing "top-level" entities. That's going to be the interesting and difficult problem in this space. -- Eric

  • If you add top level methods the global scape will become cluttered like PHP.

  • Eric,

    My big concern is how "top-level" the methods/functions are.  Are these truly global?  Or will they always be contained in within a namespace, giving the user control over which are visible within a particular scope?

    I think many people are worried a potential "global" mess - every ex-C++ programmer has run into this before. :)

  • Good discussion.

    I agree with MaBerY that PowerShell lends itself nicely for executing user-supplied code, when late-bound extensibility is desired. Since PS can access .NET classes it is a great batching tool and lowers the entry bar on the technical skill required to write simple (or quite sophisticated) scripts.

  • Interesting use of c# in scripting for Unity 3D (http://www.unity3d.com) you should check it out..

  • I don't get the entirety of this post, nor the responses of people.

    What are we talking about? Is this for adding a feature to a strongly-typed language in order to make method invocation ambiguous in order to lower development cost?

    No, to my knowledge, that's not what this post is talking about. This post has nothing to do with ambiguous method invocations. Some of the comments might be about that, but that's not what this post is about. This post is about evaluating the benefits and costs of providing a standard mechanism whereby tool providers can make it easy to let their users define what look like "top level" methods in C#. -- Eric

    Wouldn't it be smarter to lower development cost in this case by choosing a different language that already offers the feature you're talking about? Why add it to C#?

    It might be smarter. We will not know whether that is smarter or not until we evaluate the costs and benefits of a number of different alternatives. We add features to C# because we believe that the benefits of enabling the compelling scenarios outweighs the costs. -- Eric

    To me, the value of the various languages offered within the .NET family are their differences, and the boundaries around each of them. This is what provides developers choice, and funnels different classes of developers (organized by goals/priorities) to appropriate languages/grammars. To me, the power of C# is the rules of the game it employs, and in turn the higher quality of code and maintenance ensues therein.

    And this is meant to be personal, but not meant to be offenseive... I did read your About section below, and the fact that I read that you started off working on runtimes for scripting environments makes me immediately understand why you are looking to bring those ideals to C#. But if scripting is your hammer, don't let C# be your nail.

    Though that is relevant, what is probably more relevant is my years on the VSTO team, trying to figure out how to make C# extensions to Office work in a seamless and attractive way. It was very difficult to do so; the new C# 4.0 features will make it easier, and I hope we can make it even better in the future. -- Eric

    In light of features that would be useful in this sort of arena, there have got to be better, more openly reusable ideas for C#. Like, for example, why I cannot create a String populated with what looks like a class and, at runtime, compile it to get a Type. Can I not do this in VB.NET? Is this not purely a .NET problem, what possible reason other than feature selection woudl it be offered in VB.NET and not C#?

    This feature is supported by JScript.NET. Baking in that feature required careful compiler design from the ground up. It would be quite difficult to shoehorn such a feature into C# or VB, when the compilers were not designed for that. -- Eric

    In the end, there are bigger fish to fry than getting rid of "Math.". I have not encountered a senior engineer that complains (especially, to the point that they refuse to do it at all) about how extension methods also must exist in an arbitrary static class. At least, then, the class name matches the file's name, which makes maintenance straightforward.

    As I've said a couple of times, the compelling benefit of top level methods has little to do with the call side. Saving those five keystrokes is not interesting. The compelling benefit is for the convenience of the people writing the callee side. -- Eric

    And in the end, it can't always be about cranking out code faster. Quality engineering devotes time and energy to concerns with maintenance, to the point that we sacrifice productivity during invention to produce an elegant solution our business can live with once we've completed. What we have to live with is ultimately more important than how we manufacture it... ask any engineer starting a new job that has to inherit someone else's code. :)

    It's our goal to make tools that make it possible for users to produce high-quality implementations in low time. That's the definition of "productivity". We want to improve both on the quality and the time side of the equation. -- Eric

     

  • Eric,

    You have made many valid points and thus far I have been impressed with the path the C# language is taking but along my travels I have seen many people complain about the language.

    When I ask what the issue is the reply after some debate is simply "Because Microsoft are still trying to take over the world" ... personally, if a product is great it will flourish, you guys know your stuff and do it well, that's why your technology and tooling is used by mmost of the planet not because Bill threw money and beat people in to submission with lawyers.

    I think that this does however highlight that whilst Microsoft generally get it right they often make a mess of things too (eg. Vista) and this could be one of those situations if it's done wrong.

    I agree with some of the suggestions above like the math class for example, it's used everywhere so why not, even half the system namespace could likely be used this way if it was really thought out, but lets not forget that one of the main pulling points for the C# language is that core structure, it works after all and this could be fatal overkill if it's introduced to heavily.

    Whilst in principle I agree, I think its a potential very thin line to walk.

    On the other hand ....

    Just because a feature is there it doesn't have to be used right?

  • I'm responsible for making sure other developers are writing quality code.  My concern is that C# is quickly becoming a language that is too easily abused and that I'll spend more time telling junior developers what not to do other than what to do.  I understand Msoft's business need to make a highly interoperable platform that allows easy integration in heterogeneous enterprise environments.  However a new feature like this would be more appealing to me if there were someway I could turn it off (like a compiler switch) when I know that I, and everyone else on my team for that matter, shouldn't be using it.  

     Var is a great example of a well intended and quite essential new language feature that is being severely abused (imho).  Since the arrival of var, I rarely see developers type out the actual type anymore.  If I remember correctly, both Richter and Lowy recommended not even using type aliases because they could lead to confusion.  Now I don't even see those anymore.  I just see var everywhere.  It hurts readability when used like that.  Its not that big a deal, however it does seem to be an indication of things to come.  I'm just waiting to see dynamic popup for parameters everywhere.  Also, Hearing that VB and C# are going to be on the same roadmap going forward is disturbing.  Will C# become more like VB or VB like C#?  I hope the latter.  To sum up what I'm trying to say, if you guys want to enhance C#'s scripting features, please at least give me some way to turn them off.  Thanks for taking the time to read this.

  • Ryan's comments echo mine. Being able to "audit" usage of features which can be abused is [imho] VERY significant. Yet it can still be bypassed...

    I had a client a while back that had very reasonable restrictions on what .NET assemblies could be referneced by various types of developed code (e.g. no using Windows.Forms in a "Server" application.)

    One developer really wanted to use a helper from an assembly, but knew that the process would flag it if he referenced it. So he loaded it dynalically and accessed the desired routine via reflection. This made it all the way through the QA cycles and was actually deployed into product for a few months before it was detected. [He no longer works there, for this and other similar "cleverness"]

    Ryan's concern that "I'll spend more time telling junior developers what not to do other than what to do.", is already a regular and significant part of my time with many clients.....

Page 2 of 4 (51 items) 1234