- Tangible T4 Editor
-
I've been remiss generally in not blogging while we've been heads down on Visual Studio Beta1, but never more so than in respect of another T4 Editor that's recently become available.
The good folks over at Tangible Engineering have been releasing several versions of their T4 Editor for Visual Studio 2008 and now have released a shiny version for Visual Studio 2010 Beta1. How's that for speedy?
Under Visual Studio 2010 Beta1, you can find this T4 Editor by simply going to the new Extension Manager (find it under the Tools menu)
So for all you T4 fiends, you now have a choice of editors (in strictly alphabetical order)
| | Visual Studio 2010 Beta1 | Visual Studio 2008 | Visual Studio 2005 |
| Clarius Consulting | Soon, I'm sure. ;-) | Get it | Get it |
| Tangible Engineering | Get it | Get it | |
Keep on transforming those templates.
- DSL Tools 2010 Beta1 Launches!
-
Jean-Marc announced today that the latest new beta version of DSL Tools launched today hot on the heels of the recent Visual Studio 2010 and Visual Studio 2010 SDK beta releases.
The team is really proud of all the great features we're shipping in this release. I hope you enjoy them too. From Jean-Marc's blog:
- Different models can now interact with each other, (and with Visual Studio Team System Architecture UML designers), using the ModelBus. A DSL author can choose to generate a ModelBus adapter, that will expose his model to other models or tools.
- Databinding support has been added, allowing Windows.Forms and WPF form-based designers to be created by binding a standard winform or WPF-based UI directly to DSL models. This enables developers to quickly create designers such as the .ResX or .settings designers in Visual Studio.
- It is now possible to have completely or partially read only models, which can be used for instance by reviewing and commenting tools.
- A number of UI enhancements have been added, including :
- moveable labels for connectors,
- sticky toolbox (when the user double-clicks on an item in the toolbox,it's not necessary to return to the toolbox for repeated applications of the tool),
- quick navigation and editing of compartments with the keyboard
- Copy and paste of diagram elements to images (in Bitmap and .wmf/emf)
- Copy and paste of model elements in or between diagrams
- The notion of DslLibrary has been introduced. This enables factorizing and componentizing DSLs (for instance having several domain models have the same base-domain class). The authoring for this feature will not be present in Beta1
- The DSLs can now be extended by third parties after they have deployment. The authoring for this feature will not be present in Beta1
Check out the Code Gallery page for samples and beta docs.
I'll start to list out in more detail the new T4 features after I get my breath back a bit. :-)
- Customizing Generated Resources in DSL Tools
-
Most of the code we generate with DSL Tools is pretty agnostic about its location or the filename you use for the template or the generated output.
The one exception to this is the generated resource file (usually a pair called DomainModelResx.tt/DomainModelResx.resx)
The reason this puppy is a little more sensitive is that the output resx is generated as item with a compile type of Embedded Resource in the Dsl csharp project that it lives in.
This causes the project system to pick up the relative path from the project root to the resx file and use that to form the Id that is used to embed the resource into the assembly.
Naturally, we also have to reference the resource at runtime in order to make use of its contents and to do that, we need to encode its Id at the point of use. This happens to be in the DomainModel.tt/DomainModel.cs pair.
What this means is that we have to provide some logic for generating that resource Id in our T4 templates as there isn't an API in Visual Studio to get hold of it :-(
Rather than trying to be too clever, we just use a simple scheme that expects the resource to be in the default GeneratedCode folder structure.
OK, that's all a bit abstract, so here's an example.
Given a normal folder structure like this:
You'll get a resource created like this:
You can see that the GeneratedCode folder name has been encoded into the resource Id.
We'll match that in the generated DomainModel.cs file with the following snippet
1: /// <summary>
2: /// The base name of this model's resources.
3: /// </summary>
4: public const string ResourceBaseName = "MicrosoftCorp.DemoLanguage.GeneratedCode.DomainModelResx";
So now suppose you go and rearrange your DSL project and now there's an intermediate directory called Dummy? And how about you also rename the template and generated file to be simply DslResources.tt/resx?
You'll get an embedded resource like this.
So far, this is all just regular VS stuff and not much to do with DSL Tools, but if we peek into DomainModel.tt/.cs, we'll see that the generated code has no idea that it has been moved and doesn't change that ResourceBaseName constant.
So how do we fix it up? We wouldn't leave you in the lurch now would we? No, of course not.
As an aside, we've done some work in DSL 2010 to let that directory change peek through into the template, but that doesn't help with the resource file name change.
To allow fine-grained control over matching the generated code path and resource name, we added three optional parameters to the DSL directive processor used in DSL Tools' .tt files:
- GeneratedCodeFolder: allow you to specify the folder relative to the project root where the generated resources live.
- GeneratedResourceFile: allows you to specify the filename for the resource base name
- ProjectDefaultNamespace: allows you to override the project file's default namespace as the starting point of the resource base name
Here's an example of using the first two to match up our project changes
<#@ Dsl processor="DslDirectiveProcessor" requires="fileName='..\DslDefinition.dsl';GeneratedCodeFolder='Dummy.GeneratedCode';GeneratedResourceFile='DslResources'" #>
We simply catenate the three values to form the overall base name, but keeping them separate allows us to reuse them in other scenarios. Here's the result, which matches the actual embedded resource:
1: /// <summary>
2: /// The base name of this model's resources.
3: /// </summary>
4: public const string ResourceBaseName = "MicrosoftCorp.DemoLanguage.Dummy.GeneratedCode.DslResources";
These extra parameters are pretty undiscoverable options on the DSL directive processor, so I hope this tip helps someone out. It's also a good example of how you can easily expand the directive processor we generate for you with your own DSL. These three parameters were all added purely additively in a partial class next to the generated code for the dsl directive processor. (Yes, we do generate the dsl directive processor that you use for parsing dsls to generate the directive processors for other dsls :-) )
Have fun rearranging your projects.
- Software Factories Redux
-
Don has a great post on where the p&p team are going with Software Factories and GAT/GAX.
No doubt we'll be talking to those good folks about how they can take advantage of new features in DSL 2010 in their 2010 versions of Factories.
- TellMe Voice Studio Beta1
-
I just noticed that the good folks over at TellMe (a relatively new bit of Microsoft) have shipped a beta of their Voice Studio voice application dev tool, based on DSL Tools. Very cool indeed! (I seem to be saying this a lot lately)
I love the process of developing voice apps - One of the last things I did in Microsoft before joining Visual Studio was to build a voice-enabled kiosk for helping folks access local government services. It was seriously rewarding when you managed to have testers use a natural seeming set of sentences to get something done. There's just a certain Star Trek magic to the whole process.
I think part of me may end up getting a similar buzz from extracting structured data from textual DSLs - MGrammar crossed with text extraction maybe - be interesting to go the same route as voice and have a confidence match for the textual DSL and then have a process of refinement from relatively vague textual musings to specific textual models.
Many congrats to the TellMe team (or as I'm steadily learning the colloquialisms over here, GO TellMe!)
- Clarius take T4 editing to the next level
-
I see that Clarius have now got to an alpha stage for the next stage of their T4 editing toolset.
As well as their Community Edition and Pro Edition, they're now going to offer a full-featured code generation environment they're calling Visual T4 Code Generator, including tight integration with server explorer, database tables and XML as well as a multi-file generation implementation out of the box and lots more beside.
Here's a short video of their database support:
Once again, very nice indeed.
- More on T4 in ASP.Net MVC
-
Abhishek from the MVC team has just posted some detailed instructions on using the new T4 customization features in the ASP.Net MVP release candidate
Very cool.
- T4 Adoption continues with ASP.Net MVC Tools
-
Ok, the big guns are pitching in now :-)
Hot on the heels of the tooling for LINQ-to-Entities folks adopting T4 for their next release, comes another.
Scott Guthrie just blogged about a new RC of ASP.Net MVC that will shortly be coming along. It's a long post, but down in the section on view templates in the scaffolding system, he mentions that they use T4 as a one-time generation technology to set these up based on your data.
Of course, this means that the generation is fully customizable.
I'm seeing the beginnings of a trend where product groups are finding they can get often-requested features for code generation customization by going the T4 route rather than hardcoding CodeDOM trees or some other tough to modify solution.
All isn't perfect, as of course this type of customizatio tends to require comparison of product and user templates when new versions of things come out. See Oleg's excellent encapsulation techniques (Nested Template Class especially) for ways to mitigate that.
- T4 Interview in Redmond Developer News
-
A couple of months ago, John K. Waters of Redmond Developer News was kind enough to interview Rob Conery and I about T4 and code generation in general.
You can find the results here.
- DSL 2010 Feature Dives: T4 Preprocessing - Part Two - Basic Design
-
Lat time we looked at the "WHy" of T4 preprocessing - now let's look at an overview of the how.
To explain the new features, it's helpful to have a look at the current way data flows through T4 2008. Let's suppose you have a trivial template generating some trivial output:
Now to get from the template to it's output, the T4 engine goes through three stages:
In the first, Preprocess stage, the template is parsed into a series of blocks, any directives found within it are acted on (for example, to set the output extension or to include a sub-template) and then a "Template class" is generated that represents code to produce the final output of the template. In this case, the template class would look something like the following (I've simplified a bit):
T4 supplies the base class TextTransformation, which contains the Write family of helpers which simply write to a StringBuilder exposed by the GenerationEnvironment property.
The second stage takes the text of this class and compiles it using the CodeDOM to produce a temporary assembly.
The third stage loads the temp assembly, creates an instance of the Transform class and calls the TransformText method on it to produce the final output of the template.
So that's what happens at present. What do we need to change if we want to run the template in arbitrary applications?
Firstly, we need to split up the stages. Indeed, if the template is going to be built as part of some application, T4 doesn't need to get involved in stage two at all - the user will presumably have their build environment set up for compiling and deploying application code and the template class will now just become part of that code. So we'll need new APIs and tools for Stage One that return the code for the template class, and we can ignore Stage Two.
For Stage Three, the host application will need to run the template. Clearly it can just call the TransformText method as it has a trivial API - although it may have to do so via reflection if the set of templates is in any way dynamic as there is no interface implemented here, just a standard signature. (Actually, the new dynamic feature in C# 4.0 should make this a breeze). However, this tells us we'll need a way for the new Stage One API to specify more detail (such as the name and namespace) about the template class to be generated.
Finally for stage three, the base class, TextTransformation, becomes a problem. This class lives in the T4 assembly, which only ships in Visual Studio, so you will likely not have it available in your deployment environment. Luckily the contents of this base class are rather trivial helper methods on the whole. To address this, if you don't specify a base class, we'll make T4 generate the helpers directly into your template class, relieving you of the dependency. Alternatively, it's pretty easy to add a set of helpers to any base class that you add to your own deployable code, so long as they fulfill the signature that the template class needs.
Next time we'll go into detail on what APIs will be available and then fill in some details where I've oversimplified in this explanation.
- DSL 2010 Feature Dives: T4 Preprocessing - Part One - Rationale
-
Strictly, this isn't a DSL 2010 feature, but a general Visual Studio 2010 feature; T4 is only coupled organizationally to DSL Tools, not technically, but it fits into our current announcements, so I won't worry about that too much.
Over the last year or so, I've been fielding an increasing number of requests from folks who want to use T4 to produce output in scenarios other than at design-time inside Visual Studio. Just to be clear, T4 in its 2008 incarnation is not suitable for use on the server, or in any concurrent, multi-threaded scenario - there's nothing specifically wrong with the code in this regard that we know of, but it's simply not tested in this arena, and we choose not to support something that hasn't been tested. T4 also only ships inside Visual Studio SKUs, so you have some unfortunate installation and interpretation of your EULA to do if you wanted to look at using it in another scenario.
However, we're very aware of this pent-up demand, and a conversation with the Entity Framework team brought our thinking caps out.
What we came up with was an assertion that want something like the following:
In a high percentage of scenarios for using T4 at runtime (as opposed to design-time), the template only changes as part of a software installation; rather the input data for the template is changing in order to produce differing output.
For example, you can imagine T4 being used to implement some transform between EDI data and XML data in a business workflow. It's likely that the structure of the required XML document is an implementation artifact of the workflow, whereas clearly the instance documents of the XML are directly driven by the instance of EDI data that's presented.
On this basis, we realized that if we decoupled T4's preprocessing stage from its compile and run stages, we could produce template code that could repeatably take input and produce template output that we'd be happy to support running in any .Net environment, thus enabling a lot of the scenarios that customers had asked for.
We call this feature Template Preprocessing and next time I'll describe the high level design points.
- Nice step-by-step on adding a menu to DSL Tools 2008
-
Sebastian Talamoni has a nice guide to adding a menu using a vsct file.
Technorati Tags:
DSL Tools,
VSX
- New DSL Tools lab
-
Jean-Marc has just released a complete walkthrough of DSL Tools for Visual Studio 2008 aimed at DSL beginners as a lab on CodeGallery.
Here you can see the language from the lab designing mouse gestures for manipulating primitives in a simple graphics program.
Very nice indeed.
- Belated Welcome - Eyal Lantzman
-
Very poor show of me - I forgot to welcome Eyal Lantzman to the team.
Eyal recently joined the team in Cambridge, (I originally wrote "us" instead of "the team" there - note to self - must remember that I live in America now!). As part of this, he's made the big life change of moving over from Israel.
A very warm welcome Eyal!
Technorati Tags:
DSL Tools
- DSL Tools in Visual Studio 2010 - the cat is well and truly out of the bag
-
So after all the excitement of the PDC, now TechEd Europe is upon us and we're finally talking in some more depth about where we're going in Visual Studio 2010 with DSL Tools.
Firstly, quite a few of you have been pinging me to ask about the relationship between DSL Tools and Oslo. See Keith and Stuart's posts on the subject. Clearly both teams have their distinct focus and paths forward - As Keith says, I'm sure we'll be looking at ways forward that bring customer investments in either track closer together as time progresses.
The way I like to think about it that although we're both concerned about using DSLs to complete frameworks, currently Oslo is more focused on model execution to achieve that, whereas we're more focused on transformation of and generation from models to achieve it.
So, on to our VS 2010 feature set. Here's what you can expect in the next public release (from Stuart's announcement):
- Dsl Libraries. Share fragments of DSL definitions between different designers.
- Dsl extensibility. Extend the domain model and behavior for a DSL after it's been deployed, including DSLs shipped by a third party.
- Readonly. Selectively switch off the ability to edit models and model elements in a designer.
- Forms-based UI. Easily bind models to winforms and WPF-based forms UI. IMS now implements the necessary databinding interfaces.
- Modelbus. A new piece of platform to support cross-referencing between models and interaction between designers. This has been one of customers' main requests.
- T4 precompile. Precompile text templates so that they can be deployed for use on machines that do not have VS installed. (This applies to scenarios where text templates take inputs from data sources other than DSL models. We've had requests from internal teams to use text templates in scenarios which don't use models created from DSLs.)
- UI enhancements:
- Moveable labels on the connectors (moveably can be activated
- Sticky tools in the toolbox
- Quick navigation and editing of compartments
- Copy and paste of selection as BMP and EMF images
- Copy and paste of elements in the same or in different diagrams
I've spent most of my time so far in three of those areas, so I'll dive in to those along with Stuart and Jean-Marc over the next few months.
- T4 precompile
- Copy/paste
- Forms databinding
it's pretty exciting to suddenly have such a lot to talk about!
