Tuesday, March 8, 2011

Take NuGet to the next level with sample packages

NuGet has drastically simplified the process of getting .NET libraries into your projects. What used to be an error prone and painful process has become as simple as adding an assembly reference.

While it has solved an important part of the developer workflow, it has the potential to also solve another key piece of the puzzle: helping user learn to use libraries.

I found these cool packages, but now what?

There are tons of cool packages available on NuGet today, and the number is growing daily. I’ve heard of a number of users who go down the list and install all kind of packages into their projects to try them out. But if you’re not familiar with a library, how do you get started with it?

As an example to illustrate the discussion, let’s take the nifty little Clay package written by the Orchard guys. Say you have installed it into your project and want to start using it. Here is what you might do:

  • The NuGet dialog gives you a link to the ‘project URL’. Typically, it’s a link to where the project is hosted on CodePlex/BitBucket/github, and indeed this one takes you to http://clay.codeplex.com/.
  • Once you’re there, you try clicking on the Documentation tab. Unfortunately, many projects don’t have much there. But here it at least has a pointer to Bertrand’s blog posts on the topic. So you now go to his post.
  • You read through it, and after a while, you can piece together enough bits and pieces to know what it’s about and start using it into your code.

I took Clay as an example, but this is a fairly typical experience. The fact is that a lot of knowledge about immature (yet useful) projects only exists in ‘blog post series’ rather than in any formal documentation. Not ideal, but that’s how things happen.

NuGet to the rescue with Sample Packages

Luckily, there is a simple and effective solution to this problem: use NuGet to distribute basic samples that get your users on the right path with less pain.

So to illustrate this post, I went ahead and created one such package for Clay: Clay.Sample. This package depends on Clay, such that installing it also installs Clay (as well as other things Clay depends on, like Castle).

It’s a ‘source only’ package, meaning that it doesn’t contain any binaries of its own. So let’s go ahead and try it in a brand new Console app (and change it NOT to use the client profile). Go in NuGet’s ‘Add Library Reference’ dialog and search for Clay. You’ll get this:

image

After you install it, your project will look like this:

image

First, note how you got all the expected references to Clay and to its dependencies: Castle.* and log4net.

But more interestingly, it also brought in a ClaySamples source file under Samples\Clay. It contains a number of Clay samples, which I shamelessly copied from Bertrand’s post. Here is one example:

public static void AnonymousObject() {
   dynamic New = new ClayFactory();

   var person = New.Person(new {
       FirstName = "Louis",
       LastName = "Dejardin"
   });

   Console.WriteLine("{0} {1}", person.FirstName, person.LastName);
}
There are about 10 such samples in there, which demonstrate everything that the post discusses. Now go to your Console Main and make a call to a method that runs all the samples:
class Program {
   static void Main(string[] args) {
       Samples.Clay.ClaySamples.RunAll();
   }
}

While there is nothing in there that’s not in the blog post, the big advantage is that you can trivially get it into your project via NuGet, and you can then directly run/debug the samples without having to piece them together.

Of course, the blog post (or documentation) may still be worth reading for extra insight. But you may find that the samples give you all you need for now, and save the deeper reading for later.

Call to packages authors: write Sample packages!

I think this type of packages can have a huge impact on developer productivity. But for that to actually happen, those packages need to be created! And while I created the one for Clay, I am not volunteering to create all the sample packages :) Clearly, the best person to do that is the author of the package, though anyone who knows it well enough can certainly do it as well.

So if you own a NuGet package, please try to take on that task. It’s super easy, and your users will thank you for it!

Conventions, conventions, conventions

I recently blogged about using the App_Start convention for WebActivator startup code and got a great response, with almost all WebActivator users converting their existing packages to use this.

The situation here is quite similar, and calls for a similar convention, which is what I showed above. In a nutshell:

  • If your package is named Blah, call the sample package Blah.Sample. If you want multiple sample packages, you can call them Blah.Sample.Something and Blah.Sample.SomethingElse.
  • Make your Blah.Sample package dependent on Blah.
  • Within that package, just include source files. Place those file under the Samples\Blah. You can have one or more, and call them whatever you think make sense.
  • The code on there is up to you, but the general idea to to include whatever you think will help the user get started. Try to make the sample code easily runnable without too much extra setup. This may be harder for some packages, but do your best :)

Creating the package

Taking Clay as an example, here is the structure of the files before packing them into a nupkg:

├  Clay.Sample.nuspec
└──Content
└──Samples
 └──Clay
    └  ClaySamples.cs.pp

So there are just two files, the nuspec and the preprocessed sample file. Here is the nuspec:

<?xml version="1.0"?>
<package xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<metadata xmlns="http://schemas.microsoft.com/packaging/2010/07/nuspec.xsd">
<id>Clay.Sample</id>
<version>1.0</version>
<authors>Outercurve Foundation</authors>
<owners>Outercurve Foundation</owners>
<licenseUrl>http://www.opensource.org/licenses/ms-pl</licenseUrl>
<projectUrl>http://clay.codeplex.com</projectUrl>
<requireLicenseAcceptance>false</requireLicenseAcceptance>
<description>This package contains samples that demonstrate the use of the Clay library.</description>
<language>en-US</language>
<dependencies>
<dependency id="Clay" version="1.0" />
</dependencies>
</metadata>
</package>

The interesting parts here are the package Id, the description, and the dependency on Clay.

Then ClaySamples.cs.pp is a normal source file, except for a tiny bit of preprocessing for the namespace, e.g.

using System;
using ClaySharp;

namespace $rootnamespace$.Samples.Clay {
   public static class ClaySamples {
      // Sample code here
   }
}

And that’s it! Once you have that, just run ‘nuget pack’ from the folder with the nuspec, and you’ll have a sample package ready to be pushed to the feed.

Sunday, March 6, 2011

Using a custom build of MVC 3

Disclaimer: running a custom build of MVC 3 is not recommended in most scenarios. Please run against the official MVC 3 bits unless you really cannot. Be aware that using custom builds will make it harder to use 3rd party libraries built against the official bits (you’ll need to rebuild those libraries).

One nice thing about ASP.NET MVC is that you can get the full sources on CodePlex and compile them yourself.

Once you copy the sources locally, you can open the WebRuntime solution in VS 2010 and build it. As an aside, note that this solution also contains the ASP.NET WebPages sources, since MVC3 makes you of some of that logic (mostly for Razor support).

So this lets you modify the sources and build everything. However, one thing that makes the use of the resulting assemblies a bit tricky is that unlike the official bits, the bits produced by this solution are unsigned.

Let’s take this step by step.

Step 1: Create a new MVC 3 application

So let’s start by creating a new MVC 3 Web Application using the regular project template that come from installing MVC 3.

This gives you a working app, but obviously at this point you're still using the official MVC 3 bits.

Step 2: Reference your custom assemblies

The next step is to reference your custom MVC assemblies. Start by removing the System.Web.Mvc and System.Web.WebPages references. Instead, reference the version you’ve built of those same assemblies, which you’ll find under mvc3\bin\Debug (from the root of the WebRuntime solution).

Once you do that, your MVC project will build fine. However, if you try running it, you’ll get some scary looking runtime compilation error. Something like:

CS0433: The type 'System.Web.Mvc.WebViewPage<TModel>' exists in both 'c:\Windows\Microsoft.NET\assembly\GAC_MSIL\System.Web.Mvc\v4.0_3.0.0.0__31bf3856ad364e35\System.Web.Mvc.dll' and 'c:\Users\David\AppData\Local\Temp\Temporary ASP.NET Files\root\d305385c\948d4291\assembly\dl3\ef116fd6\5f110ce1_44dccb01\System.Web.Mvc.DLL'

The reason this happens is that while you’ve changed the project references to point to your assembly, the two web.config files that come with the project template are still pointing to the official assemblies left and right. Which leads us to…

Step 3: Fix up your web.config files

The project comes with two web.config files, and they each contain all kind of references to the official assemblies (which are strong name). e.g. in the root web.config, you’ll find:

<add assembly="System.Web.Mvc, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
<add assembly="System.Web.WebPages, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35" />
Luckily, the fix is as simple as yanking the strong name from a few assemblies. This is easily done using a project-wide search/replace. Specifically, do the following three replacements:

1. Replace all instances of (excluding the quotes!)

“System.Web.Mvc, Version=3.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35”

by

“System.Web.Mvc”

2. Replace all instances of

“System.Web.WebPages, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35”

by

“System.Web.WebPages”

3. Replace all instances of

“System.Web.WebPages.Razor, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31BF3856AD364E35”

by

“System.Web.WebPages.Razor”

And that should be it. Your app will now be up and running against your custom MVC bits.

Friday, February 25, 2011

App_Start folder convention for NuGet and WebActivator

[Please see the WebActivator wiki for the latest docs]

When I first blogged about WebActivator, I showed in my example using a source file named AppStart_SparkMvc.cs.pp under the Content folder in the package, which means when you install it you end up with a file named AppStart_SparkWebMvc.cs at the root of your web project.

Now suppose you install a few more packages that use the same WebActivator pattern, and you would end up with something like that at the root of your project:

AppStart_SparkWebMvc.cs
AppStart_SQLCEEntityFramework.cs
AppStart_BarPackage.cs
AppStart_BlahPackage.cs
Global.asax
Global.asax.cs
Web.config
More files...

That starts getting really ugly, and most devs like to keep the root of their app free of clutter.

We need a better convention!

The solution is simply to agree on a different convention where we put all this startup code into a folder. To match ASP.NET conventions, the obvious name to pick is App_Start. And once we do that, we no longer need to prefix the file names with AppStart, so we would have:

App_Start
SparkWebMvc.cs
SQLCEEntityFramework.cs
BarPackage.cs
BlahPackage.cs
Global.asax
Global.asax.cs
Web.config
More files...

Likewise, the full class names would change from WebApplication1.AppStart_SQLCEEntityFramework to WebApplication1.App_Start.SQLCEEntityFramework. Note that the namespace doesn’t matter a whole lot since you won’t call this code explicitly. But since existing convention is to have the namespace match the folder structure, we may as well do that here.

As of today, there are 17 packages that use WebActivator, so I’ll need to try to convince all the authors to follow this. Fun time ahead! :)

But note that it’s just a convention, with no code changes to enforce it. Nothing written here breaks any existing packages. It’s just something where by agreeing on a better convention, we make NuGet yet a little bit better!

An example: EFCodeFirst.SqlServerCompact

As an example, here is what I ended up with for the EFCodeFirst.SqlServerCompact package using this pattern.

The source file transform in the package is in Content\App_Start\SQLCEEntityFramework.cs.pp, and contains:

// namespaces, etc...

[assembly: WebActivator.PreApplicationStartMethod(
typeof($rootnamespace$.App_Start.SQLCEEntityFramework), "Start")]

namespace $rootnamespace$.App_Start {
public static class SQLCEEntityFramework {

//etc...
Note the use of $rootnamespace$ and of App_Start in the namespace.

Thursday, February 24, 2011

New features in WebActivator 1.4

[Please see the WebActivator wiki for the latest docs]

Back in October, I blogged about the WebActivator NuGet package, which allows packages to bring in some source code that runs on startup in a Web Application. It’s been a pretty popular package, as there are many scenarios where running startup logic is important. The alternative of forcing the user to modify their global.asax is just not compelling.

There have also been a few feature requests since the initial 1.0 release, and I will describe them here.

Ability to run code after Application_Start

When you use a WebActivator PreApplicationStartMethod attribute, the method it points to runs before your global.asax’s. Yep, that’s fairly obvious from the name Pre ApplicationStart :)

But in some cases, that’s just too early. Scott Hanselman ran into that when trying to register MVC areas, and I added this feature as a result (see his post).

This feature works exactly the same as the PreApplicationStartMethod attribute, except using a different attribute named… drums rolling… PostApplicationStartMethod! e.g.

[assembly: WebActivator.PostApplicationStartMethod(
typeof(TestLibrary.MyStartupCode), "CallMeAfterAppStart")]

So when does that run exactly? It runs at the time the very first HttpModule get initialized. Internally, it’s using the dynamic module registration mechanism I blogged about recently.

Ability to run code when the app shuts down

WebActivator can also help you execute cleanup logic when the app shuts down. This is done via yet another attribute that works much like the other two, e.g.

[assembly: WebActivator.ApplicationShutdownMethod(
typeof(TestLibrary.MyStartupCode), "CallMeWhenAppEnds")]

This code runs at the time Dispose is called on the last HttpModule in the app.

Support for code in App_Code in Web Sites

In a Web Site (as opposed to a Web Application), you typically put your shared code in the App_Code folder. Now if you have code in there that uses the PostApplicationStartMethod attribute, it will get called when the app starts, giving Web Sites some WebActivator love.

Please note that you can only use PostApplicationStartMethod in App_Code, and not PreApplicationStartMethod. The reason is that when PreApplicationStartMethod fires, the App_Code folder has not even been compiled!

Support for invoking the start methods outside of ASP.NET

This change came courtesy of Jakub Konecki, who needed it for unit testing purpose. This comes as a set of static methods that you can use to invoke the startup methods:

// Run all the WebActivator PreStart methods
WebActivator.ActivationManager.RunPreStartMethods();

// Run all the WebActivator PostStart methods
WebActivator.ActivationManager.RunPostStartMethods();

// Run all the WebActivator start methods
WebActivator.ActivationManager.Run();

// Run all the WebActivator shutdown methods
WebActivator.ActivationManager.RunShutdownMethods();

You can find the WebActivator sources on bitbucket.

Monday, February 14, 2011

Build your Web Application at runtime

Disclaimer: let me start by saying that the technique described in this blog is experimental, and is meant as a first step to see where this might take us. This is not in any way an officially supported technique!

If you are an ASP.NET user, you are likely aware that there are two different types of apps that you can create: Web Sites and Web Applications. Here is a quick summary of how they differ:

Web Sites

In web sites, all compilation is done at runtime rather than design time. They don’t use any VS project systems, and msbuild is never involved.

Advantages: very dynamic. You can just FTP files to the server, and everything just works. In that sense, it’s similar to ASP Classic and PHP.

Disadvantages: lack of fine control over the build process; hard to unit test; often slower in VS; not available for MVC.

Web Applications

In Web Applications, all the source code is built by VS in the designer using a standard .csproj file and msbuild. Pages and views (.aspx, .cshtml, …) are still built dynamically at runtime, so it’s sort of a mixed mode model.

Advantages: full power of msbuild, easy to unit test code, fast build in VS.

Disadvantages: once you xcopy your built app to the server, you can’t modify the code by just changing files (though you can still do this for pages & views).

What if we could get the best of both worlds?

I was recently chatting with my coworker Louis DeJardin about compilation models, and he put out the idea that we might get something interesting if we were to run msbuild on the server, which is where this came from.

In a sense, it’s sort of an ‘obvious’ thing to try if you look at the Pros can Cons of Web Sites and Web Applications. We want the full power of msbuild, but we also want the more dynamic nature of Web Sites, so the only logical thing to do is to run msbuild dynamically on the server!

Try it now using NuGet!

Before I give you more details, let me show you how you can try this in no time via NuGet:

  1. Create a new MVC app
  2. Install my ‘WebAppBuilder’ NuGet package
  3. Run the app
  4. Change the message in Controllers\HomeController.cs, and don’t rebuild
  5. Refresh the page in the browser (and then again per the message you’ll get)
  6. Now try to make a change with a compile error and refresh again

How does it all work?

There really isn’t much code at all to make this work. First, it uses the technique I described in my previous post to dynamically register a module. This is what allows it to kick in without any registration.

Whenever the appdomain starts, the module looks for the csproj file and builds it. Doing this is quite simple since msbuild is well exposed to managed code (take a look at Microsoft.Build.Execution.BuildManager). Note that it always does that on startup, with the assumption that the incremental build will be super fast if there is nothing to build.

Then if something actually got built, it sends back a simple page telling the user to refresh. This is a bit ugly as it effectively takes two refreshes to get the result, but it’s necessary since we can’t use the freshly built assembly in the same domain used to build it (since creating it causes a domain unload).

The other thing it does is listen to file change notification so it can unload the domain if any source files change. Then on the next request things get built as above.

There may be smarter ways of doing this, but this works pretty well as a proof of concept.

You can see find code on bitbucket.

Caveat: requires full trust

One big caveat of this approach is that it doesn’t work in partial trust, because launching msbuild requires full trust. This is not something that I think can be worked around easily, so I’d say it’s an inherent limitation.

Where can we take this?

Well, I’m not really sure yet, but it is certainly interesting to think about the possibilities of using this type of build model in ASP.NET.

Let me know if you think this is crazy or may have potential :)

Sunday, February 13, 2011

Register your HTTP modules at runtime without config

In ASP.NET 4, we added the concept of a PreApplicationStart method that an assembly can use to execute code early on in the appdomain without any configuration. Phil Haack covered it a while back in this post. It’s pretty simple to use. You just define a class that looks like:

public class PreApplicationStartCode {
public static void Start() {
    // Your startup code here
}
}

And then you add an assembly level attribute pointing to it:

[assembly: PreApplicationStartMethod(typeof(PreApplicationStartCode), "Start")]

With the release of MVC3 and ASP.NET Web Pages, we added another little gem: a RegisterModule() API that lets you dynamically register an IHttpModule without touching config. Sadly, the method is hidden so deep that it is hard to find by accident (it’ll get cleaned up in the next framework version).

By combining the two techniques, you have everything you need to register a module dynamically, e.g.

public class PreApplicationStartCode {
public static void Start() {
    // Register our module
    Microsoft.Web.Infrastructure.DynamicModuleHelper.DynamicModuleUtility.RegisterModule(typeof(MyModule));
}
}

I warned you it was well hidden! :)

The module type that you pass in to that is just a standard IHttpModule, e.g. here is a basic module that writes to the response on every request:

class MyModule : IHttpModule {
public void Init(HttpApplication context) {
    context.BeginRequest += (sender, e) => {
        var response = ((HttpApplication)sender).Response;
        response.Write("MyModule.BeginRequest");
    };
}

public void Dispose() { }
}

The beauty of this is that it allows you to create fully encapsulated assemblies that you can just drop into a web app’s bin folder and have them light up without having to add any ugly registration to the app.

And yes, all this works fine in partial trust!

You can download a minimal sample from here.

Thursday, February 3, 2011

Creating Console apps that use the Server Profile

When you create a Console app in Visual Studio 2010, it gets create in a way that it targets the “.NET Framework 4 Client Profile”. What that means is that it can’t use any ASP.NET component, which for ASP.NET developers is rather useless.

The standard way to fix this is to go to the Project’s properties and change the Target Framework:

image

You then get an alert telling you that your project needs to be reloaded:

image

And once you click yes, you can be on your way to other greatness.

While this works, it’s frankly painful when you have to do this many times a day. To make things worse, if you forget to do it, you often get strange failures which don’t make it obvious what the issue is, leading to frustration or worse.

I have no idea who was behind the decision to make the default be the client profile, but I’ll go on record saying that it was a dumb idea! :)

Fix this permanently using a custom project template

Luckily, it’s pretty easy to fix this by using a custom VS Project Template. @luhmann sent one to me, so I didn’t even have to write it :)

Here is what you need to do:

  1. Go to this folder: %USERPROFILE%\Documents\Visual Studio 2010\Templates\ProjectTemplates\Visual C#\
  2. Under that, create a ‘Windows’ folder if you don’t already have one (you probably don’t)
  3. Download the custom template from here, and save it into that Windows folder (but don't unzip it!).

Now when you need to create a C# Console app, you’ll see a new entry from the custom template:

image

If you use that, your console app won’t be using the evil Client Template, which will lead to greater happiness.

Note that if you really wanted, you could replace the default template by that one, but I like seeing both entries side by side as a reminder of what’s going on. And who knows, some day I might just want to use the Client Template!