Here are some techniques to dynamically modify a running Azure application. They all rely on using blob storage as source of modification and presume that the application was originally written to use these changes:

1. New/changed Silverlight applications

You can store your Silverlight xaps in blob storage and reference them from pages served by a web role. You can use either public blob storage and Shared Access Signature to control access to these blobs.(See Steve Marx' blog). You will need to make sure cross-domain policies are set up so that the application can access the the server, which will obviously not be the blob server. You can store meta-data for the applications either in blob meta-data storage or, particularly if you need to search over it in SQL Azure.

2. Customization of ASP.NET look-and-feel

You can store a variety of ASP.NET artifacts (master pages, aspx pages, graphics, etc.) in blob storage and reference it from pages you serve from a web role. Furthermore, you can use an execution-context sensitive retrieval of these references to modify the source of these references to change the appearance per user (e.g. in a multi-tenant branded web site). See my previous blog for details.

3. Dynamically loaded assemblies

You can store additional assemblies in blob, discover and load them at run-time. With this approach you need to consider the permissions of the account you are running under and whether you would want to unload/change the assemblies while executing (you would need to load separate app domains for that). This approach wasdescribed by Manu. I am attaching another sample using this approach in a web role.

4. Separate mountable disk

This approach is useful if you are creating a vm role larger than the maximum upload size of 64Gb. Create the initial vm on two or more disks, create the vm role from the main drive and upload the additional drives as mountable blobs. You will need to mount them as read-only snapshots to allow multiple instances to mount the same blob. Also, note that a single-disk vm role when created in Azure has additional disk(s) so do not assume that your additional disks will be mounted as D, E, etc.

Attached is my presentation and some samples showing these approaches.