Tuesday, February 24, 2009

Web Packaging: Creating web packages using MSBuild

 

This post is next in the series of VS 2010 articles that we have been putting together to dive into the Web Deployment improvements with VS 2010 and IIS.  I would recommend reading the the preceding posts to get an overview of all the scenarios supported:

In this post I will cover web package creation using MSBuild command line.  Many medium to large sized teams plan on automating their build process for various good reasons like predictability for QA team, time saving as compared to on-demand manual build, early bug detection with Build Verification Tests (BVTs), knowing the current state of project integration, etc… Many argue that setting up the build system is not worth the trouble for a small project running only a few months; I would suggest otherwise, believe me setting up an automated build process once will pay you back   enough just within a few weeks and will get you into a mode where in the future doing this will be so much more easier… 

Anyways, if you choose to automate your build process there are various tools and technologies available out there, some of the popular ones are:

You can certainly take your build automation process to its best by using Continuous Integration model which we will discuss in subsequent posts.

In anycase, the entire Web Deployment story in VS 2010 uses MSBuild behind the scene which means that all the UI features in Visual Studio are actually wrappers over the underlying MSBuild Targets, Tasks and Properties.  In the previous post we talked about “Creating a Web Package using VS 2010” where we discussed setting up the Package properties in “Package Tab” of the project’s property pages as shown below:

All the properties that you set up in this UI are stored in your .vbproj or .csproj file.  We also talked about this tab being “Configuration” aware, which means that you can set different properties per build environment like Debug, Testing, Staging, Release/Production etc and all of these properties will be saved in your project file.

Now if you would like to create a web package using MSBuild it is much more simpler than you can imagine:

All you have to do is open  command prompt which has MSBuild path preset (e.g. Visual Studio Command Prompt which is available under Visual Studio 2010 –> Visual Studio Tools) and type the below command:

MSBuild "YourFullyQualifiedProjectName.csproj/vbproj" /T:Package

/T:Package is the MSBuild Target named Package which we have defined as part of implementation of the Web Packaging infrastructgure.

Interestingly, when you do not specify any MSBuild target, then for most projects “Build” is the default target hence just providing below command line simply builds your project

MSBuild “YourProject.csproj”

Also note that there can be various dependencies set between targets and our “Package” target has an explicit dependency on “Build” target which means that if the “Build” was not successful then “Packaging” will not even begin, this ensures that during your automated packaging you do not land up spending resources on creating faulty web packages.

By default MSBuild uses the “Debug” configuration but if you would like to create a package for your Staging configuration all you would have to do is:

MSBuild "ProjectName.csproj/vbproj" /T:Package /P:Configuration=Staging

/P:Configuration represents the Property named Configuration which you are setting to Staging…

Diving a tiny bit deeper - If you open your project file in a text editor then you should be able to see all the properties which we talked about from UI perspective in our previous post “Creating a Web Package using VS 2010”…  All these properties will not be visible in the project file until their default values are modified (just a tiny optimization to keep the files smaller and agile :-)). These same properties are optionally settable from command line as well...  Also there are certain properties which are not manifested in the UI or in the project file by default, but are still available behind the scene to provide extensibility and fine grain control that many expect, we will go into the details of those properties in later posts as well.

Anyways, most of the time you should be able to set most of your properties in the UI and use them without much modification in the command line scenario, although it is conceivable that some of the properties may require frequent modification during automated builds e.g. “Package Location”.  Below is a sample command of how you will set up the PackageLocation property along with the Configuration property:

MSBuild "MyProjectName.csproj" /T:Package /P:Configuration=Staging;PackageLocation="D:\Vishal\Package.zip"

When I run the above command then my package for “Staging” configuration will be created in “D:\Vishal\Package.zip”

It is important to note that items passed via command line override the values set in the project file, this ensures that most common values of the properties can be stored in the project file and eventually shared by the entire team…  The ones which need to be momentarily overridden during build time can be set from the command line. 

Also it is good to remember that if you like to pass more than one property to MSBuild command then you can do so by separating multiple properties by semicolon ; as shown above for Configuration and PackageLocation.

The above command line examples can very easily be plugged into automated build systems like CC.Net, TFS, etc, we will look into the process of setting some of these environments in later posts as well.

For now, I hope you will be able to envision the prospects of creating these Web Packages in an automated fashion and share them across your teams on regular basis.

Sunday, February 08, 2009

Web Packaging: Creating a Web Package using VS 2010

In the earlier post I highlighted various investments that we are making in Visual Studio 2010 and IIS to make Web Deployment easier.  You can read that post below:

Deploying a web project with all its correct dependencies is not a trivial task. Some of the assets which need to be considered during deployment are:

  • Web Content (.aspx, .ascx, images, xml files, PDBs, Binaries etc)
  • IIS Settings (Directory browsing, Error pages, Default Documents etc)
  • Databases that the web project uses
  • GAC Assemblies and COM components which the web project depends upon
  • Registry Settings that may be used within the web project
  • Security Certificates
  • App Pools

In an enterprise environment a web application with all of its dependencies needs to move across various environments before being finally being deployed to a production server.  A typical set of transition servers are development, testing/QA, staging/pre-production and production.  Also on the production environment there are web farms where these webs need to be replicated.  Today doing all these things is more or less a manual process and involves a tons of documentation that both developers and server admins have to deal with.  Even with all the documentation the steps are certainly very much prone to errors.

To aid all these scenarios we are introducing the concept of  a "Web Package". Web Package is an atomic, transparent, self describing unit representing your web which can be easily hydrated onto any IIS Web server to reproduce your web.  VS 2010 uses MSDeploy  to create the web package from your web application.

In today's post I will be primarily focusing on creating a web package from VS 2010 which has IIS Settings as well as web content.

The package created by VS can be installed using UI in IIS Manager as well as command line, we anticipate that developers eventually will give the web packages to server administrators who will be able to inspect/verify the package and then install them on the server...  I will cover package installation topic in subsequent post...  But for now let us learn how to create a web package

Step 1: Configure your Web Application Project (WAP) to use IIS Settings

For this discussion we have BlogEngine.Web downloaded from codeplex and converted it into a WAP.  Then this project was opened in VS 2010  and the VS10 migration wizard moved the project into VS10 format.  Thanks to the multi-targeting  features in VS 2010 which can support .NET versions 2.0 till 4.0; hence it is up to you which Framework version you want to run your web against.    I have also configured this blog application to use IIS Web Server for development (Learn how to do so by clicking here). 

At the end of this step my solution explorer looks as below:

image

Step 2: Configure IIS Settings in IIS Manager

Most IIS 7 web applications use IIS integrated pipeline which is configured with "Default App Pool" of IIS.  Blog Engine .web does not use integrated mode and will throw an error shown below if made to run under "Default App Pool".

image

To get rid of this error I changed the App Pool of this application to "Classic App Pool" (Learn how to do so by clicking here) and then the application runs great as shown below:

image

App Pool mapping is just one of the IIS setting which your app may use, there are various other IIS Settings which you can configure using IIS Manager (e.g. Default document, Error pages etc etc); all of these settings are relevant based on your application scenario... The good news is that VS 10 & MSDeploy will auto detect all the changes you make to the default IIS settings and pick it up for deployment...

Essentially, at the end of this step you should have your web application up and running with all the IIS settings configured in IIS Manager. 

Step 3: Configure Package Settings

In VS 2010 we have introduced one additional property page for WAPs called "Publish" as shown below:

image

Let us look at various properties of the this tab to understand how it works:

Configuration Aware Tab: Note that the Publish tab is build configuration aware:

image

  • The Publish tab is made configuration aware as deployment settings tend to change from environment to environment; for e.g. many a times developers want to deploy their “Debug” configuration on a Test Server and include PDBs as part of this deployment. When the same web is deployed in “Release” configuration on a production server the deployment may exclude PDBs.  (Learn how to manage build configurations by clicking here)

Items to Package/Publish – This section will help you decide what type of content you would really like to package/deploy.

  • Types of Files: By default this option is set to "Only files needed to run this application" .  This is usually sufficient for your deployment as it includes all the files from your project except source code, project files and other crud files not required to be deployed...  But apart from that there are two additional options available as shown below...

image

"All files in this project" and "All files in this project folder" options are very similar to what Publish WAP options were in VS 2008...  I had written an earlier post explaining these options here...  In subsequent posts I will also dig into various other interesting ways of using these options.

  • Exclude Files from App_Data folder – “App_Data” folder is a special ASP.NET folder where many developers like to put their SQL Express DBs (.mdf/.ldf files), XML files and other content which they consider Data. In many situations on production web server a full version of SQL Server is available and using SQL Express is not all that relevant. In such scenario (and for the corresponding build configuration e.g “Release” ) a user can check the “Exclude Files from App_Data”. image
  • Exclude Generated Debug Symbols – It is important to understand that generation of debug symbols is different from deployment of the same. This check box will tell VS 10 whether you would like to package/deploy the already generated Debug Symbols (Learn more about deploying debug Symbols here). 

Package Items

image 

  • IIS Settings  - Checking this checkbox informs VS10 that you are ready to take all of your IIS Settings configured for your application in IIS Manager as a part of your web package.  I am glad to tell you that IIS 5.1, IIS 6 as well as IIS 7 environments are supported as part of this feature hence whether you are working on XP, Win2K, Win2K3, Vista or Win2K8 you should have no issue with packaging IIS Settings...  

These setting includes the "App Pool mapping" your web is configured to run against (e.g. "Classic App Pool" mapping discussed in Step 2)

  • Additional Settings -   The items in this grid are advanced properties.  It is still good to know about these coz it impacts what will be included in your package.  Most of the properties in this grid are related to the entire server and not just to your application so you should use them very carefully. 

Currently VS10 only displays "Application Pool Settings" but behind the scene it is possible to configure VS10 to support packaging root web.config, machine config , security certificates, ACLs etc...  

I wrote a small tips & trick about differences between Application Pool Mapping and Application Pool Settings which will help clarify the implications of such advanced settings; you can read more about it here.

Package Settings

image

  • Create MSDeploy Package as a ZIP file - This checkbox allows you to decide whether you would like to create your web package as a .zip file or as a folder structure. If you are concerned about the size and are moving the web package around very often then I can see you using .zip format for the package; on the other hand if you care to compare two packages using diff commands (either of source control or independently) then I can see you using the folder format.
  • Package Location - This is an important and required property as it defines the path at which Visual Studio will place your web package. If you choose to change this path make sure that you have write access to the location. Do note that the Package Location is modified based on whether you choose to create the web package as a .ZIP file or vs a folder structure.
  • Destination IIS Application Path/Name - This property allows you to give IIS Application name that you will use at the destination Web Server.
  • Destination Application Physical Path - One of the most important information which is embedded inside the web package is the physical location where the package should be installed. This property allows you to pre-specify this embedded information.  You will have an opportunity change both IIS Application Physical Path as well as Application Name at the time of deployment but in this property page you are given an opportunity to choose a default value.

Step 4: Create the "Web Package"

This is the last step in creating the web package and the simplest too...  The idea is that once you configure the above settings creating a package should be easy; in fact even if you do not go to the "Publish" tab we have tried to set smart defaults so that in most normal circumstances creating web package should be just the below two steps:

image

  • Right Click on your "Project"
  • Click on Package --> Create Package

Once you click on this command you should start getting output messages around your package creation pumped into your output window... 

When you see “Publish Succeeded” as below in the output window then your package is successfully created.

 

image

To access the package go to the location specified in the “Package Location” textbox. By default this is in obj/Configuration/Package folder under your project root directory (Configuration here implies Active Configuration like Debug/Release etc).

clip_image002

Note: "Create Package" command creates web package only for Active configuration. By default “Debug” is the active configuration inside Visual Studio. If you would like to change the Active configuration you can do so by using Build --> Configuration Manager as described here. You can certainly set properties for all available configurations by switching the configuration on top of the “Publish” tab but that action does not change the Active configuration

 

Finally, you can also automate creation of web packages via your team build environment as everything discussed above is supported via MSBuild Tasks.  In subsequent posts we will get into the details of these areas too...

Hope this helps...

Tips & Tricks: Difference between App Pool Mapping Vs App Pool Settings

Application Pool is an IIS concept and will apply to an application which uses IIS as its web server.  Learn how to make your web application to use IIS during development time by clicking here...  If you application is an IIS based application then you should be able to look at its basic settings as below:

image

In this post I quickly wanted to discuss about the difference between App Pool Mapping and the actual App Pool Settings

  • App Pool Mapping - This is a setting limited to your web application in IIS...  This instructs IIS to identify the correct App Pool which your web application should run against.  It by no way changes any settings associated with the App Pool itself i.e. you are using an app pool which was pre-created/configured and essentially the settings of those app pool will now apply to you web application too...  I had earlier written a quick tip on how to change the App Pool used by your application which you can find here...
  • App Pool Settings - App Pool settings are stored in separate configuration file in IIS and they are manifested in IIS Manager UI as below:

image

You can create, edit, delete App Pools for the machine using the above options... Although the important point to note is that the same app pool can be used by various applications on the same server and changing an App Pool setting will impact all the applications running on the server.

In anycase, if you would like to modify the App Pool Settings you can do so by clicking the "Edit Application Pool" settings as shown in the diagram above

Some of the Advanced settings which can be modified for an application pool are as shown in the figure below:

image

So in nutshell, it is important to understand that when you change app pool settings on your developer box then they will not automatically reflect on the server unless it is explicitly modified.

Also the reason why server admins are reluctant to modify a particular app pool's settings on the server is coz it may impact many other applications on the server who are using the same app pool. 

Some server admins create different app pools for different webs to ensure that other applications on the server are not impacted by individual application change requests to the app pool.

Hope this helps...

Tips & Tricks: Deploying Generated Debug Symbols for your Web

Many developers always generate debug symbols so that they can be used to debug even the production environment if need be and to a great extent this can be considered as a best practice, but that does not mean that organizations deploy their Debug Symbols.

If you would like to generate debug symbols for your application you can do so by going to the “Build” tab in the Property Pages and clicking “Advanced” bottom at the bottom. Here you will have different options for the level of debug symbols you would like to generate for your Web Application Projects (WAP)

C#

image

VB

image

Generation of Debug symbols can be configured per "Build Configuration"...  To learn more about managing build configurations click here

Hope this helps...

Tips & Tricks: Managing environment specific properties by using Configuration Manager

Many property pages of a project (File --> New --> Project --> Web Application Project) support Configuration  specific properties:

image

What this essentially means is that all the properties in that tab can be saved in the project file and will be saved per configuration.  This would mean that when your active configuration is "Debug" then all the "Debug" settings will be used.

Debug and Release configurations are available by default inVisual Studio but if you would like to add more build configurations (for various server environments like “Dev”, “QA”, “Staging”, “Production” etc then you can do so by going to the Build --> Configuration Manager.

image

You can also select your active configuration for Visual Studio 10 from the Configuration Manager UI as shown above.

The configurations are stored in the project file as shown below:

image

Note: Deleting a configuration for the solution does not delete it for every project within the solution and visa versa, so when using Configuration Manager make sure that you remove the configurations from the correct locations

Hope this helps...

Tips & Tricks: How to change the App Pool which is used by your web application

If you would like to use advanced IIS features and configuration on your development machine then you first need to make your web application use IIS Web Server for development.  You can do so as described below:

Once you app is using IIS you can go to IIS Manager by going to Start --> Run and typing Inetmgr... In IIS Manager navigate to your application (which will be typically under "Default Web Site")

Now click on the "Basic Settings" as shown below and change the app pool by clicking the "Select"  button:

image

All the available app pools on your machine will be shown in the select drop down as below:

image

e.g. Default App pool to use IIS Integrated Pipeline (Learn more by clicking here), Classic .Net App Pool for non integrated mode...

Hope this helps...

Tips & Tricks: How to use IIS as your local Web Development Server...

When you create a new web application project (WAP) by going to File --> New --> Project --> Web Application Project then the default Web server used is "Visual Studio Development Server"  (fondly named as 'Cassini')...

Cassini does not require you to run as a local administrator on the dev box and hence is something which is preferred by a lot of enterprises.  At the same time Cassini is not an exact representation of how your production web server will look like.  As your production web server is typically an IIS Web Server, Visual Studio also allows you to use IIS as your development web server...

Although many operations related to IIS require you to be a local administrator of your box...  If you would like to use IIS as your development web server than you need to make sure you are running Visual Studio in an administrator mode.

After you do so, you can right click on your WAP --> Click Properties and open the Property pages of the project.  Now you can navigate to the "Web" tab of the property page and select "IIS Web Server" as shown below...

image

You can then click the "Create Virtual Directory" button and your IIS application + VDir will be created... Going forward when you debug or run the Web Application from Visual Studio then your application should use all of the IIS Settings that you configure using IIS Management Console (Start --> Run --> Inetmgr)...

Note: Do note that Visual Studio uses IIS Metabase Compatibility mode to actually access IIS functions so you need to go to Start --> Control Panel --> Programs & Features --> Add or Remove Windows Components / Turn Windows features on or off and make sure below features are enabled:

image

Hope this helps...

Monday, February 02, 2009

Web Deployment with VS 2010 and IIS

Today, deploying a web application is not as easy as it should be. Whether you are deploying your web to a shared hosting environment and paying monthly to maintain it OR whether you have a web server/s managed by your enterprise, there are a lot of manual steps involved in getting your application from point A to point B.

If you are deploying your web application to a shared hoster then today you have to use technologies like FTP which take a long time to get your web content to the hosted server. After deploying your content you have to manually go to hoster control panel and install your database by running sql scripts and configure various IIS settings like marking a folder as an application to isolate it from the rest of the application.

If you are in an enterprise environment and you want to get a web application deployed you have to systematically document each step that your server admins and DBAs have to perform. In most circumstances you also have to ask your admins to modify the web.config files and go to IIS Manager and configure your settings apart from deploying your web content. Your DBA has to do the necessary steps of running the sql scripts in the right order to get your DB up and running. Such installations many a times take hours to complete.

With Visual Studio 2010 and IIS Web Deployment Tool (MsDeploy.exe / Web Deploy) we are introducing a set of technologies which can seamlessly deploy your applications taking care of the problems stated above. Microsoft Web Deployment Tool is a free download available on the web… You can download MSDeploy from below location:

http://blogs.iis.net/msdeploy/archive/2008/10/29/the-web-deployment-tool-beta-2-is-now-available.aspx

Do note that installing Visual Studio 2010 will automatically install MSDeploy for you. Visual Studio 2010 CTP can be downloaded from below location:

http://www.asp.net/vwd/

Web Deployment feature sets in VS 2010 can be broken down into following major areas:

1. Web Packaging - VS 2010 uses MSDeploy to create a .zip file for your application which we call as a web package. This file contains meta data + the below artifacts

· All of your IIS Settings (e.g. application pools, error pages etc)

· Web Content (e.g. .aspx, .ascx, .js, images etc)

· SQL Server DB

· Various other artifacts like Security Certs, GAC Components, Registry etc

A web package can then be taken to any server and installed either via IIS Manager UI Wizard or even via command line or API for automated deployment scenarios.

2. Web.Config Transformation – With VS 2010 web deployment we are introducing XML Document Transform (XDT) which will allow you to transform your development time web.config file to production/deployment time web.config file. The transformation is controlled by web.config TRANSFORM files named web.debug.config, web.release.config etc. The naming of these files is tied to the MSBuild configuration you are trying to deploy. The transform file will need just the changes that you really want to make to your deployed web.config… You can control the type of changes by instructing the XDT engine using simple and easy to understand syntax…

e.g. the below syntax in web.release.config will replace the connectionString section with new values in the web.config file which is produced for deployment of your release configuration.

clip_image002

3. DB Deployment – VS 2010 allows you to deploy your application along with all of its dependencies including database dependencies on SQL Server. Just by providing the connection string of your source database VS10 will automatically script its data/schema and package it for deployment. VS will also allow you to provide custom .sql scripts and also sequence them correctly to run on the server. Once your DB is packaged along with your IIS Settings and web content you can choose to deploy it to any server by providing the connection string at the install time.

4. 1-Click Publish - VS 2010 will allow you to not only package your web applications with all of its dependencies but also use IIS remote management service to publish the application to remote server. VS 10 will now allow you to create a publish profile of your hoster account or of various testing servers and save your credentials securely so that going forward you can deploy to any of these publish profiles with just one click using Web One Click toolbar. With VS 10 you will also be able to publish using MsBuild command line so that you can configure your team build environment to include publishing in continuous integration model.

To learn in further details about these technologies please view the videos here.

ALSO MAKE SURE YOU VISIT THE OVERVIEW POST FOR WEB DEPLOYMENT…