Software Development

All things programming.

Which Files Changed?

One of the complaints I’ve heard about TFS 2010 version control is that getting latest doesn’t show a listing of what was changed. This is untrue. TFS does indeed show such a listing it’s just not as obvious as in other version control systems.

After getting latest, jump over to the output window and select the “Source Control – Team Foundation” option from the “Show output from:” menu. The output window will then display the action taken and file name for each affected file.

It would be nice if TFS would present this information in a sortable/filterable grid format instead of relying on the output window but having the simple listing is typically more than enough. The output window itself is searchable so if you’re looking for changes to a specific file or folder just click in the window and press Ctrl + F to open the find dialog.

Now the next time someone claims that “TFS doesn’t show me which files changed!” you can tell them to check the output window.

LINQed Up (Part 4)

This is the fourth part of a series intended as an introduction to LINQ. The series should provide a good enough foundation to allow the uninitiated to begin using LINQ in a productive manner. In this post we’ll look at the performance implications of LINQ along with some optimization techniques.

LINQed Up Part 3 showed how to compose LINQ statements to accomplish a number of common tasks such as data transformation, joining sequences, filtering, sorting, and grouping. LINQ can greatly simplify code and improve readability but the convenience does come at a price.

LINQ’s Ugly Secret

In general, evaluating LINQ statements takes longer than evaluating a functionally equivalent block of imperative code.

Consider the following definition:

var intList = Enumerable.Range(0, 1000000);

If we want to refine the list down to only the even numbers we can do so with a traditional syntax such as:

var evenNumbers = new List<int>();

foreach (int i in intList)
{
	if (i % 2 == 0)
	{
		evenNumbers.Add(i);
	}
}

…or we can use a simple LINQ statement:

var evenNumbers = (from i in intList
					where i % 2 == 0
					select i).ToList();

I ran these tests on my laptop 100 times each and found that on average the traditional form took 0.016 seconds whereas the LINQ form took twice as long at 0.033 seconds. In many applications this difference would be trivial but in others it could be enough to avoid LINQ.

So why is LINQ so much slower? Much of the problem boils down to delegation but it’s compounded by the way we’re forced to enumerate the collection due to deferred execution.

In the traditional approach we simply iterate over the sequence once, building the result list as we go. The LINQ form on the other hand does a lot more work. The call to Where() iterates over the original sequence and calls a delegate for each item to determine if the item should be included in the result. The query also won’t do anything until we force enumeration which we do by calling ToList() resulting in an iteration over the result set to a List<int> that matches the list we built in the traditional approach.

Not Always a Good Fit

How often do we see code blocks that include nesting levels just to make sure that only a few items in a sequence are acted upon? We can take advantage of LINQ’s expressive nature to flatten much of that code into a single statement leaving just the parts that actually act on the elements. Sometimes though we’ll see a block of code and think “hey, that would be so much easier with LINQ!” but not only might a LINQ version introduce a significant performance penalty, it may also turn out to be more complicated than the original.

One such example would be Edward Tanguay’s code sample for using a generic dictionary to total enum values. His sample code builds a dictionary that contains each enum value and the number of times each is found in a list. At first glance LINQ looks like a perfect fit – the code is essentially transforming one collection into another with some aggregation. A closer inspection reveals the ugly truth. With Edward’s permission I’ve adapted his sample code to illustrate how sometimes a traditional approach may be best.

For these examples we’ll use the following enum and list:

public enum LessonStatus
{
	NotSelected,
	Defined,
	Prepared,
	Practiced,
	Recorded
}

List<LessonStatus> lessonStatuses = new List<LessonStatus>()
{
	LessonStatus.Defined,
	LessonStatus.Recorded,
	LessonStatus.Defined,
	LessonStatus.Practiced,
	LessonStatus.Prepared,
	LessonStatus.Defined,
	LessonStatus.Practiced,
	LessonStatus.Prepared,
	LessonStatus.Defined,
	LessonStatus.Practiced,
	LessonStatus.Practiced,
	LessonStatus.Prepared,
	LessonStatus.Defined
};

Edward’s traditional approach defines the target dictionary, iterates over the names in the enum to populate the dictionary with default values, then iterates over the list of enum values, updating the target dictionary with the new count.

var lessonStatusTotals = new Dictionary<string, int>();

foreach (var status in Enum.GetNames(typeof(LessonStatus)))
{
	lessonStatusTotals.Add(status, 0);
}
	
foreach (var status in lessonStatuses)
{
	lessonStatusTotals[status.ToString()]++;
}

In my tests this form took an average of 0.00003 seconds over 100 invocations. So how might it look if in LINQ? It’s just a simple grouping operation, right?

var lessonStatusTotals =
	(from l in lessonStatuses
		group l by l into g
		select new { Status = g.Key.ToString(), Count = g.Count() })
	.ToDictionary(k => k.Status, v => v.Count);

Wrong. This LINQ version isn’t functionally equivalent to the original. Did you see the problem? Take another look at the output of both forms. The dictionary created by the LINQ statement doesn’t include any enum values that don’t have corresponding entries in the list. Not only does the output not match but over 100 invocations this simple grouping query took an average of 0.0001 seconds or about three times longer than the original. Let’s try again:

var summary = from l in lessonStatuses
				group l by l into g
				select new { Status = g.Key.ToString(), Count = g.Count() };
		
var lessonStatusTotals = 
	(from s in Enum.GetNames(typeof(LessonStatus))
	 join s2 in summary on s equals s2.Status into flat
	 from f in flat.DefaultIfEmpty(new { Status = s, Count = 0 })
	 select f)
	.ToDictionary (k => k.Status, v => v.Count);

In this sample we take advantage of LINQ’s composable nature and perform an outer join to join the array of enum values to the results of the query from our last attempt. This form returns the correct result set but comes with an additional performance penalty. At an average of 0.00013 seconds over 100 invocations, This version took almost four times longer and is significantly more complicated than the traditional form.

What if we try a different approach? If we rephrase the task as “get the count of each enum value in the list” we can rewrite the query as:

var lessonStatusTotals = 
	(from s in Enum.GetValues(typeof(LessonStatus)).OfType<LessonStatus>()
	 select new
	 {
	 	Status = s.ToString(),
		Count = lessonStatuses.Count(s2 => s2 == s)
	 })
	.ToDictionary (k => k.Status, v => v.Count);

Although this form is greatly simplified from the previous one it still took an average of 0.0001 seconds over 100 invocations. The biggest problem with this query is that it uses the Count() extension method in its projection. Count() iterates over the entire collection to build its result. In this simple example Count() will be called five times, once for each enum value. The performance penalty will be amplified by the number of values in the enum and the number of enum values in the list so larger sequences will suffer even more. Clearly this is not optimal either.

A final solution would be to use a hybrid approach. Instead of joining or using Count we can compose a query that references the original summary query as a subquery.

var summary = from l in lessonStatuses
	group l by l into g
	select new { Status = g.Key.ToString(), Count = g.Count() };

var lessonStatusTotals =
	(from s in Enum.GetNames(typeof(LessonStatus))
	 let summaryMatch = summary.FirstOrDefault(s2 => s == s2.Status)
	 select new
	 {
	 	Status = s,
		Count = summaryMatch == null ? 0 : summaryMatch.Count
	 })
	.ToDictionary (k => k.Status, v => v.Count);

At an average of 0.00006 seconds over 100 iterations this approach offers the best performance of any of the LINQ forms but it still takes nearly twice as long as the traditional approach.

Of the four possible LINQ alternatives to Edward’s original sample none of them really improve readability. Furthermore, even the best performing query still took twice as long. In this example we’re dealing with sub-microsecond differences but if we were working with larger data sets the difference could be much more significant.

Query Optimization Tips

Although LINQ generally doesn’t perform as well as traditional imperative programming there are ways to mitigate the problem. Many of the usual optimization tips also apply to LINQ but there are a handful of LINQ specific tips as well.

Any() vs Count()

How often do we need to check whether a collection contains any items? Using traditional collections we’d typically look at the Count or Length property but with IEnumerable<T> we don’t have that luxury. Instead we have the Count() extension method.

As previously discussed, Count() will iterate over the full collection to determine how many items it contains. If we don’t want to do anything beyond determine that the collection isn’t empty this is clearly overkill. Luckily LINQ also provides the Any() extension method. Instead of iterating over the entire collection Any() will only iterate until a match is found.

Consider Join Order

The order in which sequences appear in a join can have a significant impact on performance. Due to how the Join() extension method iterates over the sequences the larger sequence should be listed first.

PLINQ

Some queries may benefit from Parallel LINQ (PLINQ). PLINQ partitions the sequences into segments and executes the query against the segments in parallel across multiple processors.

Bringing it All Together

As powerful as LINQ can be at the end of the day it’s just another tool in the toolbox. It provides a declarative, composable, unified, type-safe language to query and transform data from a variety of sources. When used responsibly LINQ can solve many sequence based problems in an easy to understand manner. It can simplify code and improve the overall readability of an application. In other cases such as what we’ve seen in this article it can also do more harm than good.

With LINQ we sacrifice performance for elegance. Whether the trade-off is worth while is a balancing act based on the needs of the system under development. In software where performance is of utmost importance LINQ probably isn’t a good fit. In other applications where a few extra microseconds won’t be noticed, LINQ is worth considering.

When it comes to using LINQ consider these questions:

Will using LINQ make the code more readable?
Am I willing to accept the performance difference?

If the answer to either of these questions is “no” then LINQ is probably not a good fit for your application. In my applications I find that LINQ generally does improve readability and that the performance implications aren’t significant enough to justify sacrificing readability but your mileage may vary.

Upcoming Events in Indianapolis

There are a few interesting software development related events coming up in Indianapolis over the next few weeks.

Indy TFS User Group

Date/Time:
10/13/2010 6:30 PM

Location:
Microsoft Corporation
500 E. 96th St.
Suite 460
Indianapolis, IN 46240
[Map]

Web Site:
https://www.clicktoattend.com/ invitation.aspx?code=151376

The first meeting of the Indianapolis TFS User Group will feature Paul Hacker introducing many of the Application Lifecycle Management tools in Visual Studio 2010.

I’ve been reading Professional Application Lifecycle Management with Visual Studio 2010 and am pretty excited about many of the features. I hope to use this session to expand upon what is included in the book.

This event is free to attend. Follow the link to the right to register.

IndyNDA

Date/Time:
10/14/2010 6:00 PM

Location:
Management Information Disciplines, LLC
9800 Association Court
Indianapolis, IN 46280
[Map]

Web Site:
http://indynda.org/

The October IndyNDA meeting will be presented by the group’s president, Dave Leininger. Dave will be discussing ways to graphically represent complex relationships in data.

Three special interest groups (SIGs) also meet immediately following the main event. The SIGs were on hiatus last month so I’ll be giving my introduction to dynamic programming in C# talk this month.

IndyNDA meetings are free to attend thanks to the sponsors. No registration is required. Regular attendees should note the new location.

Indy GiveCamp

Date/Time:
11/5/2010 – 11/7/2010

Location:
Management Information Disciplines, LLC
9800 Association Court
Indianapolis, IN 46280
[Map]

Web Site:
http://www.indygivecamp.org/

“Indy GiveCamp is a weekend-long collaboration between local developers, designers, database administrators, and non-profits. It is an opportunity for the technical community to celebrate and express gratitude for the contributions of these organizations by contributing code and technical know-how directly towards the needs of the groups.”

I can’t be participate in this year’s event due to prior family commitments but I’ve heard enough good things about the GiveCamp events in other cities to know that it’s a great cause. There is still a need for volunteers so if you can spare the weekend please volunteer. One of 18 charities will thank you for it.

Web Performance & Load Testing in Visual Studio 2010

Visual Studio 2010 Premium and Ultimate include tools for authoring and executing a variety of test types. Out of the box we get support for several types of tests:

Manual tests require a person to manually perform steps according to a script. Note that manual tests are actually TFS Work Items rather than code files within a test project.

Unit tests are low-level coded tests meant to exercise small areas of code (units) to ensure that code is functioning as the developer expects.

Database unit tests are intended to exercise database objects such as triggers, functions, and stored procedures.

Coded UI tests programmatically interact with the user interface components of an application to ensure proper functionality.

Web performance tests are intended to verify the functionality or performance of Web applications.

Load tests verify that an application will still perform adequately while under stress.

Generic tests allow calling tests written for another testing solution such as NUnit.

Ordered tests are a special kind of test that contain other tests that must be executed in a specific sequence.

Web performance and load tests are two of the more interesting types and are the focus of this post. Before diving into them though we should examine how Visual Studio manages tests.

Managing Tests

This section provides a high-level overview of Visual Studio’s test management capabilities. Test projects, the test view window, test categories, running tests, and working with test results will all be introduced.

Test Projects

One thing that all test types have in common is that they must be part of a test project. Test projects are language (C#, VB) specific and created by selecting the corresponding “Test Project” template from the Add New Project dialog box.

There are no restrictions on the mix of tests contained within a test project. It is perfectly acceptable to have unit tests, Web performance tests, and generic tests within a single test project. It is also acceptable for a solution to have multiple test projects.

Note: Adding a test project to a solution will also a few items to the solution’s Solution Items folder. Two of the items are .testsettings files used to control the behavior of tests under different execution scenarios. The settings used during a test run is determined by the item selected in the Test/Select Active Test Settings menu. The third item is a .vsmdi file used to manage test lists.

The Test Project template will create an empty unit test by default. This file can safely be deleted. If you want to customize what is included in new test projects change the settings in Test Tools/Test Project section of the Visual Studio options.

Adding new tests to a test project is just like adding new classes to any of the standard project types. Items can be added from either the Project menu or the test project’s context menu. Both methods allow adding each of the test types described above (except manual) to the test project.

Test View Window

The test view window is the primary interface for managing tests within a solution. From the test view window we can see a listing of all tests in a solution, filter the list to see only tests within a specific project or category, edit test properties, and select tests to run.

The test view window can be opened by selecting Test View from the Test/Windows menu.

Test Categories

As the number of tests in a solution grows it may be necessary to organize them. Test categories are a convenient way to add some order to the chaos. To categorize a test, right click on it and select properties. Locate the “Test Categories” item and click the ellipses button to open the Test Category editor. Once inside the Test Category editor categories can be selected or new categories can be created and selected.

In addition to helping organize tests categories can also be used to filter down which tests are included in a given run. The test view window includes a select box named “Filter Column” that includes an option for “Test Categories.” Selecting this option and entering a category name in the Filter Text text box is an easy way to display only the tests assigned to a particular category.

Note: The test view window can also be used to manage test lists. In Visual Studio 2010 test lists have been deprecated and are only used by TFS check-in policies.

Running Tests

The test view window makes selecting tests to run very easy. To select a single test just click it. Selecting tests multiple tests is done using the standard ctrl + click and shift + click. By default every test in the solution is included in the test view window. The list can be grouped and filtered on a variety of criteria including project and category.

Grouping is a matter of selecting the desired field from the Group By select box. Filtering is only slightly more involved. To apply a filter optionally select a field from the Filter Column select box and enter criteria in the Filter Text text box.

Once the desired tests are displayed and selected click the Run Selection button from the test view window’s toolbar. When a test run starts the test results window will be displayed.

Note: By no means is the test view window the only way to start a test run. Most test types provide their own run button in their editor. Unit tests can even be run individually by test method, test class, or namespace.

Test Results Window

Test run status is shown in the test results window. This window is displayed as soon as a test run is started and is updated as the run progresses.

The test results window displays both summary and test specific information. The summary line shows overall status of the run along with the number of tests that passed. Each of the tests included in the test run appears as a row in the results window. Individual test status is identified graphically as pending, in progress, failed, inconclusive, or aborted in the result column. Double clicking any test will display the details of the test result.

Web Performance Tests

Web performance tests use manually written or recorded scripts to verify that specific functionality of a Web application is working and performing correctly. For instance, we can record a test script to ensure that a user can log in to the application. Web performance tests are also composable so over time we can build a library of reusable scripts to combine in different ways to test different, but related activities.

In this section we’ll walk through the process of creating Web performance tests. We’ll also examine parameterization, changing request properties, comments, extraction rules, validation rules, data binding, and how to call other Web performance tests.

Creating Web Performance Tests

There are several ways to create Web performance tests:

Recording browser interactions
Manually building tests using the Web Test Editor
Writing code to create a coded Web performance test

By far the easiest method is recording browser interactions. Recorded scripts can be customized easily. They can also be used to generate coded Web performance tests. It is generally recommended to start with a recorded test therefore we will not be discussing coded Web performance tests here.

There are two main ways to record Web performance tests. Visual Studio includes a built-in Web test recorder but we can also use Fiddler to record sessions and export them to a Visual Studio Web test.

Recording in Visual Studio is generally easier as it automatically performs many tasks such as extracting dynamic parameters, setting up validation rules, and grouping dependency request whereas Fiddler offers more control of what is recorded. Regardless of which method is used, there will likely be some clean-up work when recording is complete.

Note: Recorded or manually built (non-coded) Web performance tests are XML files stored with an extension of .webtest.

Recording in Visual Studio

Invoking the Web test recorder in Visual Studio is a matter of adding a new Web performance test to the test project. Visual Studio will automatically start an instance of Internet Explorer with a docked instance of the Web Test Recorder.

Right click the test project
Select Add –> Web Performance Test.
Manually perform the steps (navigation) to include in the script
When finished, click the stop recording button
Wait for Visual Studio to detect dynamic parameters that can be promoted to Web test parameters and review its findings selecting each value that should be promoted.

Exporting Sessions from Fiddler

A full discussion of Fiddler is beyond the scope of this document but it is important to look at its filtering capability to see how we can restrict what it captures down to what is relevant to our test.

The first option we’ll typically set is to Show only Internet Explorer traffic. Selecting this option will prevent traffic from other browsers or software making http requests from showing in the sessions list.

Another option that is often useful is to change the value of the select box under Response Type and Size to “Show only HTML.” This option will prevent the session list from showing any dependency requests for additional resources like images or script files.

Note: Fiddler doesn’t record requests targeting localhost. There are workarounds but it is generally easier to target localhost by its name and parameterize the server name later.

Verify that any desired filters are set
Manually perform the steps (navigation) to include in the script
Select the appropriate sessions
File –> Export Sessions –> Selected Sessions…
Select “Visual Studio Web Test”
Click “Next”
Browse to the target location, typically the test project folder
Name the file
Deselect all Plugins
Include the newly created .webtest file in the test project

Parameterization

Most of the time a recorded script is only the starting point for constructing a Web performance test. In their initial form recorded test aren’t very flexible. Luckily we can use parameters to change that.

Whenever a parameter value is used in a request the name appears in a tokenized format. For example, if we have a context parameter named username and we wanted to reference it in a request we would use {{username}}.

Context Parameters

Context parameters are essentially variables within a Web performance test. Each step within a test can read from and write to context parameters making it possible to pass dynamic information such as new IDs from step to step.

Parameterizing Web Servers

When a test is recorded the original server name is embedded in each request. Although it is common to run tests against localhost it is often necessary to run the tests against a different server. With the server name embedded in each request we cannot satisfy this need. Fortunately, Visual Studio provides the “Parameterize Web Servers” operation that will automatically extract the protocol and server name from each request and replace it with a context parameter.

To parameterize the Web server for each request:

Click the “Parameterize Web Servers” toolbar button
Verify the server name(s)
Click OK

When the dialog closes each request will be updated to reflect the context parameter(s). You should also notice that any new parameters have been added to the test’s Context Parameters folder.

Changing the server is simply a matter of changing the value of the corresponding context parameter using the properties window. Alternatively, clicking the “Parameterize Web Servers” button again will open the “Parameterize Web Servers” dialog. From this dialog click the Change button and enter the appropriate URL.

It is possible to parameterize additional parts of the request URL but there is no direct support in Visual Studio. Should the need arise it is generally easier to manually edit the XML adding the context parameter to the context parameters element and replacing the portion of the URL being parameterized with the context parameter token.

Setting Request Properties

One of the benefits of recording Web performance tests is that the value of most of the properties for each request are generated by the recording process. For the most part modifying these values after the test is recorded isn’t necessary but occasionally we need to tweak something.

Some common scenarios for when property values might need to be adjusted are:

Needing to check response time
Needing to adjust think time to closer reflect reality
Correcting the Expected Response URL when a redirect is encountered

Note: Think times are intended to simulate the amount of time a real user would spend “thinking” before proceeding to the next step. Think times are disabled by default but are more useful in load tests.

To access a request’s properties right click the request and select the “Properties” option.

Comments

Comments allow documenting portions of a Web performance test. They can serve as helpful reminders of what a particular portion of a script is doing.

Extraction Rules

As their name implies, extraction rules are used to extract values from a response. Context parameters are used to store the extracted value for later reference.

Adding an extraction rule to a request is a matter of selecting the “Add Extraction Rule…” option from a request’s context menu. There are eight built-in extraction rule types to choose from:

Selected Option
Tag Inner Text
Extract Attribute Value
Extract Form Field
Extract HTTP Header
Extract Regular Expression
Extract Text
Extract Hidden Field

You should be aware that extraction rules can have a negative impact on test performance. To mitigate this problem extraction rules should be used judiciously.

Validation Rules

Validation rules ensure that the application under test is functioning properly by verifying that certain conditions are met.

Adding a validation rule to a request is a matter of selecting the “Add Validation Rule…” option from a request’s context menu. There are nine built-in validation rule types to choose from:

Selected Option
Tag Inner Text
Response Time Goal
Form Field
Find Text
Maximum Request Time
Required Attribute Value
Required Tag
Response URL

Like extraction rules, validation rules can have a negative impact on performance. Judicious use of validation rules can alleviate this problem.

Note: Performance degradation due to validation rules typically isn’t an issue when running a stand-alone test but can have a much larger impact on load tests. To address this problem adjust the lower the test level on the affected load test and the most important validation rules.

Binding Tests to a Data Source

There are times when static values or context parameters are sufficient but many tests can benefit from using some level of data binding to get a bit more variation in the data. To add a data source to a Web performance test either click the “Add Data Source” toolbar button or select the “Add Data Source…” option from the root node’s context menu and follow the prompts in the New Test Data Source Wizard. Once the data source is defined it can be referenced by requests.

Data can be retrieved from a data source using one of three methods:

Sequential – Iterates over the data set and loops back to the beginning if the test includes more iterations than there are records
Random – Randomly selects a record from the data source as long as necessary
Unique – Just like sequential but without looping

The steps for binding a data source are dependent upon the type of test object. In many cases once the item is selected opening the property sheet, clicking the drop down arrow, then navigating to the appropriate source and data table is all it takes. Should you need to bind to a data source manually enter the tokenized form {{dsn.table#column}}.

Transactions

In the context of a Web performance test a transaction is a group of related requests. Each of the requests inside the transaction can be tracked as a whole unit rather than individually.

To create a transaction:

Select the “Insert Transaction…” option from a request’s context menu
Enter a test name
Select the first and last items
Click OK

When the dialog closes the selected requests will be grouped under a new transaction node.

Calling Other Web Performance Tests

Parameterization, comments, extraction rules, validation rules, and data binding all directly contribute to making individual Web performance tests valuable but one of the most powerful features is making tests call other tests. For instance, if we have a variety of tests that each require first logging into an application we can isolate the login steps into a separate .webtest and have each of those tests call it before doing anything else.

Although it’s perfectly acceptable to record the common steps separately it is generally easier to start with a full test that already includes the common steps and extract them using the Web test editor. To extract a test:

Select the “Extract Web Test…” option from the root node’s context menu
Enter a test name
Select the first and last items
Comments can be very useful for identifying where one set of steps ends and the next begins
Check the “Extracted test inherits test level properties” box
Uncheck the “Copy test level plug-ins and validation rules” box
Click OK

When the dialog closes the items identified in step 3 above will be removed and a call to the newly created test will have taken their place.

When the extracted test is needed by another test it can be inserted into the test by selecting the “Insert Call to Web Test…” option from any node’s context menu.

Load Tests

Load tests verify that an application will perform under stress by repeatedly executing a series of tests and aggregating the results for reporting and analysis. Load tests are often constructed from Web performance test scripts but most other test types are also allowed.

Note: Visual Studio 2010 includes support for distributed load tests. Distributed load tests are not discussed here.

Scenarios

Load tests consist of one or more scenarios. Scenarios define the conditions for the test including how to use think times, load pattern, test mix model, network mix, and browser mix.

Think Time

One of the first options to set for a load test is how to use think times if at all. When using think times we can select to use the recorded times or to use a normal distribution based upon the recorded times. Generally speaking, it is best to use the normal distribution to get a more accurate reflection of user load.

Load tests also have an option for think time between iterations. Use this option to introduce a delay between each iteration.

Load Pattern

The load pattern defines user load through the course of a test. The two main load pattern types are constant load and step load.

Note: There is a third pattern, goal based, that adjusts load dynamically until certain performance criteria is met.

As its name implies, a constant load pattern keeps a constant load of a specified number of users throughout the duration of the test. When one iteration completes another will immediately start to keep the load at the same level.

Step load will gradually add user load to over the course of a test. Load tests using the step load pattern start will start with the specified number of users gradually adding more users up to a defined maximum.

Test Mix and Test Mix Model

The test mix defines which tests are included in the load test. Before we can define the test mix we must select a test mix model. Test mix models are control how frequently each test in the mix will be executed. There are four options for modeling the test mix.

Based on the total number of tests: Each test in the mix is assigned a percentage that indicates how many times it should be run. Each virtual user executes each test according to the defined percentages.

Based on the number of virtual users: Each test in the mix is assigned a percentage that indicates how many of the total users should invoke it.

Based on user pace: Each test in the mix is run a specified number of times per user per hour.

Based on sequential test order: Each test in the mix will be executed once per user in the order the tests appear in the mix.

Network Mix

Load tests can also simulate different network speeds according to the network mix. Available options include LAN, WAN, dial-up, cable, and some mobile phone networks. Each new virtual user will semi-randomly select a network speed according to the network mix distribution.

Note: The network emulation driver must be installed for options other than LAN. If the driver is not installed when the first non-LAN option is selected Visual Studio will prompt you to install it.

Browser Mix

When a load test is built from one or more Web performance tests we can also specify a browser mix. Most popular browsers are available for selection.

Note: If another browser is required a new .browser file can be added to the Program Files\Microsoft Visual Studio 9.0\Common7\IDE\Templates\LoadTest\Browsers folder. (http://social.msdn.microsoft.com/Forums/en/vstswebtest/thread/3f1b16d7-2343-4409-9eff-6f8c764dd93b)

Performance Counter Sets

While scenarios define the run conditions for a given load test, performance counter sets are where load tests derive their real value. Load tests can be configured to monitor performance counters on both local and remote machines through the selection of counter sets. The data from these counter sets is collected and maintained by Visual Studio. When the test run completes the collected data will be tabulated and displayed for analysis.

Run Settings

Once a scenario is configured and the performance counter sets are defined we can specify our run settings. Run settings include timing, number of iterations, sampling rate, and validation level.

For test timing we have two values to configure: Warm-up duration and run duration. Warm-up duration is the amount of time where nothing in the test is tracked. Once the warm-up timer expires data collection begins and continues until the run duration time is reached. By default there is no warm-up period and the run duration is ten minutes.

As an alternative to controlling the test by timing we can specify the number of test iterations. By default the test will execute 100 times.

The sampling rate controls how often performance counters are polled. The sampling rate is defaulted to five seconds.

As previously mentioned, validation rules have a level property that can be used to ignore certain rules depending on the test’s level setting. Just like the validation rules the validation level for the test can be set to high, medium, or low. The default validation level is high meaning that all rules will be invoked. To filter out validation rules select a lower level.

Analyzing Load Test Results

Running a load test is just like running a Web performance test but the test results are much more detailed. During the course of a test run Visual Studio is collecting performance data according to the defined performance counter sets. The Load Test Monitor is used to view and analyze the collected performance data.

By default the Load Test Monitor displays the results in graph form. The graph view provides a way to see performance data over time. This is useful for quickly identifying possible issues. To really get into the details though we need to switch over to the tabular view by clicking the “Tables” button.

The tabular view provides a select box listing several tables:

Tests displays summary information for each test in the load test.

Errors displays details about any errors encountered during the test run.

Pages lists all pages that were accessed during a test run along with summary data such as average response time.

Requests includes details for all HTTP requests issued during a test run.

Thresholds displays information about each threshold violation encountered.

Transactions provides information about Web performance transactions or unit test timers. This does not refer to database transactions.

Note: Depending on load test configuration three additional tables may be available: SQL Trace, Agents, and Test Details.

Summary

We’ve only looked at two of the available test types but it should be apparent how powerful Visual Studio’s testing facilities can be. Individually each of the test types provide a great deal of insight into the health of an application. Using multiple test types in conjunction with each other can reduce the amount of effort required to isolate trouble spots.

Web performance tests make it possible to ensure that a Web application is functioning correctly. They can validate that the correct information is appearing on pages and that pages are being processed within an acceptable time frame.

Load tests build upon other tests to determine how an application or part of an application handles being stressed. They let us see how user will experience an application under a variety of network or browser conditions.

The information provided by these tests is invaluable. Together, these tests can help us identify and address potential trouble spots before our customers do. Should customers report an issue we can use these tests to more accurately simulate their environment and confirm their findings in a controlled manner.

It is certain that building these tests can take some time but given their reusable nature it shouldn’t take too long to start building a library of tests that can be applied or adapted to multiple situations. In any case, automated testing beats synchronizing watches and hovering over a button waiting for that special moment to click…

Basic Dynamic Programming in .NET 4

.NET 4 adds some nice tools to the toolbox. Chief among them is support for dynamic languages and dynamic features in strongly typed languages. In this post we’ll examine how to use reflection to work with unknown data types then we’ll see how to use dynamics to accomplish the same task. Next, we’ll see an example of interacting with IronRuby from within a C# application. Finally, we’ll take a brief look at two of the specialized classes in the System.Dynamic namespace.

Introducing Dynamic Programming

.NET has historically been a statically typed environment. By virtue of being statically typed we get many benefits such as type safety and compile-time member checking. There are times though that we don’t know the type of a variable when we’re writing the code. Consider this code:

module Bank

type BankAccount() = class
  let mutable _balance = 0m

  member this.CurrentBalance
    with get() = _balance

  member this.Deposit(amount) =
    let lastBalance = _balance
    _balance <- _balance + amount
    ( lastBalance, _balance)

  member this.Withdrawal(amount) =
    let lastBalance = _balance
    let tmpBalance = _balance - amount
    if tmpBalance < 0m then raise(new System.Exception("Balance cannot go below zero"))
    _balance <- tmpBalance
    ( lastBalance, _balance)

  member this.Close() =
    let lastBalance = _balance
    _balance <- 0m
    ( lastBalance, _balance)
end

The code above defines a simple type in F#. Don’t worry if you’re not familiar with it – all that’s important to understand here is that the BankAccount type has a parameterless constructor, a read-only CurrentBalance property, and three methods (Deposit, Withdrawal, and Close) that each return a two item tuple.

What if we want to work with this type from a C# assembly? In many cases it will be possible to add a reference to the F# project but what if that isn’t possible? What if we’re getting a reference to the type from an IoC container, a factory method, or a COM interop component that returns object? In those cases we may not have enough information to cast the instance to a known type.

Reflecting on the Past

In the past when these situations would arise we had to resort to reflection to access an object’s members. Aside from being costly, using reflection requires a lot of extra code and often involves additional casting.

var account = BankAccountFactory.GetBankAccount();

var accountType = account.GetType();
var depositMethod = accountType.GetMethod("Deposit");
var currentBalanceProperty = accountType.GetProperty("CurrentBalance");
var withdrawalMethod = accountType.GetMethod("Withdrawal");
var closeMethod = accountType.GetMethod("Close");

Console.WriteLine(Resources.CurrentBalanceFormat, ((Tuple<decimal, decimal>)depositMethod.Invoke(account, new object[] { 1000m })).Item2);
Console.WriteLine(Resources.CurrentBalanceFormat, ((decimal)currentBalanceProperty.GetValue(account, null)));
Console.WriteLine(Resources.CurrentBalanceFormat, ((Tuple<decimal, decimal>)withdrawalMethod.Invoke(account, new object[] { 250m })).Item2);
Console.WriteLine(Resources.CurrentBalanceFormat, ((Tuple<decimal, decimal>)withdrawalMethod.Invoke(account, new object[] { 100m })).Item2);
Console.WriteLine(Resources.CurrentBalanceFormat, ((Tuple<decimal, decimal>)depositMethod.Invoke(account, new object[] { 35m })).Item2);
Console.WriteLine(Resources.CurrentBalanceFormat, ((Tuple<decimal, decimal>)closeMethod.Invoke(account, new object[] { })).Item2);

Look at those pipes! There’s a ton of code and most of it is just plumbing. Before we can do anything useful with the BankAccount instance we need to get a MemberInfo (specifically PropertyInfo or MethodInfo) instance for each member we want to use. Once we have the MemberInfo instances we need to call the Invoke or GetValue method passing the source instance, and other required arguments before casting the result of the call to the expected type! Isn’t there a better way?

Dynamic Language Runtime to the Rescue!

Prior to .NET 4 we were stuck with reflection but .NET 4 introduces the Dynamic Language Runtime (DLR) to help reduce this complexity. The DLR is exposed to C# through the dynamic type. Rather than declaring variables as type object we can define them as type dynamic.

Note: Please, please, pleeeease don’t confuse var with dynamic. in C# var uses type inference to determine the actual type of the variable at compile-time whereas dynamic defers type resolution to the DLR at run-time. Variables defined with var are still strongly typed.

The dynamic type is just like any other type in that it can be used for defining variables, fields, method return values, method arguments, etc… Variables defined as dynamic can be used like we’re working with directly with an instance of a known type. Let’s take the reflection example and revise it to use dynamics instead.

dynamic account = BankAccountFactory.GetBankAccount();

Console.WriteLine(Resources.CurrentBalanceFormat, account.Deposit(1000m).Item2);
Console.WriteLine(Resources.CurrentBalanceFormat, account.CurrentBalance);
Console.WriteLine(Resources.CurrentBalanceFormat, account.Withdrawal(250m).Item2);
Console.WriteLine(Resources.CurrentBalanceFormat, account.Withdrawal(100m).Item2);
Console.WriteLine(Resources.CurrentBalanceFormat, account.Deposit(35m).Item2);
Console.WriteLine(Resources.CurrentBalanceFormat, account.Close().Item2);

As you can see, the code is much more concise. We don’t have any of the complexity required by remoting and dynamics typically performs better than reflection. This convenience does come at a price though.

When we use dynamic types we lose all of the compile-time support that comes with a strongly typed environment. That means we lose type safety, type checking, and even IntelliSense. If something is wrong we won’t find out about it until run-time so it is especially important to have good tests around any code using dynamics.

Using Dynamic Languages

We’ve seen how to use the dynamic type so let’s take a look at using the DLR with an actual dynamic language (F# is functional, not dynamic). In this example we’ll define a class using an IronRuby script hosted within a C# application. We’ll then create and use an instance of that type.

var engine = Ruby.CreateEngine();
dynamic account = engine.Execute(
@"class BankAccount
	attr_reader :CurrentBalance

	def initialize()
		@CurrentBalance = 0.0
	end

	def Deposit(amount)
		@CurrentBalance = @CurrentBalance + amount
		@CurrentBalance
	end

	def Withdrawal(amount)
		@CurrentBalance = @CurrentBalance - amount
		@CurrentBalance
	end

	def Close()
		@CurrentBalance = 0.0
		@CurrentBalance
	end
end
return BankAccount.new"
);

Note: A full discussion of IronRuby is beyond the scope of this article. For now just know that the Ruby class exposes the Microsoft Scripting classes. The Ruby class itself is found in the IronRuby namespace in IronRuby.dll.

The IronRuby code snippet passed to engine.Execute defines a class very similar to the F# class we used earlier. The ScriptEngine‘s Execute method evaluates the script and returns a dynamic that will be bound to the IronRuby type. Once we have that reference we can use the DLR to manipulate the instance as follows:

Console.WriteLine(Resources.CurrentBalanceFormat, account.Deposit(1000m));
Console.WriteLine(Resources.CurrentBalanceFormat, account.CurrentBalance);
Console.WriteLine(Resources.CurrentBalanceFormat, account.Withdrawal(250m));
Console.WriteLine(Resources.CurrentBalanceFormat, account.Withdrawal(100m));
Console.WriteLine(Resources.CurrentBalanceFormat, account.Deposit(35m));
Console.WriteLine(Resources.CurrentBalanceFormat, account.Close());

With the exception that the IronRuby class doesn’t return a Tuple the C# code is identical to that used to work with the F# class. In both cases the DLR handles resolving properties, methods, and data types despite the fact that the underlying class is not only entirely different but is also completely unrelated. This illustrates how dynamics can also simplify working with similar classes that don’t share a common interface or base class.

The Microsoft Scripting classes don’t restrict us to using inline scripts. We can also use the ScriptEngine‘s ExecuteFile method to invoke external scripts. Unlike with the Execute method which returns dynamic ExecuteFile returns an instance of ScriptScope that can be used to dive back into the engine and provide more control for using the loaded script(s).

var scope = engine.ExecuteFile("IronRubySample.ir");
dynamic globals = engine.Runtime.Globals;
dynamic account = globals.BankAccount.@new();

Special Dynamic Types

In addition to declaring any unknown types as dynamic the .NET Framework now provides classes that allow dynamic behavior from the traditional .NET languages. Each of these types are located in the System.Dynamic namespace and instances must be defined as type dynamic to avoid static typing and take advantage of their dynamic capabilities.

Of the classes in the System.Dynamic namespace we’ll only be looking at ExpandoObject and DynamicObject here.

ExpandoObject

ExpandoObject is a dynamic type that allows members to be added or removed at run-time. The behavior of ExpandoObject is similar to that of objects in traditional dynamic languages.

public sealed class ExpandoObject : IDynamicMetaObjectProvider,
	IDictionary<string, Object>, ICollection<KeyValuePair<string, Object>>,
	IEnumerable>, IEnumerable, INotifyPropertyChanged

For the following examples assume we have an ExpandoObject defined as:

dynamic car = new ExpandoObject();
car.Make = "Toyota";
car.Model = "Prius";
car.Year = 2005;
car.IsRunning = false;

In their simplest form ExpandoObjects will only use properties:

Console.WriteLine("{0} {1} {2}", car.Year, car.Make, car.Model);

…but we can also add methods:

car.TurnOn = (Action)(() => { Console.WriteLine("Starting {0}", car.Model); car.IsRunning = true; });
car.TurnOff = (Action)(() => { Console.WriteLine("Stopping {0}", car.Model); car.IsRunning = false; });

Console.WriteLine("Is Running? {0}", car.IsRunning);
car.TurnOn();
Console.WriteLine("Is Running? {0}", car.IsRunning);
car.TurnOff();
Console.WriteLine("Is Running? {0}", car.IsRunning);

…and events:

var OnStarted =
	(Action<dynamic, EventArgs>)((dynamic c, EventArgs ea) =>
	{
		if (c.Started != null)
		{
			c.Started(c, new EventArgs());
		}
	});

var OnStopped =
	(Action<dynamic, EventArgs>)((dynamic c, EventArgs ea) =>
	{
		if (c.Stopped != null)
		{
			c.Stopped(c, new EventArgs());
		}
	});

car.Started = null;
car.Started += (Action<dynamic, EventArgs>)((dynamic c, EventArgs ea) => Console.WriteLine("{0} Started", c.Model));
car.Stopped = null;
car.Stopped += (Action<dynamic, EventArgs>)((dynamic c, EventArgs ea) => Console.WriteLine("{0} Stopped", c.Model));
car.TurnOn = (Action)(() => { car.IsRunning = true; OnStarted(car, EventArgs.Empty); });
car.TurnOff = (Action)(() => { car.IsRunning = false; OnStopped(car, EventArgs.Empty); });

Console.WriteLine("Is Running? {0}", car.IsRunning);
car.TurnOn();
Console.WriteLine("Is Running? {0}", car.IsRunning);
car.TurnOff();
Console.WriteLine("Is Running? {0}", car.IsRunning);

In addition to the standard IDynamicMetaObjectProvider interface ExpandoObject also implements several interfaces for accessing members as though they were a dictionary. The DLR will handle adding members through its binding mechanism but we need to use the dictionary syntax to remove them.

var carDict = (IDictionary<string, object>)car;
Console.WriteLine("{0} {1} {2}", carDict["Year"], carDict["Make"], carDict["Model"]);

DynamicObject

While ExpandoObject allows us to dynamically add and remove members at run-time, DynamicObject allows us to control that behavior.

public class DynamicObject : IDynamicMetaObjectProvider

Since DynamicObject ~~is an abstract class~~ doesn’t expose a public constructor we must create a derived class to take advantage of its features. A side effect of this we can also define members directly on the class and the DLR will handle resolving them correctly.

public class DynamicCar : System.Dynamic.DynamicObject
{
	public DynamicCar()
	{
		Extensions = new System.Dynamic.ExpandoObject();
	}

	private ExpandoObject Extensions { get; set; }

	public string Make { get; set; }
	public string Model { get; set; }
	public int Year { get; set; }

	public override bool TryGetMember(GetMemberBinder binder, out object result)
	{
		string name = binder.Name.ToLower();
		Console.WriteLine("Getting: {0}", name);
		return (Extensions as IDictionary<string, object>).TryGetValue(name, out result);
	}

	public override bool TrySetMember(SetMemberBinder binder, object value)
	{
		var name = binder.Name.ToLower();

		Console.WriteLine("Setting: {0} -> {1}", name, value);
		(Extensions as IDictionary<string, object>)[name] = value;
		return true;
	}

	public override bool TryInvokeMember(InvokeMemberBinder binder, object[] args, out object result)
	{
		Console.WriteLine("Invoking: {0}", binder.Name);
		return base.TryInvokeMember(binder, args, out result);
	}
}

Once the type is defined we create and use instances just like any other dynamic type:

dynamic car = new DynamicCar()
{
	Make = "Toyota",
	Model = "Prius",
	Year = 2005
};

car.IsRunning = false;
car.TurnOn = (Action)(() => car.IsRunning = true);
car.TurnOff = (Action)(() => car.IsRunning = false);

Console.WriteLine("Make: {0}", car.Make);
Console.WriteLine("Model: {0}", car.Model);
Console.WriteLine("Year: {0}", car.Year);
Console.WriteLine("IsRunning: {0}", car.IsRunning);
car.TurnOn();
car.TurnOff();

Notice how we are able to take advantage of object initializer syntax because the members we’re setting are defined on the class itself rather than being dynamic. We can still access those members normally later on despite the variable being defined as dynamic.

The output shows how we’ve changed the behavior of the dynamic members while the static members are unaffected. In this example actions affecting the dynamic members display a message.

Can’t Everything be Dynamic?

It’s true that there’s nothing preventing us from declaring everything as dynamic but it’s usually not a good idea in statically typed languages like C#. In addition to losing all compile-time support that comes from statically typed languages, code that uses dynamic typing generally performs worse than code using static typing. Generally speaking, only use dynamics when you have a good reason.

IndyNDA – Windows Phone 7 Application Platform

This month’s IndyNDA meeting is tomorrow, September 9, 2010. Come hear Bill Steele from Microsoft speak about the Windows Phone 7 Application Platform.

Following the main meeting the three special interest groups (SIGs) will meet.

ASP.NET SIG: Continuing work on an ongoing MVC knowledge base application.

C# SIG: I’ll be presenting an introduction to dynamic programming in C#.

SharePoint SIG: Topic to be determined.

Registration opens at 5:30 PM and thanks to our sponsors food and drinks are provided until 6:00. I hope to see you there.

Location:

Nine Parkwood Crossing
5th Floor Conference Room
900 EAST 96TH STREET
Indianapolis, IN 46240

TFS2010: Branching Guide

After a lunch-time discussion about branching strategies with TFS I started looking for a good branching guide. The ALM Rangers put together a thorough guide based on lessons learned from real-world deployments. The guide provides general guidance for selecting one of four increasingly complex strategies:

Basic: Main, development, and release branches
Standard: Basic + service pack branches
Advanced: Standard + hot fix branches
Mature: Single main branch with multiple development, service pack, hot fix, and release branches

Equally important is the discussion about when to create additional development branches:

Breaking changes: A change in a common library will break other parts of the system until they are updated to reflect the change
Segregated feature work: A team wants to control when its features are released to other teams
Next version development: Allows development on the next version to start before the release branch is created
“Scratch” or Temporary Branches: Prototype work, etc…

If you’re looking for a decent branching strategy guide I recommend starting here. The document is a pretty quick read and well worth the time spent reviewing it but make sure you take some time to review the supplemental materials as well. They have a lot of good information that can help clarify or extend the information in the main guide.

Guide: http://tfsbranchingguideiii.codeplex.com/
ALM Rangers: http://msdn.microsoft.com/en-us/vstudio/ee358786.aspx

TFS2010: Everyday TFS

I’ve written a bit already about working with Team Foundation Server (TFS) 2010 and have been thinking about a few other topics so I decided to wrap them all into one big post. I’ve been working with TFS for around nine months now. My first experiences with TFS were largely negative but as I’ve worked with it I’ve found that much of that negativity was due to a bias against the check-out model for version control and a lack of understanding of what else TFS could do for me and my team. Over these nine months I’ve come to really appreciate its capabilities and have found ways to work around some of its shortcomings.

The ideas here come straight from my day-to-day experiences working with TFS. Everything I’m including is something I’ve found that has made me more productive. I’m by no means a TFS expert and reading this post certainly won’t make you one either but I strongly believe that it will help you work more effectively.

In this post we’ll look at three main areas:

Tools
Version Control
Project Management

I assume at least a basic working knowledge of version control and project management concepts. As such, I don’t spend much, if any, time discussing basic concepts like performing check ins, defining changesets, or what MS Project does.

I hope you find the tips contained within to be useful and worthy of inclusion into your workflow.

Tools

One of the first thing I noticed when I started with TFS is that its basic tool support is pretty weak. Many of the tools that ship with TFS are rudimentary at best and other features such as shell integration are just missing. In this section we’ll examine some workarounds and ways to be more productive.

TFS Power Tools

The TFS Power Tools are essential utilities to simplify many day-to-day tasks and circumvent other shortcomings. I firmly believe that the first thing (ok, second to installing Visual Studio) any developer working with TFS version control on a regular basis needs to do is install the TFS Power Tools.

Installing the Power Tools

Open the Extension Manager from the Tools menu in Visual Studio 2010.
Click the Online Gallery Tab
Locate the Team Foundation Server Power Tools extension
Depending upon the extension’s popularity it may or may not be listed on the featured on the front page of the online gallery so you may need to search for it
Select the power tools and click the download button.
When prompted Run or Save the installer package
If there are any instances of Visual Studio running the installer will stop until they are closed. The installer detects previous versions of Visual Studio too so you’ll need to close those as well
Select either a typical or custom installation
The typical option should be sufficient for most users but anyone wanting to use the PowerShell Cmdlets will want to select custom install. The tools I use most frequently are the command line interface, the Visual Studio integration, and the Windows Shell Extension
Click through the remaining pages in the wizard to complete the installation
A reboot may be necessary to activate the shell extensions

Shell Integration

One of the biggest benefits of using TFS is that it integrates version control and project management capabilities into the Visual Studio IDE. For developers already spending the majority of their day in Visual Studio this is usually a good thing since Visual Studio manages a lot of the action behind the scenes. Unfortunately this benefit can also be one of its biggest drawbacks.

TFS uses a check out model for managing versioned files. Visual Studio performs a silent, implicit check out on any files being edited within the IDE but what happens when we need to edit versioned files outside of Visual Studio?

Aside from a command line utility TFS doesn’t offer any way to manage versioned files outside of Visual Studio! Unless we’re intimately familiar with the command line utility we’re forced to jump back to VS, go to Source Control Explorer, locate the file(s), and check them out before returning to the other program. This is the first place that the power tools come to the rescue.

Although the shell integration isn’t as full-featured as what we get with something like Tortoise SVN I still find it sufficient for most tasks. Depending on what I’m doing I’ll often use it before going to Source Control Explorer for simple tasks.

The shell integration is intended for routine operations like getting the latest version, checking in and out, adding, and comparing files. There are no options for branching, merging, alerts, or annotating. A huge miss in the power tools is that there’s no option for history either.

Power Tools Command Line Utility

Command line junkies will appreciate the fact that the power tools also come with a command line utility. Much of the functionality provided by the power tools is only available through tfpt.exe. We’ll discuss several of the commands available to us through the command line utility in detail later. Full documentation of the commands available through tfpt.exe is available through the /? switch.

TFS Command Line Utility

TFS itself ships with a command line utility named tf.exe. I really don’t find myself using tf.exe all that frequently mostly because of the shell integration supplied with the power tools. The only command I find myself using with any regularity is tf get.

Basic documentation about tf.exe can be found by running it with the /? switch. Full documentation is available on MSDN and is easily accessed by running tf msdn.

Changing the Compare and Merge Tools

Any version control system needs tools for comparing files and resolving conflicts and TFS is no exception. Like many of the tools that ship with TFS the compare/merge tools are pretty simplistic and lack many of the features found in other tools in the same niche. If you’re used to products such as WinMerge this can be a nuisance but TFS allows us to not only configure which tool we’d like to use for these tasks but to define them by action and file extension.

Note: What follows below are the basic steps for changing the compare and merge tools. Check your desired tool’s documentation for specific configuration information.

Custom tool configuration is rather straight forward:

Open the options dialog in Visual Studio (Tools –> Options)
Expand the Source Control node
Select the Visual Studio Team Foundation Server node
Click the “Configure User Tools…” button

The Configure User Tools dialog is pretty simple. In addition to allowing adding new tool configurations the dialog lists any configured tools and allows their modification or removal. The Configure Tool accessed through the Add or Modify buttons is where the real configuration happens.

Extension: The file extension associated with the tool
To indicate that the tool applies to all extensions use .*

Operation: Merge or Compare

Command: The full path of the tool’s executable

Arguments: Any applicable command line arguments
Use the arrow button to select tokens for files or labels.

Version Control

As developers most of our interactions with TFS will be with its version control system. Given that TFS is intended to be managed within Visual Studio the integration with Visual Studio is pretty solid. I find that most of the version control related headaches are actually due to the check-out model employed by TFS rather than TFS itself. That being said, version control in TFS isn’t particularly great and if you’re only using TFS for version control you may consider alternative versioning systems because better dedicated solutions are available.

In this section we’ll examine some of the ways to help keep the check-out model out of the way and decreasing productivity.

Keeping Your Workspace Tidy

When working on projects it’s really easy to neglect basic housekeeping. Over time workspaces can get really messy and some spring cleaning will be necessary to restore some order. At first glance cleaning a workspace can seem daunting but TFS Power Tools can make it trivial.

Unchanged Files

One big drawback of check-out model that I’ve previously discussed is that every change, even temporary ones, require files to be checked out. How many times have do we edit a file to toss in some form of debugging helper just to remove it a few minutes later? Under the check-out model all of these files are checked out but unchanged. Visual Studio itself can compound the problem by checking out some files related to projects and solutions but never changing them.

When it’s time to check in the changes all of these unchanged files will be included in the pending changes list. If any unchanged files are included in the check in TFS will detect that they are unchanged and won’t include them in the changeset. This behavior is OK but if you’re like me you’ll want to review your changes before actually checking them in just to be sure you’re not about to do something stupid like checking in a forgotten debugger; statement in a JavaScript file (not that I’ve ever done that…) One or two unchanged files aren’t too bad to deal with but sifting through more than that is just a waste of time. Wouldn’t it be nice to undo the check out on the unchanged files before reviewing them for check in?

Without the power tools the only way I know to undo the check out on the unchanged files is to manually compare the files one at a time and undo if there are no changes. The power tools provide a really handy alternative in the form of the tfpt uu command.

tfpt uu is a command line utility that will undo the check out on all of the unchanged files in a workspace at the same time. The default behavior will update the workspace and look for unchanged files before prompting to undo them but that behavior can be controlled though some command line switches. The most useful switches are /recursive that checks filespecs with full recursion and /noget that bypasses getting latest on the workspace before comparing the files. Full documentation for tfpt uu can be found using the /? switch.

The tfpt uu command has become part of my workflow. I almost always use it just before a check in and will often run it just to help keep my workspace tidy. If only there were a similar command for my desk.

Removing Unversioned Files

It is not uncommon for unversioned files to creep into our workspace. These unversioned files could come from any number of sources: compiled binaries, log files, files we intended to add but changed our mind, etc… The TFS Power Tools offer an easy way to clean up these files through the tfpt treeclean command.

tfpt treeclean will scan the workspace for all files and folders not under version control and remove them. A /preview option is available to show which files will be affected without deleting anything.

Getting Latest on Check Out

One thing we noticed with TFS is that it doesn’t automatically get the latest version of a file when it is checked out. More often than not we’d go to check in a change only to have TFS step in and inform us about changes in the repository that needed to be resolved. We’d then need to get latest on those files to merge the changes and resolve any conflicts before TFS would allow us to check it in.

We found that in many instances there would be multiple check ins for a file between the last time we got latest and when we checked it out. This meant that before we even got started making any changes we were already going to have problems trying to check in the file. So why doesn’t TFS automatically get the latest version upon checkout?

Doug Neumann from the TFS team at Microsoft explained the rationale in a forum post:

It turns out that this is by design, so let me explain the reasoning behind it. When you perform a get operation to populate your workspace with a set of files, you are setting yourself up with a consistent snapshot from source control. Typically, the configuration of source on your system represents a point in time snapshot of files from the repository that are known to work together, and therefore is buildable and testable.

As a developer working in a workspace, you are isolated from the changes being made by other developers. You control when you want to accept changes from other developers by performing a get operation as appropriate. Ideally when you do this, you’ll update the entire configuration of source, and not just one or two files. Why? Because changes in one file typically depend on corresponding changes to other files, and you need to ensure that you’ve still got a consistent snapshot of source that is buildable and testable.

This is why the checkout operation doesn’t perform a get latest on the files being checked out. Updating that one file being checked out would violate the consistent snapshot philosophy and could result in a configuration of source that isn’t buildable and testable. As an alternative, Team Foundation forces users to perform the get latest operation at some point before they checkin their changes. That’s why if you attempt to checkin your changes, and you don’t have the latest copy, you’ll be prompted with the resolve conflicts dialog.

The rationale is understandable but seems a bit idealistic. Most developers I know aren’t getting latest on the full branch before each and every check out. Instead they’re doing it once every day or so and otherwise not thinking about it. This is why there’s an option change the default behavior.

Note: The steps discussed below are for enabling getting latest on check out on a developer’s workstation. Refer to http://msdn.microsoft.com/en-us/library/bb385983.aspx for information regarding enabling the option at the server level.

To change this setting:

Open the options dialog in Visual Studio (Tools –> Options)
Expand the Source Control node
Select the Visual Studio Team Foundation Server node
Click the check box for “Get latest version of item on check out”

Although enabling the option will not eliminate the need to merge changes since it is entirely possible, perhaps even likely, that another user will check in changes prior to your check in it eliminates the need to merge with changes that were already in the repository when you checked it out. This option also won’t eliminate the need to get latest on any file dependencies but if you were only getting one file at a time prior to check in anyway does that really matter?

Managing Shelvesets

Most version control systems have some mechanism for putting aside or sharing changes without checking them in and TFS is no exception. Rather than creating patch files like we do in Subversion we “shelve” changes in TFS.

Shelving changes in TFS creates a shelveset. Shelvesets are useful for preventing unrelated changes from conflicting or transferring code to another developer without checking them in. In some ways I prefer shelving changes to creating patch files. The two biggest advantages of shelvesets are:

No need to remember where the patch file was saved since all shelvesets are in the same place
Sharing changes is easier than dealing with permissions on network shares or e-mailing attachments

Creating a shelveset is easy because there are several readily available ways to do it:

File -> Source Control -> Shelve Pending Changes…
From the Pending Changes window (View -> Other Windows -> Pending Changes) switch to the Source Files panel and click the Shelve button.
Right click on a file or folder in Solution Explorer and select Shelve Pending Changes…
Right click on a file or folder in Source Control Explorer select Shelve Pending Changes…

Each of these options will display the Shelve dialog. The dialog is virtually identical to the Check In dialog so I won’t describe its operation here.

Retrieving a shelveset isn’t quite as easy as creating one mainly because there are half as many ways to do it and they can be a bit difficult to find:

File -> Source Control -> Unshelve Pending Changes…
From the Pending Changes window (View -> Other Windows -> Pending Changes) switch to the Source Files panel and click the Unshelve button.

By default the Unshelve dialog lists the shelvesets owned by the current user. Finding shelvesets from other users is just a matter of replacing the user name in the Owner name box. Unfortunately there is no browse for user button so you’ll need to know the user name ahead of time. Clicking the details button will display a list of the changes included in the shelveset and even lets you compare the shelveset version against the local copy. Once you’ve found the desired shelveset click the unshelve button but beware, the built-in unshelve function will fail when there are local changes to any of the files in the shelveset.

The power tools provide a solution to this problem as well with the tfpt unshelve command. tfpt unshelve is a command line utility solves the lack of merge support problem. In its most basic form it displays a dialog very similar to the built-in dialog shown above. The only immediately noticeable difference is the lack of a delete button but there are more differences like the lack of compare support.

When we no longer need a shelveset we can delete it. Shelvesets can be deleted from either the built-in Unshelve dialog or from the command line using the tf shelve command with the /delete option.

Tracking Changes

Providing tools for determining where a change came from is a key component of any version control system and this is an area where I think TFS excels. TFS’s built-in tools for checking change history are actually very good.

Viewing history on a file or folder shows a list of changes typical to most version control systems. The list itself isn’t particularly interesting. It includes the changeset id, the name of the user that checked in the change, a timestamp, and the changeset comment. As expected, we can also get a particular version. We’ll examine some of the more interesting aspects of history and changesets later in the Program Management section but for now we’ll just look at the Track Changeset feature.

Track Changeset allows us to visually determine where changes came from in a multi-branch repository. When a file is branched or merged Track Changeset will graphically display the originating and target branch(es) in either a hierarchy or timeline view. By seeing how and when changes are propagated across the branches we can easily determine when and if a change was successfully applied to a given branch.

Also related to finding the source of a change is the annotation feature. Annotating a file is roughly the same as the blame feature in Subversion. Both systems show the contents of the file along with the changeset (revision in SVN). What sets TFS’s annotate apart though is its ability to drill down into any changes from merge operations to show where a change really came from.

Although the power tools offer a separate annotation utility this is one place I have to recommend the built-in tool. The power tools viewer is nice in that it shows much of the same information as the built-in viewer but doesn’t include any way to view changeset details nor does it drill down into merges. The overall appearance of the built-in tool is much more polished as well.

Project Management

Although version control is a major component of TFS it’s just one part of a much larger system. TFS is really an end-to-end project management system. If you’re only using TFS for version control you’re missing out on the visibility into the overall health of a project that the project management components can give you.

Linking Items

One way that TFS provides visibility into a project is by allowing us to define links between objects. Virtually any type of work item can be linked to another work item and TFS also allows linking to changesets or URLs. Linking individual work items will help define the project plan and structure which is important from a reporting perspective but from a development perspective having links between work items and changesets is invaluable.

In the past my team has used a separate application for defect tracking. QA would enter defects into the tracking system and if the issue required a code change we’d have to note the defect number in the revision comments and we’d quite often note the revision number in the notes on the defect. If we ever needed to track down a change later we’d need to know one of two pieces of information (defect number or revision number) and visit two systems to find why a change was made or what the change actually was. By linking changesets to work items in TFS we only ever need to visit one system to find the answer. Furthermore, if the project is structured correctly we can easily access additional details about the change by drilling deeper into other links.

Linking objects is very easy. There are several ways to link objects but for most of them we’ll usually add links by editing a work item. On the edit work item screen is a tab labeled “All Links” that contains a toolbar with several buttons. The two buttons we’re interested in are the “New” and “Link to” buttons.

As its name implies, the “New” button allows us to create a new work item and automatically create the link. Through the dialog we can create new linked item dialog we can select the link type and corresponding link details. A visual representation of what the link will look like is available for each link type. We can also specify which work item type we’re linking to for each link type. One thing we can’t do in this dialog is link to a changeset because changesets are only created by checking in changes. In order to link to a changeset we need to look at the “Link to” button.

While the “New” button allows us to link to new items the “Link to” button allows us to link to existing items. Most of the procedure is the same but linking to existing items gives us a few more options such as linking to changesets, URLs, test cases, or even individual versioned items. This is great for linking to other work items but there’s a better, less intrusive way to link to changesets to work items.

Sidebar:

When my team started using TFS for project management my director requested that the pilot group update the time remaining and time completed fields on our tasks at the end of each day. This was met with a collective groan and you can probably guess how frequently time was updated.

One reason for the poor adoption is because we aren’t conditioned to record the time anywhere so many of us (myself included) simply didn’t think about it despite his repeated requests. Even when we remembered we had to think about the day and decide how much time to deduct for lunch and other interruptions then try to remember approximately how much time we’d spent on each task. Wouldn’t it be nice to record this information when it was still fresh in our mind rather than trying to remember it at the end of the day?

When we’re selecting work items during check in we can also open them for editing with minimal interruption to the task at hand. Once I discovered this I started updating all of my tasks at check in rather than at the end of the day. With this slight change I guaranteed that my tasks were as up to date as possible with the most accurate information possible.

When we’re ready to check in some changes our workflow generally looks something like:

Open the Check In dialog box
Enter a comment describing the change (you are entering comments, right?)
Verify that the correct files are selected
Click the Check In button

We can link work items to the resulting changeset from the Check In dialog box with a simple modification to the process:

Open the Check In dialog box
Enter a comment describing the change (you are entering comments, right?)
Verify that the correct files are selected
— New Steps —
Click the Work Items button in the left panel
Click the check box for the corresponding work items (You may need to select a different query or enter some search criteria to find them)
Select the appropriate Check-in Action (Typically you’ll be selecting the Associate option)
— End New Steps —
Click the Check In button

Despite having three extra steps the revised process usually only results in a few extra clicks. Even if it’s necessary to refine the list it is far less interrupting to create the association between the work item and changeset from the Check In dialog than from the work item itself. Creating the link when we check in our changes as part of our process also ensures that we actually do establish the link rather than waiting and possibly forgetting.

Once links are established it is very easy to navigate them from the All Links tab. The links list is grouped by link type and all groups are expanded by default. Double-clicking on any item will open the appropriate window for that item. For changesets we see the changeset details window with all of its capabilities.

If it’s not immediately apparent how powerful the links are just work with it for a bit. As more links are established finding data relevant to virtually any project related question will become easier.

Microsoft Project Integration

Linking work items in TFS forms the foundation of a project’s structure. With work items we define time estimates, assign resources, identify dependencies, and project status. We can view and manage all of this information through Visual Studio but for anything but simple tasks that can be quite cumbersome. One of the most important features from a project management perspective is the integration with Microsoft Project.

Note: In order for Microsoft Project to interact with TFS Team Explorer must be installed. Refer to http://msdn.microsoft.com/en-us/library/ms181675.aspx for more information.

From inside of Visual Studio we can open work item queries in Project by either right clicking on a query in the Team Explorer window and selecting Open in Microsoft Project or we can open the query, click the Open in Microsoft Office Button, then select the Open in Microsoft Project option. We can also open an individual work item and click the Open Work Item in Microsoft Project button. Any of these actions will open Project and Project will load a new worksheet containing the appropriate work items.

From within Project we create a new worksheet and select a Team Project through either the Team toolbar or menu. Either option will display the familiar Connect to Team Project dialog. After selecting a project we need to get the work items (again through either the menu or toolbar). The Get Work Items dialog allows us to find work items by saved query, ID, or searching by title and type. Once the list is loaded we can bring them into Project by selecting individual items or all of them.

The work items load into the familiar Gantt chart and we’re free to work with them as we were working on any other Project worksheet. The big difference between working with TFS connected data and a standard worksheet is for refreshing and saving data.

Rather than saving the worksheet to save changes to the work items we need to publish them back to the server using the Publish toolbar button (or Publish Changes menu item). Similarly, if we want to update the worksheet with the latest data we’ll need to click the Refresh toolbar button (or Refresh menu item).

Aside from managing work items the Project integration allows us to manage Areas and Iterations. These can be found by clicking the Edit Areas and Iterations… item under the Team menu.

Alerts

With developers making code changes, managers updating project plans, QA entering defects, scheduled builds, etc… there is a ton of activity going on within TFS at any given time. TFS includes an alert system that helps us keep up with all of it.

Despite the appearance of the dialog box used to create some of the more common alerts the alert system is actually quite robust. Behind the basic dialog box is a full featured expression engine used to define the criteria that must be met in order for a notification to be sent.

Creating basic alerts for monitoring files and folders or work items is pretty straight-forward. Once an alert is configured notifications will be sent whenever changes are made to the associated item.

Files/Folders

Open Source Control Explorer
Locate the File or folder to be monitored
Right-click the desired item and select Alert on Change…
Verify the field values and change any where the default is not desired
Click OK

Work Items

Open the results window for a work item query
Locate the work item to be monitored
Right-click the desired item and select Alert on Change…
Verify the field values and change any where the default is not desired
Click OK

Alerts are managed through the Alerts Explorer window which is accessed through the Team menu. Existing alerts can be modified or deleted and we can also create new, more sophisticated alerts. I encourage clicking around within the Alerts Explorer window and creating a variety of alerts to see all of the possibilities.

Wrap Up

One of the ideas behind TFS is to facilitate communication across the team and by using the tools more intelligently we can make sure that it has the information it needs to accomplish that goal and help us be more productive. TFS certainly isn’t without its pain points but by incorporating the tips listed above we can reduce the impact of some of them or work around them altogether.

The TFS Power Tools will help keep the check-out model out of the way. Changing the default compare and merge tools makes resolving conflicts easier. Shelvesets let us store away or share a batch of changes without checking them in. Linking work items with changesets and other work items helps us verify that the project is on track and answer questions about what happened later on. Alerts proactively keep us informed of what’s going on. None of these are all that useful on their own but when used in combination their power multiplies and their value becomes apparent.

VS2010: The Amazing Expanding Dialog Box

Ever since Visual Studio 2010 was released I’ve noticed that the find/replace dialog was constantly expanding. After multiple openings and closings I needed to resize it to make it usable again. Apparently I wasn’t the only one that noticed since it was the one of top reported issues at Microsoft Connect.

If this problem is bothering you as much as it was me there’s good news. A patch is available to resolve the issue. I installed it over the weekend and haven’t seen the problem since.

TFS2010: Dealing with the Check-out Model

Edit (8/31/2010): The content of this post has been incorporated into my more comprehensive Everyday TFS post. If you’re looking for a general guide to being more productive with TFS on a day-to-day basis you may consider starting there instead.

TFS has been giving me flashbacks to the days when we used Visual Source Safe. After years of using Subversion one of the first things TFS did was remind me why I hate the check out model. The check-out model offers some nice features such as identifying who else might be working on a file but it really just gets in the way most of the time. When I’m in the zone the last thing I want is to have my train of thought derailed just to check out a file. I don’t want to think about source control until it’s time to check in my changes.

Visual Studio Team System helps alleviate the problem by implicitly checking out files but what happens when files are under source control in TFS but edited outside of Visual Studio? Most of the time this requires going to Visual Studio, finding the file in the Source Control Explorer, and checking out the file or issuing the tfs checkout command from a command prompt. I typically open these files through Windows Explorer or through the corresponding application’s open file dialog so neither of the options really fit into my typical workflow. Wouldn’t it be nice to check out files from Windows Explorer?

Note: Yes, it’s possible to remove the read-only attribute from the file but that’s at least as intrusive as the other methods and has the added benefit of confusing TFS.

TFS Power Tools includes shell integration. Like the shell integration for other version control systems Windows Explorer displays files and folders under TFS source control with icons decorated to indicate their status. The integration isn’t quite as full-featured as what we get with Tortoise SVN but it’s enough to be productive. Primarily it allows getting the latest version, check-in/out, undo check-out, and comparing files with the server version. Conflict resolution, adding, renaming, moving, and deleting files are also possible. I still hate the check out model but the shell integration has made it a bit less painful.

A second headache with TFS’s check out model is that EVERYTHING needs to be checked out in order to do anything with it. Under normal circumstances this is fine but what if you’re only making a small, temporary change that you know won’t be checked in – at least not on purpose anyway? I know I’m not the only one that regularly adds and removes debugger; lines in JavaScript files while I’m bug hunting. There are also those pesky files that Visual Studio automatically checks out but never actually changes. When it’s time to actually do a check-in we’re left with a bunch of unchanged files to sift though to find the ones we actually care about.

TFS is supposed to only check-in files that have actually changed but if you’re like me you like to make a quick pass over your changes before actually performing the check-in. Wading through unchanged file after unchanged file is just a frustrating waste of time. I also like to keep my digital workspace tidy (my physical desktop is a different matter) and all of these unchanged files are unnecessary clutter. TFS Power Tools offers a solution for this too.

The tfpt uu command line utility will examine the pending changes in a workspace and undo any unchanged files. By default the utility will get latest on the workspace before comparing files to the latest version. There are a handful of arguments for controlling aspects of its operation, of note are /noget which prevents the utility from getting latest and /recursive which checks the folders in the workspace recursively. For full information regarding the utility’s options refer to its documentation (tfpt uu /?).

Since I’ve started using TFS I’ve really come to rely on the functionality that the power tools provide. At this point I have very little doubt that the power tools are absolutely essential for anyone working with TFS. Both the shell integration and the tfpt uu command have become essential parts of my workflow since they help minimize the frustrations associated with the check out model.

Rate this:

Spread The Word:

LINQ’s Ugly Secret

Not Always a Good Fit

Query Optimization Tips

Any() vs Count()

Consider Join Order

PLINQ

Bringing it All Together

Rate this:

Spread The Word:

Indy TFS User Group

IndyNDA

Indy GiveCamp

Rate this:

Spread The Word:

Managing Tests

Test Projects

Test View Window

Test Categories

Running Tests

Test Results Window

Web Performance Tests

Creating Web Performance Tests

Recording in Visual Studio

Exporting Sessions from Fiddler

Parameterization

Context Parameters

Parameterizing Web Servers

Setting Request Properties

Comments

Extraction Rules

Validation Rules

Binding Tests to a Data Source

Transactions

Calling Other Web Performance Tests

Load Tests

Scenarios

Think Time

Load Pattern

Test Mix and Test Mix Model

Network Mix

Browser Mix

Performance Counter Sets

Run Settings

Analyzing Load Test Results

Summary

Rate this:

Spread The Word:

Introducing Dynamic Programming

Reflecting on the Past

Dynamic Language Runtime to the Rescue!

Using Dynamic Languages

Special Dynamic Types

ExpandoObject

DynamicObject

Can’t Everything be Dynamic?

Rate this:

Spread The Word:

Rate this:

Spread The Word:

Rate this:

Spread The Word:

Tools

TFS Power Tools

Installing the Power Tools

Shell Integration

Power Tools Command Line Utility

TFS Command Line Utility

Changing the Compare and Merge Tools

Version Control

Keeping Your Workspace Tidy

Unchanged Files

Removing Unversioned Files

Getting Latest on Check Out

Managing Shelvesets

Tracking Changes

Project Management

Linking Items

Microsoft Project Integration