LINQed Up (Part 4)

This is the fourth part of a series intended as an introduction to LINQ.  The series should provide a good enough foundation to allow the uninitiated to begin using LINQ in a productive manner.  In this post we’ll look at the performance implications of LINQ along with some optimization techniques.

LINQed Up Part 3 showed how to compose LINQ statements to accomplish a number of common tasks such as data transformation, joining sequences, filtering, sorting, and grouping.  LINQ can greatly simplify code and improve readability but the convenience does come at a price.

LINQ’s Ugly Secret

In general, evaluating LINQ statements takes longer than evaluating a functionally equivalent block of imperative code.

Consider the following definition:

var intList = Enumerable.Range(0, 1000000);

If we want to refine the list down to only the even numbers we can do so with a traditional syntax such as:

var evenNumbers = new List<int>();

foreach (int i in intList)
	if (i % 2 == 0)

…or we can use a simple LINQ statement:

var evenNumbers = (from i in intList
					where i % 2 == 0
					select i).ToList();

I ran these tests on my laptop 100 times each and found that on average the traditional form took 0.016 seconds whereas the LINQ form took twice as long at 0.033 seconds.  In many applications this difference would be trivial but in others it could be enough to avoid LINQ.

So why is LINQ so much slower?  Much of the problem boils down to delegation but it’s compounded by the way we’re forced to enumerate the collection due to deferred execution.

In the traditional approach we simply iterate over the sequence once, building the result list as we go.  The LINQ form on the other hand does a lot more work.  The call to Where() iterates over the original sequence and calls a delegate for each item to determine if the item should be included in the result.  The query also won’t do anything until we force enumeration which we do by calling ToList() resulting in an iteration over the result set to a List<int> that matches the list we built in the traditional approach.

Not Always a Good Fit

How often do we see code blocks that include nesting levels just to make sure that only a few items in a sequence are acted upon? We can take advantage of LINQ’s expressive nature to flatten much of that code into a single statement leaving just the parts that actually act on the elements. Sometimes though we’ll see a block of code and think “hey, that would be so much easier with LINQ!” but not only might a LINQ version introduce a significant performance penalty, it may also turn out to be more complicated than the original.

One such example would be Edward Tanguay’s code sample for using a generic dictionary to total enum values.  His sample code builds a dictionary that contains each enum value and the number of times each is found in a list. At first glance LINQ looks like a perfect fit – the code is essentially transforming one collection into another with some aggregation.  A closer inspection reveals the ugly truth.  With Edward’s permission I’ve adapted his sample code to illustrate how sometimes a traditional approach may be best.

For these examples we’ll use the following enum and list:

public enum LessonStatus

List<LessonStatus> lessonStatuses = new List<LessonStatus>()

Edward’s traditional approach defines the target dictionary, iterates over the names in the enum to populate the dictionary with default values, then iterates over the list of enum values, updating the target dictionary with the new count.

var lessonStatusTotals = new Dictionary<string, int>();

foreach (var status in Enum.GetNames(typeof(LessonStatus)))
	lessonStatusTotals.Add(status, 0);
foreach (var status in lessonStatuses)


In my tests this form took an average of 0.00003 seconds over 100 invocations.  So how might it look if in LINQ?  It’s just a simple grouping operation, right?

var lessonStatusTotals =
	(from l in lessonStatuses
		group l by l into g
		select new { Status = g.Key.ToString(), Count = g.Count() })
	.ToDictionary(k => k.Status, v => v.Count);


Wrong. This LINQ version isn’t functionally equivalent to the original. Did you see the problem?  Take another look at the output of both forms.  The dictionary created by the LINQ statement doesn’t include any enum values that don’t have corresponding entries in the list. Not only does the output not match but over 100 invocations this simple grouping query took an average of 0.0001 seconds or about three times longer than the original.  Let’s try again:

var summary = from l in lessonStatuses
				group l by l into g
				select new { Status = g.Key.ToString(), Count = g.Count() };
var lessonStatusTotals = 
	(from s in Enum.GetNames(typeof(LessonStatus))
	 join s2 in summary on s equals s2.Status into flat
	 from f in flat.DefaultIfEmpty(new { Status = s, Count = 0 })
	 select f)
	.ToDictionary (k => k.Status, v => v.Count);


In this sample we take advantage of LINQ’s composable nature and perform an outer join to join the array of enum values to the results of the query from our last attempt.  This form returns the correct result set but comes with an additional performance penalty.  At an average of 0.00013 seconds over 100 invocations, This version took almost four times longer and is significantly more complicated than the traditional form.

What if we try a different approach?  If we rephrase the task as “get the count of each enum value in the list” we can rewrite the query as:

var lessonStatusTotals = 
	(from s in Enum.GetValues(typeof(LessonStatus)).OfType<LessonStatus>()
	 select new
	 	Status = s.ToString(),
		Count = lessonStatuses.Count(s2 => s2 == s)
	.ToDictionary (k => k.Status, v => v.Count);


Although this form is greatly simplified from the previous one it still took an average of 0.0001 seconds over 100 invocations.  The biggest problem with this query is that it uses the Count() extension method in its projection.  Count() iterates over the entire collection to build its result.  In this simple example Count() will be called five times, once for each enum value.  The performance penalty will be amplified by the number of values in the enum and the number of enum values in the list so larger sequences will suffer even more.  Clearly this is not optimal either.

A final solution would be to use a hybrid approach.  Instead of joining or using Count we can compose a query that references the original summary query as a subquery.

var summary = from l in lessonStatuses
	group l by l into g
	select new { Status = g.Key.ToString(), Count = g.Count() };

var lessonStatusTotals =
	(from s in Enum.GetNames(typeof(LessonStatus))
	 let summaryMatch = summary.FirstOrDefault(s2 => s == s2.Status)
	 select new
	 	Status = s,
		Count = summaryMatch == null ? 0 : summaryMatch.Count
	.ToDictionary (k => k.Status, v => v.Count);


At an average of 0.00006 seconds over 100 iterations this approach offers the best performance of any of the LINQ forms but it still takes nearly twice as long as the traditional approach.

Of the four possible LINQ alternatives to Edward’s original sample none of them really improve readability.  Furthermore, even the best performing query still took twice as long.  In this example we’re dealing with sub-microsecond differences but if we were working with larger data sets the difference could be much more significant.

Query Optimization Tips

Although LINQ generally doesn’t perform as well as traditional imperative programming there are ways to mitigate the problem.  Many of the usual optimization tips also apply to LINQ but there are a handful of LINQ specific tips as well.

Any() vs Count()

How often do we need to check whether a collection contains any items?  Using traditional collections we’d typically look at the Count or Length property but with IEnumerable<T> we don’t have that luxury.  Instead we have the Count() extension method.

As previously discussed, Count() will iterate over the full collection to determine how many items it contains.  If we don’t want to do anything beyond determine that the collection isn’t empty this is clearly overkill.  Luckily LINQ also provides the Any() extension method.  Instead of iterating over the entire collection Any() will only iterate until a match is found.

Consider Join Order

The order in which sequences appear in a join can have a significant impact on performance.  Due to how the Join() extension method iterates over the sequences the larger sequence should be listed first.


Some queries may benefit from Parallel LINQ (PLINQ).  PLINQ partitions the sequences into segments and executes the query against the segments in parallel across multiple processors.

Bringing it All Together

As powerful as LINQ can be at the end of the day it’s just another tool in the toolbox.  It provides a declarative, composable, unified, type-safe language to query and transform data from a variety of sources.  When used responsibly LINQ can solve many sequence based problems in an easy to understand manner.  It can simplify code and improve the overall readability of an application.  In other cases such as what we’ve seen in this article it can also do more harm than good.

With LINQ we sacrifice performance for elegance.  Whether the trade-off is worth while is a balancing act based on the needs of the system under development.  In software where performance is of utmost importance LINQ probably isn’t a good fit.  In other applications where a few extra microseconds won’t be noticed, LINQ is worth considering.

When it comes to using LINQ consider these questions:

  • Will using LINQ make the code more readable?
  • Am I willing to accept the performance difference?

If the answer to either of these questions is “no” then LINQ is probably not a good fit for your application.  In my applications I find that LINQ generally does improve readability and that the performance implications aren’t significant enough to justify sacrificing readability but your mileage may vary.


LINQed Up (Part 3)

This is the third part of a series intended as an introduction to LINQ.  The series should provide a good enough foundation to allow the uninitiated to begin using LINQ in a productive manner.  In this post we’ll take a more detailed look at composing queries through a variety of examples and introduce some new concepts such as deferred execution.

In the previous post we covered some of the common query methods and introduced both method and query syntax.  By the end of this post you should have a solid foundation to begin composing your own queries within your applications.  The examples will use either method or query syntax depending upon their complexity.  Before we see the examples though we should take a closer look at query syntax.

Diving Into Query Syntax

As mentioned in the previous post query syntax allows us to compose LINQ expressions in a SQL-like manner.  This is accomplished through the use of several new or overloaded keywords that are converted into their method syntax equivalent at compile time.

Note: Not every method in the System.Linq.Enumerable class has a query syntax equivalent.

The main keywords used by query syntax are:

  • from
  • where
  • select
  • join
  • orderby [ascending | descending]
  • group by
  • let

Although the purpose of most of these keywords should be pretty clear it’s worth taking a closer look at each of them.


The from keyword identifies the sequence that will serve as the primary source for the query.  Every query built using query syntax must begin with a from clause.  By placing the from clause first we get full IntelliSense support on our query syntax queries.  Any type that implements IEnumerable can be used in the from clause.

Some queries will contain multiple from clauses.  This is a special syntax called SelectMany that is used to flatten multiple sequences into one.


LINQ query syntax provides a where clause that provides the same basic function as its SQL counterpart.  In the LINQ form we get the benefit of strong typing and IntelliSense.

Note: The criteria provided in the where clause can include any number of .NET operations but may be limited depending on the capabilities of the LINQ provider.


The select clause is used to project query results into a new sequence.  Select is capable of returning single values, well-known types, or anonymous types.  Select is the workhorse behind one of LINQ’s most powerful features: transforming data from one structure to another.


LINQ is capable of joining multiple sequences together much like SQL can join tables or views.  Part of what makes LINQ so powerful is its ability to join data from disparate sources in a single query.  This allows us to join simple .NET collections to XML or data returned from LINQ to SQL or Entity Framework.

One restriction on join joins in LINQ is that it requires an equality comparison.  To eliminate any ambiguity about what can be used Microsoft introduced the equals keyword to replace the == operator.


LINQ allows sorting in query syntax through the use of the orderby keyword.  By default items will be sorted in ascending order but that can be controlled through the ascending and descending keywords.  Multiple items can be sorted by separating them with a comma.

group by

Grouping in LINQ works a bit different than in SQL.  In SQL we simply specify one or more column names in the group by clause.  LINQ’s operates by projecting values into a System.Linq.IGrouping that is keyed upon the group and all group members as members of the group.  Since IGrouping implements IEnumerable it can be directly projected into another query.  Grouping can be performed upon either a data value or a calculated value.


The let keyword is a bit of an oddity.  Unlike the other query syntax keywords let doesn’t have a direct mapping to a method.  Instead, let is provided for convenience to declare query scoped variables to eliminate the need for repetitive operations within a query.

LINQ Examples

Now that we’ve covered the how LINQ works, its common methods, and syntax we should have a good foundation to see some examples of LINQ in action and understand what is happening.  In the following sections we’ll see how all of the pieces we’ve discussed so far fit together to make LINQ a powerful tool for solving a wide variety of set based problems.

Each of the examples assume the following data structures and data:

class Author
	public Author(string firstName, string lastName)
		FirstName = firstName;
		LastName = lastName;

	public string FirstName { get; set; }
	public string LastName { get; set; }

class Book
	public Book(int id, string title, string isbn13, int copyrightYear, params Author[] authors)
		ID = id;
		Title = title;
		Isbn13 = isbn13;
		CopyrightYear = copyrightYear;
		Authors = authors;

	public int ID { get; set; }
	public string Title { get; set; }
	public string Isbn13 { get; set; }
	public int CopyrightYear { get; set; }
	public IEnumerable<author> Authors { get; set; }

IEnumerable<book> GetLibrary()
	return new List<book>()
		new Book(
			"Essential LINQ",
			new Author("Charlie", "Calvert"),
			new Author("Dinesh", "Kulkarni")
		new Book(
			"Programming F#",
			new Author("Chris", "Smith")
		new Book(
			"C# 4.0 in a Nutshell",
			new Author("Joseph", "Albahari"),
			new Author("Ben", "Albahari")
		new Book(
			"WPF in Action with Visual Studio 2008",
			new Author("Arlen", "Feldman"),
			new Author("Maxx", "Daymon")
Note: I recommend using LINQPad to run these examples. If using LINQPad remember to change the language option to C# Program. If using Visual Studio to run the examples make sure your code file includes a using directive for System.Linq.

Starting Simple

This first example illustrates the most basic form of a LINQ statement.  Consider it the “Hello World” example.

var query = from b in GetLibrary()
			select b;

Even in this example we can see several of the LINQ concepts coming into play. First, query is defined using the var keyword indicating type inference. The compiler understands that GetLibrary() returns IEnumerable<Book> and the select clause is projecting instances of Book so in this case the inferred type of query is IEnumerable<Book>. We also see how the from clause defines the range variable and appears before the select clause.

Despite the simplicity of this query it LINQ really isn’t that effective here. This query can be better expressed by simply referring to GetLibrary() since all we’re doing is retrieving each item in the sequence.


LINQ includes extension methods for converting sequences to lists, dictionaries, and arrays but it is by no means restricted to those types. Through the Select clause (or method) we can return either a new instance of a well-known type or define a new anonymous type on the fly.
Transform to Array

var arr = GetLibrary().ToArray();

Transform to XML
To really demonstrate the transformation capabilities we need to see a more complex example. Here we’ll take the entire structure returned by GetLibrary() into a well-formed XML document with LINQ to XML.

var query = new XElement("library",
						 from b in GetLibrary()
						 select new XElement("book",
						 					 new XAttribute("title", b.Title),
											 new XAttribute("isbn_13", b.Isbn13),
											 new XAttribute("copyright", b.CopyrightYear),
											 new XElement("authors",
											 			  from a in b.Authors
														  select new XElement("author",
														  					  new XText(String.Format("{0}, {1}", a.LastName, a.FirstName))

Basic Filtering

Much of LINQ’s real power comes from the operations it can perform against sequences so what’s a better place to start than with it’s filtering capabilities.  Prior to LINQ if we wanted to operate against a subset of a sequence we needed to nest potentially complex logic in the body of a loop. With LINQ we can filter the sequence down to only the elements we care about before entering the loop.

var query = from b in GetLibrary()
			where b.ID == 1
			select b;

This example illustrates how easy it is to reduce a sequence to only the elements we care about. Here we’re reducing the full sequence down to only those elements with an ID of 1. Although it isn’t necessarily obvious, this query introduces a lambda expression. Let’s look at the same query using method syntax to bring out the lambda.

var query = GetLibrary().Where(b => b.ID == 1);

In either case we’ll get an IEnumerable<book> that contains only a single item. In situations like this where we know we’ll only get one item or we just want the first no matter how many items are returned we can turn to the First() or FirstOrDefault() extension methods. Each of these methods have an overload that accepts a lambda expression to filter the sequence they are acting upon. We can simplify the above queries to just get the single book we care about:

var query = GetLibrary().FirstOrDefault(b => b.ID == 1);

Filtering is also not limited to a single value nor is it restricted to values contained directly in a sequence.

var query = from b in GetLibrary()
			where b.CopyrightYear == 2009 && b.Authors.Count() > 1
			select b;

The example above reduces the sequence to only those books with a copyright year of 2009 and multiple authors. While the collection of authors is contained within each book the number of authors is determined by calling the Count() extension method.

Deferred Execution

Having seen a few queries and the role that lambda expressions play in them we have a perfect opportunity to explore deferred execution. In each of the above examples we define a variable named query and set it to the query. The query is not actually executed until it is enumerated. That is, even though we have defined the query we don’t actually determine what items make up the sequence until we force enumeration of the query by using it a loop or call a method that causes it to enumerate. To observe the effect consider the next example:

var title = "LINQ";

var query = from b in GetLibrary()
			where b.Title.Contains(title)
			select b;

title = "F#";

var book = query.FirstOrDefault();

The query references a local string variable named title that was initialized to “LINQ” prior to the definition of the query. We then change the value of title to “F#” before retrieving the first item from the new sequence. Because the value of the variable isn’t resolved in the query until the query is executed we get the book “Programming F#” rather than “Essential LINQ.”

Deferred execution is an important concept to understand in LINQ to Objects and critical for LINQ to SQL and Entity Framework.


Sorting with LINQ is pretty straight forward. Results can be sorted on one or more values in ascending or descending order.
Single Value Ascending

var query = from b in GetLibrary()
			orderby b.Title
			select b.Title;

Single Value Descending

var query = from b in GetLibrary()
			orderby b.Title descending
			select b.Title;

Multiple Values

var query = from b in GetLibrary()
			orderby b.CopyrightYear descending, b.Title
			select new { b.ID, b.Title, b.CopyrightYear };

The same query can be expressed in method syntax as follows:

var query = GetLibrary()
				.OrderByDescending (b => b.CopyrightYear)
				.ThenBy (b => b.Title)
				.Select(b => new


Sequences from one or more data sources may be joined together in a single query.  The data sources don’t even need to be of the same type.

LINQ supports both inner and outer joins. To demonstrate the join capabilities we’ll introduce an XElement that we can join to using part of each book’s ISBN.

Note: The XElement class is part of LINQ to XML and can be found in the System.Xml.Linq namespace. The LINQ to XML classes offer many advantages over the traditional XML classes in that they were specifically designed for queries and composition.
XElement GetPublishers()
	return new XElement("publishers",
						new XElement("publisher",
									 new XAttribute("id", 321),
									 new XText("Addison-Wesley")),
						new XElement("publisher",
									 new XAttribute("id", 933),
									 new XText("Manning")),
						new XElement("publisher",
									 new XAttribute("id", 596),
									 new XText("O'Reilly")),
						new XElement("publisher",
									 new XAttribute("id", 7653),
									 new XText("Tor Books")),
						new XElement("publisher",
									 new XAttribute("id", 312),
									 new XText("St. Martin's Griffin")));

Inner Join
In the next example we retrieve all of the publisher elements from our XML document. Since we don’t have the publisher ID available directly within the Book class we extract it from the ISBN and specify that the extracted publisher id is equal to the id attribute of each element. We then project a new anonymous type that includes values from both sequences.

var query = from b in GetLibrary()
			join p in GetPublishers().Descendants("publisher") on int.Parse(b.Isbn13.Split('-')[2]) equals int.Parse(p.Attribute("id").Value)
			select new { PublisherName = p.Value, b.Title };

Outer Join
Outer joins are a bit more complicated than inner joins but only slightly. In addition to specifying the join values we need to project the results of the join into another sequence that we then select from using the DefaultIfEmpty extension method.

var query = from p in GetPublishers().Descendants("publisher")
			let publisherID = int.Parse(p.Attribute("id").Value)
			join b in GetLibrary() on publisherID equals int.Parse(b.Isbn13.Split('-')[2])
				into bookPublishers
			from bp in bookPublishers.DefaultIfEmpty()
			select new
				PublisherID = publisherID,
				PublisherName = p.Value,
				Book = bp


SelectMany allows us to flatten multiple sequences. Using our Books data we can flatten the structure to extract a single sequence containing all the authors.

var query = from b in GetLibrary()
			from a in b.Authors
			orderby a.LastName
			select a;


Grouping query results by a value is fairly straight-forward. With grouping we need to identify the source sequence, specify the group ing value, then project the grouped results into another IGrouping sequence. In this example we’ll see how to group books by copyright year.

var query = from b in GetLibrary()
			group b by b.CopyrightYear into y
			select y;

Next Steps

Having read through and (hopefully) trying the examples you should have a good understanding of how to implement LINQ in your projects. While LINQ is great for simplifying code it does come at a price. In the next post we’ll examine some of the performance implications of using LINQ and look at how to optimize some queries.

LINQed Up (Part 2)

This is the second part of a series intended as an introduction to LINQ.  The series should provide a good enough foundation to allow the uninitiated to begin using LINQ in a productive manner.  In this post we’ll look at some of the common query methods in Enumerable and what LINQ looks like.

In the previous post we defined LINQ and discussed some of the features of the .NET Framework that make LINQ possible.  This post will build upon that foundation.  By the end of this post you should understand the basics of LINQ and understand how it can fit into your development toolbox.

Common Query Methods

The Enumerable class in the System.Linq namespace is central to LINQ’s functionality in that it defines all of the core extension methods that make up LINQ.  There are tons of methods in the Enumerable class but for the purpose of this post we’ll focus on just a few.  Most of the methods here have at least one overload.  We’ll examine each method taking a more generalized approach to introduce the methods and divide them into a few categories.

Sequence Operations

The methods in this section work across an entire sequence.

Method Name Description
Where Primary method used to filter a sequence.
Select Primary method used to project results from the query
Join Allows combining sequences into a single query.  Similar to a SQL join.
OrderBy/OrderByDescending Sorts a sequence.
All Indicates whether every element in a sequence meet the specified criteria.
Any Indicates whether any element in a sequence meet the specified criteria.
Count Gets the number of elements in a sequence.

Element Retrieval Operations

Each of the methods below allow retrieval of a specific element in a sequence and have two forms.  The basic form will throw an exception if the element cannot be found while the OrDefault version will return default(T).

Method Name Description
First/FirstOrDefault Retrieves the first element in a sequence.
ElementAt/ElementAtOrDefault Retrieves the element at the specified position in a sequence.
Last/LastOrDefault Retrieves the last element in a sequence.

What Does LINQ Look Like?

LINQ statements can be written in either of two forms: query syntax and method (dot) syntax.  While these forms are functionally equivalent they each have their place and the decision about which form to use will often come down to readability and is typically situational.  There is no reason to use one form exclusively.

Each of the following examples will use a sequence that contains the first ten numbers in the Fibonacci sequence.

var fibonacci = new int[] { 1, 1, 2, 3, 5, 8, 13, 21, 34, 55 };

Where We’re Coming From

Before looking at each LINQ syntax it’s helpful to see an example of what LINQ aims to address.  Consider the following code that builds a sequence of the even numbers found in the Fibonacci sequence defined above:

var evenNumbers = new List();
foreach(var i in fibonacci)
	if(i % 2 == 0)

This code, while not particularly complex involves a number of steps to accomplish a relatively simple task.  First we define a List<int> to hold the even numbers.  We then enter a loop where we check whether each value is even before adding it to the list.  This is imperative programming at its finest.  The focus of the code is on how rather than what.  Of the eight lines of code only one line is really relevant to the problem of finding even numbers in the source sequence.  LINQ addresses the imperative nature of this code by providing a functional framework to let us focus on the what rather than the how.

Method Syntax

Method syntax is a fluent interface that allows building queries using method calls.  As its name implies, method syntax calls the LINQ extension methods directly passing lambda expressions as parameters.  Compare the code below with the traditional example above.

var evenNumbers = fibonacci.Where(i => i % 2 == 0);

In this example we remove all of the imperative code and replace it with a single method call and let LINQ do all of the heavy lifting.  When the Where method executes it calls the supplied lambda expression for each value in the source sequence.  The lambda expression must return a boolean value that informs the Where method whether or not the current value meets the criteria.

Method syntax tends to be more readable for simple queries such as this example.

Query Syntax

Alternatively, query syntax introduces a SQL-like syntax for writing queries.  Here is the same example repeated using query syntax:

var evenNumbers =
    from i in fibonacci
    where i % 2 == 0
    select i;

Query syntax tends to be more verbose than method syntax but is well suited for composing more complex queries such as those that use joins.

Notice the SQL like structure of the above query. One important difference to note between SQL and query syntax is that the from and where clauses are in the opposite order as they would be in a SQL query.  In fact, query syntax requires the from clause to appear first.  By placing the from clause at the beginning of the query we get all of the benefits of IntelliSense within Visual Studio.  Ultimately though, query syntax is just some syntactic sugar.  When the code is compiled any queries using query syntax are parsed and converted to method syntax.

Next Steps

Now that we’ve seen some of the common LINQ methods and understand how to compose LINQ statements with both method and query syntax we can look at how to use the various methods.  The next post will go in-depth showing how to compose LINQ statements using the common methods and introduce some new concepts such as deferred execution.

LINQed Up (Part 1)

This is the first of a series intended as an introduction to LINQ. The series should provide a good enough foundation to allow the uninitiated to begin using LINQ in a productive manner. In this post we’ll look at what LINQ is and how it works.

My director and I were recently talking about questions he has been asking candidates for senior level .NET development positions. He mentioned that he has been asking the candidates to describe LINQ and some situations where it would be useful.  The response from each of the candidates has ranged from a blank stare to something along the lines of “it means you don’t need to write SQL anymore.”  Those responses are the inspiration for this series.

The blank stares are discouraging but the statements that constrain LINQ to a very specific use case illustrate a fundamental lack of understanding of the technology. It is true that LINQ can greatly simplify interaction with a database through LINQ to SQL or Entity Framework but those are only a small part of what LINQ can do.  In fact, the majority of the places I’ve used LINQ have no database interaction whatsoever.  LINQ has so many applications beyond database access that I find myself using at least some part of it in most of my projects and often in some unexpected places.

Let’s start with a trip through the basics.

What is LINQ?

Language INtegrated Query (LINQ) was introduced with the .NET Framework v3.5. MSDN has this to say about it:

LINQ is a set of extensions to the .NET Framework that encompass language-integrated query, set, and transform operations. It extends C# and Visual Basic with native language syntax for queries and provides class libraries to take advantage of these capabilities.

Although the description is accurate I think the language regarding “native language syntax for queries” is what leads people to mistakenly believe that the only use for LINQ is with a database. After all, we’ve been conditioned to think that queries are database operations.  That said, I offer an alternative definition:

LINQ is a set of extensions to the .NET Framework that encompass language-integrated query, set, and transform operations. It extends C# and Visual Basic with native language syntax for querying data from a variety of sources and provides class libraries to take advantage of these capabilities.

The idea that LINQ makes it possible to query data from a variety of sources is critically important to using it to its full potential. It means that LINQ is not constrained to working with databases but actually comes in several flavors:

  • LINQ to Objects
  • LINQ to XML
  • LINQ to SQL (and Entity Framework)
  • LINQ to DataSets
  • LINQ to Twitter
  • etc…

Essentially any data source can be queried with LINQ as long as there’s a corresponding provider. In addition to providing a common query language for disparate data sources these sources can be queried in a unified manner through the use of joins and subqueries.  LINQ also gives us some really powerful transformation capabilities.  Essentially LINQ is a domain specific language for working with sets of data.

How Does LINQ Work?

LINQ is made possible by several additions to the .NET Framework and in order to truly appreciate its power and elegance we need to first look at:

  • Extension Methods
  • Delegates/Lambda Expressions
  • Type Inference
  • Anonymous Types

Since these are all features of the .NET framework and/or compiler their usage is not restricted to LINQ.  Most of them are actually quite useful outside of LINQ as well.

Extension Methods

Central to the functionality of LINQ are extension methods. Extension methods allow adding capabilities to types without needing to derive a new type. They must be static methods within a static class. The type being extended must be the first parameter of the method and is modified using an overload of the this keyword.  Because extension methods add capabilities to an existing type without relying on inheritance we can even write extension methods for sealed classes.

LINQ introduces the class System.Linq.Enumerable that contains extension methods that extend the IEnumerable interface.  Microsoft could have added the signatures to the interface but that would be a breaking change and everything that previously built against IEnumerable would no longer compile until the implementations of those were provided.  By using extension methods Microsoft was able to introduce all of the LINQ query methods into the framework without breaking anything.

Activating LINQ is merely a matter of importing the System.Linq namespace.  Once the namespace is imported the extension methods are available to any type that implements IEnumerable including lists, arrays, and even strings.  There’s even a trick for using the non-generic IEnumerable with LINQ that we’ll discuss in a later post.

Delegates/Lambda Expressions

While extension methods provide the methods that make LINQ possible delegates make them work.  Most of the extension methods in the Extensible class accept one or more delegates as parameters.  Delegates have always been available in .NET but their usage and syntax has evolved over the years.

Before C# 2.0 the only way to use delegates was to have a named method.  C# 2.0 introduced anonymous methods using the delegate keyword.

Handling an event with the delegate syntax

var t = new System.Timers.Timer(1000);
t.Elapsed += delegate(object sender, System.Timers.ElapsedEventArgs ea) { Console.WriteLine("Timer elapsed"); };


Notice how the event is handled by an inline anonymous method rather than a separate named method.  LINQ makes heavy use of delegates to control query behavior. Having to include a full method signature to pass to a method would make LINQ statements virtually unreadable so clearly something else was needed. This is where lambda expressions come in to play.

C# 3.0 added support for lambda expressions. Lambda expressions are functionally equivalent to the delegate syntax above but are more developer friendly. In C# lambda expressions use the => (goes to) operator. The left side contains the list of parameters and the right side contains the method body.

Handling an event with a lambda expression

var t = new System.Timers.Timer(1000);
t.Elapsed += (s, ea) => Console.WriteLine("Timer elapsed");


In both examples the timer’s elapsed event is handled by an anonymous method and both handle the event exactly the same way but in the lambda example we have the much more concise and easier to read syntax.  The key difference between the traditional delegate syntax and a lambda expression is the lack of any type information in the parameter list of the lambda expression.  This lack of type information is a great segue into the next technology important to LINQ: type inference.

Type Inference

Type inference allows the compiler to determine the type of a variable, return type, or generic type. By letting the compiler do its job with type inference we can remove a lot of the explicit nature of type identification. Type inference gives us the ability to use anonymous types and use the var keyword to declare variables (and is required to use anonymous types).

Using the var keyword to declare variables is the subject of debate. One side is opposed to its use saying that code is too ambiguous whereas the other side likes the simplicity and convenience of it. I fall into the later group because I’ve found that as I’m first developing something I may change variable or return types multiple times as the design is flushed out.  By using the var keyword I typically only have to change the type on one place rather.  The var keyword is also required when using anonymous types.  If there’s ever any question about what type is being resolved, just hover over var in Visual Studio.

Don’t confuse use of the var keyword with the dynamic keyword in .NET 4.0 or JavaScript’s var. Variables declared with the var keyword are still strongly typed, we’re just letting the compiler figure out what the type really is.

Anonymous Types

Finally, we have anonymous types. Anonymous types are dynamically defined types with no formal definition outside of their usage.  At compile time the compiler will generate a read-only type based on the inline definition of the type.  The type name is not known until compile time and the generated type name is not valid within C# so the only way to declare a variable of an anonymous type is through the var keyword described above. Although they’re not required to use LINQ anonymous types add a lot of capabilities for projecting results from a query.

Creating an anonymous type with two properties

var anon = new { IntegerValue = 1, StringValue = "A String" };

Due to the way anonymous types are defined there are some restrictions on their use. Although there are some ways around this the rule of thumb is that anonymous types can only be used within the scope where they are declared.

Next Steps

We’ve covered what LINQ is and the main pieces of the .NET Framework make it possible.  In the next post we’ll look at the common query methods and how to construct queries.

LINQ: IEnumerable to DataTable

Over the past several months I’ve been promoting LINQ pretty heavily at work.  Several of my coworkers have jumped on the bandwagon and are realizing how much power is available to them.

This week two of my coworkers were working on unrelated projects but both needed to convert a list of simple objects to a DataTable and asked me for an easy way to do it.  LINQ to DataSet provides wonderful functionality for exposing DataTables to LINQ expressions and converting the data into another structure but it doesn’t have anything for turning a collection of objects into a DataTable.  Lucky for us LINQ makes this task really easy.

First we need to use reflection to get the properties for the type we’re converting to a DataTable.

var props = typeof(MyClass).GetProperties();

Once we have our property list we build the structure of the DataTable by converting the PropertyInfo[] into DataColumn[].  We can add each DataColumn to the DataTable at one time with the AddRange method.

var dt = new DataTable();
  props.Select(p => new DataColumn(p.Name, p.PropertyType)).ToArray()

Now that the structure is defined all that’s left is to populate the DataTable.  This is also trivial since the Add method on the Rows collection has an overload that accepts params object[] as an argument.  With LINQ we can easily build a list of property values for each object, convert that list to an array, and pass it to the Add method.

  i => dt.Rows.Add(props.Select(p =>; p.GetValue(i, null)).ToArray())

That’s all there is to it for collections of simple objects.  Those familiar with LINQ to DataSet might note that the example doesn’t use the CopyToDataTable extension method.  The main reason for adding the rows directly to the DataTable instead of using CopyToDataTable is that we’d be doing extra work.  CopyToDataTable accepts IEnumerable but constrains T to DataRow.  In order to make use of the extension method (or its overloads) we still have to iterate over the source collection to convert each item into a DataRow, add each row into a collection, then call CopyToDataTable with that collection.  By adding the rows directly to the DataTable we avoid the extra step altogether.

We can now bring the above code together into a functional example. To run this example open LINQPad, change the language selection to C# Program, and paste the code into the snippet editor.

class MyClass
  public Guid ID { get; set; }
  public int ItemNumber { get; set; }
  public string Name { get; set; }
  public bool Active { get; set; }

IEnumerable<MyClass> BuildList(int count)
  return Enumerable
    .Range(1, count)
      i =>
      new MyClass()
        ID = Guid.NewGuid(),
        ItemNumber = i,
        Name = String.Format("Item {0}", i),
        Active = (i % 2 == 0)

DataTable ConvertToDataTable<TSource>(IEnumerable<TSource> source)
  var props = typeof(TSource).GetProperties();

  var dt = new DataTable();
    props.Select(p => new DataColumn(p.Name, p.PropertyType)).ToArray()

    i => dt.Rows.Add(props.Select(p => p.GetValue(i, null)).ToArray())

  return dt;

void Main()
  var dt = ConvertToDataTable(

  // NOTE: The Dump() method below is a LINQPad extension method.
  //       To run this example outside of LINQPad this method
  //       will need to be revised.


Of course there are other ways to accomplish this and the full example has some holes but it’s pretty easy to expand. An obvious enhancement would be to rename the ConvertToDataTable method and change it to handle child collections and return a full DataSet.

LINQPad: An Essential Tool For .NET

A few days ago I was reading through my Twitter feed on my phone when I read a post about a tool called LINQPad.  This was the first time I’d heard of it so I hopped over to the LINQPad web site to see what it was all about.  After a few minutes of browsing I made a mental note to download and try it when I got back to my laptop.  I’m glad I did.

I’ve been using LINQPad for a little less than a week now and it quickly found a place in my toolkit right next to Visual Studio.  In fact, if I have VS open chances are good that LINQPad is also open.  What could a tool named “LINQPad” do to get such good placement in my toolkit?  Isn’t it just for playing with and learning LINQ?  In short, no.  LINQPad is much more than its name implies.


LINQPad does offer great support for LINQ.  Full support for LINQ to Objects, LINQ to XML, and LINQ to SQL is available out of the box.  One of the most powerful features is LINQPad’s ability to connect to a SQL Server database and automatically build classes to represent the tables and columns allowing the database to be queried using LINQ to SQL rather than traditional SQL.  No setup beyond entering the connection information is necessary.  Entity Framework and WCF Data Services are also supported.  But that’s just LINQ!  Didn’t I say it’s more than its name implies?

I find that the real power of LINQPad comes from its ability to execute any C# or VB expression, statement, or program.  This capability has some implications for ad-hoc testing and prototyping.  Instead of littering your development folder(s) with simple single-use console applications just use LINQPad to prove-out a piece of code then copy/paste the code into your project.  You can even add references to existing assemblies to expose the functionality to your ad-hoc code.

I mentioned that LINQPad supports execution of any C# or VB expression, statement, or program but what exactly does that mean?  Depending on the selected language option LINQPad will behave a bit differently.


The expression option allows a single C#/VB expression.  This is useful for testing regular expressions, playing with string formatting options, or anything else that can be expressed with a single line of code.


More often than not our ad-hoc code will need more than one line.  This is where the statement(s) option comes in.  I’ve found this useful for prototyping and solidifying the body of a method and for executing database queries.


The Program option is the most robust of the three.  It allows an entire program complete with a Main() method and classes to be written within LINQPad.  The possibilities here are endless.

Using LINQPad for ad-hoc testing should supplement rather than replace formal unit testing.  Formal unit testing included with the project’s build process is still very important for on-going development.


LINQPad is not an open source project but it is offered free of charge.  An auto-completion add-on is available for a small fee.  The software can be downloaded as either a stand-alone executable or a low-impact installer from the LINQPad web site.  All that’s really needed is .NET 3.5.

Given the power for the price I highly recommend grabbing a copy and at least trying it out.  Consider giving the LINQPad Challenge a try.  What do you have to lose?