If you are already familiar with the standard query operators in LINQ, then you have no doubt tried to do something like this:

The problem is, the “Distinct” function doesn’t let you specify a property by which you call your object ‘unique’. In other words, if you have a collection that has the same customer data in there twice… calling “Distinct” will not remove the duplicate. Note – this statement requires the following VERY IMPORTANT DISCLAIMER:

How Does LINQ Implement the Distinct Function

Before we can solve the problem explained above by extending LINQ, we have to understand why the method currently does not remove duplicates in all cases. To understand that, we have to understand how it is removing duplicates at all. – Hopefully I haven’t lost you yet

To explain in short – the Distinct function will iterate through an IEnumerable<T> one item at a time and perform the following pseudo-code:

1. Get the hash code of the current item using the GetHashCode() method which *all* objects in the .NET Framework have.
2. Check to see if this hash code has been seen before (by checking in a private dictionary hidden from our eyes).
3. If the hash code is not already in the dictionary, then we simply add the current item to the dictionary and return the object.
4. If that very same hash code has already been found, then we compare the current item with the last item with that hash to double check uniqueness and return the object if it is unique.

Here is what that code *could* look like (but it’s not exact):

By the way, did you catch the reason why two objects that represent the same data will not be de-duped? The reason is that step 1 is to use the “GetHashCode()” method to do a quick test. The here problem is that you may not consider these following objects to be different, but LINQ would:

The reason why those two objects would be considered ‘different’ is because different instances of an object will have different hash codes, even if the ‘data’ is the same. Now that we understand enough of what LINQ is doing, we can extend it to meet our simple needs.

Extending IEnumerable<T>

So, how do we add our functionality to everything that LINQ is already extending? – Very easily, extend the only interface that LINQ is extending! As a side note, I recommend putting your code in the “System.Collections.Generic” namespace so that once you add a ‘using’ (or ‘Imports’ in VB) to that namespace, you’ll automatically tap into your extensions too.

// I feel like I’m working for Microsoft when I use this namespace…
// will I get paid for this article? – Probably not
namespace System.Collections.Generic
{
public static class MyIEnumerableExtensions
{
public static IEnumerable<T> Distinct(this IEnumerable<T> source,
Func<T, object> uniqueCheckerMethod)
{
// Extension code here…
}
}
}

So, now that we have our method stub that extends any list (array or collection) by letting the developer specify how he considers an object to be unique, we need to fill in the contents of our function.

Because the “Distinct” LINQ extension has an overload that accepts an ‘IEqualityComparer’ object, we already have half of the code written for us! So, what we are going to do is create a generic ‘comparer’ class and use that to do our filtering. Here is what our extension will look like:

And, our “GenericComparer<T>” class will be as follows:

Now, I can filter the following list like so:

Download the Source File

While the source for this article is very simple (and is spread throughout the article), you may want to download it and test it out for yourself. This project (compiled for Visual Studio 2008 – .NET 3.5 SP1) has a simple Console app that demonstrates the code above. I encourage you to put some break-points in the code to see how it’s all working. Here’s the code: SingingEels_ExtendingLinqDistinct.zip

Source:singingeels.com