前言
首先给出我们需要用到的对象,如下:
public class Person
{
public string Name { get; set; }
public int Age { get; set; }
}
接下来我们添加100万条数据到集合中,如下:
var list = new List<Person>();
for (int i = 0; i < 1000000; i++)
{
list.Add(new Person() { Age = 18, Name = "迷恋自留地" });
}
for (int i = 0; i < 1000; i++)
{
list.Add(new Person() { Age = 19, Name = "迷恋自留地" });
}
第一种分组去重
年龄和名称进行分组,然后取第一条即可达到去重,如下:
var list1 = list.GroupBy(d => new { d.Age, d.Name })
.Select(d => d.FirstOrDefault())
.ToList();
第二种 HashSet去重 (扩展方法)
public static IEnumerable<TSource> Distinct<TSource, TKey>(
this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector)
{
var hashSet = new HashSet<TKey>();
foreach (TSource element in source)
{
if (hashSet.Add(keySelector(element)))
{
yield return element;
}
}
}
述扩展方法即可去重,如下:
var list2 = list.Distinct(d => new { d.Age, d.Name }).ToList();
第三种 IEqualityComparer去重 (扩展方法)
public static class Extensions
{
public static IEnumerable<T> Distinct<T>(
this IEnumerable<T> source, Func<T, T, bool> comparer)
where T : class
=> source.Distinct(new DynamicEqualityComparer<T>(comparer));
private sealed class DynamicEqualityComparer<T> : IEqualityComparer<T>
where T : class
{
private readonly Func<T, T, bool> _func;
public DynamicEqualityComparer(Func<T, T, bool> func)
{
_func = func;
}
public bool Equals(T x, T y) => _func(x, y);
public int GetHashCode(T obj) => 0;
}
}
最终通过指定属性进行比较即可去重,如下:
list = list.Distinct((a, b) => a.Age == b.Age && a.Name == b.Name).ToList();
性能比较
我们来分析其耗时情况,如下:
var list = new List<Person>();
for (int i = 0; i < 1000000; i++)
{
list.Add(new Person() { Age = 18, Name = "jeffcky" });
}
var time1 = Time(() =>
{
list.GroupBy(d => new { d.Age, d.Name })
.Select(d => d.FirstOrDefault())
.ToList();
});
Console.WriteLine($"分组耗时:{time1}");
var time2 = Time(() =>
{
list.Distinct(d => new { d.Age, d.Name }).ToList();
});
Console.WriteLine($"HashSet耗时:{time2}");
var time3 = Time(() =>
{
list.Distinct((a, b) => a.Age == b.Age && a.Name == b.Name).ToList();
});
Console.WriteLine($"委托耗时:{time3}");
static long Time(Action action)
{
var stopwatch = new Stopwatch();
stopwatch.Start();
action();
stopwatch.Stop();
return stopwatch.ElapsedMilliseconds;
}
![](https://oss-emcsprod-public.modb.pro/wechatSpider/modb_20210901_d8baa01c-0ad5-11ec-b043-00163e068ecd.png)