The Language Integrated Queryt (LINQ) technology set, introduced initially in .NET 3.5, provides a concise, symmetrical, and strongly typed manner to access a wide variety of data stores.

Understanding the Role of LINQ

The LINQ API is an attempt to provide a consistent, symmetrical way programmers can obtain and manipulate "data" in the broad sense of the term. Using LINQ, you can create directly within the C# programming language constructs called query expressions. These query expressions are based on numerous query operators that have been intentionally design to look and feel similar to a SQL expression.

  1. LINQ to Objects : This term refers to the act of applying LINQ queries to arrays and collections.
  2. LINQ to XML : This term refers to the act of using LINQ to manipulate and query XML documents.
  3. LINQ to Entities : This aspect of LINQ allows you to make use of LINQ queries within the ADO.NET Entity Framework (EF) Core API.
  4. Parallel LINQ : This allows for parallel processing of data returned from a LINQ query.

The LINQ assemblies are contained in the System.Linq namespace.

It's also important to point out that a LINQ query expression is strongly typed.

Applying LINQ Queries to Primitive Arrays

static void QueryOverString()
 {
     string[] videos = { "Morrowind", "Uncharted 2", "Fallout 3", "Daxter", "System Shock 2" };
     // Build a query expression to find the items in the array that have an embedded space.
     IEnumerable<string> subset = from g in videos where g.Contains(" ") orderby g select g;
     foreach(string s in subset)
     {
         Console.WriteLine(s);
     }
 }

Once Again, Using Extension Methods

The LINQ syntax used earlier is referred to as LINQ query expressions, which is a format that is like SQL but slightly different.

IEnumerable<string> subset = videos.Where(g => g.Contains(" ")).OrderBy(g => g).Select(g => g);

This syntax uses lambda expressions within each method to define the operation.

Reflecting Over a LINQ Result Set

static void ReflectOverQueryResults(object rs)
 {
     Console.WriteLine(rs.GetType().Name);
     Console.WriteLine(rs.GetType().Assembly.GetName().Name);
 }
 
static void QueryOverString()
 {
     string[] videos = { "Morrowind", "Uncharted 2", "Fallout 3", "Daxter", "System Shock 2" };
     IEnumerable<string> subset = from g in videos where g.Contains(" ") orderby g select g;
     ReflectOverQueryResults(subset);
     IEnumerable<string> subset2 = videos.Where(g => g.Contains(" ")).OrderBy(g => g).Select(g => g);
     ReflectOverQueryResults(subset2);
 }

  1. LINQ query result is not a List or Array.
  2. Compiler generates an internal iterator type behind the scenes.
  3. Use var - the real type is compiler-determined.
  4. All LINQ results implement IEnumerable<T>
  5. Call ToList() or ToArray() to get a concrete type.

LINQ and Implicitly Typed Local Variables

int[] numbers = { 10, 20, 30, 1, 2, 3 };
// use implicit typing here....
var subset = from i in numbers where i < 10 select i;

LINQ and Extension Methods

While System.Array does not directly implement the IEnumerable<T> interface, it indirectly gains the required functionality of this type via the static System.Linq.Enumerable class type.

This utility class defines a good number of generic extension methods ( such as Aggregate<T>(), First<T>(),Max<T>()), which System.Array acquires in the background.

The Role of Deferred Execution

When the LINQ query expressions evaluated until you iterate over the resulting sequence. Formally speaking, this is termed deferred execution. The benefit of this approach is that you can apply the same LINQ query multiple times to the same container and rest assured you are obtaining the latest and greatest results.

var subset = from i in numbers where i < 10 select i;
// LINQ statement evaluated here
foreach(var i in subset)
 {
     Console.WriteLine(i);
 }
numbers[0] = 4;
// Evaluated again!
foreach(var j in subset)
 {
     Console.WriteLine(j);
 }

Note: When a LINQ statement is selecting a single element (using First/FirstOrDefault, Single/SingleOrDefault , or any of the aggregation methods) , the sequence is empty.

DefaultIfEmpty() Method

This method returns the elements of the sequence or a default value if the sequence is empty.

// Returns all of the numbers
foreach(var i in numbers.DefaultIfEmpty(-1))
 {
     Console.WriteLine(i);
             
 }
// Returns -1 since the sequence is empty
foreach(var i in (from i in numbers where i > 99 select i).DefaultIfEmpty(-1))
 {
     Console.WriteLine(i);
 }

The Role of Immediate Execution

When you need to evaluate a LINQ expression resulting in a sequence from outside the confines of foreach logic, you can call any number of extension methods defined by that Enumerable type such as:

  1. ToArray<T>()
  2. ToDictionary<TSource,TKey>()
  3. ToList()

These methods will cause a LINQ query to execute at the exact moment you call one of the extension methods to obtain a snapshot of the data. After you have done so, the snapshot of data may be independently manipulated.

  1. First(): Return the first member of the sequence (and should always be used with an OrderBy() or OrderByDescending()).
  2. FirstOrDefault(): Returns the value for the type of item in the list if there are not any to return, such as when the original sequence is empty or the Where() clause filters our all elements.
  3. Single() : Return the first member of sequence (based on the OrderBy()/OrderByDescending(), or element order if there is not an ordering clause).
  4. SingleOrDefault(): Return the default value for the element type if there aren't any items in the sequence.

If there are no records are returned;

-> First() or Single() throw an exception.

-> FirstOrDefault()/SingleOrDefault() return null.

The difference First()/FirstOrDefault()

Single()/SingleOrDefault() -> will throw an exception if more than one element is returned from the query.

int number = (from i in numbers select i).First();
number = (from i in numbers orderby i select i).First();
number = (from i in numbers where i > 30 select i).Single();
// Return null if nothing is returned
number = (from i in numbers where i > 99 select i).FirstOrDefault();
number = (from i in numbers where i > 99 select i).SingleOrDefault();
// Get data as int[]
int[] subsetAsArray = (from i in numbers where i < 10 select i).ToArray<int>();
// Get data as List<int>
List<int> subsetList = (from i in numbers where i < 10 select i).ToList<int>();

Set Default for [First/Last/Single]OrDefault Methods

These methods SingleOrDefault(),LastOrDefault(),FirstOrDefault()) allow specification of the default value when the query doesn't return any elements. The base version of these methods automatically sets the default value (0 for a number, null for class).

int[] number = Array.Empty<int>();
var query = from i in numbers where i > 100 select i;
var n = query.FirstOrDefault(-1);
n = query.SingleOrDefault(-2);
n = query.LastOrDefault(-3);

Applying LINQ Queries to Collection Objects

LINQ query expressions can also manipulate data within members of the System.Collections.Generic namespace.

class Car
 {
     public string PetName { get; set; }
     public string Color { get; set; }
     public string Make { get; set; }
     public int Speed { get; set; }
 }
 
List<Car> myCars = new List<Car>()
 {
     new Car{PetName="Henry",Color="Silver",Speed=100,Make="BMW"},
     new Car{PetName="Daisy",Color="Tan",Speed=90,Make="BMW"},
     new Car{PetName="Mary",Color="Black",Speed=55,Make="VW"},
     new Car{PetName="Clunker",Color="Rush",Speed=5,Make="Yugo"},
     new Car{PetName="Melvin",Color="White",Speed=43,Make="Ford"},
 };

Accessing Contained Subobjects

Applying a LINQ query to a generic container is no different from doing so with a simple array, as LINQ to Object can be used on any type implementing IEnumerable<T>.

static void GetFastCars(List<Car> myCars)
 {
     var fastCars = from c in myCars where c.Speed > 55 select c;
     Console.WriteLine(fastCars.Count());
 }

Applying LINQ Queries to Nongeneric Collections

It's possible to iterate over data contained within nongeneric collections using the generic Enumerable.OfType<T> extension method.

When calling OfType<T>() from a nongeneric collection object (such as ArrayList). Simply specify the typeof item within the container to extract a compatible IEnumerable<T> object. In code, you can store this data point using an implicitly typed variable.

ArrayList myCars = new ArrayList()
 {
     new Car{PetName="Henry",Color="Silver",Speed=100,Make="BMW"},
     new Car{PetName="Daisy",Color="Tan",Speed=90,Make="BMW"},
     new Car{PetName="Mary",Color="Black",Speed=55,Make="VW"},
     new Car{PetName="Clunker",Color="Rush",Speed=5,Make="Yugo"},
     new Car{PetName="Melvin",Color="White",Speed=43,Make="Ford"},
 };
var myCarsEnum = myCars.OfType<Car>();
var fastCars = from c in myCarsEnum where c.Speed > 55 select c;

Investigating the C# LINQ Query Operators

Query Operators

  1. from,in -> Used to define the backbone for any LINQ expression, which allows you to extract a subset of data from a fitting container.
  2. where -> Used to define a restriction for which items to extract from a container.
  3. select -> Used to select a sequence from the container.
  4. join,on,equals,into -> Performs joins based on specified key.
  5. orderby,ascending,descending -> Allows the resulting subset to be ordered in a ascending or descending order.
  6. groupby -> Yields a subset with data grouped by a specified value.

class ProductInfo
 {
     public string Name { get; set; }
     public string Description { get; set; }
     public int NumberOfStock { get; set; }
     public override string ToString()
     {
         return $"{Name} {Description} {NumberOfStock}";
     }
 }
 
ProductInfo[] items = new[]
 {
     new ProductInfo{Name="Mac's coffe",Description="Coffee",NumberOfStock=24},
     new ProductInfo{Name="Milk",Description="Milk",NumberOfStock=100},
     new ProductInfo{Name="Pure Silk",Description="Blend",NumberOfStock=120},
     new ProductInfo{Name="Crunchy",Description="Chezzy",NumberOfStock=2},
     new ProductInfo{Name="Rip088",Description="From",NumberOfStock=100},
     new ProductInfo{Name="Classic",Description="Everyone",NumberOfStock=73},
 };

Basic Selection Syntax

Syntax:

var result = from matchingItem in container select matchingItem;
 
// Get Everything
var allProducts = from p in items select p;
// List only names
var names = from p in items select p.Name;

Obtaining Subsets of Data

To obtain a specific subset from a container, you can use the "where" operator.

var result = from item in container Where Expression select item;

Notice that the "where" operator expects an expression that resolves to a Boolean.

// Get only the items where we have more than 25 in stock.
var overstock = from p in items where p.NumberOfStock > 25 select p;
// Get BMWs going at least 100 MPH.
var onlyFastBMWs = from c in myCars where c.Make == "BMW" && c.Speed >= 100 select c;

Paging Data

If you need to get a certain number of records, you can use Take()/TakeWhile()/TakeLast() and Skip()/SkipWhile()/SkipLast() methods. The first set of methods returns the specified number of records (Take()), all of the records where the condition is true (TakeWhile()), or the last specified number of records (TakeLast()).

The second set of methods passes over the specified number of records(Skip()), skips all of the records where the condition is true (SkipWhile()), or skips the last specified number of records (SkipLast()).

IEnumerable<ProductInfo> list = (from p in products select p).Take(3); // The first 3
list = (from p in products select p).TakeWhile(x => x.NumberOfStock > 20);
list = (from p in products select p).TakeLast(2); // The last 2
list = (from p in products select p).Skip(3); // Skipping the first 3
list = (from p in products select p).SkipWhile(x => x.NumberOfStock > 20);
list = (from p in products select p).SkipLast(2); // All but the last 2
These methods can be combined to truly "page" the data.
list = (from p in products select p).Skip(3).Take(2);

Paging Data with Ranges 

IEnumerable<ProductInfo> list = (from p in products select p).Take(..3); // The first 3
list = (from p in products select p).Take(3..); // Skipping the first 3
list = (from p in products select p).Take(^2..); // The last 2
list = (from p in products select p).Take(3..5); // Skip 3 take 2
list = (from p in products select p).Take(..^2); // Skip the last 2

Paging Data with Chunks

Chunk() method takes one parameter (size) and then splits the source into an Enumerable of Enumerables. For ex, if we take the ProductInfo list and apply Chunk(2), the return value is a list of three lists, each inner list containing 2 records.

IEnumerable<ProductInfo[]> chunks = products.Chunk(size: 2);

Paging New Data Types

It's also possible to project new forms of data from an existing data source.

var nameDesc = from p in products select new { p.Name, p.Description };

When you need to return projected data to a caller, one approach is to transform the query result into a System.Array object using the ToArray() extension method.

static Array GetProjectedSubset(ProductInfo[] products)
 {
     var nameDesc = from p in products select new { p.Name, p.Description };
     return nameDesc.ToArray();
 }

Projecting To Different Data Types

In addition to projecting into anonymous types, you can project the results of your LINQ query into another concrete type.

class ProductInfoSmall
 {
     public string Name { get; set; }
     public string Description { get; set; }
     public override string ToString()
     {
         return $"{Name} {Description}";
     }
 }
 
IEnumerable<ProductInfoSmall> nameDesc = from p in products select new ProductInfoSmall { Name = p.Name, Description = p.Description };

Obtaining Counts Using Enumerable 

string[] videos = { "Morrowind", "Uncharted 2", "Fallout 3", "Daxter", "System Shock 3" };
int numb = (from g in videos where g.Length > 6 select g).Count();

Obtaining Nonenumerated Counts

The TryGetNonEnumeratedCount() method attempts to get the totcal count of an IEnumerable without actually enumerating the list. If the list must be enumerated over to get the count, the method fails.

IEnumerable<ProductInfo> query = from p in products select p;
var result = query.TryGetNonEnumeratedCount(out int count);
if (result)
 {
     Console.WriteLine(count);
 }

Reversing Result Sets

You can reverse the items within a result set quite simply using the Reverse() method of the Enumerable class.

IEnumerable<ProductInfo> allProducts = from p in products select p;
foreach(var prod in allProducts.Reverse())
 {
     Console.WriteLine(prod.ToString());
 }

Sorting Expressions

By default, the order will be ascending.

IEnumerable<ProductInfo> allProducts = from p in products select p;
var subset = from p in products orderby p.Name select p;

Although ascending order is the default;

IEnumerable<ProductInfo> allProducts = from p in products select p;
var subset = from p in products orderby p.Name ascending select p;
var subset = from p in products orderby p.Name descending select p;

LINQ As a Better Venn Diagramming Tool

The Enumerable class supports a set of extension methods that allows you to use 2 (or more) LINQ queries as the basis to find unions, differences, concatenations, and interactions of data. First, consider the Except() extension method, which will return a LINQ reuslt set that contains the difference between 2 containsers,which,in this case, is the value Yugo.

List<string> myCars = new List<string>{ "Yugo", "Aztec", "BMW" };
List<string> yourCars = new List<string> { "BMW", "Saab","Aztec"  };
var carDiff = (from c in myCars select c).Except(from c2 in yourCars select c2);
var carIntersect = (from c in myCars select c).Intersect(from c2 in yourCars select c2);
var carUnion = (from c in myCars select c).Union(from c2 in yourCars select c2);
var carConcat = (from c in myCars select c).Concat(from c2 in yourCars select c2);

Venn Diagramming with Selectors

The ExceptBy() extension method uses the selector to remove the records from the first set where the value of the selector exists in the second set.

var first = new (string name, int age)[] { ("Francis", 20), ("Lindset", 30), ("Ashley", 30) };
var second = new (string name, int age)[] { ("Claire",30), ("Path",30), ("Drew",33) };
var result = first.ExceptBy(second.Select(x => x.age), person => person.age);
foreach(var item in result)
 {
     Console.WriteLine(item); // {("Francis",20),("Ashley",40)}
 }

The IntersectBy() method will return a result set that contains the common data items in a set of containers based on the selector.

result = first.IntersectBy(second.Select(x => x.age), person => person.age); // {("Lindset",30)}

The UnionBy() method returns a result set that includes all the values for the selector, and the first member of the combined lists that have a value that matches the list.

result = first.UnionBy(second, person => person.age); /* 
(Francis, 20)
(Lindset, 30)
(Drew, 33)
*/

Removing Duplicates

When you call the Concat() extension method, you could very well end up redundant entries in the fetched result,

you might want to remove duplicate entries in your data by using Distinct() method.

var carContact = (from c in myCars select c).Concat(from c2 in yourCars select c2);
foreach(string s in carContact.Distinct())
 {
     Console.WriteLine(s);
 }

Removing Duplicates with Selectors

DistinctBy() method.

result = first.Concat(second).DistinctBy(x => x.age);

LINQ Aggregation Operations

LINQ queries can also be designed to perform various aggregation operations on the result set. The count() extension method is one such aggregation example. Other possibilities include obtaining an average , maximum , minimum , or sum of values using the:

  1. Max(),Min(),Average(), and Sum()

double[] winterTemes = { 2.0, -21.3, 8, -4.0 };
Console.WriteLine((from t in winterTemes select t).Max());
Console.WriteLine((from t in winterTemes select t).Min());
Console.WriteLine((from t in winterTemes select t).Sum());
Console.WriteLine((from t in winterTemes select t).Average());

Aggregation with Selectors

Console.WriteLine(products.MaxBy(x => x.NumberOfStock));
Console.WriteLine(products.MinBy(x => x.NumberOfStock));

The Internal Representation of LINQ Query Statements

LINQ query syntax is syntactic sugar. Compiler converts it to extension method calls. Both produce identical CIL code - zero performance difference.

// WAY 1 - QUERY OPERATORS
// What you normally write - compiler converts this to way 2
var subset1 = from game in videos where game.Contains(" ") orderby game select game;
 
// WAY 2 - EXTENSION METHODS
// What compiler actually generates from way 1
var subset2 = videos.Where(g => g.Contains(" ")).OrderBy(x => x).Select(x => x);
// Same thing broken into steps:
var gamesWithSpaces = videos.Where(g => g.Contains(" "));
var orderedGames = gamesWithSpaces.OrderBy(g => g);
var subset2b = orderedGames.Select(g => g);
 
// WAY 3 : EXTENSION METHODS + ANONYMOUS METHODS
Func<string, bool> searchFilter = delegate (string name) { return videos.Contains(" "); };
Func<string, string> itemToProcess = delegate (string s) { return s; };
var subset3 = videos.Where(searchFilter).OrderBy(itemToProcess).Select(itemToProcess);
 
// WAY 4 : EXTENSION METHODS + RAW DELEGATES (most verbose)
Func<string, bool> rawFilter = new Func<string, bool>(Filter);
Func<string, string> rawProcess = new Func<string, string>(ProcessItem);
var subset4 = videos.Where(rawFilter).OrderBy(rawProcess).Select(rawProcess);
 
 
private static string ProcessItem(string arg)
 {
     return arg;
 }
private static bool Filter(string arg)
 {
     return arg.Contains(" ");
 }