Tag Archives: Pro-csharp10-with-net6

Pro C#10 CHAPTER 23 Build a Data Access Layer with Entity Framework Core

CHAPTER 23

Build a Data Access Layer with Entity Framework Core

The previous chapters covered the components and capabilities of EF Core. This chapter is focused on applying what you learned about EF Core to build the AutoLot data access layer. This chapter begins with creating one project for the entities and another for the data access library code. Separation of models from the data access code is a common design decision and will be taken advantage of in the ASP.NET Core chapters.
The next step is to scaffold the existing database from Chapter 20 into entities and a derived DbContext using the EF Core command-line interface (CLI). This demonstrates the database first process. Then the project is changed to code first, where the database design is driven by the C# entities.
The entities from Chapter 20 are updated to their final version, the new entities from Chapters 21 and 22 are added into the model, and the database is updated using EF Core migrations. Then the stored procedure, database view, and user-defined functions are integrated into the EF Core migration system, providing a singular mechanism for developers to get a complete copy of the database. The final EF Core migration completes the database.
The next step is to create repositories that provide encapsulated create, read, update, and delete (CRUD) access to the database. The final step in this chapter is to add data initialization code to provide sample data, a common practice used for testing the data access layer.

Create the AutoLot.Dal and AutoLot.Models Projects
The AutoLot data access layer consists of two projects, one to hold the EF Core–specific code (the derived DbContext, context factory, repositories, migrations, etc.) and another one to hold the entities and view models. Create a new solution named Chapter23_AllProjects, add a .NET Core class library named AutoLot. Models into the solution, and add the Microsoft.EntityFrameworkCore, Microsoft.EntityFrameworkCore. SqlServer, and System.Text.Json NuGet packages to the project.
The Microsoft.EntityFrameworkCore.Abstractions package provides access to many EF Core constructs (like data annotations), is lighter weight than the Microsoft.EntityFrameworkCore package, and would normally be used for model projects. However, support for the new IEntityTypeConfiguration feature is not in the Abstractions package but the full EF Core package.
Add another .NET Core class library project named AutoLot.Dal to the solution. Add a reference to the AutoLot.Models project, and add the Microsoft.EntityFrameworkCore, Microsoft.EntityFrameworkCore. SqlServer, and Microsoft.EntityFrameworkCore.Design NuGet packages to the project.
As a refresher, the Microsoft.EntityFrameworkCore package provides the common functionality for EF Core. The Microsoft.EntityFrameworkCore.SqlServer package supplies the SQL Server data provider, and the Microsoft.EntityFrameworkCore.Design package is required for the EF Core command-line tools.

© Andrew Troelsen, Phil Japikse 2022
A. Troelsen and P. Japikse, Pro C# 10 with .NET 6, https://doi.org/10.1007/978-1-4842-7869-7_23

991

To complete all these steps using the command line, use the following (in the directory where you want the solution to be created):

dotnet new sln -n Chapter23_AllProjects

dotnet new classlib -lang c# -n AutoLot.Models -o .\AutoLot.Models -f net6.0 dotnet sln .\Chapter23_AllProjects.sln add .\AutoLot.Models
dotnet add AutoLot.Models package Microsoft.EntityFrameworkCore
dotnet add AutoLot.Models package Microsoft.EntityFrameworkCore.SqlServer dotnet add AutoLot.Models package System.Text.Json

dotnet new classlib -lang c# -n AutoLot.Dal -o .\AutoLot.Dal -f net6.0 dotnet sln .\Chapter23_AllProjects.sln add .\AutoLot.Dal
dotnet add AutoLot.Dal package Microsoft.EntityFrameworkCore
dotnet add AutoLot.Dal package Microsoft.EntityFrameworkCore.Design dotnet add AutoLot.Dal package Microsoft.EntityFrameworkCore.SqlServer

dotnet add AutoLot.Dal reference AutoLot.Models

■Note If you are not using a Windows-based machine, adjust the directory separator character for your operating system in the previous commands. This adjustment will need to be done for all the CLI commands in this chapter.

The final step in creating the projects is to turn C#’s nullable reference types off. While EF Core supports this feature, we won’t be using it in this solution. Update the project files for both projects to the following (change is in bold):


net6.0
enable
disable

Finally, update AutoLot.Dal project file to enable access to the design-time model at runtime. Update the metadata for the Microsoft.EntityFrameworkCore.Design package to the following. This change is necessary for clearing out temporal tables, covered in the “Data Initialization” section:



all

Add the Database View
Before scaffolding the entities and derived DbContext from database, add a custom database view to the AutoLot database, which will be used later in this chapter. We are adding it now to demonstrate scaffolding support for views. Connect to the AutoLot database (using either SQL Server Management Studio or Azure Data Studio) and execute the following SQL statement:

CREATE VIEW [dbo].[CustomerOrderView] AS
SELECT dbo.Customers.FirstName, dbo.Customers.LastName, dbo.Inventory.Color, dbo.Inventory.PetName, dbo.Makes.Name AS Make
FROM dbo.Orders
INNER JOIN dbo.Customers ON dbo.Orders.CustomerId=dbo.Customers.Id INNER JOIN dbo.Inventory ON dbo.Orders.CarId=dbo.Inventory.Id INNER JOIN dbo.Makes ON dbo.Makes.Id=dbo.Inventory.MakeId;

■Note In the repo’s folder for Chapter 20 are database backups for Windows and docker. If you need to restore the database, refer to the instructions in Chapter 20.

Scaffold the DbContext and Entities
The next step is to scaffold the AutoLot database using the EF Core CLI tools. Navigate to the AutoLot.Dal project directory in either a command prompt or Visual Studio’s Package Manager Console. Use the EF Core CLI tools to scaffold the AutoLot database into the entities and the DbContext-derived class with the following command, updating the connection string as necessary (all on one line):

dotnet ef dbcontext scaffold "server=.,5433;Database=AutoLot;User Id=sa;Password=P@ssw0rd;" Microsoft.EntityFrameworkCore.SqlServer --data-annotations --context ApplicationDb
Context --context-namespace AutoLot.Dal.EfStructures --context-dir EfStructures --no- onconfiguring --namespace AutoLot.Models.Entities --output-dir ..\AutoLot.Models\ Entities --force

The previous command scaffolds the database located at the provided connection string (the example command uses the connection string for the Docker container used in Chapter 20) using the SQL Server database provider. The –data-annotations flag is to prioritize data annotations where possible (over the Fluent API). The --context names the context, --context-namespaces specifies the namespace for
the context, --context-dir indicates the directory (relative to the current project) for the scaffolded context, --no-onconfiguring prevents the OnConfiguring method from being scaffolded, --output-dir is the output directory for the entities (relative to the project directory), and -n specifies the namespace for the entities. This command places all the entities in the AutoLot.Models project in the Entities folder and places the ApplicationDbContext.cs class in the EfStructures folder of the AutoLot.Dal project. The final option (--force) is used to force overwriting any existing files.

■Note The eF Core CLI commands were covered in detail in Chapter 21.

Examine the Results
After running the command to scaffold the database into C# classes, you will see six entities in the AutoLot. Models project (in the Entities folder) and one derived DbContext in the AutoLot.Dal project (in the EfStructures folder). Each table is scaffolded into a C# entity class and added as a DbSet property on the derived DbContext. Views are scaffolded into keyless entities, added as a DbSet, and mapped to the proper database view using the Fluent API.

The scaffolding command that we used specified the --data-annotations flag to prefer annotations over the Fluent API. As you examine the scaffolded classes, you will notice there are a few misses with the annotations. For example, the TimeStamp properties do not have the [Timestamp] attribute but are instead configured as RowVersion ConcurrencyTokens in the Fluent API.

■Note In my opinion, having the annotations in the class makes the code more readable than having all the configuration in the Fluent apI. If you prefer to use the Fluent apI, remove the –data-annotations option from the command.

Switch to Code First
Now that you have the database scaffolded into a derived DbContext and entities, it is time to switch from database first to code first. The process is not complicated, but also not something that should be done on a regular basis. It’s better to decide on a paradigm and stick with it. Most agile teams prefer code first, as the
emerging design of the application and its entities flows into the database. The process we are following here simulates starting a new project using EF Core targeting an existing database.
The steps involved in switching from database first to code first involves creating a DbContext factory (for the CLI tooling), creating an initial migration for the current state of the object graph, and then dropping the database and re-creating the database with either the migration or “fake” applied by tricking EF Core.

Create the DbContext Design-Time Factory
As you recall from the previous EF Core chapters, IDesignTimeDbContextFactory is used by the EF Core CLI tooling to create an instance of the derived DbContext class. Create a new class file named ApplicationDbContextFactory.cs in the AutoLot.Dal project in the EfStructures directory.
The details of the factory were covered in the previous chapter, so I’m just going to list the code here.
Make sure to update your connection string to match your environment.

namespace AutoLot.Dal.EfStructures;

public class ApplicationDbContextFactory : IDesignTimeDbContextFactory
{
public ApplicationDbContext CreateDbContext(string[] args)
{
var optionsBuilder = new DbContextOptionsBuilder();
var connectionString = @"server=.,5433;Database=AutoLot;User Id=sa;Password=P@ssw0rd;"; optionsBuilder.UseSqlServer(connectionString);
Console.WriteLine(connectionString);
return new ApplicationDbContext(optionsBuilder.Options);
}
}

Create the Initial Migration
Recall that the first migration will create three files: the two files for the migration itself and the complete model snapshot. Enter the following in a command prompt (in the AutoLot.Dal directory) to create a new

migration named Initial (using the ApplicationDbContext instance that was just scaffolded) and place the migration files in the EfStructures\Migrations folder of the AutoLot.Dal project:

dotnet ef migrations add Initial -o EfStructures\Migrations -c AutoLot.Dal.EfStructures. ApplicationDbContext

■Note It is important to make sure no changes are applied to the scaffolded files or the database until this first migration is created and applied. Changes on either side will cause the code and database to become out of sync. once applied, all changes to the database need to be completed through eF Core migrations.

To confirm that the migration was created and is waiting to be applied, execute the list command.

dotnet ef migrations list -c AutoLot.Dal.EfStructures.ApplicationDbContext

The result will show the Initial migration pending (your timestamp will be different). The connection string is shown in the output due to Console.Writeline() in the CreateDbContext() method.

Build started...
Build succeeded.
server=.,5433;Database=AutoLot;User Id=sa;Password=P@ssw0rd; 20210703194100_Initial (Pending)

Applying the Migration
The easiest method of applying the migration to the database is to drop the database and re-create it. If that is an option, you can enter the following commands and move on to the next section:

dotnet ef database drop -f
dotnet ef database update Initial -c AutoLot.Dal.EfStructures.ApplicationDbContext

If dropping and re-creating the database is not an option (e.g., it is an Azure SQL database), then EF Core needs to believe that the migration has been applied. Fortunately, this is straightforward with EF Core doing most of the work. Start by creating a SQL script from the migration by using the following command:

dotnet ef migrations script --idempotent -o FirstMigration.sql

The relevant portions of this script are the parts that create the EFMigrationsHistory table and then add the migration record into the table to indicate that it was applied. Copy those pieces to a new query in either Azure Data Studio or SQL Server Manager Studio. Here is the SQL code that you need (your timestamp will be different):

IF OBJECT_ID(N'[ EFMigrationsHistory]') IS NULL BEGIN
CREATE TABLE [ EFMigrationsHistory] ( [MigrationId] nvarchar(150) NOT NULL, [ProductVersion] nvarchar(32) NOT NULL,
CONSTRAINT [PK EFMigrationsHistory] PRIMARY KEY ([MigrationId])
);

END;
GO
...
IF NOT EXISTS(SELECT * FROM [ EFMigrationsHistory] WHERE [MigrationId] = N'20210703194100_ Initial')
BEGIN
INSERT INTO [ EFMigrationsHistory] ([MigrationId], [ProductVersion]) VALUES (N'20210703194100_Initial', N'6.0.0');
END;
GO

Now if you run the list command, it will no longer show the Initial migration as pending. With the initial migration applied, the project and database are in sync, and the development can continue code first. Before continuing with the database development, the project’s custom exceptions need to be created.

Create the GlobalUsings Files
To tidy up the code in the projects, we are going to take advantage of the new C# 10 feature for global using statements. Rename the Class1.cs files in the AutoLot.Dal and AutoLot.Models projects to GlobalUsings. cs. Clear out all the code in each of the files and replace them as follows:

//AutoLot.Dal
//GlobalUsings.cs
global using AutoLot.Dal.EfStructures; global using AutoLot.Dal.Exceptions;

global using AutoLot.Models.Entities;
global using AutoLot.Models.Entities.Configuration; global using AutoLot.Models.Entities.Base;

global using System.Data;
global using System.Linq.Expressions;

global using Microsoft.Data.SqlClient; global using Microsoft.EntityFrameworkCore;
global using Microsoft.EntityFrameworkCore.ChangeTracking; global using Microsoft.EntityFrameworkCore.Design;
global using Microsoft.EntityFrameworkCore.Metadata; global using Microsoft.EntityFrameworkCore.Migrations; global using Microsoft.EntityFrameworkCore.Query; global using Microsoft.EntityFrameworkCore.Storage; global using Microsoft.Extensions.DependencyInjection;

//AutoLot.Models
//GlobalUsings.cs
global using AutoLot.Models.Entities.Base; global using AutoLot.Models.Entities.Owned;
global using AutoLot.Models.Entities.Configuration;

global using Microsoft.EntityFrameworkCore;
global using Microsoft.EntityFrameworkCore.Metadata.Builders;

global using System.ComponentModel;
global using System.ComponentModel.DataAnnotations;
global using System.ComponentModel.DataAnnotations.Schema; global using System.Globalization;
global using System.Xml.Linq;

■Note adding all the namespaces at once will prevent the code from compiling, as all of the namespaces listed here do not yet exist. normally, you would add to this file as you develop your project. We are adding them most of them here in one shot to save space in an already long chapter.

Create Custom Exceptions
A common pattern in exception handling is to catch system exceptions (and/or EF Core exceptions, as in this example), log the exception, and then throw a custom exception. If a custom exception is caught in an upstream method, the developer knows the exception has already been logged and just needs to react to the exception appropriately in the current code block.
Create a new directory named Exceptions in the AutoLot.Dal project. In that directory, create four new class files: CustomException.cs, CustomConcurrencyException.cs, CustomDbUpdateException.cs, and CustomRetryLimitExceededException.cs. All four files are shown in the following listing:

//CustomException.cs
namespace AutoLot.Dal.Exceptions;

public class CustomException : Exception
{
public CustomException() {}
public CustomException(string message) : base(message) { } public CustomException(string message, Exception innerException)
: base(message, innerException) { }
}

//CustomConcurrencyException.cs namespace AutoLot.Dal.Exceptions;

public class CustomConcurrencyException : CustomException
{
public CustomConcurrencyException() { }
public CustomConcurrencyException(string message) : base(message) { } public CustomConcurrencyException(
string message, DbUpdateConcurrencyException innerException)
: base(message, innerException) { }
}

//CustomDbUpdateException.cs namespace AutoLot.Dal.Exceptions;

public class CustomDbUpdateException : CustomException

{
public CustomDbUpdateException() { }
public CustomDbUpdateException(string message) : base(message) { } public CustomDbUpdateException(
string message, DbUpdateException innerException)
: base(message, innerException) { }
}

//CustomRetryLimitExceededException.cs namespace AutoLot.Dal.Exceptions;

public class CustomRetryLimitExceededException : CustomException
{
public CustomRetryLimitExceededException() { }
public CustomRetryLimitExceededException(string message)
: base(message) { }
public CustomRetryLimitExceededException(
string message, RetryLimitExceededException innerException)
: base(message, innerException) { }
}

■Note Custom exception handling was covered in detail in Chapter 7.

Finalize the Entities and ViewModel
This section updates scaffolded entities to their final version, adds the additional entities from the previous two chapters, and adds a logging entity.

■Note your projects will not compile until this section is complete.

The Entities
In the Entities directory of the AutoLot.Models project, you will find six files, one for each table in the database and one for the database view. Note that the names are singular and not plural (as they are in the database). This is a change starting with EF Core 5 where the pluralizer is on by default when scaffolding entities from the database. The pluralizer, as the name describes, maps singular entity names to plural table names, and vice versa.
The previous chapters covered the EF Core conventions, data annotations, and the Fluent API in depth, so most of this section will be code listings with brief descriptions.

The BaseEntity Class
The BaseEntity class will hold the Id and TimeStamp columns that are on every entity. Create a new directory named Base in the Entities directory of the AutoLot.Models project. In this directory, create a new file named BaseEntity.cs and update the code to match the following:

namespace AutoLot.Models.Entities.Base;

public abstract class BaseEntity
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)] public int Id { get; set; }
[Timestamp]
public byte[] TimeStamp { get; set; }
}

All of the entities (except for the logging entity) will be updated to use this base class throughout the next sections.

The Owned Person Entity
The Customer, CreditRisk, and Driver entities all have FirstName and LastName properties. Entities that have the same properties in each can benefit from owned classes. While moving these two properties to an owned class is a somewhat trivial example, owned entities help to reduce code duplication and increase consistency.
Create a new directory named Owned in the Entities directory of the AutoLot.Models project. In this new directory, create a new file named Person.cs. Update the code to match the following:
namespace AutoLot.Models.Entities.Owned; [Owned]
public class Person
{
[Required, StringLength(50)]
public string FirstName { get; set; }

[Required, StringLength(50)]
public string LastName { get; set; }

[DatabaseGenerated(DatabaseGeneratedOption.Computed)] public string FullName { get; set; }
}

Notice the additional computed column that combined the names into a FullName property. Owned classes are configured as part of the owning classes, so the configuration for the column name mapping and computed column takes place as part of the Customer, CreditRisk, and Driver configuration.

The Car (Inventory) Entity
The Inventory table was scaffolded to an entity class named Inventory. We are going to change the entity name to Car while leaving the table name alone. This is an example of mapping an entity to a table with a different name. This is easy to fix: change the name of the file to Car.cs and the name of the class to Car. The Table attribute is already applied correctly, so just add the dbo schema. Note that the schema parameter is optional because SQL Server defaults to dbo, but I include it for completeness. The namespaces can also be deleted, as they are covered with the global namespaces.

[Table("Inventory", Schema = "dbo")] [Index(nameof(MakeId), Name = "IX_Inventory_MakeId")] public partial class Car
{
...
}

Next, inherit from BaseEntity, and remove the Id (and its attribute) and TimeStamp properties as well as the constructor. This is the code for the class after these changes:

namespace AutoLot.Models.Entities;

[Table("Inventory", Schema = "dbo")] [Index(nameof(MakeId), Name = "IX_Inventory_MakeId")] public partial class Car : BaseEntity
{
public int MakeId { get; set; } [Required]
[StringLength(50)]
public string Color { get; set; } [Required]
[StringLength(50)]
public string PetName { get; set; } [ForeignKey(nameof(MakeId))] [InverseProperty("Inventories")] public virtual Make Make { get; set; } [InverseProperty(nameof(Order.Car))]
public virtual ICollection Orders { get; set; }
}

Add the DisplayName attribute to the PetName property, add the Display property with the DatabaseGenerated attribute to hold the computed value from SQL Server, and add the Price and DateBuilt properties. Update the code to the following (changes in bold):

[Required] [StringLength(50)] [DisplayName("Pet Name")]
public string PetName { get; set; }

[DatabaseGenerated(DatabaseGeneratedOption.Computed)] public string Display { get; set; }

public string Price { get; set; } public DateTime? DateBuilt { get; set;}

■Note The DisplayName attribute is used by asp.neT Core and will be covered in that section.

The Make navigation property needs to be renamed to MakeNavigation, and the inverse property is using a magic string instead of the C# nameof() method. Here is the updated property:

[ForeignKey(nameof(MakeId))] [InverseProperty(nameof(Make.Cars))]
public virtual Make MakeNavigation { get; set; }

■Note This is a prime example of why naming the property the same as the class name becomes problematic. If the property name was left as Make, then the nameof function wouldn’t work properly since Make (in this instance) is referring to the property and not the type.

The scaffolded code for the Orders navigation property does use the nameof() method in the inverse property but needs an update since all reference navigation properties will have the suffix Navigation added to their names. The final change for that navigation property is to have the type of the property typed as IEnumerable instead of ICollection and initialized with a new List. This is not
a required change, as ICollection will also work. I prefer to use the lower-level IEnumerable on collection navigation properties (since IQueryable and ICollection both derive from IEnumerable). Update the code to match the following:

[InverseProperty(nameof(Order.CarNavigation))]
public virtual IEnumerable Orders { get; set; } = new List();

Add the collection navigation property for the Driver and CarDriver entities and the reference navigation property for the Radio entity:
[InverseProperty(nameof(Driver.Cars))]
public virtual IEnumerable Drivers { get; set; } = new List();

[InverseProperty(nameof(CarDriver.CarNavigation))]
public virtual IEnumerable CarDrivers { get; set; } = new List();

[InverseProperty(nameof(Radio.CarNavigation))] public virtual Radio RadioNavigation { get; set; }

Next, add a NotMapped property that will display the Make value of the Car. If the related Make information was retrieved from the database with the Car record, the Make Name will be displayed. If the related data was not retrieved, the property displays “Unknown.” As a reminder, NotMapped properties are not part of the database and exist only on the entity. Add the following:

[NotMapped]
public string MakeName => MakeNavigation?.Name ?? "Unknown";

Add an override for the ToString() method to display vehicle information.

public override string ToString()
{
// Since the PetName column could be empty, supply
// the default name of No Name.
return $"{PetName ?? "No Name"} is a {Color} {MakeNavigation?.Name} with ID {Id}.";
}

Add the Required and DisplayName attributes to the MakeId. Even though the MakeId property is considered by EF Core to be required since it is non-nullable, I always add it for readability and UI framework support. Update the code to match the following:

[Required] [DisplayName("Make")]
public int MakeId { get; set; }

The next change is to add the non-nullable bool IsDrivable property with a nullable backing field and a display name.

private bool? _isDrivable;

[DisplayName("Is Drivable")] public bool IsDrivable
{
get => _isDrivable ?? true; set => _isDrivable = value;
}

The final step for the Car class is to add the EntityTypeConfiguration attribute:

[Table("Inventory", Schema = "dbo")] [Index(nameof(MakeId), Name = "IX_Inventory_MakeId")] [EntityTypeConfiguration(typeof(CarConfiguration))] public class Car : BaseEntity
{
...
}

That completes the updates to the Car entity class and is listed here in its entirety:

namespace AutoLot.Models.Entities;

[Table("Inventory", Schema = "dbo")] [Index(nameof(MakeId), Name = "IX_Inventory_MakeId")] [EntityTypeConfiguration(typeof(CarConfiguration))] public partial class Car : BaseEntity
{
[Required] [StringLength(50)]
public string Color { get; set; } public string Price { get; set; }

private bool? _isDrivable; [DisplayName("Is Drivable")] public bool IsDrivable
{
get => _isDrivable ?? true; set => _isDrivable = value;
}

public DateTime? DateBuilt { get; set; }

[DatabaseGenerated(DatabaseGeneratedOption.Computed)] public string Display { get; set; }

[Required] [StringLength(50)] [DisplayName("Pet Name")]
public string PetName { get; set; }

[Required] [DisplayName("Make")]
public int MakeId { get; set; } [ForeignKey(nameof(MakeId))] [InverseProperty(nameof(Make.Cars))]
public virtual Make MakeNavigation { get; set; }

[InverseProperty(nameof(Radio.CarNavigation))] public virtual Radio RadioNavigation { get; set; }

[InverseProperty(nameof(Driver.Cars))]
public virtual IEnumerable Drivers { get; set; } = new List();

[InverseProperty(nameof(CarDriver.CarNavigation))]
public virtual IEnumerable CarDrivers { get; set; } = new List();

[InverseProperty(nameof(Order.CarNavigation))]
public virtual IEnumerable Orders { get; set; } = new List();

[NotMapped]
public string MakeName => MakeNavigation?.Name ?? "Unknown"; public override string ToString()
{
// Since the PetName column could be empty, supply
// the default name of No Name.
return $"{PetName ?? "No Name"} is a {Color} {MakeNavigation?.Name} with ID {Id}.";
}
}

Update the ApplicationDbContext Class
Since the Inventory class was renamed to Car, the ApplicationDbContext class must be updated. Locate the DbSet property and update the line to the following:

public virtual DbSet Cars { get; set; }

The CarConfiguration Class
Just as we did in Chapter 21, we will use IEntityTypeConfiguration to hold the Fluent API code. This keeps the configuration for each entity in its own class, significantly decreasing the size of the ApplicationDbContext OnModelCreating() method. Start by creating a new directory named Configuration under the Entities directory. In this new directory, add a new file named
CarConfiguration.cs, make it public, and implement the IEntityTypeConfiguration interface, like this:

namespace AutoLot.Models.Entities.Configuration;

public class CarConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
}
}

Next, move the contents of the configuration for the Car entity (note that it will still be named Inventory) from the OnModelCreating() method in the ApplicationDbContext into the Configure() method of the CarConfiguration class. The scaffolded Fluent API code for each entity is wrapped in a construct similar to the following:

modelBuilder.Entity(entity =>
{
//Fluent API code here
});

Each IEntityTypeConfiguration is strongly typed to an entity, so the outer code for each entity is not needed, just the scaffolded inner code. Move the entire block, and then delete the entity specifier. Replace the entity variable in each of the inner code blocks with the builder variable, and then add the additional Fluent API code so the Configure() method looks like this:

public void Configure(EntityTypeBuilder builder)
{
builder.HasQueryFilter(c => c.IsDrivable);

builder.Property(p => p.IsDrivable)
.HasField("_isDrivable")
.HasDefaultValue(true);
builder.Property(e => e.DateBuilt).HasDefaultValueSql("getdate()"); builder.Property(e => e.Display)
.HasComputedColumnSql("[PetName] + ' (' + [Color] + ')'", stored: true);

CultureInfo provider = new CultureInfo("en-us");
NumberStyles style = NumberStyles.Number | NumberStyles.AllowCurrencySymbol; builder.Property(p => p.Price)
.HasConversion(
v => decimal.Parse(v, style, provider), v => v.ToString("C2"));

builder.HasOne(d => d.MakeNavigation)
.WithMany(p => p.Cars)
.HasForeignKey(d => d.MakeId)
.OnDelete(DeleteBehavior.ClientSetNull)
.HasConstraintName("FK_Inventory_Makes_MakeId");

builder
.HasMany(p => p.Drivers)
.WithMany(p => p.Cars)
.UsingEntity( j => j
.HasOne(cd => cd.DriverNavigation)
.WithMany(d => d.CarDrivers)
.HasForeignKey(nameof(CarDriver.DriverId))
.HasConstraintName("FK_InventoryDriver_Drivers_DriverId")
.OnDelete(DeleteBehavior.Cascade), j => j
.HasOne(cd => cd.CarNavigation)
.WithMany(c => c.CarDrivers)
.HasForeignKey(nameof(CarDriver.CarId))
.HasConstraintName("FK_InventoryDriver_Inventory_InventoryId")
.OnDelete(DeleteBehavior.ClientCascade), j =>
{
j.HasKey(cd => new { cd.CarId, cd.DriverId });
});
}

The Inventory table will be configured as a temporal table, so add the following to the
Configure() method:

public void Configure(EntityTypeBuilder builder)
{
//omitted for brevity
builder.ToTable( b => b.IsTemporal(t =>
{
t.HasPeriodEnd("ValidTo"); t.HasPeriodStart("ValidFrom"); t.UseHistoryTable("InventoryAudit");
}));
}

Make sure all the code in the OnModelBuilding() method (in the ApplicationDbContext.cs class) that configures the Inventory class is deleted, and add the following single line of code in its place:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
...
}

The Driver Entity
Chapter 21 added a new entity named Driver and set up a many-to-many relationship with the Car entity. Since this table wasn’t in the Chapter 20 database, it is not in our scaffolded code. Add a new file named Driver.cs to the Entities folder and update the code in the file to match the following:
namespace AutoLot.Models.Entities; [Table("Drivers", Schema = "dbo")]
[EntityTypeConfiguration(typeof(DriverConfiguration))]
public class Driver : BaseEntity
{
public Person PersonInformation{ get; set; } = new Person(); [InverseProperty(nameof(Car.Drivers))]
public virtual IEnumerable Cars { get; set; } = new List();

[InverseProperty(nameof(CarDriver.DriverNavigation))]
public virtual IEnumerable CarDrivers { get; set; } = new List();
}

Update the ApplicationDbContext Class
Since this is a new table, a new DbSet property must be added into the ApplicationDbContext
class. Add the following to the DbSet properties:

public virtual DbSet Drivers { get; set; }

The DriverConfiguration Class
Add a new file named DriverConfiguration.cs into the Configuration folder, and update the code to the following:

namespace AutoLot.Models.Entities.Configuration;

public class DriverConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.OwnsOne(o => o. PersonInformation, pd =>
{
pd.Property(nameof(Person.FirstName))
.HasColumnName(nameof(Person.FirstName))
.HasColumnType("nvarchar(50)"); pd.Property(nameof(Person.LastName))
.HasColumnName(nameof(Person.LastName))
.HasColumnType("nvarchar(50)"); pd.Property(p => p.FullName)
.HasColumnName(nameof(Person.FullName))
.HasComputedColumnSql("[LastName] + ', ' + [FirstName]");
});

builder.Navigation(d => d.PersonInformation).IsRequired(true);
}
}

The Driver entity uses the Person-owned property, so it cannot be configured as a temporal table.

Update the ApplicationDbContext OnModelCreating() method:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new DriverConfiguration().Configure(modelBuilder.Entity());
}

The CarDriver Entity
Continuing with configuration the many-to-many relationship between Car and Driver, add a new class named CarDriver. Update the code to the following:

namespace AutoLot.Models.Entities;

[Table("InventoryToDrivers", Schema = "dbo")] [EntityTypeConfiguration(typeof(CarDriverConfiguration))] public class CarDriver : BaseEntity
{
public int DriverId { get; set; } [ForeignKey(nameof(DriverId))]
public virtual Driver DriverNavigation { get; set; }

[Column("InventoryId")]
public int CarId { get; set; } [ForeignKey(nameof(CarId))]
public virtual Car CarNavigation { get; set; }
}

Update the ApplicationDbContext Class
Since this is a new table, a new DbSet property must be added into the ApplicationDbContext
class. Add the following to the DbSet properties:

public virtual DbSet CarsToDrivers { get; set; }

The CarDriverConfiguration Class
Because of the query filter for nondrivable cars on the Car class, the related tables (CarDriver and Order) need to have the same query filter applied to their navigation properties. Add a new file named CarDriverConfiguration.cs into the Configuration folder, and update the code to the following:

namespace AutoLot.Models.Entities.Configuration;

public class CarDriverConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.HasQueryFilter(cd=>cd.CarNavigation.IsDrivable);
}
}

The InventoryToDrivers table will be configured as a temporal table, so add the following to the
Configure() method:

public void Configure(EntityTypeBuilder builder)
{
//omitted for brevity
builder.ToTable( b => b.IsTemporal(t =>
{
t.HasPeriodEnd("ValidTo"); t.HasPeriodStart("ValidFrom"); t.UseHistoryTable("InventoryToDriversAudit");
}));
}

Update the ApplicationDbContext OnModelCreating() method:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new DriverConfiguration().Configure(modelBuilder.Entity());
new CarDriverConfiguration().Configure(modelBuilder.Entity());
}

The Radio Entity
Chapter 21 also added a new entity named Radio and set up a one-to-one relationship with the Car entity. Add a new file named Radio.cs to the Entities folder and update the code in the file to match the following:
namespace AutoLot.Models.Entities; [Table("Radios", Schema = "dbo")]
[EntityTypeConfiguration(typeof(RadioConfiguration))] public class Radio : BaseEntity
{
public bool HasTweeters { get; set; } public bool HasSubWoofers { get; set; } [Required, StringLength(50)]
public string RadioId { get; set; } [Column("InventoryId")]
public int CarId { get; set; } [ForeignKey(nameof(CarId))]
public virtual Car CarNavigation { get; set; }
}

Update the ApplicationDbContext Class
Since this is a new table, a new DbSet property must be added into the ApplicationDbContext class. Add the following to the DbSet properties:

public virtual DbSet Radios { get; set; }

The RadioConfiguration Class
Create a new class named RadioConfiguration, implement the IEntityTypeConfiguration
interface, and add the code from the ApplicationDbContext OnModelBuilding() method:

namespace AutoLot.Models.Entities.Configuration;

public class RadioConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.HasQueryFilter(r=>r.CarNavigation.IsDrivable); builder.HasIndex(e => e.CarId, "IX_Radios_CarId")
.IsUnique();

builder.HasOne(d => d.CarNavigation)
.WithOne(p => p.RadioNavigation)
.HasForeignKey(d => d.CarId);
}
}

The Radios table will be configured as a temporal table, so add the following to the
Configure() method:

public void Configure(EntityTypeBuilder builder)
{
//omitted for brevity
builder.ToTable( b => b.IsTemporal(t =>
{
t.HasPeriodEnd("ValidTo"); t.HasPeriodStart("ValidFrom"); t.UseHistoryTable("RadiosAudit");
}));
}

Update the OnModelCreating() method in the ApplicationDbContext:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new DriverConfiguration().Configure(modelBuilder.Entity());
new CarDriverConfiguration().Configure(modelBuilder.Entity());
new RadioConfiguration().Configure(modelBuilder.Entity());
}

The Customer Entity
The Customers table was scaffolded to an entity class named Customer. Inherit from BaseEntity and remove the Id and TimeStamp properties. Delete the constructor and add the Table attribute with schema. Remove the FirstName and LastName properties as they will be replaced by the Person-owned entity. This is where the class code stands at this time:

namespace AutoLot.Models.Entities; [Table("Customers", Schema = "dbo")] public partial class Customer : BaseEntity
{
[InverseProperty(nameof(CreditRisk.Customer))]
public virtual ICollection CreditRisks { get; set; } [InverseProperty(nameof(Order.Customer))]
public virtual ICollection Orders { get; set; }
}

Like the Car entity, there are still some issues with this code that need to be fixed, and the owned entity must be added. The inverse property attributes need to be updated with the Navigation suffix and the types changed to an IEnumerable and initialized. Update the code to match the following:

[InverseProperty(nameof(CreditRisk.CustomerNavigation))]
public virtual IEnumerable CreditRisks { get; set; } = new List();

[InverseProperty(nameof(Order.CustomerNavigation))]
public virtual IEnumerable Orders { get; set; } = new List();

The next step is to add the owned property. The relationship will be further configured in the Fluent API.

public Person PersonInformation { get; set; } = new Person();

The final step is to add the EntityTypeConfiguration attribute. Here is the complete class, with the final update in bold:
namespace AutoLot.Models.Entities; [Table("Customers", Schema = "dbo")]
[EntityTypeConfiguration(typeof(CustomerConfiguration))]
public partial class Customer : BaseEntity
{
public Person PersonInformation { get; set; } = new Person(); [InverseProperty(nameof(CreditRisk.CustomerNavigation))]
public virtual IEnumerable CreditRisks { get; set; } = new List(); [InverseProperty(nameof(Order.CustomerNavigation))]
public virtual IEnumerable Orders { get; set; } = new List();
}

The CustomerConfiguration Class
Add a new file named CustomerConfiguration.cs into the Configuration folder, and update the code to the following:

namespace AutoLot.Models.Entities.Configuration;

public class CustomerConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.OwnsOne(o => o. PersonInformation, pd =>
{

});

pd.Property(nameof(Person.FirstName))
.HasColumnName(nameof(Person.FirstName))
.HasColumnType("nvarchar(50)"); pd.Property(nameof(Person.LastName))
.HasColumnName(nameof(Person.LastName))
.HasColumnType("nvarchar(50)"); pd.Property(p => p.FullName)
.HasColumnName(nameof(Person.FullName))
.HasComputedColumnSql("[LastName] + ', ' + [FirstName]");

builder.Navigation(d => d. PersonInformation).IsRequired(true);
}
}

The Customer entity uses the Person-owned property, so it cannot be configured as a temporal table. Delete the Customer configuration code in the ApplicationDbContext OnModelCreating() method,
and add the configuration line for the Customer:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new DriverConfiguration().Configure(modelBuilder.Entity());
new CarDriverConfiguration().Configure(modelBuilder.Entity()); new RadioConfiguration().Configure(modelBuilder.Entity());
new CustomerConfiguration().Configure(modelBuilder.Entity());
}

The Make Entity
The Makes table was scaffolded to an entity class named Make. Inherit from BaseEntity and remove the Id and TimeStamp properties. Delete the constructor and add the Table attribute with schema. Here is the current state of the entity:

namespace AutoLot.Models.Entities;

[Table("Makes", Schema = "dbo")]

public partial class Make : BaseEntity
{
[Required] [StringLength(50)]
public string Name { get; set; } [InverseProperty(nameof(Inventory.Make))]
public virtual ICollection Inventories { get; set; }
}

The following code shows the Cars navigation property corrected, renaming the
Inventory/Inventories references to Car/Cars, the type changed to IEnumerable, and an initializer added:

[InverseProperty(nameof(Car.MakeNavigation))]
public virtual IEnumerable Cars { get; set; } = new List();

The MakeConfiguration Class
Create a new class named MakeConfiguration, implement the IEntityTypeConfiguration interface, and add the code from the ApplicationDbContext OnModelBuilding() method:

namespace AutoLot.Models.Entities.Configuration;

public class MakeConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
}
}

The Makes table will be configured as a temporal table, so add the following to the Configure() method:

public void Configure(EntityTypeBuilder builder)
{
builder.ToTable( b => b.IsTemporal(t =>
{
t.HasPeriodEnd("ValidTo"); t.HasPeriodStart("ValidFrom"); t.UseHistoryTable("MakesAudit");
}));
}

Delete the scaffolded configuration for the Make entity from the ApplicationDbContext Configure()
method to complete the Make entity.
Update the OnModelCreating() method in the ApplicationDbContext: protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new DriverConfiguration().Configure(modelBuilder.Entity());
new CarDriverConfiguration().Configure(modelBuilder.Entity());

new RadioConfiguration().Configure(modelBuilder.Entity());
new CustomerConfiguration().Configure(modelBuilder.Entity());
new MakeConfiguration().Configure(modelBuilder.Entity());
}

The CreditRisk Entity
The CreditRisks table was scaffolded to an entity class named CreditRisk. Inherit from BaseEntity and remove the Id and TimeStamp properties. Delete the constructor and add the Table attribute with schema. Remove the FirstName and LastName properties, as they will be replaced by the Person-owned entity. Here is the updated class code:

namespace AutoLot.Models.Entities;

[Table("CreditRisks", Schema = "dbo")] [Index(nameof(CustomerId), Name = "IX_CreditRisks_CustomerId")] [EntityTypeConfiguration(typeof(CreditRiskConfiguration))] public partial class CreditRisk : BaseEntity
{
public int CustomerId { get; set; } [ForeignKey(nameof(CustomerId))] [InverseProperty("CreditRisks")]
public virtual Customer Customer { get; set; }
}

Fix the navigation property by using the nameof() method in the InverseProperty attribute and add the Navigation suffix to the property name.

[ForeignKey(nameof(CustomerId))] [InverseProperty(nameof(Customer.CreditRisks))]
public virtual Customer CustomerNavigation { get; set; }

The final change is to add the owned property. The relationship will be further configured in the Fluent API.

public Person PersonInformation { get; set; } = new Person();

■Note as discussed when the CreditRisk table was introduced, having the Person-owned class and a navigation property to the Customer table feels like an odd design, and in truth, it is. all of these tables were created to teach a different aspect of eF Core, and this is no different. Consider the extra FirstName/LastName as a place to put the uncreditworthy individual’s alias.

That completes the CreditRisk entity.

The CreditRiskConfiguration Class
Add a new file named CreditRiskConfiguration.cs into the Configuration folder, and update the code to the following:

namespace AutoLot.Models.Entities.Configuration;

public class CreditRiskConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.HasOne(d => d.CustomerNavigation)
.WithMany(p => p.CreditRisks)
.HasForeignKey(d => d.CustomerId)
.HasConstraintName("FK_CreditRisks_Customers");

builder.OwnsOne(o => o.PersonInformation, pd =>
{

});

pd.Property(nameof(Person.FirstName))
.HasColumnName(nameof(Person.FirstName))
.HasColumnType("nvarchar(50)"); pd.Property(nameof(Person.LastName))
.HasColumnName(nameof(Person.LastName))
.HasColumnType("nvarchar(50)"); pd.Property(p => p.FullName)
.HasColumnName(nameof(Person.FullName))
.HasComputedColumnSql("[LastName] + ', ' + [FirstName]");

builder.Navigation(d => d.PersonInformation).IsRequired(true);
}
}

The CreditRisk entity uses the Person-owned property, so it cannot be configured as a temporal table.

Update the ApplicationDbContext OnModelCreating() method by removing the CreditRisk configuration and adding the line for the CreditRiskConfiguration class:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new DriverConfiguration().Configure(modelBuilder.Entity());
new CarDriverConfiguration().Configure(modelBuilder.Entity()); new RadioConfiguration().Configure(modelBuilder.Entity());
new CustomerConfiguration().Configure(modelBuilder.Entity()); new MakeConfiguration().Configure(modelBuilder.Entity());
new CreditRiskConfiguration().Configure(modelBuilder.Entity());
}

The Order Entity
The Orders table was scaffolded to an entity class named Order. Inherit from BaseEntity and remove the Id and
TimeStamp properties. Delete the constructor and add the Table attribute with schema. Here is the current code:

namespace AutoLot.Models.Entities;

[Table("Orders", Schema = "dbo")]
[Index(nameof(CarId), Name = "IX_Orders_CarId")]
[Index(nameof(CustomerId), nameof(CarId), Name = "IX_Orders_CustomerId_CarId", IsUnique
= true)]
public partial class Order : BaseEntity
{
public int CarId { get; set; } [ForeignKey(nameof(CarId))] [InverseProperty(nameof(Inventory.Orders))] public virtual Inventory Car { get; set; }

public int CustomerId { get; set; } [ForeignKey(nameof(CustomerId))] [InverseProperty("Orders")]
public virtual Customer { get; set; }
}

The Car and Customer navigation properties need the Navigation suffix added to their property names. The Car navigation property needs the type corrected to Car from Inventory. The inverse property needs the nameof() method to use Car.Orders instead of Inventory.Orders. The Customer navigation property needs to use the nameof() method for the InverseProperty.

[ForeignKey(nameof(CarId))] [InverseProperty(nameof(Car.Orders))]
public virtual Car CarNavigation { get; set; }

[ForeignKey(nameof(CustomerId))] [InverseProperty(nameof(Customer.Orders))]
public virtual Customer CustomerNavigation { get; set; }

The final step is to add the EntityTypeConfiguration attribute with the update in bold:

namespace AutoLot.Models.Entities;

[Table("Orders", Schema = "dbo")] [Index(nameof(CarId), Name = "IX_Orders_CarId")]
[Index(nameof(CustomerId), nameof(CarId), Name = "IX_Orders_CustomerId_CarId", IsUnique
= true)] [EntityTypeConfiguration(typeof(OrderConfiguration))] public partial class Order : BaseEntity
{
...
}

That completes the Order entity.

The OrderConfiguration Class
Add a new file named OrderConfiguration.cs into the Configuration folder, and update the code to the following:

namespace AutoLot.Models.Entities.Configuration;

public class OrderConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.HasIndex(cr => new { cr.CustomerId, cr.CarId }).IsUnique(true); builder.HasQueryFilter(e => e.CarNavigation!.IsDrivable); builder.HasOne(d => d.CarNavigation)
.WithMany(p => p.Orders)
.HasForeignKey(d => d.CarId)
.OnDelete(DeleteBehavior.ClientSetNull)
.HasConstraintName("FK_Orders_Inventory"); builder.HasOne(d => d.CustomerNavigation)
.WithMany(p => p.Orders)
.HasForeignKey(d => d.CustomerId)
.HasConstraintName("FK_Orders_Customers");
}
}

The Orders table will be configured as a temporal table, so add the following to the Configure() method:

public void Configure(EntityTypeBuilder builder)
{
//omitted for brevity
builder.ToTable( b => b.IsTemporal(t =>
{
t.HasPeriodEnd("ValidTo"); t.HasPeriodStart("ValidFrom"); t.UseHistoryTable("OrdersAudit");
}));
}

Update the ApplicationDbContext OnModelCreating() method by removing the Order configuration and adding the line for the OrderConfiguration class:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new DriverConfiguration().Configure(modelBuilder.Entity());
new CarDriverConfiguration().Configure(modelBuilder.Entity()); new RadioConfiguration().Configure(modelBuilder.Entity());
new CustomerConfiguration().Configure(modelBuilder.Entity()); new MakeConfiguration().Configure(modelBuilder.Entity());
new CreditRiskConfiguration().Configure(modelBuilder.Entity());
new OrderConfiguration().Configure(modelBuilder.Entity());
}

■Note at this time, both the autoLot.models project and the autoLot.dal project should build properly.

The SeriLogEntry Entity
The database needs an additional table to hold log records. The ASP.NET Core projects later in this book use the SeriLog logging framework, and one of the options is to write log records to a SQL Server table. We are going to add the table now, knowing it will be used a few chapters from now.
The table does not relate to any other tables and does not use the BaseEntity class. Add a new class file named SeriLogEntry.cs in the Entities folder. The code is listed in its entirety here:
namespace AutoLot.Models.Entities; [Table("SeriLogs", Schema = "Logging")]
[EntityTypeConfiguration(typeof(SeriLogEntryConfiguration))] public class SeriLogEntry
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)] public int Id { get; set; }
public string Message { get; set; }
public string MessageTemplate { get; set; } [MaxLength(128)]
public string Level { get; set; } [DataType(DataType.DateTime)]
public DateTime TimeStamp { get; set; } public string Exception { get; set; } public string Properties { get; set; } public string LogEvent { get; set; } public string SourceContext { get; set; } public string RequestPath { get; set; } public string ActionName { get; set; }
public string ApplicationName { get; set; } public string MachineName { get; set; } public string FilePath { get; set; }
public string MemberName { get; set; } public int LineNumber { get; set; } [NotMapped]
public XElement PropertiesXml => (Properties != null)? XElement.Parse(Properties):null;
}

■Note The TimeStamp property in this entity is not the same as the TimeStamp property in the BaseEntity class. The names are the same, but in this table it holds the date and time of when the entry was logged and not the rowversion used in the other entities.

Update the ApplicationDbContext Class
Since this is a new table, a new DbSet property must be added into the
ApplicationDbContext class. Add the following to the DbSet properties:

public virtual DbSet SeriLogEntries { get; set; }

The SerilogEntryConfiguration Class
Add a new file named SeriLogEntryConfiguration.cs into the Configuration folder, and update the code to the following:

namespace AutoLot.Models.Entities.Configuration;

public class SeriLogEntryConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.Property(e => e.Properties).HasColumnType("Xml"); builder.Property(e => e.TimeStamp).HasDefaultValueSql("GetDate()");
}
}

Update the ApplicationDbContext OnModelCreating() method by adding the line for the
SeriLogEntryConfiguration class:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new DriverConfiguration().Configure(modelBuilder.Entity());
new CarDriverConfiguration().Configure(modelBuilder.Entity()); new RadioConfiguration().Configure(modelBuilder.Entity());
new CustomerConfiguration().Configure(modelBuilder.Entity());
new CreditRiskConfiguration().Configure(modelBuilder.Entity()); new OrderConfiguration().Configure(modelBuilder.Entity());
new SeriLogEntryConfiguration().Configure(modelBuilder.Entity());
}

The View Models
The CustomerOrderView was scaffolded into a keyless entity along with the database tables. Another term used for keyless entities is view models, as they are designed to view data, usually from more than one table. This section will update the scaffolded entity into its final form as well as add a new view model to view temporal data. Start by adding a new folder named ViewModels in the AutoLot.Models project.

The CustomerOrderViewModel
Move the CustomerOrderView.cs class from the Entities folder into this folder and rename the file to
CustomerOrderViewModel.cs and the class to CustomerOrderViewModel. Add an EntityTypeConfiguration

attribute for the soon to be created configuration class. Also implement the INonPersisted interface (which will be created next):

namespace AutoLot.Models.ViewModels;

[Keyless] [EntityTypeConfiguration(typeof(CustomerOrderViewModelConfiguration))] public partial class CustomerOrderViewModel : INonPersisted
{
}

Add a new NotMapped property named FullDetail, as follows:

[NotMapped]
public string FullDetail => $"{FirstName} {LastName} ordered a {Color} {Make} named
{PetName}";

The FullDetail property is decorated with the NotMapped data annotation. Recall that this informs EF Core that this property is not to be included in the data coming from the database.
Next, add the four new properties for the Car entity to the view model:

public bool? IsDrivable { get; set; } public string Display { get; set; } public string Price {get;set; }
public DateTime? DateBuilt {get;set; }

Next add an override for the ToString() method. The ToString() override is also ignored by EF Core:

public override string ToString() => FullDetail;

That completes the changes to the view model. The complete file is shown here:
namespace AutoLot.Models.ViewModels; [Keyless]
[EntityTypeConfiguration(typeof(CustomerOrderViewModelConfiguration))] public partial class CustomerOrderViewModel : INonPersisted
{
[Required] [StringLength(50)]
public string FirstName { get; set; } [Required]
[StringLength(50)]
public string LastName { get; set; } [Required]
[StringLength(50)]
public string Color { get; set; } [Required]
[StringLength(50)]
public string PetName { get; set; } [Required]

[StringLength(50)]
public string Make { get; set; } public bool? IsDrivable { get; set; } public string Display { get;set; } [NotMapped]
public string FullDetail => $"{FirstName} {LastName} ordered a {Color} {Make} named
{PetName}";
public override string ToString() => FullDetail;
}

As a reminder, the KeyLess data annotation indicates this is an entity that works with data that does not have a primary key.

Add the INonPersisted Interface
Create a new folder named Interfaces in the ViewModels folder. In this folder, add a new interface named
INonPersisted, and update the code to the following:

namespace AutoLot.Models.ViewModels.Interfaces; public interface INonPersisted { }
Update the ApplicationDbContext Class
Since the CustomerOrderView class was renamed to CustomerOrderViewModel, the ApplicationDbContext class must be updated. Locate the DbSet< CustomerOrderView> property and update the line to the following:

public virtual DbSet< CustomerOrderViewModel> CustomerOrderViewModels { get; set; }

The CustomerOrderViewModelConfiguration Class
Create new folder named Configuration in the ViewModels folder. In this folder, add a new file named CustomerOrderViewModelConfiguration.cs into the Configuration folder, and update the code to the following:

namespace AutoLot.Models.ViewModels.Configuration;

public class CustomerOrderViewModelConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.ToView("CustomerOrderView");
}
}

Update the ApplicationDbContext OnModelCreating() method by deleting the configuration for the
CustomerOrderView class and adding the line for the CustomerOrderViewModelConfiguration class:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new DriverConfiguration().Configure(modelBuilder.Entity()); new RadioConfiguration().Configure(modelBuilder.Entity());
new CustomerConfiguration().Configure(modelBuilder.Entity());
new CreditRiskConfiguration().Configure(modelBuilder.Entity()); new OrderConfiguration().Configure(modelBuilder.Entity());
new SeriLogEntryConfiguration().Configure(modelBuilder.Entity());
new CustomerOrderViewModelConfiguration()
.Configure(modelBuilder.Entity());
}

Update the GlobalUsings.cs Files
The new namespaces need to be added into the GlobalUsings.cs files in the AutoLot.Dal and AutoLot. Models projects. Add the following global using statements to each file:

//AutoLot.Models
global using AutoLot.Models.ViewModels.Configuration; global using AutoLot.Models.ViewModels.Interfaces;

//AutoLot.Dal
global using AutoLot.Models.ViewModels;
global using AutoLot.Models.ViewModels.Configuration; global using AutoLot.Models.ViewModels.Interfaces;

The TemporalViewModel
Recall from the previous chapter that when working with temporal data, it helps to have a class that stores the row along with the row’s from and to dates. Create a new class named TemporalViewModel in the Entities folder. Update the code to the following:

namespace AutoLot.Models.ViewModels;

public class TemporalViewModel where T: BaseEntity, new()
{
public T Entity { get; set; }
public DateTime ValidFrom { get; set; } public DateTime ValidTo { get; set; }
}

Since this class is only going to be used to store the results of queries on temporal tables, it does not need to be configured in ApplicationDbContext.

Update the ApplicationDbContext
It is time to update the ApplicationDbContext.cs file. Delete the default constructor, as we won’t need it. It is only used in conjunction with an OnConfiguring() method, which, as discussed earlier, is considered not a good practice to use. The next constructor takes an instance of the DbContextOptions object and is fine for now. The event hooks for DbContext and ChangeTracker will be added later in this chapter.

Add the Mapped Database Functions
Recall that user-defined database functions can be mapped to C# functions for use in LINQ queries. Add the following functions to ApplicationDbContext for the two user-defined functions:

[DbFunction("udf_CountOfMakes", Schema = "dbo")] public static int InventoryCountFor(int makeId)
=> throw new NotSupportedException();

[DbFunction("udtf_GetCarsForMake", Schema = "dbo")] public IQueryable GetCarsFor(int makeId)
=> FromExpression(() => GetCarsFor(makeId));

Handling DbContext and ChangeTracker Events
Navigate to the constructor of ApplicationDbContext and add the three DbContext events discussed in the previous chapter.

public ApplicationDbContext(DbContextOptions options)
: base(options)
{
SavingChanges += (sender, args) =>
{
string cs = ((ApplicationDbContext)sender).Database!.GetConnectionString(); Console.WriteLine($"Saving changes for {cs}");
};
SavedChanges += (sender, args) =>
{
string cs = ((ApplicationDbContext)sender).Database!.GetConnectionString(); Console.WriteLine($"Saved {args!.EntitiesSavedCount} changes for {cs}");
};
SaveChangesFailed += (sender, args) =>
{
Console.WriteLine($"An exception occurred! {args.Exception.Message} entities");
};
}

Next, add handlers for the ChangeTracker, StateChanged, and Tracked events.

public ApplicationDbContext(DbContextOptions options)
: base(options)
{
...
ChangeTracker.Tracked += ChangeTracker_Tracked; ChangeTracker.StateChanged += ChangeTracker_StateChanged;
}

As a refresher, EntityTrackedEventArgs holds a reference to the entity that triggered the event and whether it came from a query (loaded from the database) or was added programmatically. Add the following event handler in ApplicationDbContext:

private void ChangeTracker_Tracked(object sender, EntityTrackedEventArgs e)
{
var source = (e.FromQuery) ? "Database" : "Code"; if (e.Entry.Entity is Car c)
{
Console.WriteLine($"Car entry {c.PetName} was added from {source}");
}
}

The StateChanged event is fired when a tracked entity’s state changes. In the following event handler, if the entity’s NewState is Unchanged, the OldState is examined to see whether the entity was added or modified. Add the following event handler into ApplicationDbContext:

private void ChangeTrackerStateChanged(object sender, EntityStateChangedEventArgs e)
{
if (e.Entry.Entity is not Car c)
{
return;
}
var action = string.Empty;
Console.WriteLine($"Car {c.PetName} was {e.OldState} before the state changed to
{e.NewState}"); switch (e.NewState)
{
case EntityState.Unchanged: action = e.OldState switch
{
EntityState.Added => "Added", EntityState.Modified => "Edited",
=> action
};
Console.WriteLine($"The object was {action}"); break;
}
}

Override the Conventions
Add the override for ConfigureConventions. As a reminder, the following overrides will default strings to nvarchar(50) and will ignore entities that implement the INonPersisted interface. Any data annotations or Fluent API commands that contradict those two settings will override the configured conventions:

protected override void ConfigureConventions(ModelConfigurationBuilder configurationBuilder)
{
configurationBuilder.Properties().HaveMaxLength(50); configurationBuilder.IgnoreAny();
}

Override the SaveChanges Method
Recall that the SaveChanges() method on the base DbContext class persists the data updates, additions, and deletions to the database. Overriding that method in the derived DbContext enables exception handing to be encapsulated in one place. Add the following override to the SaveChanges() method:

public override int SaveChanges()
{
try
{
return base.SaveChanges();
}
catch (DbUpdateConcurrencyException ex)
{
//A concurrency error occurred
//Should log and handle intelligently
throw new CustomConcurrencyException("A concurrency error happened.", ex);
}
catch (RetryLimitExceededException ex)
{
//DbResiliency retry limit exceeded
//Should log and handle intelligently
throw new CustomRetryLimitExceededException("There is a problem with SQL Server.", ex);
}
catch (DbUpdateException ex)
{
//Should log and handle intelligently
throw new CustomDbUpdateException("An error occurred updating the database", ex);
}
catch (Exception ex)
{
//Should log and handle intelligently
throw new CustomException("An error occurred updating the database", ex);
}
}

Create the Next Migration and Update the Database
At this point in the chapter, we are ready to create another migration to update the database. Enter the following commands in the AutoLot.Dal project directory (each command must be entered on one line):

dotnet ef migrations add UpdatedEntities -o EfStructures\Migrations -c AutoLot.Dal.
EfStructures.ApplicationDbContext

dotnet ef database update UpdatedEntities -c AutoLot.Dal.EfStructures.ApplicationDbContext

Use EF Migrations to Create/Update Database Objects
While the CustomerOrderViewModel was scaffolded from the CustomerOrderView in the database, the view itself is not represented in the C# code. If you were to drop the database and re-create it using the EF Core migrations, the view will not exist. For database objects, you have two options: maintain them separately and apply them using SSMS/Azure Data Studio or leverage the EF Core migrations to handle their creation.
Recall that each EF Core migration file has an Up() method (for applying the migration to the database) and a Down() method (for rolling the changes back). The MigrationBuilder also has a Sql() method that executes SQL statements directly against the database. By adding the CREATE and DROP statements into the Up() and Down() methods of a migration, the migration system will handle applying (and rolling back) database changes.

Add the MigrationHelpers Class
A helper class will hold all the SQL statements used by the custom migration. This separation prevents losing the code if the migration is removed from the system. Create a new static class named MigrationHelpers. cs in the EfStructures folder of the AutoLot.Dal project.
Add a new file named MigrationHelpers.cs in the EfStructures folder of the AutoLot.Dal project.
Add a using statement for Microsoft.EntityFrameworkCore.Migrations, make the class public and static, and add the following methods, which use the MigrationBuilder to execute SQL statements against the database:
namespace AutoLot.Dal.EfStructures; public static class MigrationHelpers
{
public static void CreateCustomerOrderView(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(@"exec (N'
CREATE VIEW [dbo].[CustomerOrderView] AS
SELECT c.FirstName, c.LastName, i.Color, i.PetName, i.DateBuilt, i.IsDrivable, i.Price, i.Display, m.Name AS Make
FROM dbo.Orders o
INNER JOIN dbo.Customers c ON c.Id = o.CustomerId INNER JOIN dbo.Inventory i ON i.Id = o.CarId INNER JOIN dbo.Makes m ON m.Id = i.MakeId')");
}
public static void DropCustomerOrderView(MigrationBuilder migrationBuilder)
{

migrationBuilder.Sql("EXEC (N' DROP VIEW [dbo].[CustomerOrderView] ')");
}

public static void CreateSproc(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(@"exec (N'
CREATE PROCEDURE [dbo].[GetPetName] @carID int,
@petName nvarchar(50) output AS
SELECT @petName = PetName from dbo.Inventory where Id = @carID')");
}
public static void DropSproc(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql("EXEC (N' DROP PROCEDURE [dbo].[GetPetName]')");
}

public static void CreateFunctions(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql(@"exec (N'
CREATE FUNCTION [dbo].[udtf_GetCarsForMake] ( @makeId int ) RETURNS TABLE
AS RETURN (
SELECT Id, IsDrivable, DateBuilt, Color, PetName, MakeId, TimeStamp, Display, Price
FROM Inventory WHERE MakeId = @makeId
)')");
migrationBuilder.Sql(@"exec (N'
CREATE FUNCTION [dbo].[udf_CountOfMakes] ( @makeid int ) RETURNS int
AS BEGIN
DECLARE @Result int
SELECT @Result = COUNT(makeid) FROM dbo.Inventory WHERE makeid = @makeid RETURN @Result
END')");
}
public static void DropFunctions(MigrationBuilder migrationBuilder)
{
migrationBuilder.Sql("EXEC (N' DROP FUNCTION [dbo].[udtf_GetCarsForMake]')"); migrationBuilder.Sql("EXEC (N' DROP FUNCTION [dbo].[udf_CountOfMakes]')");
}
}

■Note The CREATE statements are included in a sQL server exec statement so they will successfully run when the migrations are scripted. each migration process is wrapped in an IF block, and creation statements must be wrapped in exec statements when executed inside an IF.

Create and Update the Migration
Calling the dotnet migrations add command when there aren’t any model changes will still create the properly timestamped migration files with empty Up() and Down() methods. Execute the following to create the empty migration (but do not apply the migration):

dotnet ef migrations add SQL -o EfStructures\Migrations -c AutoLot.Dal.EfStructures. ApplicationDbContext

Open the newly created migration class and notice that the Up() and Down() methods are empty. That is the key to this technique. Using a blank migration that is updated using the MigrationHelpers methods prevents mixing custom code with EF Core–generated code. The static methods to create the database objects go into the migration’s Up() method, and the methods to drop the database objects go into the migration’s Down() method. When this migration is applied, the SQL Server objects are created, and when the migration is rolled back, the SQL Server objects are dropped. Here is the updated migration code listing:
namespace AutoLot.Dal.EfStructures.Migrations' public partial class SQL : Migration
{
protected override void Up(MigrationBuilder migrationBuilder)
{
MigrationHelpers.CreateCustomerOrderView(migrationBuilder); MigrationHelpers.CreateSproc(migrationBuilder); MigrationHelpers.CreateFunctions(migrationBuilder);
}

protected override void Down(MigrationBuilder migrationBuilder)
{
MigrationHelpers.DropCustomerOrderView(migrationBuilder); MigrationHelpers.DropSproc(migrationBuilder); MigrationHelpers.DropFunctions(migrationBuilder);
}
}

Apply the Migration
If you dropped your database to run the initial migration, you can apply this migration and move on. Apply the migration by executing the following command:

dotnet ef database update -c AutoLot.Dal.EfStructures.ApplicationDbContext

If you did not drop your database for the first migration, the procedure and view already exist, and the create SQL statements for those database objects will fail, rolling the entire migration back. The simple fix is to drop the procedure and view using SSMS or Azure Data Studio, like this:

DROP VIEW [dbo].[CustomerOrderView] GO
DROP PROCEDURE [dbo].[GetPetName] GO

Now, apply the migration to add the SQL objects into the AutoLot database:

dotnet ef database update SQL -c AutoLot.Dal.EfStructures.ApplicationDbContext

■Note you could also write code that will first check for an object’s existence and drop it if it already exists, but I find that overkill for an issue happens only once when transitioning from database first to code first.

Add the Repositories
A common data access design pattern is the repository pattern. As described by Martin Fowler, the core of this pattern is to mediate between the domain and data mapping layers. Having a generic base repository that contains the common data access code helps to eliminate duplication of code. Having specific repositories and interfaces that derive from a base repository also works well with the dependency injection framework in ASP.NET Core.

■Note This next section is not meant to be (nor does it pretend to be) a literal interpretation of mr. Fowler’s design pattern. If you are interested in the original pattern that motivated this version, you can find more information on the repository pattern at www.martinfowler.com/eaaCatalog/repository.html.

Each of the domain entities in the AutoLot data access layer will have a strongly typed repo to encapsulate all the data access work. To start, create a folder named Repos in the AutoLot.Dal project to hold all the classes.

Add the IBaseViewRepo Interface
The IBaseViewRepo interface exposes three methods for getting data from a view model. Make a new folder named Base in the Repos directory. Add a new interface into the Repos\Base folder named IBaseViewRepo. Update the code to match the following:

namespace AutoLot.Dal.Repos.Base;

public interface IBaseViewRepo: IDisposable where T : class,new()
{
ApplicationDbContext Context {get;} IEnumerable ExecuteSqlString(string sql); IEnumerable GetAll();
IEnumerable GetAllIgnoreQueryFilters();
}

Add the BaseViewRepo Implementation
Next, add a class named BaseViewRepo to the Repos\Base directory. This class will implement the IBaseViewRepo interface and provide the implementation for the interface. Make the class generic with type T and constrain the type to class and new(), which limits the types to classes that have a parameterless constructor. Implement the IBaseViewRepo interface, as follows:

namespace AutoLot.Dal.Repos.Base;

public abstract class BaseViewRepo : IBaseViewRepo where T : class, new()
{
//implementation goes here
}

The repo needs an instance of the ApplicationDbContext injected into a constructor. When used with the ASP.NET Core dependency injection (DI) container, the container will handle the lifetime of the context. A second constructor, used for integration testing, will accept an instance of DbContextOptions and use that to create an instance of the ApplicationDbContext. That context will need to be disposed since it isn’t being managed by a DI container. Because this class is abstract, both constructors are protected. Add the following code for the public ApplicationDbContext, the two constructors, and the Dispose pattern:

private readonly bool _disposeContext; public ApplicationDbContext Context { get; }

protected BaseViewRepo(ApplicationDbContext context)
{
Context = context;
_disposeContext = false;
}

protected BaseViewRepo(DbContextOptions options) : this(new ApplicationDbContext(options))
{
_disposeContext = true;
}

public virtual void Dispose()
{
Dispose(true); GC.SuppressFinalize(this);
}
private bool _isDisposed;
protected virtual void Dispose(bool disposing)
{
if (_isDisposed)
{
return;
}

if (disposing)
{
if (_disposeContext)
{
Context.Dispose();
}
}
_isDisposed = true;
}

~BaseViewRepo()
{
Dispose(false);
}

The DbSet properties of ApplicationDbContext can be referenced by using the DbContext.Set() method. Create a public property named Table of type DbSet and set the value in the initial constructor, like this:

public DbSet Table { get; }
protected BaseViewRepo(ApplicationDbContext context)
{
Context = context;
Table = Context.Set();
_disposeContext = false;
}

Implement the Read Methods
The next series of methods returns records using LINQ statements or a SQL query. The GetAll() methods return all the records from the table. The first retrieves them in database order, and the second turns off all query filters.

public virtual IEnumerable GetAll()
=> Table.AsQueryable();

public virtual IEnumerable GetAllIgnoreQueryFilters()
=> Table.AsQueryable().IgnoreQueryFilters();

The ExecuteSqlString() method is there to execute FromSqlRaw() queries:

public IEnumerable ExecuteSqlString(string sql) => Table.FromSqlRaw(sql);

Add the IBaseRepo Interface
The IBaseRepo interface exposes many of the common methods used in data access with the DbSet properties where T is of type BaseEntity. Add a new interface into the Repos\Base folder named IBaseRepo. The full interface is listed here:

namespace AutoLot.Dal.Repos.Base;

public interface IBaseRepo : IBaseViewRepo where T : BaseEntity, new()
{
T Find(int? id);
T FindAsNoTracking(int id);
T FindIgnoreQueryFilters(int id);
void ExecuteParameterizedQuery(string sql, object[] sqlParametersObjects); int Add(T entity, bool persist = true);
int AddRange(IEnumerable entities, bool persist = true); int Update(T entity, bool persist = true);

int UpdateRange(IEnumerable entities, bool persist = true); int Delete(int id, byte[] timeStamp, bool persist = true);
int Delete(T entity, bool persist = true);
int DeleteRange(IEnumerable entities, bool persist = true); int SaveChanges();
}

Add the BaseRepo Implementation
Next, add a class named BaseRepo to the Repos\Base directory. This class implements the IBaseRepo interface, inherits from the BaseViewRepo abstract class, and provides the core functionality for the type-specific repos that will be built for each entity. Make the class generic with type T and constrain the type to BaseEntity and new(), which limits the types to classes that inherit from BaseEntity and have a parameterless constructor. Implement the IBaseRepo interface, as follows:

namespace AutoLot.Dal.Repos.Base;

public abstract class BaseRepo : BaseViewRepo, IBaseRepo where T : BaseEntity, new()
{
//implementation goes here
}

The repo leverages the BaseViewRepo for the handling of the ApplicationDbContext instance as well as the Dispose() pattern implementation. Add the following code for the two constructors:

protected BaseRepo(ApplicationDbContext context) : base(context) {}
protected BaseRepo(DbContextOptions options) : this(new ApplicationDbContext(options))
{
}

Implement the SaveChanges Method
The BaseRepo has a SaveChanges() method that calls into the overridden SaveChanges() method on the
ApplicationDbContext class. Add the following code to the BaseRepo class:

public int SaveChanges()
{
try
{
return Context.SaveChanges();
}
catch (CustomException ex)
{
//Should handle intelligently - already logged throw;
}
catch (Exception ex)
{

//Should log and handle intelligently
throw new CustomException("An error occurred updating the database", ex);
}
}

Implement the Common Read Methods
The next series of methods returns records using LINQ statements. The Find() method takes the primary key value(s) and searches the ChangeTracker first. If the entity is already being tracked, the tracked instance is returned. If not, the record is retrieved from the database.

public virtual T Find(int? id) => Table.Find(id);

The two additional Find() methods augment the Find() base method. The next method demonstrates retrieving a record but not adding it to the ChangeTracker using AsNoTrackingWithIdentityResolution(). Add the following code to the class:

public virtual T FindAsNoTracking(int id) => Table.AsNoTrackingWithIdentityResolution().FirstOrDefault(x => x.Id == id);

The next variation removes the query filters from the entity and then uses the shorthand version (skipping the Where() method) to get FirstOrDefault(). Add the following to the class:

public virtual T FindIgnoreQueryFilters(int id) => Table.IgnoreQueryFilters().FirstOrDefault(x => x.Id == id);

The final method’s next variation is used to execute a parameterized stored procedure. Add the final method to the class:

public virtual void ExecuteParameterizedQuery(string sql, object[] sqlParametersObjects)
=> Context.Database.ExecuteSqlRaw(sql, sqlParametersObjects);

The Add, Update, and Delete Methods
The next block of code to be added wraps the matching Add()/AddRange(), Update()/UpdateRange(), and Remove()/RemoveRange() methods on the specific DbSet property. The persist parameter determines whether the repo executes SaveChanges() immediately when the repository methods are called. All the methods are marked virtual to allow for downstream overriding. Add the following code to your class:

public virtual int Add(T entity, bool persist = true)
{
Table.Add(entity);
return persist ? SaveChanges() : 0;
}
public virtual int AddRange(IEnumerable entities, bool persist = true)
{
Table.AddRange(entities);
return persist ? SaveChanges() : 0;
}
public virtual int Update(T entity, bool persist = true)

{
Table.Update(entity);
return persist ? SaveChanges() : 0;
}
public virtual int UpdateRange(IEnumerable entities, bool persist = true)
{
Table.UpdateRange(entities);
return persist ? SaveChanges() : 0;
}
public virtual int Delete(T entity, bool persist = true)
{
Table.Remove(entity);
return persist ? SaveChanges() : 0;
}
public virtual int DeleteRange(IEnumerable entities, bool persist = true)
{
Table.RemoveRange(entities);
return persist ? SaveChanges() : 0;
}

There is one more Delete() method that doesn’t follow the same pattern. This method uses EntityState to conduct the delete operation, which is used fairly often in ASP.NET Core operations to cut down on the network traffic. It is listed here:

public int Delete(int id, byte[] timeStamp, bool persist = true)
{
var entity = new T {Id = id, TimeStamp = timeStamp}; Context.Entry(entity).State = EntityState.Deleted; return persist ? SaveChanges() : 0;
}

This concludes the BaseRepo class, and now it’s time to build the repo for temporal table support.

Add the ITemporalTableBaseRepo Interface
The ITemporalTableBaseRepo interface exposes the temporal capabilities of EF Core. Add a new interface into the Repos\Base folder named ITemporalTableBaseRepo. The full interface is listed here:

namespace AutoLot.Dal.Repos.Base;

public interface ITemporalTableBaseRepo : IBaseRepo where T : BaseEntity, new()
{
IEnumerable<TemporalViewModel> GetAllHistory(); IEnumerable<TemporalViewModel> GetHistoryAsOf(DateTime dateTime);
IEnumerable<TemporalViewModel> GetHistoryBetween(DateTime startDateTime, DateTime endDateTime);
IEnumerable<TemporalViewModel> GetHistoryContainedIn(DateTime startDateTime, DateTime endDateTime);
IEnumerable<TemporalViewModel> GetHistoryFromTo(DateTime startDateTime, DateTime endDateTime);
}

Add the TemporalTableBaseRepo Implementation
Next, add a class named TemporalTableBaseRepo to the Repos\Base directory. This class implements the ITemporalTableBaseRepo interface, inherits from BaseRepo, and provides the functionality for using temporal tables. Also make the class generic with type T and constrain the type to BaseEntity and new(). Implement the ITemporalTableBaseRepo interface, as follows:

namespace AutoLot.Dal.Repos.Base;

public abstract class TemporalTableBaseRepo : BaseRepo, ITemporalTableBaseRepo where T : BaseEntity, new()
{
//implementation goes here
}

The repo leverages the BaseRepo for the handling of the ApplicationDbContext instance as well as the Dispose() pattern implementation. Add the following code for the two constructors:

protected TemporalTableBaseRepo(ApplicationDbContext context) : base(context) {} protected TemporalTableBaseRepo(DbContextOptions options)
: this(new ApplicationDbContext(options))
{
}

Implement the Helper Methods
There are two helper methods in this class. The first converts the current time (based on the TimeZoneInfo of the executing computer) to UTC time, and the second encapsulates the execution of the query and the projection to the TemporalViewModel. Add the following code to the class to convert the current time to UTC:

internal static DateTime ConvertToUtc(DateTime dateTime)
=> TimeZoneInfo.ConvertTimeToUtc(dateTime, TimeZoneInfo.Local);

The next method takes in an IQueryable, adds the OrderBy clause for the ValidFrom field, and projects the results into a collection of TemporalViewModel instances:

internal static IEnumerable<TemporalViewModel> ExecuteQuery(IQueryable query)
=> query.OrderBy(e => EF.Property(e, "ValidFrom"))
.Select(e => new TemporalViewModel
{
Entity = e,
ValidFrom = EF.Property(e, "ValidFrom"), ValidTo = EF.Property(e, "ValidTo")
});

Implement the Temporal Methods
The final step is to implement the five temporal-based methods of the interface. They take in the required data parameters, call the relevant EF Core temporal method, and then pass execution to the ExecuteQuery() helper method:

public IEnumerable<TemporalViewModel> GetAllHistory()
=> ExecuteQuery(Table.TemporalAll());

public IEnumerable<TemporalViewModel> GetHistoryAsOf(DateTime dateTime)
=> ExecuteQuery(Table.TemporalAsOf(ConvertToUtc(dateTime)));

public IEnumerable<TemporalViewModel> GetHistoryBetween( DateTime startDateTime, DateTime endDateTime)
=> ExecuteQuery(Table.TemporalBetween(ConvertToUtc(startDateTime), ConvertToUtc(endDateTime)));
public IEnumerable<TemporalViewModel> GetHistoryContainedIn( DateTime startDateTime, DateTime endDateTime)
=> ExecuteQuery(Table.TemporalContainedIn(ConvertToUtc(startDateTime), ConvertToUtc(endDateTime)));
public IEnumerable<TemporalViewModel> GetHistoryFromTo( DateTime startDateTime, DateTime endDateTime)
=> ExecuteQuery(Table.TemporalFromTo(ConvertToUtc(startDateTime), ConvertToUtc(endDa teTime)));

Now that all the base repos are completed, it’s time to build the entity-specific repos.

Entity-Specific Repo Interfaces
Each entity and view model will have a strongly typed repository derived from BaseRepo and an interface that implements IRepo. Add a new folder named Interfaces under the Repos directory in the AutoLot. Dal project. In this new directory, add the following interfaces:
ICarDriverRepo ICarRepo.cs ICreditRiskRepo.cs
ICustomerOrderViewModelRepo.cs ICustomerRepo.cs
IDriverRepo IMakeRepo.cs IOrderRepo.cs IRadioRepo
The next sections complete the interfaces.

■Note The interfaces in this section are simple and not indicative of a real-world application. When building production applications with this pattern, the entity specific interfaces will usually hold a significant number of additional methods. These examples are kept simple to show the pattern and how it is used.

The CarDriver Repository Interface
Open the ICarDriverRepo.cs interface. This interface doesn’t add any functionality beyond what is provided in the TemporalTableBaseRepo. Update the code to the following:

namespace AutoLot.Dal.Repos.Interfaces;

public interface ICarDriverRepo : ITemporalTableBaseRepo
{
}

The Car Repository Interface
Open the ICarRepo.cs interface. Change the interface to public and define the repo as follows:

namespace AutoLot.Dal.Repos.Interfaces;

public interface ICarRepo : ITemporalTableBaseRepo
{
IEnumerable GetAllBy(int makeId); string GetPetName(int id);
}

The Credit Risk Interface
Open the ICreditRiskRepo.cs interface. This interface doesn’t add any functionality beyond what is provided in the BaseRepo. Update the code to the following:

namespace AutoLot.Dal.Repos.Interfaces;

public interface ICreditRiskRepo : IBaseRepo
{
}

The CustomerOrderViewModel Repository Interface
Open the ICustomerOrderViewModelRepo.cs interface. This interface doesn’t add any functionality beyond what is provided in the BaseViewRepo. Notice that it implements IBaseViewRepo and not IBaseRepo. Update the code to the following:

namespace AutoLot.Dal.Repos.Interfaces;

public interface ICustomerOrderViewModelRepo : IBaseViewRepo
{
}

The Customer Repository Interface
Open the ICustomerRepo.cs interface. This interface doesn’t add any functionality beyond what is provided in the BaseRepo. Update the code to the following:

namespace AutoLot.Dal.Repos.Interfaces;

public interface ICustomerRepo : IBaseRepo
{
}

The Driver Repository Interface
Open the IDriverRepo.cs interface. This interface doesn’t add any functionality beyond what is provided in the BaseRepo. Update the code to the following:

namespace AutoLot.Dal.Repos.Interfaces;

public interface IDriverRepo : IBaseRepo
{
}

The Make Repository Interface
Open the IMakeRepo.cs interface. This interface doesn’t add any functionality beyond what is provided in the TemporalTableBaseRepo. Update the code to the following:

namespace AutoLot.Dal.Repos.Interfaces;

public interface IMakeRepo : ITemporalTableBaseRepo
{
}

The Order Repository Interface
Open the IOrderRepo.cs interface. This interface doesn’t add any functionality beyond what is provided in the TemporalTableBaseRepo. Update the code to the following:

namespace AutoLot.Dal.Repos.Interfaces;

public interface IOrderRepo : IBaseRepo
{
}

The Radio Repository Interface
Open the IRadioRepo.cs interface. Update the code to the following:

namespace AutoLot.Dal.Repos.Interfaces;

public interface IRadioRepo : ITemporalTableBaseRepo
{
}

This completes the interfaces for the entity-specific repositories.

Implement the Entity-Specific Repositories
The implemented repositories gain most of their functionality from the base class. This section covers the functionality added to or overridden from the base repository. In the Repos directory of the AutoLot.Dal project, add the following repo classes:
CarDriverRepo.cs CarRepo.cs CreditRiskRepo.cs
CustomerOrderViewModelRepo.cs CustomerRepo.cs
DriverRepo.cs MakeRepo.cs OrderRepo.cs
The next sections complete the repositories.

■Note you will notice that none of the repository classes has error handling or logging code. This is intentional to keep the examples focused. you will want to make sure you are handling (and logging) errors in your production code.

The CarDriver Repository
Open the CarDriverRepo.cs file, change the class to public, inherit from TemporalTableBaseRepo, and implement ICarDriverRepo.

namespace AutoLot.Dal.Repos;

public class CarDriverRepo : TemporalTableBaseRepo, ICarDriverRepo
{
//implementation code goes here
}

Each of the repositories must implement the two constructors from the BaseRepo. The first constructor will be used by ASP.NET Core and its built-in dependency injection process. The second will be used by the integration tests (covered in the next chapter) and in the Windows Presentation Foundation (WPF) chapters.

public CarDriverRepo(ApplicationDbContext context) : base(context)
{
}
internal CarDriverRepo(DbContextOptions options) : base(options)
{
}

Next, create an internal method that includes the CarNavigation and DriverNavigation properties. Note that the return type is IIncludableQueryable<CarDriver, Driver>. When using multiple includes, the exposed type uses the base type (CarDriver) and the final Included type (Driver). This method will be used by the public methods of the repo.

internal IIncludableQueryable<CarDriver, Driver> BuildBaseQuery()
=> Table.Include(c => c.CarNavigation).Include(d=>d.DriverNavigation);

Override the GetAll(), GetAllIgnoreQueryFilters(), and Find() methods to utilize the internal method:

public override IEnumerable GetAll()
=> BuildBaseQuery();

public override IEnumerable GetAllIgnoreQueryFilters()
=> BuildBaseQuery().IgnoreQueryFilters();

public override CarDriver Find(int? id)
=> BuildBaseQuery().IgnoreQueryFilters().Where(x => x.Id == id).FirstOrDefault();

The Car Repository
Open the CarRepo.cs file and change the class to public, inherit from TemporalTableBaseRepo, and implement ICarRepo and the standard constructors:

namespace AutoLot.Dal.Repos;
public class CarRepo : TemporalTableBaseRepo, ICarRepo
{
public CarRepo(ApplicationDbContext context) : base(context)
{
}
internal CarRepo(DbContextOptions options) : base(options)
{
}
//remaining implementation code goes here
}

Next, create an internal method that includes the MakeNavigation and an OrderBy() for the PetName
property. Note that the type returned is an IOrderedQueryable. This will be used by the public methods:

internal IOrderedQueryable BuildBaseQuery()
=> Table.Include(x => x.MakeNavigation).OrderBy(p=>p.PetName);

Add overrides for GetAll() and GetAllIgnoreQueryFilters(), using the base query, to include the
MakeNavigation property and order by the PetName values.

public override IEnumerable GetAll()
=> BuildBaseQuery();
public override IEnumerable GetAllIgnoreQueryFilters()
=> BuildBaseQuery().IgnoreQueryFilters();

Implement the GetAllBy() method. This method gets all the Inventory records with the specified MakeId:

public IEnumerable GetAllBy(int makeId)
=> BuildBaseQuery().Where(x => x.MakeId == makeId);

Add an override for Find() to include the MakeNavigation property and ignore query filters.

public override Car Find(int? id)
=> Table
.IgnoreQueryFilters()
.Where(x => x.Id == id)
.Include(m => m.MakeNavigation)
.FirstOrDefault();

Finally, add the method to get a car’s PetName value using the stored procedure. This uses the
ExecuteParameterizedQuery() method of the base repo and returns the value of the OUT parameter.

public string GetPetName(int id)
{
var parameterId = new SqlParameter
{
ParameterName = "@carId", SqlDbType = SqlDbType.Int, Value = id,
};

var parameterName = new SqlParameter
{
ParameterName = "@petName", SqlDbType = SqlDbType.NVarChar, Size = 50,
Direction = ParameterDirection.Output
};

ExecuteParameterizedQuery("EXEC [dbo].[GetPetName] @carId, @petName OUTPUT", new[] {parameterId, parameterName});
return (string)parameterName.Value;
}

The CreditRisk Repository
Open the CreditRiskRepo.cs file and change the class to public, inherit from BaseRepo, implement ICreditRiskRepo, and add the two required constructors.

namespace AutoLot.Dal.Repos;
public class CreditRiskRepo : BaseRepo, ICreditRiskRepo
{
public CreditRiskRepo(ApplicationDbContext context) : base(context)
{
}
internal CreditRiskRepo(DbContextOptions options)
: base(options)
{
}
}

The CustomerOrderViewModel Repository
Open the CustomerOrderViewModelRepo.cs file and change the class to public, inherit from
BaseViewRepo, implement ICreditRiskRepo, and add the two required constructors.

namespace AutoLot.Dal.Repos;

public class CustomerOrderViewModelRepo
: BaseViewRepo, ICustomerOrderViewModelRepo
{
public CustomerOrderViewModelRepo(ApplicationDbContext context) : base(context)
{
}
internal CustomerOrderViewModelRepo(DbContextOptions options)
: base(options)
{
}
}

The Customer Repository
Open the CustomerRepo.cs file and change the class to public, inherit from BaseRepo, implement ICustomerRepo, and add the two required constructors.

namespace AutoLot.Dal.Repos;

public class CustomerRepo : BaseRepo, ICustomerRepo
{
public CustomerRepo(ApplicationDbContext context)
: base(context)
{
}
internal CustomerRepo(DbContextOptions options)

: base(options)
{
}
}

The final step is to add the method that returns all Customer records with their orders sorted by
LastName. Add the following method to the class:

public override IEnumerable GetAll()
=> Table
.Include(c => c.Orders)
.OrderBy(o => o.PersonInformation.LastName);

The Driver Repository
Open the DriverRepo.cs file and change the class to public, inherit from BaseRepo, implement
IDriverRepo, and add the two constructors.

namespace AutoLot.Dal.Repos;

public class DriverRepo : BaseRepo, IDriverRepo
{
public DriverRepo(ApplicationDbContext context) : base(context)
{
}
internal DriverRepo(DbContextOptions options) : base(options)
{
}
//remaining implementation code goes here
}

Next, create an internal method that includes the order by the LastName and then FirstName properties of the PersonInformation-owned class. Note that the return type is IOrderedQueryable. This will be used by the public functions:

internal IOrderedQueryable BuildQuery()
=> Table
.OrderBy(m => m.PersonInformation.LastName)
.ThenBy(f => f.PersonInformation.FirstName);

Override the GetAll() and GetAllIgnoreQueryFilters() methods to utilize the internal method:
public override IEnumerable GetAll() => BuildQuery(); public override IEnumerable GetAllIgnoreQueryFilters()
=> BuildQuery().IgnoreQueryFilters();

The Make Repository
Open the MakeRepo.cs class and change the class to public, inherit from TemporalTableBaseRepo, implement IMakeRepo, and add the two required constructors.

namespace AutoLot.Dal.Repos;

public class MakeRepo : TemporalTableBaseRepo, IMakeRepo
{
public MakeRepo(ApplicationDbContext context)
: base(context)
{
}

internal MakeRepo( DbContextOptions options)
: base(options)
{
}
}

Next, create an internal method that includes the order by the Name property. Note that the return type is IOrderedQueryable:

internal IOrderedQueryable BuildQuery()
=> Table.OrderBy(m => m. Name);

The final methods to override are the GetAll() and GetAllIgnoreQueryFilters() methods, sorting the
Make values by name.
public override IEnumerable GetAll() => BuildQuery(); public override IEnumerable GetAllIgnoreQueryFilters()
=> BuildQuery().IgnoreQueryFilters();

The Order Repository
Open the OrderRepo.cs class and change the class to public, inherit from TemporalTableBaseRepo, implement IOrderRepo, and add the two constructors:

namespace AutoLot.Dal.Repos;

public class OrderRepo : TemporalTableBaseRepo, IOrderRepo
{
public OrderRepo(ApplicationDbContext context)
: base(context)
{
}

internal OrderRepo( DbContextOptions options)

: base(options)
{
}
}

The Radio Repository
Open the RadioRepo.cs file and change the class to public, inherit from TemporalTableBaseRepo, implement ICreditRiskRepo, and add the two required constructors.

namespace AutoLot.Dal.Repos;

public class RadioRepo : TemporalTableBaseRepo, IRadioRepo
{
public RadioRepo(ApplicationDbContext context) : base(context)
{
}
internal RadioRepo(DbContextOptions options)
: base(options)
{
}
}

That completes all the repositories. The next section will create the code to drop, create, seed, and clear the database.

Update the GlobalUsings.cs Files
The new repository namespaces need to be added into the GlobalUsings.cs file in the AutoLot.Dal project. Add the following global using statements to the file:

//AutoLot.Dal
global using AutoLot.Dal.Repos.Base; global using AutoLot.Dal.Repos.Interfaces;

Programmatic Database and Migration Handling
The Database property of DbContext provides programmatic methods to drop and create the database as well as run all the migrations. Table 23-1 describes the methods related to these operations.

Table 23-1. Programmatically Working with the Database

Member of
Database Meaning in Life
EnsureDeleted() Drops the database if it exists. Does nothing if it does not exist.
EnsureCreated() Creates the database if it doesn’t exist. Does nothing if it does. Creates the tables and columns based on the classes reachable from the DbSet properties. Does not apply any migrations. Note: This should not be used in conjunction with migrations.
Migrate() Creates the database if it doesn’t exist. Applies all migrations to the database.

As mentioned in the table, the EnsureCreated() method will create the database if it doesn’t exist and then creates the tables, columns, and indices based on the entity model. It does not apply any migrations. If you are using migrations (as we are), this will present errors when working with the database, and you will have to trick out EF Core (as we did earlier) to believe the migrations have been applied. You will also have to apply any custom SQL objects to the database manually. When you are working with migrations, always use the Migrate() method to programmatically create the database and not the EnsureCreated() method.

Drop, Create, and Clean the Database
During development, it can be beneficial to drop and re-create the development database and then seed it with sample data. This creates a stable and predictable database setup useful when testing (manual or automated). Create a new folder named Initialization in the AutoLot.Dal project. In this folder, create a new class named SampleDataInitializer.cs. Make the class public and static as shown here:

namespace AutoLot.Dal.Initialization; public static class SampleDataInitializer
{
//implementation goes here
}

Create a method named DropAndCreateDatabase that takes an instance of ApplicationDbContext as the single parameter. This method uses the Database property of ApplicationDbContext to first delete the database (using the EnsureDeleted() method) and then creates the database (using the Migrate() method).

internal static void DropAndCreateDatabase(ApplicationDbContext context)
{
context.Database.EnsureDeleted(); context.Database.Migrate();
}

■Note This process works very well when you are using a local database (e.g., in a docker container, on your local drive, etc.). This does not work when using sQL azure, as the eF Core commands cannot create sQL azure database instances. If you are using sQL azure, use the ClearData() method instead, detailed next.

Create another method named ClearData() that deletes all the data in the database and resets the identity values for each table’s primary key. The method loops through a list of domain entities and uses the DbContext Model property to get the schema and table name each entity is mapped to. Then it executes a delete statement and resets the identity for each table using the ExecuteSqlRaw() method on the DbContext Database property. If the table is temporal, it then clears out the history data.

internal static void ClearData(ApplicationDbContext context)
{
var entities = new[]
{
typeof(Order).FullName, typeof(Customer).FullName, typeof(CarDriver).FullName, typeof(Driver).FullName, typeof(Radio).FullName, typeof(Car).FullName, typeof(Make).FullName, typeof(CreditRisk).FullName
};
var serviceCollection = new ServiceCollection(); serviceCollection.AddDbContextDesignTimeServices(context);
var serviceProvider = serviceCollection.BuildServiceProvider(); var designTimeModel = serviceProvider.GetService(); foreach (var entityName in entities)
{
var entity = context.Model.FindEntityType(entityName); var tableName = entity.GetTableName();
var schemaName = entity.GetSchema(); context.Database.ExecuteSqlRaw($"DELETE FROM {schemaName}.{tableName}");
context.Database.ExecuteSqlRaw($"DBCC CHECKIDENT (\"{schemaName}.{tableName}\", RESEED, 1);");
if (entity.IsTemporal())
{
var strategy = context.Database.CreateExecutionStrategy(); strategy.Execute(() =>
{
using var trans = context.Database.BeginTransaction();
var designTimeEntity = designTimeModel.FindEntityType(entityName); var historySchema = designTimeEntity.GetHistoryTableSchema();
var historyTable = designTimeEntity.GetHistoryTableName(); context.Database.ExecuteSqlRaw(
$"ALTER TABLE {schemaName}.{tableName} SET (SYSTEMVERSIONING = OFF)");
context.Database.ExecuteSqlRaw($"DELETE FROM {historySchema}.{historyTable}"); context.Database.ExecuteSqlRaw($"ALTER TABLE {schemaName}.{tableName} SET (SYSTEM
VERSIONING = ON (HISTORY_TABLE={historySchema}.{historyTable}))");
trans.Commit();
});
}
}
}

■Note The ExecuteSqlRaw() method of the database façade should be used carefully to prevent potential sQL injection attacks.

Now that you can drop and create the database and clear the data, it’s time to create the methods that will add the sample data.

Data Initialization
We are going to build our own data seeding system that can be run on demand. The first step is to create the sample data and then add the methods into the SampleDataInitializer used to load the sample data into the database.

Create the Sample Data
Add a new file named SampleData.cs to the Initialization folder. Make the class public and static: namespace AutoLot.Dal.Initialization;
public static class SampleData
{
//implementation goes here
}

The file consists of eight static methods that create the sample data.

public static List Customers => new()
{
new() { Id = 1, PersonInformation = new() { FirstName = "Dave", LastName = "Brenner" } }, new() { Id = 2, PersonInformation = new() { FirstName = "Matt", LastName = "Walton" } }, new() { Id = 3, PersonInformation = new() { FirstName = "Steve", LastName = "Hagen" } }, new() { Id = 4, PersonInformation = new() { FirstName = "Pat", LastName = "Walton" } }, new() { Id = 5, PersonInformation = new() { FirstName = "Bad", LastName = "Customer" } },
};

public static List Makes => new()
{
new() { Id = 1, Name = "VW" },
new() { Id = 2, Name = "Ford" },
new() { Id = 3, Name = "Saab" },
new() { Id = 4, Name = "Yugo" },
new() { Id = 5, Name = "BMW" },
new() { Id = 6, Name = "Pinto" },
};

public static List Drivers => new()
{
new() { Id = 1, PersonInformation = new() { FirstName = "Fred", LastName = "Flinstone" } }, new() { Id = 2, PersonInformation = new() { FirstName = "Barney", LastName = "Rubble" } }
};

public static List Inventory => new()
{
new() { Id = 1, MakeId = 1, Color = "Black", PetName = "Zippy" },
new() { Id = 2, MakeId = 2, Color = "Rust", PetName = "Rusty" },
new() { Id = 3, MakeId = 3, Color = "Black", PetName = "Mel" },
new() { Id = 4, MakeId = 4, Color = "Yellow", PetName = "Clunker" },
new() { Id = 5, MakeId = 5, Color = "Black", PetName = "Bimmer" },
new() { Id = 6, MakeId = 5, Color = "Green", PetName = "Hank" },
new() { Id = 7, MakeId = 5, Color = "Pink", PetName = "Pinky" },
new() { Id = 8, MakeId = 6, Color = "Black", PetName = "Pete" },
new() { Id = 9, MakeId = 4, Color = "Brown", PetName = "Brownie" },
new() { Id = 10, MakeId = 1, Color = "Rust", PetName = "Lemon", IsDrivable = false },
};

public static List Radios => new()
{
new() { Id = 1, CarId = 1, HasSubWoofers = true, RadioId = "SuperRadio 1", HasTweeters = true }, new() { Id = 2, CarId = 2, HasSubWoofers = true, RadioId = "SuperRadio 2", HasTweeters = true }, new() { Id = 3, CarId = 3, HasSubWoofers = true, RadioId = "SuperRadio 3", HasTweeters = true }, new() { Id = 4, CarId = 4, HasSubWoofers = true, RadioId = "SuperRadio 4", HasTweeters = true }, new() { Id = 5, CarId = 5, HasSubWoofers = true, RadioId = "SuperRadio 5", HasTweeters = true }, new() { Id = 6, CarId = 6, HasSubWoofers = true, RadioId = "SuperRadio 6", HasTweeters = true }, new() { Id = 7, CarId = 7, HasSubWoofers = true, RadioId = "SuperRadio 7", HasTweeters = true }, new() { Id = 8, CarId = 8, HasSubWoofers = true, RadioId = "SuperRadio 8", HasTweeters = true }, new() { Id = 9, CarId = 9, HasSubWoofers = true, RadioId = "SuperRadio 9", HasTweeters = true }, new() { Id = 10, CarId = 10, HasSubWoofers = true, RadioId = "SuperRadio 10", HasTweeters = true },
};

public static List CarsAndDrivers => new()
{
new() { Id = 1, CarId = 1, DriverId = 1 }, new() { Id = 2, CarId = 2, DriverId = 2 }
};

public static List Orders => new()
{
new() { Id = 1, CustomerId = 1, CarId = 5 }, new() { Id = 2, CustomerId = 2, CarId = 1 }, new() { Id = 3, CustomerId = 3, CarId = 4 }, new() { Id = 4, CustomerId = 4, CarId = 7 }, new() { Id = 5, CustomerId = 5, CarId = 10 },
};

public static List CreditRisks => new()
{
new()
{
Id = 1,
CustomerId = Customers[4].Id, PersonInformation = new()

{
FirstName = Customers[4].PersonInformation.FirstName, LastName = Customers[4].PersonInformation.LastName
}
}
};

Load the Sample Data
The internal SeedData() method in the SampleDataInitializer class adds the data from the SampleData
methods into an instance of ApplicationDbContext and then persists the data to the database.

internal static void SeedData(ApplicationDbContext context)
{
try
{
ProcessInsert(context, context.Customers, SampleData.Customers); ProcessInsert(context, context.Makes, SampleData.Makes); ProcessInsert(context, context.Drivers, SampleData.Drivers); ProcessInsert(context, context.Cars, SampleData.Inventory); ProcessInsert(context, context.Radios, SampleData.Radios); ProcessInsert(context, context.CarsToDrivers, SampleData.CarsAndDrivers); ProcessInsert(context, context.Orders, SampleData.Orders); ProcessInsert(context, context.CreditRisks, SampleData.CreditRisks);
}
catch (Exception ex)
{
Console.WriteLine(ex);
//Set a break point here to determine what the issues is throw;
}
static void ProcessInsert( ApplicationDbContext context, DbSet table,
List records) where TEntity : BaseEntity
{
if (table.Any())
{
return;
}
IExecutionStrategy strategy = context.Database.CreateExecutionStrategy(); strategy.Execute(() =>
{
using var transaction = context.Database.BeginTransaction(); try
{
var metaData = context.Model.FindEntityType(typeof(TEntity).FullName); context.Database.ExecuteSqlRaw(
$"SET IDENTITY_INSERT {metaData.GetSchema()}.{metaData.GetTableName()} ON"); table.AddRange(records);
context.SaveChanges();

context.Database.ExecuteSqlRaw(
$"SET IDENTITY_INSERT {metaData.GetSchema()}.{metaData.GetTableName()} OFF"); transaction.Commit();
}
catch (Exception)
{
transaction.Rollback();
}
});
}
}

The SeedData() method uses a local function to process the data. It first checks to see whether the table has any records and, if not, proceeds to process the sample data. An ExecutionStrategy is created from the database façade, and this is used to create an explicit transaction, which is needed to turn identity insert on and off. The records are added, and if all is successful, the transaction is committed; otherwise, it’s rolled back.
The next two methods are public and used to reset the database. InitializeData() drops and re- creates the database before seeding it, and the ClearDatabase() method just deletes all the records, resets the identity, and then seeds the data.

public static void InitializeData(ApplicationDbContext context)
{
DropAndCreateDatabase(context); SeedData(context);
}

public static void ClearAndReseedDatabase(ApplicationDbContext context)
{
ClearData(context);
SeedData(context);
}

The initialization code will be exercised heavily in the next chapter.

Summary
This chapter used the knowledge gained in the previous chapter to complete the data access layer for the AutoLot database. You used the EF Core command-line tools to scaffold an existing database, updated the model to its final version, and then created migrations and applied them. Repositories were added for the encapsulation of the data access, and database initialization code with sample data can drop and create the database in a repeatable, reliable manner. The next chapter focuses on test-driving the data access layer.

Pro C#10 CHAPTER 22 Exploring Entity Framework Core

CHAPTER 22

Exploring Entity Framework Core

The previous chapter covered the components of EF Core. This chapter will dive into the capabilities of EF Core, starting with create, read, update, and delete (CRUD) operations. After covering the CRUD operations, specific EF Core features are explored, including global query filters, mixing SQL queries with LINQ, projections, and more.

■ Note This chapter’s code picks up where the previous chapter left off, so you can continue to use the code from Chapter 21 if you have been coding along with the text. If you are starting with this chapter and want to code along with the text, start with the previous chapter’s code from the repo.

To get started, clear out the Program.cs file and add the following Console.WriteLine(): Console.WriteLine(" More Fun with Entity Framework Core ");
Creating Records
Records are added to the database by creating them in code, adding them to their DbSet, and calling SaveChanges()/SaveChangesAsync() on the context. When SaveChanges() is executed, the ChangeTracker reports all the added entities, and EF Core (along with the database provider) creates the appropriate SQL statement(s) to insert the record(s). You saw examples of this earlier in this chapter when the sample records were added into the database.
As a reminder from Chapter 21, SaveChanges() executes in an implicit transaction, unless an explicit transaction is used. If the save was successful, the server-generated values are then queried to set the values on the entities. The handling of server-generated values is covered in depth later in this chapter.
All of the SQL statements shown in this section were collected using SQL Server Profiler.

■ Note Records can also be added using the derived DbContext. These examples will all use the DbSet collection properties to add the records. Both DbSet and DbContext have async versions of Add()/AddRange(). Only the synchronous versions are shown.

© Andrew Troelsen, Phil Japikse 2022
A. Troelsen and P. Japikse, Pro C# 10 with .NET 6, https://doi.org/10.1007/978-1-4842-7869-7_22

921

Entity State
When an entity is created through code but not yet added to a DbSet, the EntityState is Detached. Once a new entity is added to a DbSet, the EntityState is set to Added. After SaveChanges() executes successfully, the EntityState is set to Unchanged.
The following code shows a newly created Make record and its EntityState:
static void AddRecords()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var newMake = new Make
{
Name = "BMW"
};
Console.WriteLine($"State of the {newMake.Name} is {context.Entry(newMake).State}");
}
After calling the AddRecords() method from the top-level statements, you should see the
following output:

More Fun with Entity Framework Core State of the BMW is Detached

Add a Single Record Using Add
To add a new Make record to the database, create a new entity instance and call the Add() method of the appropriate DbSet. To trigger the persistence of the data, SaveChanges() of the derived DbContext class must also be called. The following code adds the new Make record to the database:

static void AddRecords()
{
...
context.Makes.Add(newMake);
Console.WriteLine($"State of the {newMake.Name} is {context.Entry(newMake).State}"); context.SaveChanges();
Console.WriteLine($"State of the {newMake.Name} is {context.Entry(newMake).State}");
}
Running the program again, you will see the following output in the console window. After the entity
was added into the Change Tracker (using the Add() method), the state was changed to Added. The message about saving changes comes from the SavingChanges event handler, and the message “Saved 1 entities” came from the SavedChanges event handler. After SaveChanges() is called on the context, the entity’s state is changed to Unchanged:

More Fun with Entity Framework Core State of the BMW is Detached
State of the BMW is Added
Saving changes for server=.,5433;Database=AutoLotSamples;User Id=sa;Password=P@ssw0rd; Saved 1 entities
State of the BMW is Unchanged

The executed SQL statement for the insert is shown here. The format of the query is due to the batching process used by EF Core to improve the performance of database operations. Batching is covered later in this chapter. All of the values passed into the SQL statement are parameterized to help reduce the threat of scripting attacks. Also notice that the recently added entity is queried for the database-generated properties. EF Core’s handling of server-managed values is also covered later in this chapter.

exec sp_executesql N'SET NOCOUNT ON; INSERT INTO [Makes] ([Name]) VALUES (@p0);

SELECT [Id], [TimeStamp] FROM [Makes]
WHERE @@ROWCOUNT = 1 AND [Id] = scope_identity(); ',N'@p0 nvarchar(50)',@p0=N'BMW'

Add a Single Record Using Attach
When an entity’s primary key is mapped to an identity column in SQL Server, EF Core will treat that entity instance as Added when added to the ChangeTracker if the primary key property’s value is zero. The following code adds the new Car record using the Attach() method instead of the Add() method. Note that SaveChanges() must still be called for the data to be persisted.

static void AddRecords()
{
...
var newCar = new Car()
{
Color = "Blue",
DateBuilt = new DateTime(2016, 12, 01), IsDrivable = true,
PetName = "Bluesmobile", MakeId = newMake.Id
};
Console.WriteLine($"State of the {newCar.PetName} is {context.Entry(newCar).State}"); context.Cars.Attach(newCar);
Console.WriteLine($"State of the {newCar.PetName} is {context.Entry(newCar).State}"); context.SaveChanges();
Console.WriteLine($"State of the {newCar.PetName} is {context.Entry(newCar).State}");
}

Running the program again, you will see the following same progression of states as you saw with the
Make entity:

More Fun with Entity Framework Core
...
State of the Bluesmobile is Detached State of the Bluesmobile is Added
Saving changes for server=.,5433;Database=AutoLotSamples;User Id=sa;Password=P@ssw0rd; Saved 1 entities
State of the Bluesmobile is Unchanged

The executed SQL statement for the insert is shown here:

exec sp_executesql N'SET NOCOUNT ON;
INSERT INTO [dbo].[Inventory] ([Color], [DateBuilt], [IsDrivable], [MakeId], [PetName]) VALUES (@p0, @p1, @p2, @p3, @p4);

SELECT [Id], [Display], [TimeStamp] FROM [dbo].[Inventory]
WHERE @@ROWCOUNT = 1 AND [Id] = scope_identity();

',N'@p0 nvarchar(50),@p1 datetime2(7),@p2 bit,@p3 int,@p4 nvarchar(50)',@p0=N'Blue', @p1='2016-12-01 00:00:00',@p2=1,@p3=1,@p4=N'Bluesmobile'

Add Multiple Records at Once
To insert multiple records in a single transaction, use the AddRange() method of a DbSet property, as shown in the following example:

static void AddRecords()
{
...
var cars = new List
{
new() { Color = "Yellow", MakeId = newMake.Id, PetName = "Herbie" }, new() { Color = "White", MakeId = newMake.Id, PetName = "Mach 5" }, new() { Color = "Pink", MakeId = newMake.Id, PetName = "Avon" }, new() { Color = "Blue", MakeId = newMake.Id, PetName = "Blueberry" },
};
context.Cars.AddRange(cars); context.SaveChanges();
}

Even though four records were added, EF Core generated only one SQL statement for the inserts. The SQL statement for the inserts is shown here:

exec sp_executesql N'SET NOCOUNT ON;
DECLARE @inserted0 TABLE ([Id] int, [_Position] [int]); MERGE [dbo].[Inventory] USING (
VALUES (@p0, @p1, @p2, 0),

(@p3, @p4, @p5, 1),
(@p6, @p7, @p8, 2),
(@p9, @p10, @p11, 3)) AS i ([Color], [MakeId], [PetName], _Position) ON 1=0 WHEN NOT MATCHED THEN
INSERT ([Color], [MakeId], [PetName]) VALUES (i.[Color], i.[MakeId], i.[PetName]) OUTPUT INSERTED.[Id], i._Position
INTO @inserted0;

SELECT [t].[Id], [t].[DateBuilt], [t].[Display], [t].[IsDrivable], [t].[TimeStamp] FROM [dbo].[Inventory] t
INNER JOIN @inserted0 i ON ([t].[Id] = [i].[Id]) ORDER BY [i].[_Position];

',N'@p0 nvarchar(50),@p1 int,@p2 nvarchar(50),@p3 nvarchar(50),@p4 int,@p5 nvarchar(50), @p6 nvarchar(50),@p7 int,@p8 nvarchar(50),@p9 nvarchar(50),@p10 int,@p11 nvarchar(50)', @p0=N'Yellow',@p1=1,@p2=N'Herbie',@p3=N'White',@p4=1,@p5=N'Mach 5',@p6=N'Pink',@p7=1, @p8=N'Avon',@p9=N'Blue',@p10=1,@p11=N'Blueberry'

Identity Column Considerations When Adding Records
When an entity has a numeric property that is defined as the primary key, that property (by default) gets mapped to an Identity column in SQL Server. EF Core considers any entity with the default value (zero) for the key property to be new, and any entity with a nondefault value to already exist in the database. If you create a new entity and set the primary key property to a nonzero number and attempt to add it to the database, EF Core will fail to add the record because identity insert is not enabled.
For SQL Server, identity insert is enabled by issuing the SET IDENTITY_INSERT command within an explicit transaction. This command requires the database schema and table name, not the C# namespace and entity name. To get the database information for an entity, use the FindEntityType() method of
the Model property for the derived DbContext. Once you have the EntityType, use the GetSchema() and
GetTableName() methods:

static void AddRecords()
{
...
IEntityType metadata = context.Model.FindEntityType(typeof(Car).FullName); var schema = metadata.GetSchema();
var tableName = metadata.GetTableName();
}

Recall from the previous chapter that when using an ExecutionStrategy, explicit transactions must execute within the scope of that strategy. For reference, here is the following sample from the previous chapter:

var strategy = context.Database.CreateExecutionStrategy(); strategy.Execute(() =>
{
using var trans = context.Database.BeginTransaction(); try
{
//actionToExecute(); trans.Commit();

Console.WriteLine($"Insert succeeded");
}
catch (Exception ex)
{

}
});

trans.Rollback();
Console.WriteLine($"Insert failed: {ex.Message}");

When adding a record using identity insert, the actionToExecute() placeholder in the previous code block is replaced by code to turn identity insert on, add the record(s), and then save the changes. If that all succeeds, the transaction is committed. If any part of it fails, the transaction is rolled back. In the finally block, identity insert is turned back off.
EF Core provides two methods to execute commands directly against the database. The ExecuteSqlRaw() method executes the string exactly as it is written, while ExecuteSqlInterpolated() uses C# string interpolation to create a parameterized query. For this example, use the ExecuteSqlRaw() version. Here is the updated code, with the new lines in bold:

var strategy = context.Database.CreateExecutionStrategy(); strategy.Execute(() =>
{
using var trans = context.Database.BeginTransaction(); try
{
context.Database.ExecuteSqlRaw(
$"SET IDENTITY_INSERT {schema}.{tableName} ON"); var anotherNewCar = new Car()
{
Id = 27,
Color = "Blue",
DateBuilt = new DateTime(2016, 12, 01), IsDrivable = true,
PetName = "Bluesmobile", MakeId = newMake.Id
};
context.Cars.Add(anotherNewCar); context.SaveChanges(); trans.Commit();
Console.WriteLine($"Insert succeeded");
}
catch (Exception ex)
{
trans.Rollback();
Console.WriteLine($"Insert failed: {ex.Message}");
}
finally
{

}
});

context.Database.ExecuteSqlRaw(
$"SET IDENTITY_INSERT {schema}.{tableName} OFF");

■Note when using known values, as in this example, the ExecuteSqlRaw() method is safe to use. however, if you are collecting inputs from users, you should use the ExecuteSqlInterpolated() version for added protection.

The previous code executed the following commands against the database:

SET IDENTITY_INSERT dbo.Inventory ON SAVE TRANSACTION [ EFSavePoint];
exec sp_executesql N'SET NOCOUNT ON;
INSERT INTO [dbo].[Inventory] ([Id], [Color], [DateBuilt], [IsDrivable], [MakeId], [PetName])
VALUES (@p0, @p1, @p2, @p3, @p4, @p5); SELECT [Display], [TimeStamp]
FROM [dbo].[Inventory]
WHERE @@ROWCOUNT = 1 AND [Id] = @p0;

',N'@p0 int,@p1 nvarchar(50),@p2 datetime2(7),@p3 bit,@p4 int,@p5 nvarchar(50)',@p0=27, @p1=N'Blue',@p2='2016-12-01 00:00:00',@p3=1,@p4=1,@p5=N'Bluesmobile'
SET IDENTITY_INSERT dbo.Inventory OFF

Adding an Object Graph
When adding an entity to the database, child records can be added in the same call without specifically adding them into their own DbSet. This is accomplished by adding them into the collection navigation property for the parent record. For example, a new Make entity is created, and a child Car record is added to the Cars property on the Make. When the Make entity is added to the DbSet property, EF Core automatically starts tracking the child Car record as well, without having to add it into the DbSet property explicitly. Executing SaveChanges() saves the Make and Car together. The following test demonstrates this:

static void AddRecords()
{
...
var anotherMake = new Make {Name = "Honda"};
var car = new Car { Color = "Yellow", PetName = "Herbie" };
//Cast the Cars property to List from IEnumerable ((List) anotherMake.Cars).Add(car); context.Makes.Add(make);
context.SaveChanges();
}

The executed SQL statements are shown here:

exec sp_executesql N'SET NOCOUNT ON; INSERT INTO [Makes] ([Name])
VALUES (@p0);
SELECT [Id], [TimeStamp] FROM [Makes]

WHERE @@ROWCOUNT = 1 AND [Id] = scope_identity(); ',N'@p0 nvarchar(50)',@p0=N'Honda'
exec sp_executesql N'SET NOCOUNT ON;
INSERT INTO [dbo].[Inventory] ([Color], [MakeId], [PetName]) VALUES (@p1, @p2, @p3);
SELECT [Id], [DateBuilt], [Display], [IsDrivable], [TimeStamp] FROM [dbo].[Inventory]
WHERE @@ROWCOUNT = 1 AND [Id] = scope_identity();

',N'@p1 nvarchar(50),@p2 int,@p3 nvarchar(50)',@p1=N'Yellow',@p2=2,@p3=N'Herbie'

Notice how EF Core retrieved the Id for the new Make record and automatically included that in the insert statement for the Car record.

Add Many-to-Many Records
With the new EF Core support for many-to-many tables, records can be added directly from one entity to the other without going through the pivot table. Now you can write the following code to add Driver records directly to the Car records:

//M2M
var drivers = new List
{
new() { PersonInfo = new Person { FirstName = "Fred", LastName = "Flinstone" } }, new() { PersonInfo = new Person { FirstName = "Wilma", LastName = "Flinstone" } }, new() { PersonInfo = new Person { FirstName = "BamBam", LastName = "Flinstone" } }, new() { PersonInfo = new Person { FirstName = "Barney", LastName = "Rubble" } }, new() { PersonInfo = new Person { FirstName = "Betty", LastName = "Rubble" } }, new() { PersonInfo = new Person { FirstName = "Pebbles", LastName = "Rubble" } },
};
var carsForM2M = context.Cars.Take(2).ToList();
//Cast the IEnumerable to a List to access the Add method
//Range support works with LINQ to Objects, but is not translatable to SQL calls ((List)carsForM2M[0].Drivers).AddRange(drivers.Take(..3)); ((List)carsForM2M[1].Drivers).AddRange(drivers.Take(3..)); context.SaveChanges();

When the SaveChanges() method is executed, two insert statements are executed. The first inserts the six Driver records into the Drivers table, and then the second inserts the six records into the InventoryDriver table (the pivot table). Here is the insert statement for the pivot table:

exec sp_executesql N'SET NOCOUNT ON;
DECLARE @inserted0 TABLE ([InventoryId] int, [DriverId] int, [_Position] [int]); MERGE [dbo].[InventoryToDrivers] USING (
VALUES (@p12, @p13, 0), (@p14, @p15, 1),
(@p16, @p17, 2),
(@p18, @p19, 3),
(@p20, @p21, 4),

(@p22, @p23, 5)) AS i ([InventoryId], [DriverId], _Position) ON 1=0 WHEN NOT MATCHED THEN
INSERT ([InventoryId], [DriverId]) VALUES (i.[InventoryId], i.[DriverId])
OUTPUT INSERTED.[InventoryId], INSERTED.[DriverId], i._Position INTO @inserted0;

SELECT [t].[Id], [t].[TimeStamp] FROM [dbo].[InventoryToDrivers] t
INNER JOIN @inserted0 i ON ([t].[InventoryId] = [i].[InventoryId]) AND ([t].[DriverId] = [i].[DriverId])
ORDER BY [i].[_Position];

',N'@p12 int,@p13 int,@p14 int,@p15 int,@p16 int,@p17 int,@p18 int,@p19 int,@p20 int,@p21 int,@p22 int,@p23 int',@p12=1,@p13=1,@p14=1,@p15=2,@p16=1,@p17=3,@p18=2,@p19=4,@p20=2, @p21=5,@p22=2,@p23=6

This is a much better experience than past versions of EF Core when using many-to-many relationships, where you had to manage the pivot table yourself.

Add Sample Records
The final step is to add a series of Make and Car records for the query examples covered in the next section. The method creates several Make and Car entities and adds them to the database.
Start by creating the Make entities and add them to the Makes DbSet property of the derived ApplicationDbContext, and then call the SaveChanges() method. Repeat this process for the Car records, using the Cars DbSet property:

static void LoadMakeAndCarData()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); List makes = new()
{
new() { Name = "VW" },
new() { Name = "Ford" },
new() { Name = "Saab" },
new() { Name = "Yugo" },
new() { Name = "BMW" },
new() { Name = "Pinto" },
};
context.Makes.AddRange(makes); context.SaveChanges(); List inventory = new()
{
new() { MakeId = 1, Color = "Black", PetName = "Zippy" }, new() { MakeId = 2, Color = "Rust", PetName = "Rusty" }, new() { MakeId = 3, Color = "Black", PetName = "Mel" }, new() { MakeId = 4, Color = "Yellow", PetName = "Clunker" }, new() { MakeId = 5, Color = "Black", PetName = "Bimmer" }, new() { MakeId = 5, Color = "Green", PetName = "Hank" }, new() { MakeId = 5, Color = "Pink", PetName = "Pinky" },

new() { MakeId = 6, Color = "Black", PetName = "Pete" }, new() { MakeId = 4, Color = "Brown", PetName = "Brownie" },
new() { MakeId = 1, Color = "Rust", PetName = "Lemon", IsDrivable = false },
};
context.Cars.AddRange(inventory); context.SaveChanges();
}

Clear the Sample Data
Later in this chapter, deleting records will be covered in depth. For now, we will create a method that clears out the sample data so when the examples are run multiple times, the previous executions don’t interfere with the examples.
Create a new method called ClearSampleData(). The method uses the FindEntityType() method on the Model property of the ApplicationDbContext to get the table and schema name and then deletes the records. After the records are deleted, the code uses the DBCC CHECKIDENT command to reset the identity for each table.

static void ClearSampleData()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var entities = new[]
{
typeof(Driver).FullName, typeof(Car).FullName, typeof(Make).FullName,
};
foreach (var entityName in entities)
{
var entity = context.Model.FindEntityType(entityName); var tableName = entity.GetTableName();
var schemaName = entity.GetSchema(); context.Database.ExecuteSqlRaw($"DELETE FROM {schemaName}.{tableName}");
context.Database.ExecuteSqlRaw($"DBCC CHECKIDENT (\"{schemaName}.{tableName}\", RESEED, 0);");
}
}

Add a call to this method at the beginning of the top-level statements to reset the database each time the program is executed. Also add a call after the AddRecords() method to clean up the examples that add individual records:

Console.WriteLine(" More Fun with Entity Framework Core "); ClearSampleData();
AddRecords();
ClearSampleData();
LoadMakeAndCarData();

Querying Data
Querying data using EF Core is typically accomplished using LINQ queries. As a reminder, when using LINQ to query the database for a list of entities, the query isn’t executed until the query is iterated over, converted to a List (or an array), or bound to a list control (like a data grid). For single-record queries, the statement is executed immediately when the single record call (First(), Single(), etc.) is used.

■Note This book is not a complete lInQ reference but shows just a few examples. For more examples of lInQ queries, microsoft has published 101 lInQ samples at https://code.msdn.microsoft.com/101- LINQ-Samples-3fb9811b.

New in EF Core 5, you can call the ToQueryString() method in most LINQ queries to examine the query that gets executed against the database. The main exception is any immediate execution queries, such as First()/FirstOrDefault(). For split queries, the ToQueryString() method returns only the first query that will be executed.

Get All Records
To get all the records for a table, simply use the DbSet property directly without any LINQ statements. For immediate execution, add ToList() to the DbSet property. Add the following method to the Program.cs file:

static void QueryData()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null);
//Return all of the cars IQueryable cars = context.Cars; foreach (Car c in cars)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
//Clean up the context context.ChangeTracker.Clear();
List cars2 = context.Cars.ToList(); foreach (Car c in cars2)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
}

Notice that the type returned is an IQueryable when using the DbSet, and the return type is
List when using the ToList() method.

Filter Records
The Where() method is used to filter records from the DbSet. Multiple Where() methods can be fluently chained to dynamically build the query. Chained Where() methods are always combined as and clauses in the created query. In the following example, the generated query for cars2 and cars3 are identical. To create an or statement, you must use the same Where() clause.

static void FilterData()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null);
//Return all yellow cars
IQueryable cars = context.Cars.Where(c=>c.Color == "Yellow");
Console.WriteLine("Yellow cars"); foreach (Car c in cars)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
context.ChangeTracker.Clear();

//Return all yellow cars with a petname of Clunker
IQueryable cars2 = context.Cars.Where(c => c.Color == "Yellow" && c.PetName == "Clunker");
Console.WriteLine("Yellow cars and Clunkers"); foreach (Car c in cars2)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
context.ChangeTracker.Clear();

//Return all yellow cars with a petname of Clunker
IQueryable cars3 = context.Cars.Where(c=>c.Color == "Yellow").Where(c=>c.PetName == "Clunker");
Console.WriteLine("Yellow cars and Clunkers"); foreach (Car c in cars3)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
context.ChangeTracker.Clear();

//Return all yellow cars or cars with PetName of Clunker
IQueryable cars4 = context.Cars.Where(c=>c.Color == "Yellow" || c.PetName == "Clunker");
Console.WriteLine("Yellow cars or Clunkers"); foreach (Car c in cars4)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
context.ChangeTracker.Clear();
}

Notice that the type returned is also an IQueryable when using a Where clause.

One improvement in EF Core 6 is the handling of converting string.IsNullOrWhiteSpace() into SQL. Examine the added code to the end of the FilterData() method:

static void FilterData()
{
...
IQueryable cars5 = context.Cars.Where(c => !string.IsNullOrWhiteSpace(c.Color)); Console.WriteLine("Cars with colors");
foreach (Car c in cars5)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
context.ChangeTracker.Clear();
}

Prior to EF Core 6, the resulting query was a mix of LTRIM/RTRIM commands. With the improvements in EF Core 6, the executed query is much cleaner:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i]. [MakeId], [i].[PetName], [i].[TimeStamp]
FROM [dbo].[Inventory] AS [i] WHERE [i].[Color] <> N''

Sort Records
The OrderBy() and OrderByDescending() methods set the sort(s) for the query in either ascending or descending order, respectively. If subsequent sorts are required, use the ThenBy() and/or the ThenByDescending() methods.

static void SortData()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null);
//Return all cars ordered by color
IOrderedQueryable cars = context.Cars.OrderBy(c=>c.Color); Console.WriteLine("Cars ordered by Color");
foreach (Car c in cars)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
context.ChangeTracker.Clear();
//Return all cars ordered by color then Petname
IOrderedQueryable cars1 = context.Cars.OrderBy(c=>c.Color).ThenBy(c=>c.PetName); Console.WriteLine("Cars ordered by Color then PetName");
foreach (Car c in cars1)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
context.ChangeTracker.Clear();
//Return all cars ordered by color Descending
IOrderedQueryable cars2 = context.Cars.OrderByDescending(c=>c.Color);

Console.WriteLine("Cars ordered by Color descending"); foreach (Car c in cars2)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
context.ChangeTracker.Clear();
}
Notice that the datatype returned from a LINQ query with an OrderBy()/OrderByDescending() is
IOrderedQueryable.
Ordering by ascending and descending can be intermixed, as is shown here:

static void SortData()
{
...
//Return all cars ordered by Color then by PetName descending
IOrderedQueryable cars3 = context.Cars.OrderBy(c=>c.Color).ThenByDescending(c=> c.PetName);
Console.WriteLine("Cars ordered by Color then by PetName descending"); foreach (Car c in cars3)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
context.ChangeTracker.Clear();
}

Reverse Sort Records
The Reverse() method reverses the entire sort order, as shown here:

static void SortData()
{
...
//Return all cars ordered by Color then Petname in reverse
IQueryable cars1 = context.Cars.OrderBy(c=>c.Color).ThenBy(c=>c.PetName).Reverse(); Console.WriteLine("Cars ordered by Color then PetName in reverse");
foreach (Car c in cars1)
{
Console.WriteLine($"{c.PetName} is {c.Color}");
}
context.ChangeTracker.Clear();
}

Notice that the datatype returned from a LINQ query with a Reverse() clause is IQueryable, and not IOrderedQueryable.
The preceding LINQ query gets translated into the following:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp] FROM [dbo].[Inventory] AS [i]
ORDER BY [i].[Color] DESC, [i].[PetName] DESC

Paging
EF Core provides paging capabilities using Skip() and Take(). Skip() skips the specified number of records while Take() retrieves the specified number of records.
Using the Skip() method with SQL Server executes a query with an OFFSET command. The OFFSET command is the SQL Server version of skipping records that would normally be returned from the query. Add the following method to the Program.cs file:

static void Paging()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); Console.WriteLine("Paging");
//Skip the first two records
var cars = context.Cars.Skip(2).ToList();
}

■Note The SQl Server OFFSET command has decreasing performance the more records are skipped. most applications probably won’t be using eF Core (or any ORm) with massive amounts of data, but make sure you performance test any calls that use Skip(). If there is a performance issue, it might be better to drop down to a FromSqlRaw()/FromSqlInterpolated() to optimize the query.

The example code skips the first two records and returns the rest. The slightly edited (for readability) query is shown here:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp] FROM [dbo].[Inventory] AS [i]
ORDER BY (SELECT 1) OFFSET 2 ROWS

Notice that the generated query adds an ORDER BY clause, even though the LINQ statement did not have any ordering. This is because the SQL Server OFFSET command cannot be used without an ORDER BY clause.
The Take() method generates a SQL Server query that uses the TOP command. The following addition to the Paging() method uses the Clear() method on the ChangeTracker to reset the ApplicationDbContext and then uses the Take() method to return two records.

static void Paging()
{
...
context.ChangeTracker.Clear();
//Take the first two records
cars = context.Cars.Take(2).ToList();
}

The executed query is shown here:

SELECT TOP(2) [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp] FROM [dbo].[Inventory] AS [i]

■ Note Recall from Chapter 13 that with .neT 6/C#10, the Take() method can take a range. This capability is not supported in eF Core.

Combing the Skip() and Take() method enables paging of the data. For example, if you have a page size of two (due to our small database size) and you need to get the second page, execute the following LINQ query:

static void Paging()
{
...
//Skip the first two records and take the next two records cars = context.Cars.Skip(2).Take(2).ToList();
}

When combining Skip() and Take(), SQL Server doesn’t use the TOP command, but another version of the OFFSET command, as shown here:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp]
FROM [dbo].[Inventory] AS [i] ORDER BY (SELECT 1)
OFFSET 2 ROWS FETCH NEXT 2 ROWS ONLY

Retrieve a Single Record
There are three main methods (with OrDefault variants) for returning a single record with a query: First()/FirstOrDefault(), Last()/LastOrDefault(), and Single()/SingleOrDefault(). While all three return a single record, their approaches all differ. The three methods and their variants are detailed here:
•First() returns the first record that matches the query condition and any ordering clauses. If no ordering is specified, then the record returned is based on database order. If no record is returned, an exception is thrown.
•The FirstOrDefault() behavior matches First() except that if no records match the query, the method returns the default value for the type (null).
•Single() returns the first record that matches the query condition and any ordering clauses. If no ordering is specified, then the record returned is based on database order. If no records or more than one record matches the query, then an exception is thrown.
•The SingleOrDefault() behavior matches Single() except that if no records match the query, the method returns the default value for the type (null).

•Last() returns the last record that matches the query condition and any ordering clauses. If no ordering is specified, an exception is thrown. If no record is returned, an exception is thrown.
•The LastOrDefault() behavior matches Last() except that if no records match the query, the method returns the default value for the type (null).
All the methods can also take an Expression<Func<T, bool>> to filter the result set. This means you can place the Where() expression inside the call for the First()/Single() methods as long as there is only a single Where() clause. The following statements are equivalent:

Context.Cars.Where(c=>c.Id < 5).First(); Context.Cars.First(c=>c.Id < 5);

Using First
When using the parameterless form of First() and FirstOrDefault(), the first record (based on database order or any preceding ordering clauses) will be returned. The following example gets the first record based on database order:

static void SingleRecordQueries()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); Console.WriteLine("Single Record with database Sort");
var firstCar = context.Cars.First(); Console.WriteLine($"{firstCar.PetName} is {firstCar.Color}"); context.ChangeTracker.Clear();
}

The preceding LINQ query gets translated into the following:

SELECT TOP(1) [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp] FROM [dbo].[Inventory] AS [i]

The following code gets the first record based on Color order:

static void SingleRecordQueries()
{
...
Console.WriteLine("Single Record with OrderBy sort");
var firstCarByColor = context.Cars.OrderBy(c => c.Color).First(); Console.WriteLine($"{firstCarByColor.PetName} is {firstCarByColor.Color}"); context.ChangeTracker.Clear();
}

The preceding LINQ query gets translated into the following:

SELECT TOP(1) [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp]

FROM [dbo].[Inventory] AS [i] ORDER BY [i].[Color]

The following code shows First() being used with a Where() clause and then using First() as the
Where() clause:

static void SingleRecordQueries()
{
...
Console.WriteLine("Single Record with Where clause");
var firstCarIdThree = context.Cars.Where(c => c.Id == 3).First(); Console.WriteLine($"{firstCarIdThree.PetName} is {firstCarIdThree.Color}"); context.ChangeTracker.Clear();

Console.WriteLine("Single Record Using First as Where clause"); var firstCarIdThree1 = context.Cars.First(c => c.Id == 3);
Console.WriteLine($"{firstCarIdThree1.PetName} is {firstCarIdThree1.Color}"); context.ChangeTracker.Clear();
}

Both preceding statements get translated into the following SQL:

SELECT TOP(1) [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp] FROM [dbo].[Inventory] AS [i]
WHERE [i].[Id] = 3
The following example shows that an exception is thrown if there isn’t a match when using First(): static void SingleRecordQueries()
{
...
Console.WriteLine("Exception when no record is found"); try
{
var firstCarNotFound = context.Cars.First(c => c.Id == 27);
}
catch (InvalidOperationException ex)
{
Console.WriteLine(ex.Message);
}
context.ChangeTracker.Clear();
}

The preceding LINQ query gets translated into the following:

SELECT TOP(1) [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp] FROM [dbo].[Inventory] AS [i]
WHERE [i].[Id] = 27

When using FirstOrDefault(), instead of an exception, the result is a null when no data is returned.

static void SingleRecordQueries()
{
...
Console.WriteLine("Return Default (null) when no record is found"); var firstCarNotFound = context.Cars.FirstOrDefault(c => c.Id == 27); Console.WriteLine(firstCarNotFound == true); context.ChangeTracker.Clear();
}

The preceding LINQ query gets translated into the same SQL as the previous example.

■ Note also recall from Chapter 13 that with .neT 6/C#10, the OrDefault() methods can specify a default value when nothing is returned from the query. Unfortunately, this capability is also not supported in eF Core.

Using Last
When using the parameterless form of Last() and LastOrDefault(), the last record (based on any preceding ordering clauses) will be returned. When using Last(), the LINQ query must have an OrderBy()/ OrderByDescending() clause or an InvalidOperationException will be thrown:

static void SingleRecordQueries()
{
...
Console.WriteLine("Exception with Last and no order by"); try
{
context.Cars.Last();
}
catch (InvalidOperationException ex)
{
Console.WriteLine(ex.Message);
}
}

The following test gets the last record based on Color order:

static void SingleRecordQueries()
{
...
Console.WriteLine("Get last record sorted by Color"); var lastCar = context.Cars.OrderBy(c=>c.Color).Last(); Console.WriteLine(firstCarNotFoundDefault == null); context.ChangeTracker.Clear();
}

EF Core reverses the ORDER BY statements and then takes the top(1) to get the result. Here is the executed query:

SELECT TOP(1) [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp] FROM [dbo].[Inventory] AS [i]
ORDER BY [i].[Color] DESC

Using Single
Conceptually, Single()/SingleOrDefault() works the same as First()/FirstOrDefault(). The main difference is that Single()/SingleOrDefault() returns Top(2) instead of Top(1) and throws an exception if two records are returned from the database.
The following tests retrieves the single record where Id == 1:

static void SingleRecordQueries()
{
...
Console.WriteLine("Get single record");
var singleCar = context.Cars.Single(c=>c.Id == 3); context.ChangeTracker.Clear();
}

The preceding LINQ query gets translated into the following:

SELECT TOP(2) [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp] FROM [dbo].[Inventory] AS [i]
WHERE [i].[Id] = 3

Single() throws an exception if no records are returned or more than one record is returned:

static void SingleRecordQueries()
{
...
Console.WriteLine("Exception when more than one record is found"); try
{
context.Cars.Single(c => c.Id > 1);
}
catch (InvalidOperationException ex)
{
Console.WriteLine(ex.Message);
}
context.ChangeTracker.Clear();

Console.WriteLine("Exception when no records are found"); try
{
context.Cars.Single(c => c.Id == 27);
}

catch (InvalidOperationException ex)
{
Console.WriteLine(ex.Message);
}
context.ChangeTracker.Clear();}
}

When using SingleOrDefault(), an exception is also thrown if more than one record is returned:

static void SingleRecordQueries()
{
...
Console.WriteLine("Exception when more than one record is found"); try
{
context.Cars.SingleOrDefault(c => c.Id > 1);
}
catch (InvalidOperationException ex)
{
Console.WriteLine(ex.Message);
}
context.ChangeTracker.Clear();
}

When using SingleOrDefault(), instead of an exception, the result is null when no data is returned.

static void SingleRecordQueries()
{
...
var defaultWhenSingleNotFoundCar = context.Cars.SingleOrDefault(c => c.Id == 27); context.ChangeTracker.Clear();
}

Using Find
The Find() method also returns a single record but behaves a little differently than the other single record methods. The Find() method’s parameter(s) represent the primary key(s) of the entity. It then looks in the ChangeTracker for an instance of the entity with the matching primary key and returns it if it’s found. If not, it will then make a call to the database to retrieve the record.

static void SingleRecordQueries()
{
...
var foundCar = context.Cars.Find(27); context.ChangeTracker.Clear();
}

If the entity has a compound primary key, then pass in the values representing the compound key:

var item = context.MyClassWithCompoundKey.Find(27,3);

Aggregation Methods
EF Core also supports server-side aggregate methods (Max(), Min(), Count(), and Average()). All aggregate methods can be used in conjunction with Where() methods and return a single value. The aggregation queries execute on the server side. Global query filters affect aggregate methods as well and can be disabled with IgnoreQueryFilters(). Global query filters are covered later in this chapter.
Note that each of the aggregate methods is a terminating function. In other words, they end the LINQ statement when executed since each method returns a single numeric value. Query execution also happens immediately, like the single record methods discussed earlier.
All of the SQL statements shown in this section were collected using SQL Server Profiler as the
ToQueryString() method doesn’t work with aggregation.
This first example counts all the Car records in the database.
static void Aggregation()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var count = context.Cars.Count();
}
The executed SQL is shown here:

SELECT COUNT(*)
FROM [dbo].[Inventory] AS [i]

The Count() method can contain the filter expression, just like First() and Single(). The following examples demonstrate the Count() method with a where condition. The first adds the expression directly into the Count() method, and the second adds the Count() method to the end of the LINQ statement after the Where() method.

static void Aggregation()
{
...
var countByMake = context.Cars.Count(x=>x.MakeId == 1); Console.WriteLine($"Count: {countByMake}");
var countByMake2 = context.Cars.Where(x=>x.MakeId == 1).Count(); Console.WriteLine($"Count: {countByMake2}");
}

Both lines of code create the same SQL calls to the server, as shown here:

SELECT COUNT(*)
FROM [dbo].[Inventory] AS [i] WHERE [i].[MakeId] = 1

The next examples show Min(), Max(), and Average(). Each method takes an expression to specify the property that is being operated on:

static void Aggregation()
{
...
var max = context.Cars.Max(x => x.Id);

var min = context.Cars.Min(x => x.Id);
var avg = context.Cars.Average(x => x.Id); Console.WriteLine($"Max ID: {max} Min ID: {min} Ave ID: {avg}");}
--Generated SQL
SELECT MAX([i].[Id]) FROM [dbo].[Inventory] AS [i]
SELECT MIN([i].[Id]) FROM [dbo].[Inventory] AS [i]
SELECT AVG(CAST([i].[Id] AS float)) FROM [dbo].[Inventory] AS [i]

Any( ) and All( )
The Any() and All() methods check a set of records to see whether any records match the criteria (Any()) or whether all records match the criteria (All()). Just like the aggregation methods, the Any() method (but not the All() method) can be added to the end of a LINQ query with Where() methods, or the filter expression can be contained in the method itself. Any() and All() methods execute on the server side, and a Boolean is returned from the query. Both are terminating functions. Global query filters affect Any() and All() methods functions as well and can be disabled with IgnoreQueryFilters().
The ToQueryString() method doesn’t work with the Any()/All() functions either, so all of the SQL statements shown in this section were collected using SQL Server Profiler.
This first sample checks if any car records have a MakeId of 1.

static void AnyAndAll()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var resultAny = context.Cars.Any(x => x.MakeId == 1);
//This executes the same query as the preceding line
var resultAnyWithWhere = context.Cars.IgnoreQueryFilters().Where(x => x.MakeId == 1).Any();

Console.WriteLine($"Exist? {resultAny}"); Console.WriteLine($"Exist? {resultAnyWithWhere}");
}

The executed SQL for the first example is shown here:

SELECT CASE WHEN EXISTS (
SELECT 1
FROM [dbo].[Inventory] AS [i]
WHERE [i].[MakeId] = 1) THEN CAST(1 AS bit) ELSE CAST(0 AS bit)
END
This second example checks if all car records have a specific MakeId. static void AnyAndAll()
{
...
var resultAll = context.Cars.All(x => x.MakeId == 1); Console.WriteLine($"All? {resultAll}");
}

The executed SQL for the first theory test is shown here:

SELECT CASE
WHEN NOT EXISTS ( SELECT 1
FROM [dbo].[Inventory] AS [i]
WHERE [i].[MakeId] <> 1) THEN CAST(1 AS bit) ELSE CAST(0 AS bit)
END

Getting Data from Stored Procedures
The final data retrieval pattern to examine is getting data from stored procedures. While there are some gaps in EF Core in relation to stored procedures (compared to EF 6), remember that EF Core is built on top of ADO.NET. We just need to drop down a layer and remember how we called stored procedures pre-ORM.
The first step is to create the stored procedure in our database:

CREATE PROCEDURE [dbo].[GetPetName] @carID int,
@petName nvarchar(50) output
AS
SELECT @petName = PetName from dbo.Inventory where Id = @carID

The following method creates the required parameters (input and output), leverages the
ApplicationDbContext Database property, and calls ExecuteSqlRaw():

static string GetPetName(ApplicationDbContext context, int id)
{
var parameterId = new SqlParameter
{
ParameterName = "@carId",
SqlDbType = System.Data.SqlDbType.Int, Value = id,
};

var parameterName = new SqlParameter
{
ParameterName = "@petName",
SqlDbType = System.Data.SqlDbType.NVarChar, Size = 50,
Direction = ParameterDirection.Output
};

var result = context.Database
.ExecuteSqlRaw("EXEC [dbo].[GetPetName] @carId, @petName OUTPUT",parameterId, parameterName);
return (string)parameterName.Value;
}

The next step is to get the Car records (to get the Id values for each of the Car records), loop though them, and use the stored procedure to get their PetName:

static void GetDataFromStoredProc()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var cars = context.Cars.IgnoreQueryFilters().ToList();
foreach (var c in cars)
{
Console.WriteLine($"PetName: {GetPetName(context,c.Id)}");
}
}

■ Note This example is somewhat contrived, as getting all of the Car records also gets the PetName properties for the records. getting all of the records is a convenient way to demonstrate calling the stored procedure multiple times.

When the code is run, EF Core executes the following SQL for each Car in the list (only one is shown here):

declare @p4 nvarchar(50) set @p4=N'Hank'
exec sp_executesql N'EXEC [dbo].[GetPetName] @carId, @petName OUTPUT',
N'@carId int,@petName nvarchar(50) output',@carId=1,@petName=@p4 output select @p4

Querying Related Data
Entity navigation properties are used to load an entity’s related data. The related data can be loaded eagerly (one LINQ statement, one SQL query), eagerly with split queries (one LINQ statement, multiple SQL queries), explicitly (multiple LINQ calls, multiple SQL queries), or lazily (one LINQ statement, multiple on- demand SQL queries).
In addition to the ability to load related data using the navigation properties, EF Core will automatically fix up entities as they are loaded into the Change Tracker. For example, assume all the Make records are loaded into the DbSet collection property. Next, all the Car records are loaded into DbSet. Even though the records were loaded separately, they will be accessible to each other through the navigation properties.

Eager Loading
Eager loading is the term for loading related records from multiple tables in one database call. This is analogous to creating a query in T-SQL linking two or more tables with joins. When entities have navigation properties and those properties are used in the LINQ queries, the translation engine uses joins to get data from the related tables and loads the corresponding entities. This is usually much more efficient than executing one query to get the data from one table and then running additional queries for each of the related tables. For those times when it is less efficient to use one query, EF Core 5 introduced query splitting, covered next.

The Include() and ThenInclude() (for subsequent navigation properties) methods are used to traverse the navigation properties in LINQ queries. If the relationship is required, the LINQ translation engine will create an inner join. If the relationship is optional, the translation engine will create a left join.
For example, to load all the Car records with their related Make information, execute the following LINQ query:

static void RelatedData()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var carsWithMakes = context.Cars.Include(c => c.MakeNavigation).ToList(); context.ChangeTracker.Clear();
}

The previous LINQ executes the following query against the database:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp],
[m].[Id], [m].[Name], [m].[TimeStamp] FROM [Dbo].[Inventory] AS [i]
INNER JOIN [dbo].[Makes] AS [m] ON [i].[MakeId] = [m].[Id]

■ Note The SELECT statement is returning all the fields for the Inventory and Makes tables. eF Core then wires up the data correctly, returning the correct object graph.

The MakeNavigation property is a required relationship since the MakeId property of the Car entity is non-nullable. Because it’s required, the Make table is joined to the Inventory table with an INNER JOIN. If the navigation property was optional (MakeId was defined with a nullable int), the join would be an OUTER JOIN. The next example demonstrates optional relationships in the queries generated.
Multiple Include() statements can be used in the same query to join more than one entity to the original. To work down the navigation property tree, use ThenInclude() after an Include(). For example, to get all the Make records with their related Car records and the Driver records for the Cars, use the following statement:

var makesWithCarsAndDrivers = context.Makes.Include(c=>c.Cars).ThenInclude(d=>d.Drivers). ToList();

■ Note The call to Clear() on the ChangeTracker is added to make sure that previous code examples don’t interfere with the results of the code being discussed.

The previous LINQ executes the following query against the database:

SELECT [m].[Id], [m].[Name], [m].[TimeStamp], [t0].[Id], [t0].[Color], [t0].[DateBuilt],
[t0].[Display], [t0].[IsDrivable], [t0].[MakeId], [t0].[PetName], [t0].[TimeStamp], [t0].[CarId],
[t0].[DriverId], [t0].[Id0], [t0].[FirstName], [t0].[LastName], [t0].[TimeStamp0] FROM [Makes] AS [m]

LEFT JOIN (
SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i]. [MakeId],
[i].[PetName], [i].[TimeStamp], [t].[CarId], [t].[DriverId], [t].[Id] AS [Id0], [t].[FirstName],
[t].[LastName], [t].[TimeStamp] AS [TimeStamp0] FROM [dbo].[Inventory] AS [i]
LEFT JOIN (
SELECT .[CarId], .[DriverId], [d].[Id], [d].[FirstName], [d].[LastName], [d]. [TimeStamp]
FROM [CarDriver] AS
INNER JOIN [Drivers] AS [d] ON .[DriverId] = [d].[Id]
) AS [t] ON [i].[Id] = [t].[CarId]
) AS [t0] ON [m].[Id] = [t0].[MakeId]
ORDER BY [m].[Id], [t0].[Id], [t0].[CarId], [t0].[DriverId], [t0].[Id0]

One thing that might seem odd is the ORDER BY clause, since the LINQ query did not include any ordering. When using chained includes (with the Include()/ThenInclude() statements), the LINQ translation engine will add an ORDER BY clause based on the order of the tables included and their primary and foreign keys. This is in addition to any ordering you specified in the LINQ query.
Take the following updated example:

var orderedMakes = context.Makes.Include(c => c.Cars).ThenInclude(d => d.Drivers). OrderBy(d=>d.Name).ToList();

The SQL produced will be sorted by any (and all) ordering clauses in the LINQ query, suffixed with the autogenerated ORDER BY clauses:

SELECT [m].[Id], [m].[Name], [m].[TimeStamp], [t0].[Id], [t0].[Color], [t0].[DateBuilt],
[t0].[Display], [t0].[IsDrivable], [t0].[MakeId], [t0].[PetName], [t0].[TimeStamp], [t0].[CarId],
[t0].[DriverId], [t0].[Id0], [t0].[FirstName], [t0].[LastName], [t0].[TimeStamp0] FROM [Makes] AS [m]
LEFT JOIN (
SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i]. [MakeId],
[i].[PetName], [i].[TimeStamp], [t].[CarId], [t].[DriverId], [t].[Id] AS [Id0], [t].[FirstName],
[t].[LastName], [t].[TimeStamp] AS [TimeStamp0] FROM [dbo].[Inventory] AS [i]
LEFT JOIN (
SELECT .[CarId], .[DriverId], [d].[Id], [d].[FirstName], [d].[LastName], [d]. [TimeStamp]
FROM [CarDriver] AS
INNER JOIN [Drivers] AS [d] ON .[DriverId] = [d].[Id]
) AS [t] ON [i].[Id] = [t].[CarId]
) AS [t0] ON [m].[Id] = [t0].[MakeId]
ORDER BY [m].[Name], [m].[Id], [t0].[Id], [t0].[CarId], [t0].[DriverId], [t0].[Id0]

Filtered Include
Introduced in EF Core 5, the included data can be filtered and sorted. The allowable operations on the collection navigation are Where(), OrderBy(), OrderByDescending(), ThenBy(), ThenByDescending(), Skip(), and Take(). For example, if you want to get all Make records, but only the related Car records where the color is yellow, you filter the navigation property in the lambda expression, like this:

//Add to keep the demos clean context.ChangeTracker.Clear();
var makesWithYellowCars = context.Makes
.Include(x => x.Cars.Where(x=>x.Color == "Yellow"))
.ToList();

The query that is executed is as follows:

SELECT [m].[Id], [m].[Name], [m].[TimeStamp], [t].[Id], [t].[Color], [t].[DateBuilt],
[t].[Display], [t].[IsDrivable], [t].[MakeId], [t].[PetName], [t].[TimeStamp] FROM [Makes] AS [m]
LEFT JOIN (
SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp]
FROM [dbo].[Inventory] AS [i] WHERE [i].[Color] = N'Yellow'
) AS [t] ON [m].[Id] = [t].[MakeId]
ORDER BY [m].[Id], [t].[Id]

Eager Loading with Split Queries
When a LINQ query contains a lot of includes, there can be a negative performance impact. To resolve this situation, EF Core 5 introduced split queries. Instead of executing a single query, EF Core will split the LINQ query into multiple SQL queries and then wire up all the related data. For example, the previous query can be expected as multiple SQL queries by adding AsSplitQuery() into the LINQ query, like this:

//Add to keep the demos clean context.ChangeTracker.Clear();
var splitMakes = context.Makes.AsSplitQuery()
.Include(x => x.Cars.Where(x=>x.Color == "Yellow")).ToList();

The queries that are executed are shown here:

SELECT [m].[Id], [m].[Name], [m].[TimeStamp] FROM [dbo].[Makes] AS [m]
ORDER BY [m].[Id]

SELECT [t].[Id], [t].[Color], [t].[DateBuilt], [t].[Display], [t].[IsDrivable],
[t].[MakeId], [t].[PetName], [t].[TimeStamp], [m].[Id] FROM [Makes] AS [m]
INNER JOIN (
SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i]. [MakeId],

[i].[PetName], [i].[TimeStamp] FROM [dbo].[Inventory] AS [i]
WHERE [i].[Color] = N'Yellow'
) AS [t] ON [m].[Id] = [t].[MakeId] ORDER BY [m].[Id]

There is a downside to using split queries: if the data changes between executing the queries, then the data returned will be inconsistent.

Many-to-Many Queries
The new EF Core support for designing many-to-many tables carries over to querying data with LINQ. Prior to the many-to-many support, queries would have to go through the pivot table. Now you can write the following LINQ statement to get the Car and related Driver records:

var carsAndDrivers = context.Cars.Include(x => x.Drivers).Where(x=>x.Drivers.Any());

As you can see from the generated SQL select statement, EF Core takes care of working through the pivot table to get the Car and Driver records matched up correctly:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDeleted], [i].
[IsDrivable], [i].[MakeId], [i].[PetName], [i].[Price], [i].[TimeStamp], [i].[ValidFrom],
[i].[ValidTo], [t].[InventoryId], [t].[DriverId], [t].[Id], [t].[TimeStamp], [t].[Id0], [t]. [TimeStamp0], [t].[FirstName], [t].[LastName]
FROM [dbo].[Inventory] AS [i] LEFT JOIN (
SELECT [i1].[InventoryId], [i1].[DriverId], [i1].[Id], [i1].[TimeStamp], [d0].[Id] AS
[Id0], [d0].[TimeStamp] AS [TimeStamp0], [d0].[FirstName], [d0].[LastName] FROM [dbo].[InventoryToDrivers] AS [i1]
INNER JOIN [dbo].[Drivers] AS [d0] ON [i1].[DriverId] = [d0].[Id]
) AS [t] ON [i].[Id] = [t].[InventoryId]
WHERE ([i].[IsDrivable] = CAST(1 AS bit)) AND EXISTS ( SELECT 1
FROM [dbo].[InventoryToDrivers] AS [i0]
INNER JOIN [dbo].[Drivers] AS [d] ON [i0].[DriverId] = [d].[Id] WHERE [i].[Id] = [i0].[InventoryId])
ORDER BY [i].[Id], [t].[InventoryId], [t].[DriverId]

Explicit Loading
Explicit loading is loading data along a navigation property after the core object is already loaded. This process involves executing an additional database call to get the related data. This can be useful if your application selectively needs to get the related records instead of always pulling them.
The process starts with an entity that is already loaded and uses the Entry() method on the derived DbContext. When querying against a reference navigation property (e.g., getting the Make information for a car), use the Reference() method. When querying against a collection navigation property, use the
Collection() method. The query is deferred until Load(), ToList(), or an aggregate function (e.g., Count(), Max()) is executed.

The following examples show how to get the related Make data as well as any Drivers for a Car record:

//Get the Car record
var car = Context.Cars.First(x => x.Id == 1);
//Get the Make information
context.Entry(car).Reference(c => c.MakeNavigation).Load();
//Get any Drivers the Car is related to context.Entry(car).Collection(c => c.Drivers).Query().Load();

The previous statements generate the following queries:

//Get the Car record
SELECT TOP(1) [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp]
FROM [dbo].[Inventory] AS [i] WHERE [i].[Id] = 1
//Get the Make information
SELECT [m].[Id], [m].[Name], [m].[TimeStamp] FROM [Makes] AS [m]
WHERE [m].[Id] = 5
//Get any Drivers the Car is related to
SELECT [t].[Id], [t].[FirstName], [t].[LastName], [t].[TimeStamp], [i].[Id], [t].[CarId], [t].[DriverId],
[t0].[CarId], [t0].[DriverId], [t0].[Id], [t0].[Color], [t0].[DateBuilt], [t0].[Display], [t0].[IsDrivable], [t0].[MakeId], [t0].[PetName], [t0].[TimeStamp]
FROM [dbo].[Inventory] AS [i] INNER JOIN (
SELECT [d].[Id], [d].[FirstName], [d].[LastName], [d].[TimeStamp], .[CarId], . [DriverId]
FROM [CarDriver] AS
INNER JOIN [Drivers] AS [d] ON .[DriverId] = [d].[Id]
) AS [t] ON [i].[Id] = [t].[CarId] LEFT JOIN (
SELECT [c0].[CarId], [c0].[DriverId], [t1].[Id], [t1].[Color], [t1].[DateBuilt], [t1]. [Display],
[t1].[IsDrivable], [t1].[MakeId], [t1].[PetName], [t1].[TimeStamp] FROM [CarDriver] AS [c0]
INNER JOIN (
SELECT [i0].[Id], [i0].[Color], [i0].[DateBuilt], [i0].[Display], [i0].[IsDrivable], [i0].[MakeId], [i0].[PetName], [i0].[TimeStamp]
FROM [dbo].[Inventory] AS [i0]
WHERE [i0].[IsDrivable] = CAST(1 AS bit)
) AS [t1] ON [c0].[CarId] = [t1].[Id] WHERE [t1].[Id] = 1
) AS [t0] ON [t].[Id] = [t0].[DriverId] WHERE [i].[Id] = 1
ORDER BY [i].[Id], [t].[CarId], [t].[DriverId], [t].[Id], [t0].[CarId], [t0].[DriverId], [t0].[Id]

As you can see, that third and final query is doing a lot of work to simply get the Driver records for the selected Car record. This shows us two important facts: 1) If you can write it all on one query using eager loading, it is usually better to do so, which negates the need to go back to the database to get the related

records, and 2) EF Core doesn’t always create the best queries. I’ve already shown you how to use eager loading in the previous section. Later in this chapter, I will show you how to use SQL statements with or without additional LINQ statements to retrieve data from the database. This is useful when EF Core creates suboptimal queries.

Lazy Loading
Lazy loading is loading a record on-demand when a navigation property is used to access a related record that is not yet loaded into memory. Lazy loading is a feature from EF 6 that was added back into EF Core with version 2.1. While it might sound like a good idea to turn this on, enabling lazy loading can cause performance problems in your application by making potentially unnecessary round-trips to your database.
Lazy loading can be useful in smart client (WPF, WinForms) applications but is recommended to not be used in web or service applications. For this reason, lazy loading is off by default in EF Core (it was enabled by default in EF 6).
To use lazy loading, the navigation properties to be lazily loaded must be marked as virtual. This is because the navigation properties are wrapped with a proxy. This proxy will then have EF Core make a call to the database if the navigation property has not been loaded when it is referenced in your application.
To use lazy loading with proxies, the derived DbContext must be correctly configured. Start by adding the Microsoft.EntityFrameworkCore.Proxies package to your project. You must then opt in to using
the lazy loading proxies in the derived DbContext options. While this would normally be set in your application code when configuring your derived DbContext, we are going to opt in to the proxies using the ApplicationDbContextFactory class that we built earlier. Remember that this class is meant for design- time use and shouldn’t be used in your application code. However, for learning and exploring, it will work just fine.
Open the ApplicationDbContextFactory.cs file and navigate to the CreateDbContext() method. We will take advantage of the args parameter to indicate that we want the method to return a derived DbContext configured to use the lazy loading proxies. Update the CreateDbContext() method to the following:

public ApplicationDbContext CreateDbContext(string[] args)
{
var optionsBuilder = new DbContextOptionsBuilder();
var connectionString = @"server=.,5433;Database=AutoLotSamples;User Id=sa;Password=P @ssw0rd;";
if (args != null && args.Length == 1 && args[0].Equals("lazy", StringComparison. OrdinalIgnoreCase))
{
optionsBuilder = optionsBuilder.UseLazyLoadingProxies();
}
optionsBuilder = optionsBuilder.UseSqlServer(connectionString); Console.WriteLine(connectionString);
return new ApplicationDbContext(optionsBuilder.Options);
}

Next, update the Car class to the following:

[Table("Inventory", Schema = "dbo")] [Index(nameof(MakeId), Name = "IX_Inventory_MakeId")] [EntityTypeConfiguration(typeof(CarConfiguration))] public class Car : BaseEntity
{
private string _color;

[Required, StringLength(50)] public string Color
{
get => _color;
set => _color = value;
}
private bool? _isDrivable;

public bool IsDrivable
{
get => _isDrivable ?? true; set => _isDrivable = value;
}
[DatabaseGenerated(DatabaseGeneratedOption.Computed)] public string Display { get; set; }
public DateTime? DateBuilt { get; set; } [Required, StringLength(50)]
public string PetName { get; set; } public int MakeId { get; set; } [ForeignKey(nameof(MakeId))]
public virtual Make MakeNavigation { get; set; } public virtual Radio RadioNavigation { get; set; }

[InverseProperty(nameof(Driver.Cars))]
public virtual IEnumerable Drivers { get; set; } = new List();

[InverseProperty(nameof(CarDriver.CarNavigation))]
public virtual IEnumerable CarDrivers { get; set; } = new List();
}

Now that the properties are marked virtual, they can be used with lazy loading. Add the following method to your Program.cs file (notice that we are not using the args parameter of the CreateDbContext() method yet) and call the method from your top-level statements:

static void LazyLoadCar()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null);

var query = context.Cars.AsQueryable(); var cars = query.ToList();
var make = cars[0].MakeNavigation; Console.WriteLine(make.Name);
}

When you run this sample, you will receive a null reference exception when trying to access the Name property of the Make instance. This is because the Make record wasn’t loaded, and we are not using the proxy-enabled version of the derived DbContext(). Update the method to pass in the “lazy” argument to CreateDbContext(), which enables the lazy loading proxy support:

var context = new ApplicationDbContextFactory().CreateDbContext(new string[] {"lazy"});

When you run the code again, you may be surprised to receive an InvalidOperationException. When using lazy proxies, all navigation properties on the models must be marked as virtual, even ones not directly involved in the executing code block. So far, we have updated only the Car entity. Update the rest of the models to the following (updates in bold):

//CarDriver
[Table("InventoryToDrivers", Schema = "dbo")] public class CarDriver : BaseEntity
{
public int DriverId {get;set;} [ForeignKey(nameof(DriverId))]
public virtual Driver DriverNavigation {get;set;}

[Column("InventoryId")] public int CarId {get;set;} [ForeignKey(nameof(CarId))]
public virtual Car CarNavigation {get;set;}
}

//Driver.cs
[Table("Drivers", Schema = "dbo")] [EntityTypeConfiguration(typeof(DriverConfiguration))] public class Driver : BaseEntity
{
public Person PersonInfo { get; set; } = new Person(); [InverseProperty(nameof(Car.Drivers))]
public virtual IEnumerable Cars { get; set; } = new List(); [InverseProperty(nameof(CarDriver.DriverNavigation))]
public virtual IEnumerable CarDrivers { get; set; } = new List();
}
//Make.cs
[Table("Makes", Schema = "dbo")] public class Make : BaseEntity
{
...
[InverseProperty(nameof(Car.MakeNavigation))]
public virtual IEnumerable Cars { get; set; } = new List();
}
//Radio.cs
[Table("Radios", Schema = "dbo")] [EntityTypeConfiguration(typeof(RadioConfiguration))] public class Radio : BaseEntity
{
...
[ForeignKey(nameof(CarId))]
public virtual Car CarNavigation { get; set; }
}

■ Note even though the Owned Person class represents a relationship, it is not a navigation property, and properties that are an Owned type do not need to be marked virtual.

Now, when you run the program again, the Make of the car will be printed to the console. When watching the SQL Server activity in profiler, you can clearly see that there were two queries executed:

--Get initial Inventory/Car records
SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp]
FROM [dbo].[Inventory] AS [i]
--Get the Make record for the first Inventory/Car record
--Parameters removed for readability
SELECT [m].[Id], [m].[Name], [m].[TimeStamp] FROM [Makes] AS [m]
WHERE [m].[Id] = 5

If you want to learn more about lazy loading and how to use it with EF Core, consult the documentation here: https://docs.microsoft.com/en-us/ef/core/querying/related-data/lazy.

Updating Records
Records are updated by loading them into DbSet as a tracked entity, changing them through code, and then calling SaveChanges() on the context. When SaveChanges() is executed, the ChangeTracker reports all of the modified entities, and EF Core (along with the database provider) creates the appropriate SQL statement(s) to update the record(s).

Entity State
When a tracked entity is edited, EntityState is set to Modified. After the changes are successfully saved, the state is returned to Unchanged.

Update Tracked Entities
Updating a single record is much like adding a single record, except that the initial record is retrieved from the database and not created through code. Load the record from the database into a tracked entity, make some changes, and call SaveChanges(). Note that you do not have to call the Update()/UpdateRange() methods on the DbSet, since the entities are already tracked. The following code updates only one record, but the process is the same if multiple tracked entities are updated and saved.

static void UpdateRecords()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var car = context.Cars.First();
car.Color = "Green"; context.SaveChanges();
}

The executed SQL statement is listed here:

exec sp_executesql N'SET NOCOUNT ON; UPDATE [dbo].[Inventory] SET [Color] = @p0 WHERE [Id] = @p1 AND [TimeStamp] = @p2; SELECT [Display], [TimeStamp]
FROM [dbo].[Inventory]
WHERE @@ROWCOUNT = 1 AND [Id] = @p1;

',N'@p1 int,@p0 nvarchar(50),@p2 varbinary(8)',@p1=2,@p0=N'Green',@p2=0x0000000000000867

■ Note The previous where clause checked not only the Id column but also the TimeStamp column. This is eF Core using concurrency checking, covered later in this chapter.

Update Nontracked Entities
Untracked entities can also be used to update database records. The process is similar to updating tracked entities except that the entity is created in code (and not queried), and EF Core must be notified that the entity should already exist in the database and needs to be updated. There are two ways to notify EF Core that this entity needs to be processed as an update. The first is to call the Update() method on the DbSet, which sets the state to Modified, like this:

context.Cars.Update(updatedCar);

The second is to use the context instance and the Entry() method to set the state to Modified, like this:

context.Entry(updatedCar).State = EntityState.Modified;

Either way, SaveChanges() must still be called for the values to persist.

■ Note you might be wondering when you might update an entity that is not tracked. Think of an aSp.neT Core post call that sends the values for an entity over hTTp. The updated data needs to be persisted, and using this technique negates the need for another call to the database to get the entity into the ChangeTracker.

The following example reads a record in as nontracked (simulating a postback in ASP.NET Core) and changes one property (Color). Then it sets the state as Modified by calling the Update() method on DbSet.

static void UpdateRecords()
{
...
var carToUpdate = context.Cars.AsNoTracking().First(x => x.Id == 1); carToUpdate.Color = "Orange";
context.Cars.Update(carToUpdate); context.SaveChanges();
}

Since the entity is not tracked, EF Core updates all property values in the generated SQL:

exec sp_executesql N'SET NOCOUNT ON;
UPDATE [dbo].[Inventory] SET [Color] = @p0, [DateBuilt] = @p1, [IsDrivable] = @p2, [MakeId] = @p3, [PetName] = @p4
WHERE [Id] = @p5 AND [TimeStamp] = @p6; SELECT [Display], [TimeStamp]
FROM [dbo].[Inventory]
WHERE @@ROWCOUNT = 1 AND [Id] = @p5;

',N'@p5 int,@p0 nvarchar(50),@p1 datetime2(7),@p2 bit,@p3 int,@p4 nvarchar(50),
@p6 varbinary(8)',@p5=1,@p0=N'Orange',@p1='2021-06-21 01:12:46.5800000',@p2=1,@p3=5, @p4=N'Hank',@p6=0x000000000000088F

The next example follows the same logic, but instead of calling the Update() method, the code manually changes the EntityState to Modified and then calls SaveChanges(). The Clear() method is called on the ChangeTracker to make sure there isn’t any crossover from the different execution paths. The generated SQL is the same as the previous example.

static void UpdateRecords()
{
context.ChangeTracker.Clear();
var carToUpdate2 = context.Cars.AsNoTracking().First(x => x.Id == 1); carToUpdate2.Color = "Orange";
context.Entry(carToUpdate2).State = EntityState.Modified; context.SaveChanges();
}

Deleting Records
One or more entities are marked for deletion by calling Remove() (for a single entity) or RemoveRange() (for a list of entities) on the appropriate DbSet property or by setting the state for the entity/entities to Deleted. The removal process will cause cascade effects on navigation properties based on the rules configured in OnModelCreating() (or by EF Core conventions). If deletion is prevented due to cascade policy, an exception is thrown.

Entity State
When the Remove() method is called on an entity that is being tracked, its EntityState is set to Deleted. After the SaveChanges() method is successfully executed, the entity is removed from the ChangeTracker, and its state is changed to Detached. Note that the entity still exists in your application unless it has gone out of scope and been garbage collected.

Delete Tracked Records
The delete process mirrors the update process. Once an entity is tracked, call Remove() on that instance, and then call SaveChanges() to remove the record from the database.

static void DeleteRecords()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); ClearSampleData();
LoadMakeAndCarData();
var car = context.Cars.First(x=>x.Color != "Green"); context.Cars.Remove(car);
context.SaveChanges();
}
The executed SQL call for the delete is listed here:

exec sp_executesql N'SET NOCOUNT ON; DELETE FROM [dbo].[Inventory]
WHERE [Id] = @p0 AND [TimeStamp] = @p1; SELECT @@ROWCOUNT;
',N'@p0 int,@p1 varbinary(8)',@p0=2,@p1=0x00000000000008EB
As a point of emphasis, after SaveChanges() is called, the entity instance still exists but is no longer in the ChangeTracker. When checking the EntityState, the state will be Detached.

static void DeleteRecords()
{
...
Console.WriteLine($"{car.PetName}'s state is {context.Entry(car).State}");
}

Delete Nontracked Entities
Untracked entities can delete records the same way untracked entities can update records. The difference is that the entity is tracked by calling Remove()/RemoveRange() or setting the state to Deleted and then calling SaveChanges().
The following example follows the same pattern for updating nontracked entities. It reads a record in as nontracked and then uses the Remove() method on DbSet (the first example) or manually changes the EntityState to Deleted (the second example). In each case, the code calls SaveChanges() to persist the deletion. The Clear() call on the ChangeTracker makes sure there isn’t any pollution between the first example and the second.

static void DeleteRecords()
{
...
context.ChangeTracker.Clear();
var carToDelete = context.Cars.AsNoTracking().First(x=>x.Color != "Green"); context.Cars.Remove(carToDelete);
context.SaveChanges();
context.ChangeTracker.Clear();
var carToDelete2 = context.Cars.AsNoTracking().First(x=>x.Color != "Green"); context.Entry(carToDelete2).State = EntityState.Deleted; context.SaveChanges();
}

Catch Cascade Delete Failures
EF Core will throw a DbUpdateException when an attempt to delete a record fails due to the cascade rules. The following test shows this in action:

static void DeleteRecords()
{
...
context.ChangeTracker.Clear(); var make = context.Makes.First(); context.Makes.Remove(make);
try
{
context.SaveChanges();
}
catch (DbUpdateException ex)
{
Console.WriteLine(ex.Message);
}
}

This completes the content on Create, Read, Update, and Delete (CRUD) operations using EF Core. The next section covers notable EF Core features that provide benefit to data access code and developer productivity.

Notable EF Core Features
Many features from EF 6 have been replicated in EF Core, with more being added in every release. Many of those features have been greatly improved in their EF Core implementation, both in functionality and
performance. In addition to bringing forward features from EF 6, EF Core has added many new features. The following are some of the more notable features in EF Core (in no particular order).

Global Query Filters
Global query filters enable a where clause to be added into all LINQ queries for a particular entity. For example, a common database design pattern is to use soft deletes instead of hard deletes. A field is added to the table to indicate the deleted status of the record. If the record is “deleted,” the value is set to true (or 1), but not removed from the database. This is called a soft delete. To filter out the soft-deleted records from normal operations, every where clause must check the value of this field. Remembering to include this filter in every query can be time-consuming, if not problematic.
EF Core enables adding a global query filter to an entity that is then applied to every query involving that entity. For the soft delete example described earlier, you set a filter on the entity class to exclude the soft- deleted records. No longer do you have to remember to include the where clause to filter out the soft-deleted records in every query you write.

Presume that all Car records that are not drivable should be filtered out of the normal queries. Open the
CarConfiguration class and add the following line to the Configure() method:

public void Configure(EntityTypeBuilder builder)
{
...
builder.HasQueryFilter(c=>c.IsDrivable == true);
...
}

With the global query filter in place, queries involving the Car entity will automatically filter out the nondrivable cars. Take the following LINQ query that retrieves all cars except those excluded by the query filter:

static void QueryFilters()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var cars = context.Cars.ToList();
Console.WriteLine($"Total number of drivable cars: {cars.Count}");
}

The generated SQL is as follows:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp]
FROM [dbo].[Inventory] AS [i]
WHERE [i].[IsDrivable] = CAST(1 AS bit)

■ Note Query filters are not additive. The last one in wins. If you were to add another query filter for the Car
entity, it would replace the existing query filter.

If you need to retrieve all records, including those filtered with the global query filter, add the
IgnoreQueryFilters() method into the LINQ query:

static void QueryFilters()
{
...
var allCars = context.Cars.IgnoreQueryFilters().ToList(); Console.WriteLine($"Total number of cars: {allCars.Count}"); var radios = context.Radios.ToList();
var allRadios = context.Radios.IgnoreQueryFilters().ToList();
}
With the IgnoreQueryFilters() added to the LINQ query, the generated SQL statement no longer has
the where clause excluding the soft-deleted records:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp]
FROM [dbo].[Inventory] AS [i]

It is important to note that calling IgnoreQueryFilters() removes the query filter for every entity in the LINQ query, including any that are involved in Include() or ThenInclude() statements.

■ Note global query filters are entirely an eF Core construct. no changes are made to the database. even with the filter to exclude nondrivable cars in place, if you open up SSmS/azure Data Studio and run a query to select all Inventory (remember the Car entity is mapped to the Inventory table) records, you would see all records, even the nondrivable records.

Global Query Filters on Navigation Properties
Global query filters can also be set on navigation properties. Suppose you want to filter out any radios that are in a Car that is not drivable. The query filter is created on the Radio entity’s CarNavigation navigation property, like this (using the RadioConfiguration class):

public void Configure(EntityTypeBuilder builder)
{
...
builder.HasQueryFilter(e=>e.CarNavigation.IsDrivable);
...
}

When executing a standard LINQ query, any orders that contain a nondrivable car will be excluded from the result. Here is the LINQ statement and the generated SQL statement:

//C# Code
static void QueryFilters()
{
...
var radios = context.Radios.ToList();
}

/ Generated SQL query /
SELECT [r].[Id], [r].[InventoryId], [r].[HasSubWoofers],
[r].[HasTweeters], [r].[RadioId], [r].[TimeStamp] FROM [Radios] AS [r]
INNER JOIN (
SELECT [i].[Id], [i].[IsDrivable] FROM [dbo].[Inventory] AS [i]
WHERE [i].[IsDrivable] = CAST(1 AS bit)
) AS [t] ON [r].[InventoryId] = [t].[Id] WHERE [t].[IsDrivable] = CAST(1 AS bit)

To remove the query filter, use IgnoreQueryFilters(). The following is the updated LINQ statements and the subsequent generated SQL:

//C# Code
static void QueryFilters()
{
...
var allRadios = context.Radios.IgnoreQueryFilters().ToList();
}

/ Generated SQL query /
SELECT [r].[Id], [r].[InventoryId], [r].[HasSubWoofers], [r].[HasTweeters], [r].[RadioId], [r].[TimeStamp]
FROM [Radios] AS [r]

A word of caution here: EF Core does not detect cyclic global query filters, so use care when adding query filters to navigation properties.

Explicit Loading with Global Query Filters
Global query filters are also in effect when loading related data explicitly. For example, if you wanted to load the Car records for a Make, the IsDrivable filter will prevent nondrivable cars from being loaded into memory. Take the following code snippet as an example:

static void RelatedDataWithQueryFilters()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var make = context.Makes.First(x => x.Id == 1);
//Get the Cars collection context.Entry(make).Collection(c => c.Cars).Load(); context.ChangeTracker.Clear();
}

By now it should be no surprise that the generated SQL query includes the filter for nondrivable cars.

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp] FROM [dbo].[Inventory] AS [i]
WHERE ([i].[IsDrivable] = CAST(1 AS bit)) AND ([i].[MakeId] = 1)

There is a slight catch to ignoring query filters when explicitly loading data. The type returned by the Collection() method is CollectionEntry<Make,Car> and does not explicitly implement the IQueryable interface. To call IgnoreQueryFilters(), you must first call Query(), which returns an IQueryable.

static void RelatedDataWithQueryFilters()
{
...
//Get the Cars collection
context.Entry(make).Collection(c => c.Cars).Query().IgnoreQueryFilters().Load();
}

It should come as no surprise now that the SQL generated by EF Core does not include the filter for nondrivable cars:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[TimeStamp]
FROM [dbo].[Inventory] AS [i] WHERE [i].[MakeId] = 1

The same process applies when using the Reference() method to retrieve data from a reference navigation property. First call Query() and then call IgnoreQueryFilters().

Raw SQL Queries with LINQ
Sometimes getting the correct LINQ statement for a complicated query can be harder than just writing the SQL directly. Or the generated SQL from your LINQ query is suboptimal. Fortunately, EF Core
has a mechanism to allow raw SQL statements to be executed on a DbSet. The FromSqlRaw() and FromSqlRawInterpolated() methods take in a string that replaces the LINQ query. This query is executed on the server side.
If the raw SQL statement is nonterminating (e.g., neither a stored procedure, user-defined function, a statement that uses a common table expression, nor ends with a semicolon), then additional LINQ statements can be added to the query. The additional LINQ statements, such as Include(), OrderBy(), or
Where() clauses, will be combined with the original raw SQL call and any global query filters, and the entire
query is executed on the server side.
When using one of the FromSql variants, the query must be written using the data store schema and table name, and not the entity names. FromSqlRaw() will send the string in just as it is written. FromSqlInterpolated() uses C# string interpolation, and each interpolated string is translated in the SQL parameter. You should use the interpolated version whenever you are using variables for the added protection inherent in parameterized queries.
To get the database schema and table name, use the Model property on the derived DbContext. The Model exposes a method called FindEntityType() that returns an IEntityType, which in turn has methods to get the schema and table name. This was used earlier in the chapter to set identity insert for SQL Server. The following code displays the schema and table name:

static void UsingFromSql()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); IEntityType metadata = context.Model.FindEntityType(typeof(Car).FullName); Console.WriteLine(metadata.GetSchema()); Console.WriteLine(metadata.GetTableName());
}

Presuming the global query filter from the previous section is set on the Car entity, the following LINQ statement will get the first inventory record where the Id is one, include the related Make data, and filter out nondrivable cars:

int carId = 1;
var car = context.Cars
.FromSqlInterpolated($"Select * from dbo.Inventory where Id = {carId}")
.Include(x => x.MakeNavigation)
.First();

■ Note Unfortunately, the table and schema name cannot be added to the query using C# string interpolation, as SQl Server doesn’t support the parameterization of those items.

The LINQ to SQL translation engine combines the raw SQL statement with the rest of the LINQ statements and executes the following query:

SELECT TOP(1) .[Id], .[Color], .[DateBuilt], .[Display],
.[IsDrivable], .[MakeId], .[PetName], .[TimeStamp],
[m].[Id], [m].[Name], [m].[TimeStamp] FROM (
Select * from dbo.Inventory where Id = @p0
) AS
INNER JOIN [Makes] AS [m] ON .[MakeId] = [m].[Id]
WHERE .[IsDrivable] = CAST(1 AS bit)',N'@p0 int',@p0=1

Know that there are a few rules that must be observed when using raw SQL with LINQ.
• The SQL query must return data for all properties of the entity type.
• The column names must match the properties they are mapped to (an improvement over EF 6 where mappings were ignored).
• The SQL query can’t contain related data.

Projections
In addition to using raw SQL queries with LINQ, view models can be populated with projections. A projection is where another object type is composed at the end of a LINQ query, projecting the data into another datatype. A projection can be a subset of the original data (e.g., getting all the Id values of Car entities that match a where clause) or a custom type, like the CarMakeViewModel.
To get the list of all the primary keys of the Car records, add the following method to your top-level statements:

static void Projections()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); List ids = context.Cars.Select(x => x.Id).ToList();
}

The SQL generated for this statement is shown here. Notice the global query filter is honored in the projection:

SELECT [i].[Id]
FROM [dbo].[Inventory] AS [i]
WHERE [i].[IsDrivable] = CAST(1 AS bit)

To populate a custom type, use the new keyword in the Select() method. The values of the new type are populated using object initialization, and the LINQ to SQL translation engine takes care of ensuring the navigation properties used for the new type are retrieved from the database. This is done using eager loading, which is covered in detail later in this chapter. Update the Projections() method with the

following code to create a list of CarMakeViewModel entities (note the additional using statement that must be added to the top of your file):

static void Projections()
{
...
var vms = context.Cars.Select(x => new CarMakeViewModel
{
CarId = x.Id, Color = x.Color,
DateBuilt = x.DateBuilt.GetValueOrDefault(new DateTime(2020, 01, 01)), Display = x.Display,
IsDrivable = x.IsDrivable, Make = x.MakeNavigation.Name, MakeId = x.MakeId,
PetName = x.PetName
});
foreach (CarMakeViewModel c in vms)
{
Console.WriteLine($"{c.PetName} is a {c.Make}");
}
}

The SQL generated for this statement is shown next. Notice the inner join to retrieve the Make Name
property and that the global query filter is once again honored:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display],
[i].[IsDrivable], [m].[Name], [i].[MakeId], [i].[PetName] FROM [dbo].[Inventory] AS [i]
INNER JOIN [Makes] AS [m] ON [i].[MakeId] = [m].[Id]
WHERE [i].[IsDrivable] = CAST(1 AS bit)

Handling Database-Generated Values
In addition to change tracking and the generation of SQL queries from LINQ, a significant advantage to using EF Core over raw ADO.NET is the seamless handling of database-generated values. After adding or updating an entity, EF Core queries for any database-generated data and automatically updates the entity with the correct values. In raw ADO.NET, you would need to do this yourself.
For example, the Inventory table has an integer primary key that is defined in SQL Server as an Identity column. Identity columns are populated by SQL Server with a unique number (from a sequence) when a record is added, and this primary key is not allowed to be updated during normal updates of the record (excluding the special case of having identity insert enabled). Additionally, the Inventory table has a Timestamp column used for concurrency checking. Concurrency checking is covered next, but for now just know that the Timestamp column is maintained by SQL Server and updated on any add or edit action. We also added two columns with default values to the table, DateBuilt and IsDrivable, and one computed column, Display.
The following code adds a new Car to the Inventory table, similar to the code used earlier in the chapter:

static void AddACar()
{
//The factory is not meant to be used like this, but it's demo code 🙂

var context = new ApplicationDbContextFactory().CreateDbContext(null); var car = new Car
{
Color = "Yellow", MakeId = 1, PetName = "Herbie"
};
context.Cars.Add(car); context.SaveChanges();
}

When SaveChanges() is executed, there are two queries run against the database. The first, shown here, inserts the new record into the table:

--Process the insert
INSERT INTO [Dbo].[Inventory] ([Color], [MakeId], [PetName]) VALUES (N'Yellow', 1, N'Herbie');

The second returns the values for the primary key and all other server-generated data. In this case, the query is returning the Id, DateBuilt, Display, IsDrivable, and Timestamp values. EF Core then updates the entity with the server-generated values.

--Return the server maintained values to EF Core
SELECT [Id], [DateBuilt], [Display], [IsDrivable], [TimeStamp] FROM [dbo].[Inventory]
WHERE @@ROWCOUNT = 1 AND [Id] = scope_identity();

■ Note eF Core actually executes parameterized queries, but I have simplified the SQl examples for readability.

When adding a record that assigns values to the properties with defaults, you will see that EF Core does not query for those properties, as the entity already has the correct values. Take the following example code:

static void AddACarWithDefaultsSet()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var car = new Car
{
Color = "Yellow", MakeId = 1,
PetName = "Herbie", IsDrivable = true,
DateBuilt = new DateTime(2021,01,01)
};
context.Cars.Add(car); context.SaveChanges();
}

When SaveChanges() is executed, the following two SQL statements are run. Notice that only the Id
(primary key), Display, and TimeStamp values are retrieved:

--Insert the values
INSERT INTO [dbo].[Inventory] ([Color], [DateBuilt], [IsDrivable], [MakeId], [PetName]) VALUES (N'Yellow','2021-01-01 00:00:00',1,1, N'Herbie');
--Get the database managed values SELECT [Id], [Display], [TimeStamp] FROM [dbo].[Inventory]
WHERE @@ROWCOUNT = 1 AND [Id] = scope_identity();

When updating records, the primary key values are already known, so only the nonprimary key fields that are database controlled are returned. Using the previous Car example, only the updated Timestamp value is queried and returned when the record is updated. The following code retrieves a Car record from the database, changes the color, and then saves the updated Car:

static void UpdateACar()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var car = context.Cars.First(c => c.Id == 1);
car.Color = "White"; context.SaveChanges();
}

The SaveChanges() command executes the following SQL, which first saves the updates and then returns the new Display and TimeStamp values. Once again the SQL is simplified from the parameterized version. Don’t worry about the TimeStamp value in the where clause; that will be explained in the next section.

//Update the Car Record
UPDATE [dbo].[Inventory] SET [Color] = N'White' WHERE [Id] = 1 AND [TimeStamp] = 0x00000000000007E1;
//Return the updated Display and TimeStamp values to EF Core SELECT [Display], [TimeStamp]
FROM [dbo].[Inventory]
WHERE @@ROWCOUNT = 1 AND [Id] = 1;

This process also works when adding and/or updating multiple items into the database. EF Core knows how to wire up the values retrieved from the database to the correct entities in your collection.

Concurrency Checking
Concurrency issues arise when two separate processes (users or systems) attempt to update the same record at roughly the same time. For example, User 1 and User 2 both get the data for Customer A. User 1 updates the address and saves the change. User 2 updates the credit rating and attempts to save the same record.
If the save for User 2 works, the changes from User 1 will be reverted, since the address was changed after User 2 retrieved the record. Another option is to fail the save for User 2, in which case User 1’s changes are persisted, but User 2’s changes are not.

How this situation is handled depends on the requirements for the application. Solutions range from doing nothing (second update overwrites the first) to using optimistic concurrency (the second update fails) to more complicated solutions such as checking individual fields. Except for the choice of doing nothing (universally considered a bad programming idea), developers need to know when concurrency issues arise so they can be handled appropriately.
Fortunately, many modern databases have tooling to help the development team handle concurrency issues. SQL Server has a built-in data type called timestamp, a synonym for rowversion. If a column is defined with a data type of timestamp, when a record is added to the database, the value for the column
is created by SQL Server, and when a record is updated, the value for the column is updated as well. The value is virtually guaranteed to be unique and is entirely controlled by SQL Server, so you don’t have to do anything but “opt in.”
EF Core can leverage the SQL Server timestamp data type by implementing a Timestamp property on an entity (represented as byte[] in C#). Entity properties defined with the Timestamp attribute or Fluent API designation are added to the where clause when updating or deleting records. Instead of just using the primary key value(s), the generated SQL adds the value of the timestamp property to the where clause, as you saw in the previous example. This limits the results to those records where the primary key and the timestamp values match. If another user (or the system) has updated the record, the timestamp values will not match, and the update or delete statement will not update the record. Here is the previous update example highlighting the query using the Timestamp column:

UPDATE [dbo].[Inventory] SET [Color] = N'White' WHERE [Id] = 1 AND [TimeStamp] = 0x00000000000007E1;

Databases (such as SQL Server) report the number of records affected when adding, updating, or deleting records. If the database reports a number of records affected that is different than the number of records the ChangeTracker expected to be changed, EF Core throws a DbUpdateConcurrencyException and rolls the entire transaction back. The DbUpdateConcurrencyException contains information for all the records that did not persist, including the original values (when the entity was loaded from the database) and the current values (as the user/system updated them). There is also a method to get the current database values (this requires another call to the server). With this wealth of information, the developer can then handle the concurrency error as the application requires. The following code shows this in action:

static void ThrowConcurrencyException()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); try
{
//Get a car record (doesn't matter which one) var car = context.Cars.First();
//Update the database outside of the context context.Database.ExecuteSqlInterpolated($"Update dbo.Inventory set Color='Pink' where Id
= {car.Id}");
//update the car record in the change tracker and then try and save changes car.Color = "Yellow";
context.SaveChanges();
}
catch (DbUpdateConcurrencyException ex)
{
//Get the entity that failed to update var entry = ex.Entries[0];

//Get the original values (when the entity was loaded) PropertyValues originalProps = entry.OriginalValues;
//Get the current values (updated by this code path) PropertyValues currentProps = entry.CurrentValues;
//get the current values from the data store –
//Note: This needs another database call
PropertyValues databaseProps = entry.GetDatabaseValues();
}
}

Connection Resiliency
Transient errors are difficult to debug and more difficult to replicate. Fortunately, many database providers have a built-in retry mechanism for glitches in the database system (tempdb issues, user limits, etc.) that can be leveraged by EF Core. For SQL Server, SqlServerRetryingExecutionStrategy catches errors
that are transient (as defined by the SQL Server team), and if enabled on the derived DbContext through
DbContextOptions, EF Core automatically retries the operation until the maximum retry limit is reached.
For SQL Server, there is a shortcut method that can be used to enable SqlServerRetryingExecutionStrategy with all the defaults. The method used with SqlServerOptions is EnableRetryOnFailure() and is demonstrated here:

public ApplicationDbContext CreateDbContext(string[] args)
{
var optionsBuilder = new DbContextOptionsBuilder();
var connectionString = @"server=.,5433;Database=AutoLot50;User Id=sa;Password=P@ssw0rd;"; optionsBuilder.UseSqlServer(connectionString, options => options.EnableRetryOnFailure()); return new ApplicationDbContext(optionsBuilder.Options);
}

The maximum number of retries and the time limit between retries can be configured per the application’s requirements. If the retry limit is reached without the operation completing, EF Core will notify the application of the connection problems by throwing a RetryLimitExceededException. This
exception, when handled by the developer, can relay the pertinent information to the user, providing a better experience.

try
{
Context.SaveChanges();
}
catch (RetryLimitExceededException ex)
{
//A retry limit error occurred
//Should handle intelligently
Console.WriteLine($"Retry limit exceeded! {ex.Message}");
}

When using an execution strategy, explicit transactions must be created within the context of the execution strategy:

static void TransactionWithExecutionStrategies()

{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var strategy = context.Database.CreateExecutionStrategy(); strategy.Execute(() =>
{
using var trans = context.Database.BeginTransaction(); try
{
//actionToExecute(); trans.Commit();
}
catch (Exception ex)
{
trans.Rollback();
}
});
}

For database providers that don’t provide a built-in execution strategy, custom execution strategies can also be created. For more information, refer to the EF Core documentation: https://docs.microsoft.com/ en-us/ef/core/miscellaneous/connection-resiliency.

Database Function Mapping
SQL Server functions can be mapped to C# methods and be included in LINQ statements. The C# method is merely a placeholder as the server function gets folded into the generated SQL for the query. Support for
table-valued function mapping has been added in EF Core to the already existing support for scalar function mapping.
EF Core already supports many built-in SQL Server functions. The C# null coalescing operator (??) translates to the SQL Server coalesce function. String.IsNullOrEmpty() translates to a null check and uses the SQL Server len function to check for an empty string.
To see mapping a user-defined function in action, create a user-defined function that returns the number of Car records based on MakeId:

CREATE FUNCTION udf_CountOfMakes ( @makeid int ) RETURNS int
AS BEGIN
DECLARE @Result int
SELECT @Result = COUNT(makeid) FROM dbo.Inventory WHERE makeid = @makeid RETURN @Result
END GO

To use this in C#, create a new function in the derived DbContext class. The C# body of this function is never executed; it’s merely a placeholder that is mapped to the SQL Server function. Note that this method can be placed anywhere, but it is usually placed in the derived DbContext class for discoverability:

[DbFunction("udf_CountOfMakes", Schema = "dbo")] public static int InventoryCountFor(int makeId)
=> throw new NotSupportedException();

This function can now be used in LINQ queries and becomes part of the generated SQL. To see this in action, add the following code to your top-level statements:

static void UsingMappedFunctions()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var makes = context.Makes
.Where(x=>ApplicationDbContext.InventoryCountFor(x.Id)>1).ToList();
}

When this code is executed, the following SQL is executed:

SELECT [m].[Id], [m].[Name], [m].[TimeStamp] FROM [Makes] AS [m]
WHERE [dbo].udf_CountOfMakes > 1

EF Core also supports mapping table-valued functions. Add the following function to your database:

CREATE FUNCTION udtf_GetCarsForMake ( @makeId int ) RETURNS TABLE
AS RETURN (
-- Add the SELECT statement with parameter references here
SELECT Id, IsDrivable, DateBuilt, Color, PetName, MakeId, TimeStamp, Display FROM Inventory WHERE MakeId = @makeId
) GO

Add the following code to your ApplicationDbContext class:

[DbFunction("udtf_GetCarsForMake", Schema = "dbo")] public IQueryable GetCarsFor(int makeId)
=> FromExpression(()=>GetCarsFor(makeId));

The FromExpression() call allows the function to be called directly on the derived DbContext instead of using a regular DbSet. Add the following code to your top-level statements to exercise the table-valued function:

static void UsingMappedFunctions()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null);
var makes = context.Makes.Where(x=>ApplicationDbContext.InventoryCountFor(x.Id)>1). ToList();
var cars = context.GetCarsFor(1).ToList();
}

The following SQL gets executed (notice that the global query filter is honored when using the database function):

exec sp_executesql N'SELECT [u].[Id], [u].[Color], [u].[DateBuilt],
[u].[Display], [u].[IsDrivable], [u].[MakeId], [u].[PetName], [u].[TimeStamp]
FROM [dbo].[udtf_GetCarsForMake](@ makeId_1) AS [u] WHERE [u].[IsDrivable] = CAST(1 AS bit)',
N'@ makeId_1 int',@ makeId_1=1

For more information on database function mapping, consult the documentation: https://docs. microsoft.com/en-us/ef/core/querying/user-defined-function-mapping.

EF.Functions
The static EF class was created as a placeholder for CLR methods that get translated to database functions, using the same mechanism as database function mapping covered in the previous section. The main difference is that all the implementation details are handled by the database providers. Table 22-1 lists the functions available.

Table 22-1. Functions Available Through EF.Functions

Function Meaning in Life
Like() An implementation of the SQL LIKE operation. Case sensitivity and syntax are dependent on the database. For SQL Server, the comparison is not case sensitive, and the wildcard operator (%) must be provided.
Random() Introduced in EF Core 6, this returns a pseudorandom number between 0 and 1, inclusive. Maps to SQL Server’s RAND function.
Collate() Specifies the collation to be used in a LINQ query.
Contains() Maps to the SQL Server CONTAINS function. Table must be full-text indexed to use Contains().
FreeText() Maps to the server FREETEXT store function. Table must be full-text indexed to use FreeText().
DataLength() Returns number of bytes used to represent an expression.
(continued)

Table 22-1. (continued)

Function Meaning in Life
DateDiffYear() DateDiffMonth() DateDiffWeek() DateDiffDay() DateDiffHour() DateDiffMinute() DateDiffSecond() DateDiffMillisecond() DateDiffMicrosecond() DateDiffNansecond() Counts the number time interval of boundaries crossed between start date and end date. Maps to SQL Server’s DATEDIFF function.
DateFromParts() Initializes a new instance of the DateTime structure for specified year, month, day. Maps to SQL Server’s DATEFROMPARTS function.
DateTime2FromParts() Initializes a new instance of the DateTime structure for specified year, month, day, hour, minute, second, fractions, and precision. Maps to SQL Server’s DATETIME2FROMPARTS function.
DateTimeFromParts() Initializes a new instance of the DateTime structure for specified year, month, day, hour, minute, second,
millisecond. Maps to SQL Server’s DATETIMEFROMPARTS
function.
DateTimeOffsetFromParts() Initializes a new instance of the DateTimeOffset structure for specified year, month, day, hour, minute, second, fractions, hourOffset, minuteOffset, and precision. Maps to SQL Server’s DATETIMEOFFSETFROMPARTS function.
SmallDateTimeFromParts() Initializes a new instance of the DateTime structure to the specified year, month, day, hour, and minute. Corresponds to the SQL Server’s SMALLDATETIMEFROMPARTS function.
TimeFromParts() Initializes a new instance of the TimeSpan structure for the specified hour, minute, second, fractions, and
precision. Maps to SQL Server’s TIMEFROMPARTS function.
IsDate() Validate if a given string is a date. Maps to SQL Server’s
ISDATE().

To demonstrate using the EF.Functions.Like method, add the following local function into your top- level statements:

static void UseEFFunctions()
{
Console.WriteLine("Using Like");
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null);

//Same as contains
var cars = context.Cars.IgnoreQueryFilters().Where(x=>EF.Functions.Like(x. PetName,"%Clunk%")).ToList();
foreach (var c in cars)
{
Console.WriteLine($"{c.PetName} was found");
}
//Same as Starts with
cars = context.Cars.IgnoreQueryFilters().Where(x=>EF.Functions.Like(x.PetName,"Clun%")). ToList();
foreach (var c in cars)
{
Console.WriteLine($"{c.PetName} was found");
}
//Same as Ends with
cars = context.Cars.IgnoreQueryFilters().Where(x=>EF.Functions.Like(x.PetName,"%er")). ToList();
foreach (var c in cars)
{
Console.WriteLine($"{c.PetName} was found");
}
}

Notice that the SQL Server wildcard character (%) is used just like in a T-SQL query.

Batching of Statements
EF Core has significantly improved the performance when saving changes to the database by executing the statements in one or more batches. This decreases trips between the application and the database, increasing performance and potentially reducing cost (e.g., for cloud databases where customers are charged by the transaction).
EF Core batches the create, update, and delete statements using table-valued parameters. The number of statements that EF batches depends on the database provider. For example, for SQL Server, batching
is inefficient below 4 statements and above 40, so EF Core will max out the number of statements at 42. Regardless of the number of batches, all statements still execute in a transaction. The batch size can also be configured through DbContextOptions, but the recommendation is to let EF Core calculate the batch size for most (if not all) situations.
If you were to insert four cars in one transaction like this:

static void Batching()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var cars = new List
{
new Car { Color = "Yellow", MakeId = 1, PetName = "Herbie" }, new Car { Color = "White", MakeId = 2, PetName = "Mach 5" }, new Car { Color = "Pink", MakeId = 3, PetName = "Avon" },
new Car { Color = "Blue", MakeId = 4, PetName = "Blueberry" },
};

context.Cars.AddRange(cars); context.SaveChanges();
}

EF Core would batch up the statements into a single call. The query generated is shown here:

exec sp_executesql N'SET NOCOUNT ON;
DECLARE @inserted0 TABLE ([Id] int, [_Position] [int]); MERGE [dbo].[Inventory] USING (
VALUES (@p0, @p1, @p2, 0),
(@p3, @p4, @p5, 1),
(@p6, @p7, @p8, 2),
(@p9, @p10, @p11, 3)) AS i ([Color], [MakeId], [PetName], _Position) ON 1=0 WHEN NOT MATCHED THEN
INSERT ([Color], [MakeId], [PetName]) VALUES (i.[Color], i.[MakeId], i.[PetName]) OUTPUT INSERTED.[Id], i._Position
INTO @inserted0;

SELECT [t].[Id], [t].[DateBuilt], [t].[Display], [t].[IsDrivable], [t].[TimeStamp] FROM [dbo].[Inventory] t
INNER JOIN @inserted0 i ON ([t].[Id] = [i].[Id]) ORDER BY [i].[_Position];

',N'@p0 nvarchar(50),@p1 int,@p2 nvarchar(50),@p3 nvarchar(50),@p4 int,@p5 nvarchar(50), @p6 nvarchar(50),@p7 int,@p8 nvarchar(50),@p9 nvarchar(50),@p10 int,@p11 nvarchar(50)', @p0=N'Yellow',@p1=1,@p2=N'Herbie',@p3=N'White',@p4=2,@p5=N'Mach 5',@p6=N'Pink',@p7=3, @p8=N'Avon',@p9=N'Blue',@p10=4,@p11=N'Blueberry'

Value Converters
Value converters are used to automatically convert data when retrieved and saved to the database. EF Core ships with a long list of built-in value converters (you can see the complete list here: https://docs. microsoft.com/en-us/dotnet/api/microsoft.entityframeworkcore.storage.valueconversion). In
addition to the built-in value converters, you can also create your own. For example, you can store the price of a Car in the database as a numeric value but display the price as a currency string.
Table 22-2 lists many of the built-in value converters available in EF Core. In addition to the converters that ship with EF Core, you can create your own.

Table 22-2. Some of the EF Core Value Converters

Value Converter Meaning in Life
BoolToStringConverter Converts Boolean values to and from two string values.
BytesToStringConverter Converts arrays of bytes to and from strings.
CharToStringConverter Converts a Char to and from a single-character String.
DateTimeOffsetToBinaryConverter Converts DateTime to and from binary representation into a long. The DateTime is truncated beyond 0.1 millisecond precision.
DateTimeOffsetToBytesConverter Converts DateTime to and from arrays of bytes.
DateTimeOffsetToStringConverter Converts DateTimeOffset to and from strings.
DateTimeToBinaryConverter Converts DateTime using ToBinary(). This will preserve the
DateTimeKind.
DateTimeToStringConverter Converts DateTime to and from strings.
DateTimeToTicksConverter Converts DateTime to and from Ticks.
EnumToNumberConverter<TEnum,TNumber> Converts enum values to and from their underlying numeric representation.
EnumToStringConverter Converts enum values to and from their string representation.
GuidToStringConverter Converts a Guid to and from a String using the standard “8-4- 4-4-12” format.
NumberToStringConverter Converts numeric values to and from their string representation.
StringToBoolConverter Converts strings to and from Boolean values.
StringToBytesConverter Converts strings to and from arrays of bytes.
StringToCharConverter Converts strings to and from Char values.
StringToDateTimeConverter Converts strings to and from DateTime values.
StringToDateTimeOffsetConverter Converts strings to and from DateTimeOffset values.
StringToEnumConverter Converts strings to and from enum values.
StringToGuidConverter Converts strings to and from a Guid using the standard “8-4-4- 4-12” format.
StringToNumberConverter Converts strings to and from numeric values.
StringToTimeSpanConverter Converts strings to and from TimeSpan values.
StringToUriConverter Converts strings to and from Uri values.
TimeSpanToStringConverter Converts TimeSpan to and from strings.
TimeSpanToTicksConverter Converts TimeSpan to and from Ticks.
UriToStringConverter Converts a Uri to and from a String.
ValueConverter Defines conversions from an object of one type in a model to an object of the same or different type in the store.

Many of the built-in value converters don’t require any configuration. For many situations, EF Core will automatically convert from a C# bool to a SQL Server bit data type without any intervention on your part.

■ Note Value converters are set on the model. while they allow mapping disparate data types between your model and the database, the converters will be used only when using eF Core to query the database. Querying the database directly will return the database type, and not the converted type.

For example, to return a string value from the database instead of a decimal, start by updating the Car
class to have a string Price property:

public class Car : BaseEntity
{
...
public string Price { get; set; }
...
}

If you are following along, make sure to update the database by creating and applying a migration:

dotnet ef migrations add Price -o Migrations -c AutoLot.Samples.ApplicationDbContext dotnet ef database update Price -c AutoLot.Samples.ApplicationDbContext

Add the following global using statements into the GlobalUsings.cs file:

global using Microsoft.EntityFrameworkCore.Storage.ValueConversion; global using System.Globalization;

Next, add code into the CarConfiguration class’s Configure() method to use the built-in
StringToNumberConverter, like this:

public void Configure(EntityTypeBuilder builder)
{
...
builder.Property(p=>p.Price).HasConversion(new StringToNumberConverter());
...
}

If you want more control over the values, for example to format the price as currency, you can create a custom converter. Update the value conversion in the CarConfiguration class to the following:

public void Configure(EntityTypeBuilder builder)
{
...
CultureInfo provider = new CultureInfo("en-us");
NumberStyles style = NumberStyles.Number | NumberStyles.AllowCurrencySymbol; builder.Property(p => p.Price)
.HasConversion(
v => decimal.Parse(v, style, provider),

v => v.ToString("C2"));
...
}

The first parameter is converting the values going into the database, and the second parameter is formatting the data coming out of the database.
Adding the Price column breaks the table-valued function from earlier in this chapter. To correct this, update the function to include the new column:

ALTER FUNCTION udtf_GetCarsForMake ( @makeId int ) RETURNS TABLE
AS
RETURN (
-- Add the SELECT statement with parameter references here
SELECT Id, IsDrivable, DateBuilt, Color, PetName, MakeId, TimeStamp, Display, Price
FROM Inventory WHERE MakeId = @makeId
)

For more information on value conversion, please read the documentation at https://docs. microsoft.com/en-us/ef/core/modeling/value-conversions.

Shadow Properties
Shadow properties are properties that aren’t explicitly defined on your model but exist due to EF Core. The value and state of these properties are maintained entirely by the Change Tracker. One example of the use of shadow properties is to represent foreign keys for navigation properties if the foreign key isn’t defined as part of the entity. Another example is with temporal tables, discussed next.
Shadow properties that aren’t added to an entity by EF Core can be defined only through the Fluent API using the Property() method. If the name of the property passed into the Property() method matches an existing property for the entity (either a previously defined shadow property or an explicit property), the Fluent API code configures the existing property. Otherwise, a shadow property is created for the entity. To add a shadow property of type bool? named IsDeleted to the Car entity with the default value of true, add the following code to the Configure() method of the CarConfiguration class:

public void Configure(EntityTypeBuilder builder)
{
builder.Property<bool?>("IsDeleted").IsRequired(false).HasDefaultValue(true);
//omitted for brevity
}

Shadow properties can be accessed only through the Change Tracker, so they must be loaded from the database as tracked entities. For example, the following code will not compile:

static void ShadowProperties()
{
Console.WriteLine("Shadow Properties");
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var newCar = new Car
{

Color = "Blue",
PetName = "TestRecord",
MakeId = context.Makes.First().Id,
//Can't do this (compile error):
//IsDeleted = false
};
}

Once the record is added to the Change Tracker, though, the IsDeleted property can be accessed:

static void ShadowProperties()
{
Console.WriteLine("Shadow Properties");
//omitted for brevity context.Cars.Add(newCar);
context.Entry(newCar).Property("IsDeleted").CurrentValue = true;
}

Shadow properties can also be accessed in LINQ queries. The following code first loads all the Car records into the Change Tracker and then sets IsDeleted = false for every other record. After updating the database, the next LINQ query retrieves all of the deleted cars:

static void ShadowProperties()
{
Console.WriteLine("Shadow Properties");
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null);
//omitted for brevity

var cars = context.Cars.ToList();
foreach (var c in cars.Where(c=>c.Id % 2 == 0))
{
context.Entry(c).Property("IsDeleted").CurrentValue = false;
}
context.SaveChanges();
var nonDeletedCars = context.Cars.Where(c=> !EF.Property(c,"IsDeleted")).ToList(); foreach (Car c in nonDeletedCars)
{
Console.WriteLine($"{c.PetName} is deleted? {context.Entry(c).Property("IsDeleted"). CurrentValue}");
}
}

Adding the IsDeleted column also breaks the table-valued function from earlier in this chapter. Update the function to include the new column:

ALTER FUNCTION udtf_GetCarsForMake ( @makeId int ) RETURNS TABLE
AS
RETURN (

-- Add the SELECT statement with parameter references here
SELECT Id, IsDrivable, DateBuilt, Color, PetName, MakeId, TimeStamp, Display, Price, IsDeleted
FROM Inventory WHERE MakeId = @makeId
)

SQL Server Temporal Table Support
SQL Server temporal tables automatically keep track of all data ever stored in the table. This is accomplished using a history table into which a timestamped copy of the data is stored whenever a change or deletion is made to the main table. Historical data is then available for querying, auditing, or restoring. EF Core 6 has added support for creating temporal tables, converting normal tables to temporal tables, querying historical data, and restoring data from a point in time.

Configure Temporal Tables
To add default temporal table support, use the ToTable() Fluent API method. This method can also be used to specify the table name and schema, but in our example, it’s not needed due to the Table attribute on the Car entity. Update the CarConfiguration class’s Configure() method to add a ToTable() call:

public void Configure(EntityTypeBuilder builder)
{
//specify table name and schema – not needed because of the Table attribute
//builder.ToTable("Inventory", "dbo", b=> b.IsTemporal()); builder.ToTable(b=> b.IsTemporal());
//omitted for brevity
}

■ Note Split tables are not supported for use as temporal tables. This includes any tables generated from entities using Owned entities.

After creating a new EF Core migration and updating the database, the Inventory table is converted to a system-versioned temporal table with two additional datetime2 columns, PeriodEnd and PeriodStart. This also creates a history table named History (CarHistory, in this example) in the same schema as the main table (dbo, in this example). This history table is a clone of the updated Inventory table and stores the history of any changes to the Inventory table.
The names for the additional columns and the table (as well as the schema for the history table) can be controlled in the IsTemporal() method. The following example uses the names ValidFrom and ValidTo for PeriodStart and PeriodEnd, respectively; names the table InventoryAudit; and places the table in the audits schema:

builder.ToTable(b => b.IsTemporal(t =>
{
t.HasPeriodEnd("ValidTo"); t.HasPeriodStart("ValidFrom"); t.UseHistoryTable("InventoryAudit", "audits");
}));

■ Note at the time of this writing, neither eF Core nor SQl Server will create the schema for the history table if the schema doesn’t exist. make sure you are using an existing schema or update the migration to ensure the schema exists by adding the following:migrationBuilder.EnsureSchema("audits");

After this change is migrated to the database, when you examine the tables using Azure Data Studio (or SSMS), you will see the Inventory table is labeled as System-Versioned, and when you expand the node, you will see the audits.InventoryAudit table. The InventoryAudit table is marked as History, as shown in Figure 22-1.

Figure 22-1. The Inventory and InventoryAudit tables

If you expand the Columns node, you will see the two new fields added to the table, and expanding the
InventoryAudit table, you will see that the table is a clone of the Inventory table, as shown in Figure 22-2.

Figure 22-2. The Inventory and InventoryAudit columns

An important note regarding temporal tables: regular queries will not return the timestamp fields as EF Core configures them as HIDDEN. If you execute the following query, you will see only the regular fields returned:

SELECT * FROM dbo.Inventory

To return the new fields, they must be explicitly specified, like this:

SELECT *,ValidFrom, ValidTo FROM dbo.Inventory

When using LINQ to query the table, EF Core includes the timestamp properties in the query. For example, the following query works as expected without any modification to the Car class:

var c = context.Cars.First();

Notice in the generated SQL the inclusion of the two timestamp columns:

SELECT TOP(1) [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDeleted],
[i].[IsDrivable], [i].[MakeId], [i].[PetName], [i].[Price], [i].[TimeStamp],
[i].[ValidFrom], [i].[ValidTo]
FROM [dbo].[Inventory] AS [i]
WHERE [i].[IsDrivable] = CAST(1 AS bit)

EF Core adds the additional columns onto your entity as shadow properties. As demonstrated in the previous section, shadow properties exist as columns in the database table, but are not explicitly defined on the entity.
The problem happens when you are instantiating classes using FromSqlRaw()/FromSqlInterpolated(), stored procedures, or user-defined functions. As you have learned when populating entities by any means other than LINQ, the query must return every property that is expected. Even though the ValidFrom and ValidTo properties are not defined on the Car entity, EF Core still expects them to be returned.
This breaks some previous code in this chapter. To fix it up, first go back to the UsingFromSql() local function and update the FromSqlInterpolated() call to the following:

var car = context.Cars
.FromSqlInterpolated($"Select *,ValidFrom,ValidTo from dbo.Inventory where Id = {carId}")
.Include(x => x.MakeNavigation)
.First();

Next, update the table-valued function udtf_GetCarsForMake() to the following:

ALTER FUNCTION udtf_GetCarsForMake ( @makeId int ) RETURNS TABLE
AS RETURN (
-- Add the SELECT statement with parameter references here SELECT Id, IsDrivable, DateBuilt, Color, PetName, MakeId,
TimeStamp, Display, Price, IsDeleted, ValidFrom, ValidTo
FROM Inventory WHERE MakeId = @makeId
)

Main Table and History Table Interactions
Because the PeriodStart and PeriodEnd column values are maintained by SQL Server, you can continue to use the Car entity as you did prior to adding in the temporal table support. Updates and deletions always
target the main table, and queries that don’t reference the shadow properties retrieve the current state of the table. While there doesn’t appear to be any differences between a nontemporal and a temporal table when used with normal LINQ-based CRUD operations, behind the scenes there is constant interaction with the history table.
To see this in action, the following local function adds, updates, and then deletes a record from the
Inventory table using normal EF Core interactions:

static void TemporalTables()
{
Console.WriteLine("Temporal Tables");
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null);
//ensure there is at least one Make record var make = new Make { Name = "Honda" }; context.Makes.Add(make); context.SaveChanges();
//create a Car record to work with var car = new Car
{
Color = "LightBlue",
MakeId = context.Makes.First().Id, PetName = "Sky",
IsDrivable = true,
DateBuilt = new DateTime(2021, 01, 01)
};
context.Cars.Add(car); context.SaveChanges(); car.Color = "Cloudy"; context.Cars.Update(car); context.SaveChanges(); context.Cars.Remove(car); context.SaveChanges();
}

Each insert, edit, or deletion is tracked in the history table. When a record is inserted into the table, the ValidFrom value is set to the time the insertion transaction started, and the ValidTo is set to the max date for a datetime2, which is 12/31/9999 23:59:59.9999999. Nothing exists in the history table.

■ Note all times are stored in UTC time and are recorded at the start of the transaction. Therefore, all rows in a transaction will have the same time recorded in their appropriate tracking fields.

If you step through the code and pause right after the SaveChanges() call that inserted the new Car entity, you can execute the following queries in Azure Data Studio (or SSMS) to verify that there is a record in the Inventory table and nothing in the InventoryAudit table, as well as examine the ValidFrom and ValidTo values:

SELECT ,ValidFrom,ValidTo FROM dbo.Inventory SELECT FROM audits.InventoryAudit

■ Note The history table does not hide the timestamp fields, so using the SELECT * syntax will retrieve all the columns. It’s only the main table that hides the timestamp fields.

When the record is updated, a copy of the record to be updated is added into the history table (prior to the update statement), and the ValidTo value is set to the start of the transaction. In the main table, the ValidFrom value is set to the time of the update transaction started, and the ValidTo is set to the max date. If you query the tables after the SaveChanges() call that updates the Car record, you can see the two records, one in each table.
When a record is deleted, a copy of the record to be deleted is added into the history table (prior to the delete statement), and the ValidTo value is set to the start of the transaction. In the main table, the record is simply deleted. Now when you query the two tables, you will see there isn’t any data in the Inventory table and two records in the InventoryAudit table.

Querying Temporal Tables
The previous example showed querying both the Inventory and InventoryAudit tables. While this showed the interactions between the two tables, there isn’t any need to query the history table. SQL Server added the FOR SYSTEM_TIME clause that uses the main table and the history table to reconstruct the state of the data at the time(s) specified. There are five subclauses that can be used with the FOR SYSTEM_TIME clause, and they are listed in Table 22-3.

Table 22-3. For System Time Subclauses When Querying Temporal Tables (T-SQL)

Sub Clause Qualifying Rows Meaning in Life
AS OF PeriodFrom <= date_time and PeriodTo > date_time Returns rows that were current at the specified point in time. Internally, a union between the main table and the history table to return the valid rows.
FROM TO PeriodFrom < enddate time and PeriodTo > start_date_time Returns all row versions that were current within the specified time range. Note that the boundaries are exclusive.
BETWEEN TO PeriodFrom <= enddate time and PeriodTo > start_datetime Returns all row versions that were current within the specified time range. Note that the PeriodFrom boundary is inclusive and the PeriodTo boundary is exclusive.
CONTAINED IN (<start
datetime>, <end datetime>) PeriodFrom >= start date_time and PeriodTo

= end_date_time Returns all rows that were active only within the specified time range. Note that the boundaries are inclusive.
ALL Returns all records.

The FOR SYSTEM_TIME clause filters out records that have a period of validity that is zero (PeriodFrom = PeriodTo). To make our example a little more meaningful, add some Thread.Sleep() calls after each of the SaveChanges() calls:

static void TemporalTables()
{
..//omitted for brevity context.Cars.Add(car); context.SaveChanges(); Thread.Sleep(5000); car.Color = "Cloudy"; context.Cars.Update(car); context.SaveChanges(); Thread.Sleep(5000); context.Cars.Remove(car); context.SaveChanges();
}

EF Core 6 has added query operators that are translated by the SQL Server provider into FOR SYSTEM_ TIME and the subclauses listed in Table 22-3. Table 22-4 shows the new operators.

Table 22-4. EF Core Support for the System Time Subclauses When Querying Temporal Tables

Sub Clause Translated To Meaning in Life
TemporalAsOf() AS OF Returns rows that were current at the specified point in time. Internally, a union between the main table and the history table to return the valid rows.
TemporalFromTo() FROM TO Returns all row versions that were current within the specified time range. Note that the boundaries are exclusive.
TemporalBetween() BETWEEN TO Returns all row versions that were current within the specified time range. Note that the PeriodFrom boundary is inclusive and the PeriodTo boundary is exclusive.
TemporalContainedIn() CONTAINED IN (, ) Returns all rows that were active only within the specified time range. Note that the boundaries are inclusive.
TemporalAll() ALL Returns all records.

Remember that PeriodStart and PeriodEnd (ValidFrom and ValidTo in our example) are shadow properties, and not defined in the Car entity. Therefore, the following query won’t work:

//This doesn't work with shadow properties
var cars = context.Cars.TemporalAll().OrderBy(e => e.ValidFrom);

Instead, you need to use the EF.Property<>() method to access the shadow properties:

var cars = context.Cars.TemporalAll().OrderBy(e => EF.Property(e, "ValidFrom"));

It is important to note that if you want the historical from and to dates, you must retrieve those explicitly in your query using the EF.Property<>() method. The following LINQ statement that returns all the current and historical data, using a projection to capture each row and its time values:

var cars = context.Cars
.TemporalAll()
.OrderBy(e => EF.Property(e, "ValidFrom"))
.Select(
e => new
{
Car = e,
ValidFrom = EF.Property(e, "ValidFrom"), ValidTo = EF.Property(e, "ValidTo")
});
foreach (var c in cars)
{
Console.WriteLine(
$"{c.Car.PetName} was painted {c.Car.Color} was active from {c.ValidFrom} to {c.ValidTo}");
}

When you examine the generated SQL, you can see that the call uses the FOR SYSTEM_TIME clause with the ALL subclause:

SELECT [i].[Id], [i].[Color], [i].[DateBuilt], [i].[Display], [i].[IsDeleted], [i].[IsDrivable],
[i].[MakeId], [i].[PetName], [i].[Price], [i].[TimeStamp], [i].[ValidFrom], [i].[ValidTo] FROM [dbo].[Inventory] FOR SYSTEM_TIME ALL AS [i]
WHERE [i].[IsDrivable] = CAST(1 AS bit) ORDER BY [i].[ValidFrom]

As a final note on querying temporal tables, all queries using one of the temporal operators are nontracking queries. If you wanted to restore a record that was deleted, for example, you would use one of the temporal operators to get the historical record and call Add() on the DbSet and then call SaveChanges().

Clearing Temporal Tables
At this point, you might be wondering how you completely clear temporal tables. The short answer is that you can’t, not without removing the versioning, clearing the historical data, and then adding the versioning back in. When versioning is turned off, the history table and the main table become disassociated. Then you can delete all records from the main table (which no longer records the history) and the history table;
then you can turn versioning back on, and the tables will be re-associated. To do this in Azure Data Studio or SSMS, enter the following commands:

ALTER TABLE [dbo].[Inventory]
SET (SYSTEM_VERSIONING = OFF)
DELETE FROM [dbo].[Inventory]
DELETE FROM [audits].[InventoryAudit] ALTER TABLE [dbo].[Inventory]
SET (SYSTEM_VERSIONING = ON (HISTORY_TABLE=audits.InventoryAudit))

If the history table has a custom name (like audits.InventoryAudit), it must be specified when turning versioning back on, or a new history table will be created.

To clear the history table using EF Core, the same statements are executed in an explicit transaction using the ExecuteSqlRaw() command on the context’s database façade:

var strategy = context.Database.CreateExecutionStrategy(); strategy.Execute(() =>
{
using var trans = context.Database.BeginTransaction(); context.Database.ExecuteSqlRaw($"ALTER TABLE dbo.Inventory SET (SYSTEM_VERSIONING = OFF)"); context.Database.ExecuteSqlRaw($"DELETE FROM audits.InventoryHistory"); context.Database.ExecuteSqlRaw($"ALTER TABLE dbo.Inventory SET (SYSTEM_VERSIONING = ON (HISTORY_TABLE={historySchema}.{historyTable}))");
trans.Commit();
});

Making this more generic is a bit more complicated. There is a method (IsTemporal()) on the entity that checks if a table is temporal, and two methods to get the history table name (GetHistoryTableName()) and schema (GetHistoryTableSchema()). While IsTemporal() works at runtime, the methods to get the table name and schema do not work against the runtime model. The runtime model contains just what is needed for EF Core (and your code) to execute, while the design-time model contains everything. To use these methods, you have to get an instance of the design-time model at runtime.
To enable access to the design time model at runtime, the project file must be updated. The Microsoft.
EntityFrameworkCore.Design package is a DevelopmentDependency package. This means that the dependency won’t flow into other projects, and you can’t, by default, reference its types. To reference its types in your code, update the package metadata in the project file by removing the tag:



all

To get the design-time model, create a new ServiceCollection and add the DbContextDesignTimeServices.
After building the service provider, you can then get an instance of the design-time model:

var serviceCollection = new ServiceCollection(); serviceCollection.AddDbContextDesignTimeServices(context);
var serviceProvider = serviceCollection.BuildServiceProvider(); var designTimeModel = serviceProvider.GetService();

■ Note If you are not familiar with the ServiceCollection, it is used by .neT Core for dependency injection. Dependency injection will be covered in depth in the aSp.neT Core chapters, later in this book.

With this in place, you can update the calls to clear the historical data as follows:

var strategy = context.Database.CreateExecutionStrategy(); strategy.Execute(() =>
{
using var trans = context.Database.BeginTransaction();
var designTimeEntity = designTimeModel.FindEntityType(entityName); var historySchema = designTimeEntity.GetHistoryTableSchema();

var historyTable = designTimeEntity.GetHistoryTableName(); context.Database.ExecuteSqlRaw(
$"ALTER TABLE {schemaName}.{tableName} SET (SYSTEM_VERSIONING = OFF)");
context.Database.ExecuteSqlRaw($"DELETE FROM {historySchema}.{historyTable}"); context.Database.ExecuteSqlRaw(
$"ALTER TABLE {schemaName}.{tableName} SET (SYSTEMVERSIONING = ON (HISTORY
TABLE={historySchema}.{historyTable}))");
trans.Commit();
});

Here is the updated ClearSampleData() method in its entirety:

static void ClearSampleData()
{
//The factory is not meant to be used like this, but it's demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var entities = new[]
{
typeof(Car).FullName, typeof(Make).FullName,
};
var serviceCollection = new ServiceCollection(); serviceCollection.AddDbContextDesignTimeServices(context);
var serviceProvider = serviceCollection.BuildServiceProvider(); var designTimeModel = serviceProvider.GetService();

foreach (var entityName in entities)
{
var entity = context.Model.FindEntityType(entityName); var tableName = entity.GetTableName();
var schemaName = entity.GetSchema(); context.Database.ExecuteSqlRaw($"DELETE FROM {schemaName}.{tableName}");
context.Database.ExecuteSqlRaw($"DBCC CHECKIDENT (\"{schemaName}.{tableName}\", RESEED, 0);");
if (entity.IsTemporal())
{
var strategy = context.Database.CreateExecutionStrategy(); strategy.Execute(() =>
{
using var trans = context.Database.BeginTransaction();
var designTimeEntity = designTimeModel.FindEntityType(entityName); var historySchema = designTimeEntity.GetHistoryTableSchema();
var historyTable = designTimeEntity.GetHistoryTableName(); context.Database.ExecuteSqlRaw(
$"ALTER TABLE {schemaName}.{tableName} SET (SYSTEM_VERSIONING = OFF)");
context.Database.ExecuteSqlRaw($"DELETE FROM {historySchema}.{historyTable}");

context.Database.ExecuteSqlRaw($"ALTER TABLE {schemaName}.{tableName} SET (SYSTEM_ VERSIONING = ON (HISTORY_TABLE={historySchema}.{historyTable}))");
trans.Commit();
});
}
}
}

Summary
This chapter began with a long look at create, read, update, and delete (CRUD) operations using EF Core and then explored several EF Core features to help with developer productivity.
Now that you have a solid foundation for how EF Core works, the next chapter will build the AutoLot
data access layer.

Pro C#10 CHAPTER 21 Introducing Entity Framework Core

PART VII

Entity Framework Core

CHAPTER 21

Introducing Entity Framework Core

The previous chapter examined the fundamentals of ADO.NET. As you saw, ADO.NET enables .NET programmers to work with relational data. While ADO.NET is an effective tool for working with data, it isn’t necessarily an efficient tool. The efficiency that I am referring to is developer efficiency. To help with the developer efficiency, Microsoft introduced a new framework for data access called the Entity Framework (or simply, EF) in .NET 3.5 Service Pack 1.
EF provides the capability to interact with data from relational databases using an object model that maps directly to the business objects (or domain objects) in your application. For example, rather than treating a batch of data as a collection of rows and columns, you can operate on a collection of strongly typed objects termed entities. These entities are held in specialized collection classes that are LINQ aware, enabling data access operations using C# code. The collection classes provide querying against the data store using the same LINQ grammar you learned about in Chapter 13.
In addition to working with your data as the application domain model (instead of a normalized database model), EF provides efficiencies such as state tracking, unit of work operations, and intrinsic transaction support.
Entity Framework Core is a complete rewrite of Entity Framework 6. It is built on top of the .NET 6 Framework, enabling EF Core to run on multiple platforms. Rewriting EF Core has enabled the team to add new features and performance improvements to EF Core that couldn’t be reasonably implemented in EF 6.
Re-creating an entire framework from scratch requires a hard look at which features will be supported in the new framework and which features will be left behind. One of the features of EF 6 that is not in EF Core (and not likely to ever be added) is support for the Entity Designer. EF Core only supports what is called code-first development. The name is really a terrible name since it infers you can’t use EF Core with an existing database. It really means “without a designer,” but that wasn’t the name that was chosen. EF Core can be used with existing databases that can be scaffolded into entity classes and a derived DbContext, or you can use EF Core to create/update your database from your entity classes and derived DbContext. I will cover both of these scenarios shortly.
With each release, EF Core has added more features that existed in EF 6 as well as new features that never existed in EF 6. The 3.1 release significantly shortened the list of essential features that are missing from EF Core (as compared to EF 6), and 5.0 closed the gap even more. The release of EF Core 6.0 has solidified the framework, and now, for most projects, EF Core has everything you need.
This chapter and the next three will introduce you to data access using EF Core. You will learn about the following: creating a domain model, mapping entity classes and properties to the database tables and columns, implementing change tracking, using the EF Core command-line interface (CLI) for scaffolding and migrations, as well as the role of the DbContext class. You will also learn about relating entities with navigation properties, transactions, and concurrency checking, just to name a few of the features explored. The fourth and final chapter on EF Core exercises the data access layer by using a series of integration tests. These tests demonstrate using EF Core for create, read, update, and delete (CRUD) operations.
By the time you complete these chapters, you will have the final version of the data access layer for our
AutoLot database. Before we get into EF Core, let’s talk about object-relational mappers in general.

© Andrew Troelsen, Phil Japikse 2022
A. Troelsen and P. Japikse, Pro C# 10 with .NET 6, https://doi.org/10.1007/978-1-4842-7869-7_21

861

■ Note Four chapters are not nearly enough to cover all of Entity Framework Core, as entire books (some the size of this one) are dedicated to just EF Core. The intent of these chapters is to give you a working knowledge to get you started using EF Core for your line-of-business applications.

Object-Relational Mappers
ADO.NET provides you with a fabric that lets you select, insert, update, and delete data with connections, commands, and data readers. While this is all well and good, these aspects of ADO.NET force you to treat the fetched data in a manner that is tightly coupled to the physical database schema. Recall, for example, when getting records from the database, you open a connection, create and execute a command object, and then use a data reader to iterate over each record using database-specific column names.
When you use ADO.NET, you must always be mindful of the physical structure of the back-end database. You must know the schema of each data table, author potentially complex SQL queries to interact with data table(s), track changes to the retrieved (or added) data, etc. This can force you to author some fairly verbose C# code because C# itself does not speak the language of the database schema directly.
To make matters worse, the way in which a physical database is usually constructed is squarely focused on database constructs such as foreign keys, views, stored procedures, and data normalization, not object- oriented programming.
Another concern for application developers is change tracking. Getting the data from the database is one step of the process, but any changes, additions, and/or deletions must be tracked by the developer so they can be persisted back to the data store.
The availability of object-relational mapping frameworks (commonly referred to as ORMs) in .NET greatly enhanced the data access story by managing the bulk of CRUD data access tasks for the developer. The developer creates a mapping between the .NET objects and the relational database, and the ORM manages connections, query generation, change tracking, and persisting the data. This leaves the developer free to focus on the business needs of the application.

■ Note It is important to remember that ORMs are not magical unicorns riding on rainbows. Every decision involves trade-offs. ORMs reduce the amount of work for developers creating data access layers but can also introduce performance and scaling issues if used improperly. Use ORMs for CRUD operations and use the power of your database for set-based operations.

Even though the different ORMs have slight differences in how they operate and how they are used, they all have essentially the same pieces and parts and strive for the same goal—to make data access operations easier. Entities are classes that are mapped to the database tables. A specialized collection type contains one or more entities. A change tracking mechanism tracks the state of the entities and any changes, additions, and/or deletions made to them, and a central construct controls operations as the ringleader.

Understanding the Role of the Entity Framework Core
Under the covers, EF Core uses the ADO.NET infrastructure you have already examined in the previous chapter. Like any ADO.NET interaction with a data store, EF Core uses an ADO.NET data provider for data store interactions. Before an ADO.NET data provider can be used by EF Core, it must be updated to fully integrate with EF Core. Due to this added functionality, you might have fewer EF Core data providers available than ADO.NET data providers.

The benefit of EF Core using the ADO.NET database provider pattern is that it enables you to combine EF Core and ADO.NET data access paradigms in the same project, augmenting your capabilities. For example, using EF Core to provide the connection, schema, and table name for bulk copy operations leverages the mapping capabilities of EF Core and the BCP functionality built into ADO.NET. This blended approach makes EF Core just another tool in your tool chest.
When you see how much of the basic data access plumbing is handled for you in a convenient and efficient manner, EF Core will most likely become your go-to mechanism for data access.

■ Note Many third-party databases (e.g., Oracle and MySQL) provide EF-aware data providers. If you are not using SQL Server, consult your database vendor for details or navigate to https://docs.microsoft.com/ en-us/ef/core/providers for a list of available EF Core data providers.

EF Core best fits into the development process in forms-over-data (or API-over-data) situations.
Operations on small numbers of entities using the unit of work pattern to ensure consistency is the sweet spot for EF Core. It is not very well suited for large-scale data operations such as extract-transform-load (ETL) data warehouse applications or large reporting situations.

The Building Blocks of the Entity Framework
The main components of EF Core are DbContext, ChangeTracker, the DbSet specialized collection type, the database providers, and the application’s entities. To work through this chapter, create a new
Console Application named AutoLot.Samples and add the Microsoft.EntityFrameworkCore, Microsoft. EntityFrameworkCore.Design, and Microsoft.EntityFrameworkCore.SqlServer packages. Remember to disable nullable reference types in the project file:


Exe
net6.0
enable
disable

The Microsoft.EntityFrameworkCore package provides the common functionality for EF Core. The Microsoft.EntityFrameworkCore.SqlServer package supplies the SQL Server data provider, and the Microsoft.EntityFrameworkCore.Design package is required for the EF Core command-line tools.

■ Note If you prefer to use the nuget package Manager Console to run the EF Core commands, install the Microsoft.EntityFrameworkCore.Tools package. This text does not cover the nuget-style commands since the CLI works across all platforms and doesn’t rely on Visual Studio.

Add a new file named GlobalUsings.cs, clear out the template code, and update the file to match the following:

global using Microsoft.EntityFrameworkCore;
global using Microsoft.EntityFrameworkCore.ChangeTracking; global using Microsoft.EntityFrameworkCore.Design;

global using Microsoft.EntityFrameworkCore.Metadata;
global using Microsoft.EntityFrameworkCore.Metadata.Builders;

global using System.ComponentModel.DataAnnotations;
global using System.ComponentModel.DataAnnotations.Schema;

The DbContext Class
The DbContext class is the ringleader component of EF Core and provides access to the database through the Database property. DbContext manages the ChangeTracker instance, exposes the virtual
OnModelCreating() method for access to the Fluent API, holds all the DbSet properties, and supplies the SaveChanges method to persist data to the data store. It is not used directly, but through a custom class that inherits DbContext. It is in the derived class that the DbSet properties are placed.
Table 21-1 shows some of the more commonly used members of DbContext.

Table 21-1. Common Members of DbContext

Member of DbContext Meaning in Life
Database Provides access to database-related information and functionality, including execution of SQL statements.
Model The metadata about the shape of entities, the relationships between them, and how they map to the database. Note: This property is usually not interacted with directly.
ChangeTracker Provides access to information and operations for entity instances this
DbContext is tracking.
DbSet Not truly a member of DbContext, but properties added to the custom derived DbContext class. The properties are of type DbSet and are used to query and save instances of application entities. LINQ queries against DbSet properties are translated into SQL queries.
Entry() Provides access to change tracking information and operations for the entity, such as explicitly loading related entities or changing the EntityState. Can also be called on an untracked entity to change the state to tracked.
Set() Creates an instance of the DbSet property that can be used to query and persist data.
SaveChanges()/SaveChange sAsync() Saves all entity changes to the database and returns the number of records affected. Executes in a transaction (implicit or explicit).
Add()/AddRange() Update()/UpdateRange() Remove()/RemoveRange() Methods to add, update, and remove entity instances. Changes are persisted only when SaveChanges() is executed successfully. Async versions are available as well. Note: While available on the derived DbContext, these methods are usually called directly on the DbSet properties.
Find() Finds an entity of a type with the given primary key values. Async versions are available as well. Note: While available on the derived DbContext, these methods are usually called directly on the DbSet properties.
(continued)

Table 21-1. (continued)

Member of DbContext Meaning in Life
Attach()/AttachRange() Begins tracking an entity (or list of entities). Async versions are available as well. Note: While available on the derived DbContext, these methods are usually called directly on the DbSet properties.
SavingChanges() Event fired at the beginning of a call to SaveChanges()/SaveChangesAsync().
SavedChanges() Event fired at the end of a call to SaveChanges()/SaveChangesAsync().
SaveChangesFailed Event fired if a call to SaveChanges()/SaveChangesAsync() fails.
OnModelCreating() Called when a model has been initialized, but before it’s finalized. Methods from the Fluent API are placed in this method to finalize the shape of the model.
OnConfiguring() A builder used to create or modify options for DbContext. Executes each time a DbContext instance is created. Note: It is recommended not to use this and instead use DbContextOptions to configure the DbContext instance at runtime and use an instance of IDesignTimeDbContextFactory at design time.

Creating a Derived DbContext
The first step in EF Core is to create a custom class that inherits from DbContext. Then add a constructor that accepts a strongly typed instance of DbContextOptions (covered next) and passes the instance through to the base class. Add a file named ApplicationDbContext.cs and update the code to match the following:

namespace AutoLot.Samples;

public class ApplicationDbContext : DbContext
{
public ApplicationDbContext(DbContextOptions options)
: base(options)
{
}
}

This is the class that is used to access the database and work with entities, the change tracker, and all components of EF Core.

Configuring the DbContext
The DbContext instance is configured using an instance of the DbContextOptions class. The DbContextOptions instance is created using DbContextOptionsBuilder, as the DbContextOptions class is not meant to be directly constructed in your code. Through the DbContextOptionsBuilder instance, the database provider is selected (along with any provider-specific settings), and EF Core DbContext general options (such as logging) are set. Then the instance of the DbContextOptions is injected into the base DbContext at runtime. You will see this in action in the next section.
This dynamic configuration capability enables changing settings at runtime simply by selecting different options (e.g., MySQL instead of the SQL Server provider) and creating a new instance of your derived DbContext.

The Design-Time DbContext Factory
The design-time DbContext factory is a class that implements the IDesignTimeDbContextFactory interface, where T is the derived DbContext class. The interface has one method, CreateDbContext(), that you must implement to create an instance of your derived DbContext. This class is not meant for production use, but only during development, and exists primarily for the EF Core command-line tools, which you will explore shortly. In the examples in this and the next chapter, it will be used to create new instances of the ApplicationDbContext.

■ Note It is considered bad practice to use the DbContext factory to create instances of your derived DbContext class. Remember that this is demo code meant for teaching, and using it in this way keeps the demo code cleaner. you will see how to properly instantiate your derived DbContext class in the chapters on windows presentation Foundation and aSp.nET Core.

The following ApplicationDbContextFactory class uses the CreateDbContext() method to create a strongly typed DbContextOptionsBuilder for the ApplicationDbContext class, sets the database provider to the SQL Server provider (using the Docker instance connection string from Chapter 20), and then creates and returns a new instance of the ApplicationDbContext:

namespace AutoLot.Samples;

public class ApplicationDbContextFactory : IDesignTimeDbContextFactory
{
public ApplicationDbContext CreateDbContext(string[] args)
{
var optionsBuilder = new DbContextOptionsBuilder();
var connectionString = @"server=.,5433;Database=AutoLotSamples;User Id=sa;Password= P@ssw0rd;Encrypt=False;";
optionsBuilder.UseSqlServer(connectionString); Console.WriteLine(connectionString);
return new ApplicationDbContext(optionsBuilder.Options);
}
}

■ Note The database name used in these samples is AutoLotSamples, and not AutoLot, which was the name used in Chapter 20. The autoLot database will be updated to its final form starting with Chapter 20.

Again, the context factory is designed for the EF Core command-line interface to create an instance of the derived DbContext class, and not for production use. The command-line interface uses the factory when performing actions such as creating or applying database migrations. One major reason that you don’t want to use this in production is the hard-coded connection string. Since this is for design-time use, using a set connection string that points to the development database works perfectly.
The CreateDbContext() method takes a string array as argument. While not used in earlier versions, support for passing in arguments from the command line into the IDesignTimeDbContextFactory CreateDbContext() method was added in EF Core 5.

OnModelCreating
The base DbContext class exposes the OnModelCreating method that is used to shape your entities using the Fluent API. This will be covered in depth later in this chapter, but for now, add the following code to the ApplicationDbContext class:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
// Fluent API calls go here
}

Saving Changes
To persist any changes (add, update, or delete) to entities, call the SaveChanges() (or SaveChangesAsync()) method on the derived DbContext. The SaveChanges()/SaveChangesAsync() methods wrap the database calls in an implicit transaction and persist them as a unit of work. Transactions are covered next, and the change tracker is covered later in this section.
Add the following global using statement to the GlobalUsings.cs file:

global using AutoLot.Samples;

Clear out any code in the Program.cs file, and update it to match the following:
Console.WriteLine("Fun with Entity Framework Core"); static void SampleSaveChanges()
{
//The factory is not meant to be used like this, but it’s demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null);
//make some changes context.SaveChanges();
}

There will be many examples of saving changes through the rest of this chapter (and book).

Transaction and Save Point Support
As mentioned previously, EF Core wraps each call to SaveChanges()/SaveChangesAsync() in an implicit transaction. By default, the transaction uses the isolation level of the database. For more control, you can enlist the derived DbContext into an explicit transaction instead of using the default implicit transaction. To execute in an explicit transaction, create a transaction using the Database property of the derived DbContext. Conduct your operation(s) as usual and then commit or roll back the transaction. Here is a code snippet that demonstrates this:

static void TransactedSaveChanges()
{
//The factory is not meant to be used like this, but it’s demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); using var trans = context.Database.BeginTransaction();

try
{
//Create, change, delete stuff context.SaveChanges(); trans.Commit();
}
catch (Exception ex)
{
trans.Rollback();
}
}

Save points for EF Core transactions were introduced in EF Core 5. When SaveChanges()/SaveChange sAsync() is called and a transaction is already in progress, EF Core creates a save point in that transaction. If the call fails, the transaction is rolled back to the save point and not the beginning of the transaction. Save points can also be managed programmatically by calling CreateSavePoint() and RollbackToSavepoint() on the transaction, like this:

static void UsingSavePoints()
{
//The factory is not meant to be used like this, but it’s demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); using var trans = context.Database.BeginTransaction();
try
{
//Create, change, delete stuff trans.CreateSavepoint("check point 1"); context.SaveChanges();
trans.Commit();
}
catch (Exception ex)
{
trans. RollbackToSavepoint("check point 1");
}
}

Explicit Transactions and Execution Strategies
When an execution strategy is active (covered in the next chapter in the “Connection Resiliency” section), before creating an explicit transaction, you must get a reference to the current execution strategy in use. Then call the Execute() method on the strategy to create an explicit transaction.

static void TransactionWithExecutionStrategies()
{
//The factory is not meant to be used like this, but it’s demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null); var strategy = context.Database.CreateExecutionStrategy(); strategy.Execute(() =>
{
using var trans = context.Database.BeginTransaction();

try
{
//actionToExecute(); trans.Commit();
Console.WriteLine("Insert succeeded");
}
catch (Exception ex)
{

}
});
}

trans.Rollback();
Console.WriteLine($"Insert failed: {ex.Message}");

Saving/Saved Changes Events
EF Core 5 introduced three new events that are triggered by the SaveChanges()/SaveChangesAsync() methods. The SavingChanges event fires when SaveChanges() is called but before the SQL statements are executed against the data store. The SavedChanges event fires after SaveChanges() has completed. The SaveChangesFailed event fires if the call to SaveChanges() was unsuccessful. The following (trivial) code examples in the ApplicationDbContext class constructor show the events and their handlers in action:

public ApplicationDbContext(DbContextOptions options)
: base(options)
{
SavingChanges += (sender, args) =>
{
Console.WriteLine($"Saving changes for {((DbContext)sender).Database.
GetConnectionString()}");
};
SavedChanges += (sender, args) =>
{
Console.WriteLine($"Saved {args.EntitiesSavedCount} entities");
};
SaveChangesFailed += (sender, args) =>
{
Console.WriteLine($"An exception occurred! {args.Exception.Message} entities");
};
}

The DbSet Class
For each entity type (T) in your object model, you add a property of type DbSet to the derived DbContext
class. The DbSet class is a specialized collection property used to interact with the database provider to read, add, update, or delete records in the database. Each DbSet provides a number of core services for the database interactions, including translating LINQ queries executed against a DbSet property into database queries by the database provider. Table 21-2 describes some of the core members of the DbSet class.

Table 21-2. Common Members and Extension Methods of DbSet

Member of DbSet Meaning in Life
Add()/AddRange() Begins tracking the entity/entities in the Added state. Item(s) will be added when SaveChanges() is called. Async versions are available as well.
AsAsyncEnumerable() Returns the collection as IAsyncEnumerable.
AsQueryable() Returns the collection as IQueryable.
Find() Searches for the entity in the ChangeTracker by primary key. If not found in the change tracker, the data store is queried for the object. An async version is available as well.
Update()UpdateRange() Begins tracking the entity/entities in the Modified state. Item(s) will be updated when SaveChanges is called. Async versions are available as well.
Remove()RemoveRange() Begins tracking the entity/entities in the Deleted state. Item(s) will be removed when SaveChanges() is called. Async versions are available as well.
Attach()AttachRange() Begins tracking the entity/entities. Entities with numeric primary keys defined as an identity and value equaling zero are tracked as Added. All others are tracked as Unchanged. Async versions are available as well.
FromSqlRaw() FromSqlInterpolated() Creates a LINQ query based on a raw or interpolated string representing a SQL query. Can be combined with additional LINQ statements for server-side execution.
AsQueryable() Returns an IQueryable instance from DbSet.

The DbSet type implements IQueryable, which enables the use of LINQ queries to retrieve records from the database. In addition to extension methods added by EF Core, DbSet supports the same extension methods you learned about in Chapter 13, such as ForEach(), Select(), and All().
You will be adding the DbSet properties to ApplicationDbContext in the “Entities” section.

■ Note Many of the methods listed in Table 21-2 are named the same as the methods in Table 21-1. The main difference is that the DbSet methods already know the type to operate on and have the list of entities. The DbContext methods must determine what to act on using reflection. It is much more common to use the methods on the DbSet properties rather than the more general methods on the derived DbContext.

The ChangeTracker
The ChangeTracker instance tracks the state for objects loaded into DbSet within a DbContext instance. Table 21-3 lists the possible values for the state of an object.

Table 21-3. Entity State Enumeration Values

Value Meaning in Life
Added The entity is being tracked but does not yet exist in the database.
Deleted The entity is being tracked and is marked for deletion from the database.
Detached The entity is not being tracked by the change tracker.
Modified The entry is being tracked and has been changed.
Unchanged The entity is being tracked, exists in the database, and has not been modified.

If you need to check the state of an object, use the following code:

EntityState state = context.Entry(entity).State;

You can also programmatically change the state of an object using the same mechanism. To change the state to Deleted (for example), use the following code:

context.Entry(entity).State = EntityState.Deleted;

ChangeTracker Events
There are two events that can be raised by ChangeTracker. The first is StateChanged, and the second is Tracked. The StateChanged event fires when an entity’s state is changed. It does not fire when an entity is first tracked. The Tracked event fires when an entity starts being tracked, either by being programmatically added to a DbSet instance or when returned from a query.
Update the constructor for the ApplicationDbContext class to the following to specify the event handlers for the StateChanged and Tracked events:

public ApplicationDbContext(DbContextOptions options)
: base(options)
{
...
ChangeTracker.StateChanged += ChangeTracker_StateChanged; ChangeTracker.Tracked += ChangeTracker_Tracked;
}

The StateChanged Event
As mentioned, the StateChanged event fires when the state of an entity changes, but not when an entity is first tracked. The OldState and NewState are exposed through the EntityStateChangedEventArgs. The following example writes to the console anytime an entity is updated:

private void ChangeTracker_StateChanged(object sender, EntityStateChangedEventArgs e)
{
if (e.OldState == EntityState.Modified && e.NewState == EntityState.Unchanged)
{
Console.WriteLine($"An entity of type {e.Entry.Entity.GetType().Name} was updated.");
}
}

The Tracked Event
The Tracked event fires when ChangeTracker starts to track an entity. The FromQuery property of the EntityTrackedEventArgs indicates if the entity was loaded via a database query or programmatically. The following example writes to the console anytime an entity is loaded from the database:

private void ChangeTracker_Tracked(object sender, EntityTrackedEventArgs e)
{
if (e.FromQuery)
{
Console.WriteLine($"An entity of type {e.Entry.Entity.GetType().Name} was loaded from the database.");
}
}

Resetting DbContext State
EF Core 5 added the ability to reset a derived DbContext back to its original state. The ChangeTracker. Clear() method clears out all entities from the DbSet collections by setting their state to Detached. The main benefit of this is to improve performance. As with any ORM, there is some overhead that comes with instantiating a derived DbContext class. While that overhead isn’t usually significant, the ability to clean up an already instantiated context could help with performance in some scenarios.

Entities
The strongly typed classes that map to database tables are officially called entities. The collection of entities in an application comprises a conceptual model of a physical database. Formally speaking, this model is termed an entity data model (EDM), usually referred to simply as the model. The model is mapped to the application/business domain. The entities and their properties are mapped to the tables and columns using Entity Framework Core conventions, configuration, and the Fluent API (code). Entities do not need to map directly to the database schema. You are free to structure your entity classes to fit your application needs and then map your unique entities to your database schema.
This loose coupling between the database and your entities means you can shape the entities to match your business domain, independent of the database design and structure. For example, take the simple Inventory table in the AutoLot database and the Car entity class from the previous chapter. The names are different, yet the Car entity can be mapped to the Inventory table. EF Core examines the configuration of your entities in the model to map the client-side representation of the Inventory table (in our example, the Car class) to the correct columns of the Inventory table.
The next several sections detail how EF Core conventions, data annotations, and code (using the Fluent API) map entities, properties, and the relationships between entities in the mode to the tables, columns, and foreign key relationships in your database. Each of these topics is covered in depth later in this chapter.

Entity Properties and Database Columns
When using a relational data store, EF Core uses data from a table’s columns to populate an entity’s properties when reading from the data store and writes from the entity’s properties to a table’s columns when persisting data. If the property is an automatic property, EF Core reads and writes through the getter and setter. If the property has a backing field, EF Core will read and write to the backing field instead of the public property, even if the backing field is private. While EF Core can read and write to private fields, there still must be a public read-write property that encapsulates the backing field.

Two scenarios where the backing field support is advantageous are when using the INotifyPropertyChanged pattern in Windows Presentation Foundation (WPF) applications and when database default values clash with .NET default values. Using EF Core with WPF is covered in Chapter 28, and database default values are covered later in this chapter.

Table Mapping Schemes
There are two class to table mapping schemes available in EF Core: table-per-hierarchy (TPH) and table-per- type (TPT). TPH mapping is the default and maps an inheritance hierarchy to a single table. Introduced in EF Core 5, TPT maps each class in the hierarchy to its own table.

■ Note Classes can also be mapped to views and raw SQL queries. These are referred to as query types and are covered later in this chapter.

Table-Per-Hierarchy Mapping
Consider the following example, which shows the Car class from Chapter 20 split into two classes: a base class (BaseEntity) for the Id and TimeStamp properties, and the rest of the properties in the Car class. The code for the TPH examples are in the AutoLot.TPH project in the source for this chapter.

//BaseEntity.cs
namespace AutoLot.TPH.Models;

public abstract class BaseEntity
{
public int Id { get; set; }
public byte[] TimeStamp { get; set; }
}

//Car.cs
namespace AutoLot.TPH.Models;

public class Car : BaseEntity
{
public string Color { get; set; } public string PetName { get; set; } public int MakeId { get; set; }
}

To make EF Core aware that an entity class is part of the object model, add a DbSet property for the entity. Create an ApplicationDbContext class and update it to the following:

using Microsoft.EntityFrameworkCore; using AutoLot.TPH.Models;

namespace AutoLot.TPH;

public class ApplicationDbContext : DbContext
{
public ApplicationDbContext(DbContextOptions options)
: base(options) { }
public DbSet Cars { get; set; }
}

Note the DbSet property in the ApplicationDbContext class. This informs EF Core that the Car class maps to the Cars table in the database (more on this in the “Entity Conventions” section). Also notice that there isn’t a DbSet property for the BaseEntity class. This is because in the TPH scheme, the entire hierarchy becomes a single table. The properties of the tables up the inheritance chain are folded into the table with the DbSet property. This is shown by the following SQL:

CREATE TABLE [dbo].[Cars](
[Id] [INT] IDENTITY(1,1) NOT NULL, [Color] NVARCHAR NULL, [PetName] NVARCHAR NULL, [MakeId] [INT] NOT NULL, [TimeStamp] VARBINARY NULL,
CONSTRAINT [PK_Cars] PRIMARY KEY CLUSTERED
(
[Id] ASC
)
WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]

Table-per-Type Mapping
To explore the TPT mapping scheme, the BaseEntity and Car classes can be used, even with the base class marked as abstract. Since TPH is the default, EF Core must be instructed to map each class to a table. This can be done with data annotations (shown later in this chapter) or the Fluent API. To use the TPT mapping scheme, use the following Fluent API code in the OnModelCreating() method of the ApplicationDbContext. These examples are in the AutoLot.TPT project in the chapter’s code samples.

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity().ToTable("BaseEntities"); modelBuilder.Entity().ToTable("Cars");
}

EF Core will create two tables, shown here. The indexes also show that the tables have a one-to-one mapping between the BaseEntities and Cars tables.

CREATE TABLE [dbo].[BaseEntities]( [Id] [INT] IDENTITY(1,1) NOT NULL, [TimeStamp] VARBINARY NULL,
CONSTRAINT [PK_BaseEntities] PRIMARY KEY CLUSTERED

(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] GO

CREATE TABLE [dbo].[Cars]( [Id] [INT] NOT NULL,
[Color] NVARCHAR NULL, [PetName] NVARCHAR NULL, [MakeId] [INT] NOT NULL,
CONSTRAINT [PK_Cars] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS
= ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] GO

ALTER TABLE [dbo].[Cars] WITH CHECK ADD CONSTRAINT [FK_Cars_BaseEntities_Id] FOREIGN KEY([Id])
REFERENCES [dbo].[BaseEntities] ([Id]) GO

ALTER TABLE [dbo].[Cars] CHECK CONSTRAINT [FK_Cars_BaseEntities_Id] GO

■ Note Table-per-type mapping can have significant performance implications that should be considered before using this mapping scheme. For more information, refer to the documentation: https://docs.microsoft.com/en-us/ef/core/performance/modeling-for- performance#inheritance-mapping.

Navigation Properties and Foreign Keys
Navigation properties represent how entity classes relate to each other and enable code to traverse from one entity instance to another. By definition, a navigation property is any property that maps to a nonscalar type as defined by the database provider. In practice, navigation properties map to another entity (called reference navigation properties) or a collection of another entity (called collection navigation properties).
On the database side, navigation properties are translated into foreign key relationships between tables. One-to-one, one-to-many, and (new in EF Core 5) many-to-many relationships are directly supported in EF Core. Entity classes can also have navigation properties that point back to themselves, representing self- referencing tables.

■ Note I find it helpful to consider objects with navigation properties as linked lists, and if the navigation properties are bidirectional, the objects act like doubly linked lists.

Before covering the details of navigation properties and entity relationship patterns, refer to Table 21-4.
These terms are used in all three relationship patterns.

Table 21-4. Terms Used to Describe Navigation Properties and Relationships

Term Meaning in Life
Principal entity The parent of the relationship.
Dependent entity The child of the relationship.
Principal key The property/properties used to define the principal entity. Can be the primary key or an alternate key. Keys can be configured using a single property or multiple properties.
Foreign key The property/properties held by the child entity to store the principal key.
Required relationship Relationship where the foreign key value is required (non-nullable).
Optional relationship Relationship where the foreign key value is not (nullable).

Missing Foreign Key Properties
If an entity with a reference navigation property does not have a property for the foreign key value, EF Core will create the necessary property/properties on the entity. These are known as shadow foreign key
properties and are named in the format of or . This is true for all the relationship types (one-to-many, one-to-one, many-to-many). It is a much cleaner approach to build your entities with explicit foreign key property/properties than to make EF Core create them for you.

One-to-Many Relationships
To create a one-to-many relationship, the entity class on the one side (the principal) adds a collection property of the entity class that is on the many side (the dependent). The dependent entity should also have properties for the foreign key back to the principal. If not, EF Core will create shadow foreign key properties, as explained earlier.
For example, in the database created in Chapter 20, the Makes table (represented by the Make entity class) and Inventory table (represented by the Car entity class) have a one-to-many relationship.

■ Note For these initial examples, the Car class will map to a table named Cars. Later in this chapter the
Car class will be mapped to the Inventory table.

Back in the AutoLot.Samples project, create a new folder named Models. Add the following BaseEntity. cs, Make.cs, and Car.cs files as shown in the code listing. The bold code shows the bidirectional navigation properties representing the one-to-many relationship:

//BaseEntity.cs
namespace AutoLot.Samples.Models;

public abstract class BaseEntity
{
public int Id { get; set; }
public byte[] TimeStamp { get; set; }
}

//Make.cs
namespace AutoLot.Samples.Models;

public class Make : BaseEntity
{
public string Name { get; set; }
public IEnumerable Cars { get; set; } = new List();
}

//Car.cs
namespace AutoLot.Samples.Models;

public class Car : BaseEntity
{
public string Color { get; set; } public string PetName { get; set; } public int MakeId { get; set; }
public Make MakeNavigation { get; set; }
}

■ Note when scaffolding an existing database (as you will do in the next chapter), EF Core names reference navigation properties the same as the property type name (e.g., public Make Make {get; set;}). This can cause issues with navigation and IntelliSense, as well as make the code difficult to work with. I prefer to add the suffix Navigation to reference navigation properties for clarity, as shown in the previous example.

In the Car/Make example, the Car entity is the dependent entity (the many of the one-to-many), and the
Make entity is the principal entity (the one of the one-to-many).
Update the GlobalUsings.cs file to include the new namespace for the models:

global using AutoLot.Samples.Models;

Next, add the DbSet and DbSet properties to ApplicationDbContext, as shown here:

public DbSet Cars { get; set; } public DbSet Makes { get; set; }

When the database is updated using EF Core migrations, the following tables are created:

CREATE TABLE [dbo].[Makes](
[Id] [int] IDENTITY(1,1) NOT NULL,
[Name] nvarchar NULL, [TimeStamp] varbinary NULL,
CONSTRAINT [PK_Makes] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] GO

CREATE TABLE [dbo].[Cars](
[Id] [int] IDENTITY(1,1) NOT NULL,
[Color] nvarchar NULL, [PetName] nvarchar NULL, [TimeStamp] varbinary NULL, [MakeId] [int] NOT NULL,
CONSTRAINT [PK_Cars] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] GO

ALTER TABLE [dbo].[Cars] WITH CHECK ADD CONSTRAINT [FK_Cars_Makes_MakeId] FOREIGN KEY([MakeId]) REFERENCES [dbo].[Makes] ([Id])
ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Cars] CHECK CONSTRAINT [FK_Cars_Makes_MakeId]
GO

Note the foreign key and check constraints created on the dependent (Cars) table.

■ Note Updating the database using EF Core migrations is covered later in this chapter. If you are already familiar with EF Core migrations or want to skip ahead to the section on EF Core CLI commands to learn about migrations before continuing on with this section, the specific commands to create these tables are here: dotnet ef migrations add Initial -o Migrations -c AutoLot.Samples.ApplicationDbContextdotnet ef database update Initial -c AutoLot.Samples.ApplicationDbContext

One-to-One Relationships
In one-to-one relationships, both entities have a reference navigation property to the other entity. While one-to-many relationships clearly denote the principal and dependent entity, when building one-to-one
relationships, EF Core must be informed which side is the principal entity. This can be done either by having

a clearly defined foreign key to the principal entity or by indicating the principal using the Fluent API. If EF Core is not informed through one of those two methods, it will choose one based on its ability to detect a foreign key. In practice, you should clearly define the dependent by adding foreign key properties. This removes any ambiguity and ensures that your tables are properly configured.
Add a new class named Radio.cs and update the code to the following:
namespace AutoLot.Samples.Models; public class Radio : BaseEntity
{
public bool HasTweeters { get; set; } public bool HasSubWoofers { get; set; } public string RadioId { get; set; } public int CarId { get; set; }
public Car CarNavigation { get; set; }
}

Add the Radio navigation property to the Car class:
namespace AutoLot.Samples.Models; public class Car : BaseEntity
{

public Radio RadioNavigation { get; set; }
}

Since Radio has a foreign key to the Car class (based on convention, covered shortly), Radio is the dependent entity, and Car is the principal entity. EF Core creates the required unique index on the foreign key property in the dependent entity implicitly. If you want to change the name of the index, that can be accomplished using data annotations or the Fluent API.
Add the DbSet property to the ApplicationDbContext class:

public DbSet Cars { get; set; } public DbSet Makes { get; set; } public DbSet Radios { get; set; }

When the database is updated using the following EF Core migrations, the Cars table is unchanged, and the following Radios table is created:

dotnet ef migrations add Radio -o Migrations -c AutoLot.Samples.ApplicationDbContext dotnet ef database update Radio -c AutoLot.Samples.ApplicationDbContext

The following shows the added Radios table:

CREATE TABLE [dbo].[Radios](
[Id] [int] IDENTITY(1,1) NOT NULL,
[HasTweeters] [bit] NOT NULL, [HasSubWoofers] [bit] NOT NULL, [RadioId] nvarchar NULL, [TimeStamp] varbinary NULL, [CarId] [int] NOT NULL,

CONSTRAINT [PK_Radios] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] GO
ALTER TABLE [dbo].[Radios] WITH CHECK ADD CONSTRAINT [FK_Radios_Cars_CarId] FOREIGN KEY([CarId])
REFERENCES [dbo].[Cars] ([Id]) ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Radios] CHECK CONSTRAINT [FK_Radios_Cars_CarId]
GO

Note the foreign key and check constraints created on the dependent (Radios) table (shown in bold).

Many-to-Many Relationships
In many-to-many relationships, both entities have a collection navigation property to the other entity. This is implemented in the data store with a join table in between the two entity tables. This join table, by default, is named after the two tables using , but can be changed programmatically through the Fluent API. The join entity has one-to-many relationships to each of the entity tables.
Start by creating a Driver class, which will have a many-to-many relationship with the Car class. On the
Driver side, this is implemented with a collection navigation property to the Car class:
namespace AutoLot.Samples.Models; public class Driver : BaseEntity
{
public string FirstName { get; set; } public string LastName { get; set; }
public IEnumerable Cars { get; set; } = new List();
}

Add the DbSet property to the ApplicationDbContext class:

public DbSet Cars { get; set; } public DbSet Makes { get; set; } public DbSet Radios { get; set; } public DbSet Drivers { get; set; }

Next, update the Car class to have a collection navigation property to the new Driver class:
namespace AutoLot.Samples.Models; public class Car : BaseEntity
{
public string Color { get; set; } public string PetName { get; set; } public int MakeId { get; set; }
public Make MakeNavigation { get; set; }

public Radio RadioNavigation { get; set; }
public IEnumerable Drivers { get; set; } = new List();
}

To update the database, use the following migration commands (again, migrations will be fully explained later in this chapter):

dotnet ef migrations add Drivers -o Migrations -c AutoLot.Samples.ApplicationDbContext dotnet ef database update Drivers -c AutoLot.Samples.ApplicationDbContext

When the database is updated, the Cars table is unchanged, and the Drivers and CarDriver tables are created. Here are the definitions for the new tables:

CREATE TABLE [dbo].[Drivers](
[Id] [INT] IDENTITY(1,1) NOT NULL, [FirstName] NVARCHAR NULL, [LastName] NVARCHAR NULL, [TimeStamp] VARBINARY NULL,
CONSTRAINT [PK_Drivers] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS
= ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] GO

CREATE TABLE [dbo].[CarDriver]( [CarsId] [int] NOT NULL, [DriversId] [int] NOT NULL,
CONSTRAINT [PK_CarDriver] PRIMARY KEY CLUSTERED (
[CarsId] ASC, [DriversId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS
= ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] GO
ALTER TABLE [dbo].[CarDriver] WITH CHECK ADD CONSTRAINT [FK_CarDriver_Cars_CarsId] FOREIGN KEY([CarsId]) REFERENCES [dbo].[Cars] ([Id]) ON DELETE CASCADE
GO

ALTER TABLE [dbo].[CarDriver] CHECK CONSTRAINT [FK_CarDriver_Cars_CarsId] GO

ALTER TABLE [dbo].[CarDriver] WITH CHECK ADD CONSTRAINT [FK_CarDriver_Drivers_DriversId] FOREIGN KEY([DriversId]) REFERENCES [dbo].[Drivers] ([Id]) ON DELETE CASCADE
GO

ALTER TABLE [dbo].[CarDriver] CHECK CONSTRAINT [FK_CarDriver_Drivers_DriversId] GO

Note the compound primary key, the check constraints (foreign keys), and the cascade behavior are all created by EF Core to make sure the CarDriver table is configured as a proper join table.

Many-to-Many Prior to EF Core 5
The equivalent Car-Driver many-to-many relationship can be accomplished by creating the three tables explicitly and is how it must be done in EF Core versions earlier than EF Core 5. Here is an abbreviated example:

public class Driver
{
...
public IEnumerable CarDrivers { get; set; }
}

public class Car
{
...
public IEnumerable CarDrivers { get; set; }
}
public class CarDriver
{
public int CarId {get;set;}
public Car CarNavigation {get;set;}
public int DriverId {get;set;}
public Driver DriverNavigation {get;set;}
}

Cascade Behavior
Most data stores (like SQL Server) have rules controlling the behavior when a row is deleted. If the related (dependent) records should also be deleted, this is referred to as cascade delete. In EF Core, there are three actions that can occur when a principal entity (with dependent entities loaded into memory) is deleted.
•Dependent records are deleted.
•Dependent foreign keys are set to null.
•The dependent entity remains unchanged.
The default behavior is different between optional and required relationships. The behavior can also be configured to one of seven values, although only five are recommended for use. The behavior is
configured with the DeleteBehavior enum using the Fluent API. The options available in the enumeration are listed here:
•Cascade
•ClientCascade
•ClientNoAction (not recommended for use)
•ClientSetNull
•NoAction (not recommended for use)
•SetNull
•Restrict

In EF Core, the specified behavior is triggered only after an entity is deleted and SaveChanges() is called on the derived DbContext. See the “Query Execution” section for more details about when EF Core interacts with the data store.

Optional Relationships
Recall from Table 21-4 that optional relationships are where the dependent entity can set the foreign key value(s) to null. For optional relationships, the default behavior is ClientSetNull. Table 21-5 shows the cascade behavior with dependent entities and the effect on database records when using SQL Server.

Table 21-5. Cascade Behavior with Optional Relationships

Delete Behavior Effect on Dependents (In
Memory) Effect on Dependents (In Database)
Cascade Entities are deleted. Entities are deleted by the database.
ClientCascade Entities are deleted. For databases that do not support cascade delete, EF Core deletes the entities.
ClientSetNull
(default) Foreign key property/ properties set to null. None.
SetNull Foreign key property/ properties set to null. Foreign key property/properties set to null.
Restrict None. None.

Required Relationships
Required relationships are where the dependent entity cannot set the foreign key value(s) to null. For required relationships, the default behavior is Cascade. Table 21-6 shows the cascade behavior with dependent entities and the effect on database records when using SQL Server.

Table 21-6. Cascade Behavior with Required Relationships

Delete Behavior Effect on Dependents (In
Memory) Effect on Dependents (In Database)
Cascade
(default) Entities are deleted. Entities are deleted.
ClientCascade Entities are deleted. For databases that do not support cascade delete, EF Core deletes the entities.
ClientSetNull SaveChanges throws exception. None.
SetNull SaveChanges throws exception. SaveChanges throws exception.
Restrict None. None.

Entity Conventions
There are many conventions that EF Core uses to define an entity and how it relates to the data store. The conventions are always enabled unless overruled by data annotations or code in the Fluent API. Table 21-7 lists some of the more important EF Core conventions.

Table 21-7. Some of the EF Core Conventions

Convention Meaning in Life
Included tables All classes with a DbSet property and all classes that can be reached (through navigation properties) by a DbSet class are created in the database.
Included columns All public properties with a getter and setter (including automatic properties) are mapped to columns.
Table name Maps to the name of the DbSet property name in the derived DbContext. If no DbSet
exists, the class name is used.
Schema Tables are created in the data store’s default schema (dbo on SQL Server).
Column name Column names are mapped to the property names of the class.
Column data type Data types are selected based on the .NET data type and translated by the database provider (SQL Server). DateTime maps to datetime2(7), and string maps to nvarchar(max). Strings as part of a primary key map to nvarchar(450).
Column nullability Non-nullable data types are created as Not Null persistence columns. EF Core honors C# 8 nullability.
Primary key Properties named Id or Id will be configured as the primary key. Keys of type short, int, long, or Guid have values controlled by the data store. Numerical values are created as Identity columns (SQL Server).
Relationships Relationships between tables are created when there are navigation properties between two entity classes.
Foreign key Properties named Id are foreign keys for navigation properties of type

.

The previous navigation property examples all leverage EF Core conventions to build the relationships between the tables.

Mapping Properties to Columns
By convention, the public read-write properties map to columns of the same name. The data type matches the data store’s equivalent of the property’s CLR data type. Non-nullable properties are set to not null in the data store, and nullable properties (including nullable reference types introduced in C# 8) are set to allow null.
EF Core also supports reading and writing to property backing fields. EF Core expects the backing field to be named using one of the following conventions (in order of precedence):
•_
•_ •m_
•m_

If the Color property of the Car class is updated to use a backing field, it would (by convention) need to be named one of _color, _Color, m_color, or m_Color, like this:

private string _color; public string Color
{
get => _color;
set => _color = value;
}

Overriding EF Core Conventions
New in EF Core 6, the conventions can be overridden using the ConfigureConventions() method. For example, if you want string properties to default to a certain size (instead of nvarchar(max)), you can add the following code into the ApplicationDbContext class:

protected override void ConfigureConventions(ModelConfigurationBuilder configurationBuilder)
{
configurationBuilder.Properties().HaveMaxLength(50);
}

When a new migration is created and executed, you will see that all of the string properties are updated to nvarchar(50).

Entity Framework Data Annotations
Data annotations are C# attributes that are used to further shape your entities. Table 21-8 lists some of the most commonly used data annotations for defining how your entity classes and properties map to database tables and fields. Data annotations override any conflicting conventions. There are many more annotations that you can use to refine the entities in the model, as you will see throughout the rest of this chapter and book.

Table 21-8. Some Data Annotations Supported by the Entity Framework Core (*New Attributes in EF Core 6)

Data Annotation Meaning in Life
Table Defines the schema and table name for the entity.
EntityTypeConfiguration* In conjunction with the IEntityTypeConfiguration interface, allows an entity to be configured in its own class using the Fluent API. The use of this attribute is covered in the Fluent API section.
Keyless Indicates an entity does not have a key (e.g., representing a database view).
Column Defines the column name for the entity property.
BackingField Specifies the C# backing field for a property.
Key Defines the primary key for the entity. Key fields are implicitly also [Required].
Index Placed on a class to specify a single column or multicolumn index. Allows for specifying the index is unique.
(continued)

Table 21-8. (continued)

Data Annotation Meaning in Life
Owned Declares that the class will be owned by another entity class.
Required Declares the property as not nullable in the database.
ForeignKey Declares a property that is used as the foreign key for a navigation property.
InverseProperty Declares the navigation property on the other end of a relationship.
StringLength Specifies the max length for a string property.
TimeStamp Declares a type as a rowversion in SQL Server and adds concurrency checks to database operations involving the entity.
ConcurrencyCheck Flags field to be used in concurrency checking when executing updates and deletes.
DatabaseGenerated Specifies if the field is database generated or not. Takes a
DatabaseGeneratedOption value of Computed, Identity, or None.
DataType Provides for a more specific definition of a field than the intrinsic data type.
Unicode* Maps string property to Unicode/non-Unicode database column without specifying the native datatype.
Precision* Specifies precision and scale for the database column without specifying the native datatype.
NotMapped Excludes the property or class in regard to database fields and tables.

The following code shows the BaseEntity class with annotations that declare the Id field as the primary key. The second data annotation on the Id property indicates that it is an Identity column in SQL Server.
The TimeStamp property will be a SQL Server timestamp/rowversion property (for concurrency checking, covered later in this chapter).

public abstract class BaseEntity
{
[Key, DatabaseGenerated(DatabaseGeneratedOption.Identity)]
public int Id { get; set; }
[Timestamp]
public byte[] TimeStamp { get; set; }
}

Here is the Car class and the data annotations that shape it in the database, explained after the code sample:

[Table("Inventory", Schema="dbo")] [Index(nameof(MakeId), Name = "IX_Inventory_MakeId")] public class Car : BaseEntity
{
private string _color; [Required, StringLength(50)] public string Color

{
get => _color;
set => _color = value;
}
[Required, StringLength(50)]
public string PetName { get; set; } public int MakeId { get; set; } [ForeignKey(nameof(MakeId))]
public Make MakeNavigation { get; set; } public Radio RadioNavigation { get; set; } [InverseProperty(nameof(Driver.Cars))]
public IEnumerable Drivers { get; set; }
}

The Table attribute maps the Car class to the Inventory table in the dbo schema. The Column attribute (not shown in this example) works in a similar fashion and is used to change a column name or data type. The Index attribute creates an index on the foreign key MakeId. The two text fields (Color and PetName)
are set to be Required and a max StringLength of 50 characters. The InverseProperty and ForeignKey
attributes are explained in the next section.
The changes from the EF Core conventions are as follows:
•Renaming the table from Cars to Inventory.
•Changing the TimeStamp column from varbinary(max) to the SQL Server timestamp data type.
•Setting the nullability for the Color and PetName columns from null to not null.
•Explicitly setting the size of the Color and PetName columns to nvarchar(50). This was already handled when the EF Core conventions for string properties were overridden but included here for visibility.
•Renaming the index on the MakeId column.
The rest of the annotations used match the configuration defined by the EF Core conventions. To confirm the changes, we examine the table created by EF Core:

CREATE TABLE [dbo].[Inventory](
[Id] [INT] IDENTITY(1,1) NOT NULL, [Color] [NVARCHAR](50) NOT NULL, [PetName] [NVARCHAR](50) NOT NULL, [MakeId] [INT] NOT NULL, [TimeStamp] [TIMESTAMP] NULL,
CONSTRAINT [PK_Inventory] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS
= ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY] GO

ALTER TABLE [dbo].[Inventory] ADD DEFAULT (N'') FOR [Color] GO

ALTER TABLE [dbo].[Inventory] ADD DEFAULT (N'') FOR [PetName] GO

ALTER TABLE [dbo].[Inventory] WITH CHECK ADD CONSTRAINT [FK_Inventory_Makes_MakeId] FOREIGN KEY([MakeId]) REFERENCES [dbo].[Makes] ([Id]) ON DELETE CASCADE
GO

ALTER TABLE [dbo].[Inventory] CHECK CONSTRAINT [FK_Inventory_Makes_MakeId] GO

■ Note If you have been following along and running migrations with each of these changes, you might be surprised to see a failure when updating the TimeStamp column to the SQL Server timestamp data type. This is because SQL Server does not allow an existing column’s datatype to be changed to the timestamp datatype from another datatype. The column must be dropped and then re-added with the new timestamp datatype. EF Core sees the column as already existing and issues an alter statement and not the paired drop/add commands that are required to make the change. To update your database, comment out the TimeStamp property on the base class, create a new migration and apply it, and then uncomment the TimeStamp property and create another migration and apply it.

Note the defaults added to the Color and PetName columns. If there was any data that had null values for either of these columns, it would cause the migration to fail. This change ensures the change to not null will succeed by placing an empty string in those columns if they were null at the time of the migration being applied.
To change the Radio’s CarId property to map to a field named InventoryId, and make the RadioId
required and explicitly set the size to 50, update the Radio entity to the following (note the changes in bold):

namespace AutoLot.Samples.Models;

[Table("Radios", Schema="dbo")]
public class Radio : BaseEntity
{
public bool HasTweeters { get; set; } public bool HasSubWoofers { get; set; } [Required, StringLength(50)]
public string RadioId { get; set; }
[Column("InventoryId")]
public int CarId { get; set; }
[ForeignKey(nameof(CarId))]
public Car CarNavigation { get; set; }
}

The migration commands and the resulting table are shown here:

dotnet ef migrations add UpdateRadio -o Migrations -c AutoLot.Samples.ApplicationDbContext dotnet ef database update UpdateRadio -c AutoLot.Samples.ApplicationDbContext

CREATE TABLE [dbo].[Radios](
[Id] [INT] IDENTITY(1,1) NOT NULL,

[HasTweeters] [BIT] NOT NULL, [HasSubWoofers] [BIT] NOT NULL, [RadioId] [NVARCHAR](50) NOT NULL,
[InventoryId] [INT] NOT NULL, [TimeStamp] [TIMESTAMP] NULL,
CONSTRAINT [PK_Radios] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS
= ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] GO

ALTER TABLE [dbo].[Radios] ADD DEFAULT (N'') FOR [RadioId] GO

ALTER TABLE [dbo].[Radios] WITH CHECK ADD CONSTRAINT [FK_Radios_Inventory_InventoryId] FOREIGN KEY([InventoryId]) REFERENCES [dbo].[Inventory] ([Id]) ON DELETE CASCADE
GO

ALTER TABLE [dbo].[Radios] CHECK CONSTRAINT [FK_Radios_Inventory_InventoryId] GO

As a final step in updating our models, update the Name property on the Make entity to be required, as well as set the max length of to 50, and do the same for the FirstName and LastName properties on the Driver entity:

//Make.cs
namespace AutoLot.Samples.Models;

[Table("Makes", Schema="dbo")]
public class Make : BaseEntity
{
[Required, StringLength(50)]
public string Name { get; set; }
[InverseProperty(nameof(Car.MakeNavigation))]
public IEnumerable Cars { get; set; } = new List();
}
//Driver.cs
namespace AutoLot.Samples.Models;

[Table("Drivers", Schema="dbo")]
public class Driver : BaseEntity
{
[Required, StringLength(50)]
public string FirstName { get; set; }

[Required, StringLength(50)]
public string LastName { get; set; }
[InverseProperty(nameof(Car.Drivers))]
public IEnumerable Cars { get; set; } = new List();
}

Annotations and Navigation Properties
The ForeignKey annotation lets EF Core know which property is the backing field for the navigation property. By convention, Id would automatically be set as the foreign key property. In the preceding examples, it is explicitly set for readability. This supports different naming styles as well as having more than one foreign key to the same table. Note that in a one-to-one relationship, only the dependent entity has a foreign key.
InverseProperty informs EF Core of how the entities relate by indicating the navigation property on the other entity that navigates back to this entity. InverseProperty is required when an entity relates to another entity more than once and also (in my honest opinion) makes the code more readable.

The Fluent API
The Fluent API configures the application entities through C# code. The methods are exposed by the ModelBuilder instance available in the DbContext OnModelCreating() method. The Fluent API is the most powerful of the configuration methods and overrides any conventions or data annotations that are in conflict. Some configuration options are available only using the Fluent API, such as setting default values and cascade behavior for navigation properties.

Class and Property Methods
The Fluent API is a superset of the data annotations when shaping your individual entities. It supports all of the functionality contained in the data annotations, but has additional capabilities, such as specifying composite keys and indices, and defining computed columns.

Class and Property Mapping
The following code shows the previous Car example with the Fluent API equivalent to the data annotations used (omitting the navigation properties, which will be covered next).

modelBuilder.Entity(entity =>
{
entity.ToTable("Inventory","dbo");
});

The following maps the CarId property of the Radio class to the InventoryId column of the Makes table:

modelBuilder.Entity(entity =>
{
entity.Property(e => e.CarId).HasColumnName("InventoryId");
});

Keys and Indices
To set the primary key for an entity, use the HasKey() method, as shown here:

modelBuilder.Entity(entity =>
{
entity.ToTable("Inventory","dbo");
entity.HasKey(e=>e.Id);
});

To set a composite key, select the properties that are in the key in the expression for the HasKey() method. For example, if the primary key for the Car entity should be the Id columns and an OrganizationId property, you would set it like this:

modelBuilder.Entity(entity =>
{
entity.ToTable("Inventory","dbo");
entity.HasKey(e=> new { e.Id, e.OrganizationId});
});

The process is the same for creating indices, except that it uses the HasIndex() Fluent API method. For example, to create an index named IX_Inventory_MakeId, use the following code:

modelBuilder.Entity(entity =>
{
entity.ToTable("Inventory","dbo"); entity.HasKey(e=>e.Id);
entity.HasIndex(e => e.MakeId, "IX_Inventory_MakeId");
});

To make the index unique, use the IsUnique() method. The IsUnique() method takes an optional bool
that defaults to true:

entity.HasIndex(e => e.MakeId, "IX_Inventory_MakeId").IsUnique();

Field Size and Nullability
Properties are configured by selecting them using the Property() method and then using additional methods to configure the property. You already saw an example of this with the mapping of the CarId property to the InventoryId column.
The IsRequired() takes an optional bool that defaults to true and defines the nullability of the database column. The HasMaxLength() method sets the size of the column. Here is the Fluent API code that sets the Color and PetName properties as required with a max length of 50 characters:

modelBuilder.Entity(entity =>
{
...
entity.Property(e => e.Color)
.IsRequired()
.HasMaxLength(50); entity.Property(e => e.PetName)
.IsRequired()
.HasMaxLength(50);
});

Default Values
The Fluent API provides methods to set default values for columns. The default value can be a value type or a SQL string. For example, to set the default Color for a new Car to Black, use the following:

modelBuilder.Entity(entity =>
{
...
entity.Property(e => e.Color)
.IsRequired()
.HasMaxLength(50)
.HasDefaultValue("Black");
});

To set the value to a database function (like getdate()), use the HasDefaultValueSql() method.
Presume that a DateTime property named DateBuilt has been added to the Car class:

public class Car : BaseEntity
{
...
public DateTime? DateBuilt { get; set; }
}

The default value should be the current date using the SQL Server getdate() method. To configure this property to have this default, use the following Fluent API code:

modelBuilder.Entity(entity =>
{
...
entity.Property(e => e.DateBuilt)
.HasDefaultValueSql("getdate()");
});

SQL Server will use the result of the getdate() function if the DateBuilt property on the entity does not have a value when saved to the database.
A problem exists when a Boolean or numeric property has a database default value that contradicts the CLR default value. For example, if a Boolean property (such as IsDrivable) has a default set to true in the database, the database will set the value to true when inserting a record if a value isn’t specified for that column. This is, of course, the expected behavior on the database side of the equation. However, the default
CLR value for Boolean properties is false, which causes an issue due to how EF Core handles default values.
For example, add a bool property named IsDrivable to the Car class. If you are following along, make sure to create and apply a new migration to update the database.

//Car.cs
public class Car : BaseEntity
{
...
public bool IsDrivable { get; set; }
}

Before discussing default values, let’s examine the EF Core behavior for the Boolean datatype. Take the following code to create a new Car record with the IsDrivable set to false:

context.Cars.Add(new() { MakeId = 1, Color = "Rust", PetName = "Lemon", IsDrivable = false }); context.SaveChanges();

Here is the generated SQL for the insert:

exec sp_executesql N'SET NOCOUNT ON;
INSERT INTO [dbo].[Inventory] ([Color], [IsDrivable], [MakeId], [PetName], [Price]) VALUES (@p0, @p1, @p2, @p3, @p4);
SELECT [Id], [DateBuilt], [Display], [IsDeleted], [TimeStamp], [ValidFrom], [ValidTo] FROM [dbo].[Inventory]
WHERE @@ROWCOUNT = 1 AND [Id] = scope_identity();

',N'@p0 nvarchar(50),@p1 bit,@p2 int,@p3 nvarchar(50),@p4 decimal(18,2)',@p0=N'Rust',@p1=0, @p2=1,@p3=N'Lemon',@p4=NULL

Now set the default for the property’s column mapping to true in the ApplicationDbContext’s OnModelCreating() method (once again creating and applying a new database migration):

//ApplicationDbContext
protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity(entity =>
{
...
entity.Property(e => e.IsDrivable).HasDefaultValue(true);
});

Executing the same code to insert the previous Car record generates different SQL:

exec sp_executesql N'SET NOCOUNT ON;
INSERT INTO [dbo].[Inventory] ([Color], [MakeId], [PetName], [Price]) VALUES (@p0, @p1, @p2, @p3);
SELECT [Id], [DateBuilt], [Display], [IsDeleted], [IsDrivable], [TimeStamp], [ValidFrom], [ValidTo]
FROM [dbo].[Inventory]
WHERE @@ROWCOUNT = 1 AND [Id] = scope_identity();

',N'@p0 nvarchar(50),@p1 int,@p2 nvarchar(50),@p3 decimal(18,2)',@p0=N'Rust',@p1=1, @p2=N'Lemon',@p3=NULL

Notice that the IsDrivable column is not included in the insert statement. EF Core knows that the IsDrivable property’s value is the CLR default, and it knows that the column has a SQL Server default, so the column isn’t included in the statement. Therefore, when you save a new record with IsDrivable = false, the value is ignored, and the database default will be used. This means that the value for IsDrivable will always be true!

EF Core does alert you to this problem when you create a migration. In this particular example, this warning is output:

The 'bool' property 'IsDrivable' on entity type 'Car' is configured with a database- generated default. This default will always be used for inserts when the property has the value 'false', since this is the CLR default for the 'bool' type. Consider using the
nullable 'bool?' type instead, so that the default will only be used for inserts when the property value is 'null'.

One solution for this is to make your public property (and therefore the column) nullable, since the default value for a nullable value type is null, so setting a Boolean property to false works as expected. However, changing the nullability of the property might not fit the business need.
Another solution is provided by EF Core and its support for backing fields. Recall from earlier that if a backing field exists (and is identified as the backing field for the property through convention, data
annotation, or Fluent API), then EF Core will use the backing field for read-write actions and not the public property.
If you update IsDrivable to use a nullable backing field (but keep the property non-nullable), EF Core will read-write from the backing field and not the property. The default value for a nullable bool is null and not false. This change now makes the property work as expected.

public class Car
{
...
private bool? _isDrivable; public bool IsDrivable
{
get => _isDrivable ?? true; set => _isDrivable = value;
}
}

The Fluent API is used to inform EF Core of the backing field.

modelBuilder.Entity(entity =>
{
entity.Property(p => p.IsDrivable)
.HasField("_isDrivable")
.HasDefaultValue(true);
});

■ Note The HasField() method is not necessary in this example since the name of the backing field follows the naming conventions. I included it to show how to use the Fluent apI to set it and to keep the code readable.

EF Core translates the field to the following SQL definition:

CREATE TABLE [dbo].[Inventory](
...
[IsDrivable] [BIT] NOT NULL,
... GO
ALTER TABLE [dbo].[Inventory] ADD DEFAULT (CONVERT([BIT],(1))) FOR [IsDrivable]
GO

Although not shown in the previous examples, numeric properties work the same way. If you are setting a non-zero default value, the backing field (or property itself if not using a backing field) must be nullable.
As a final note, the warning will still appear even when the fields are properly configured with nullable backing fields. The warning can be suppressed; however, I recommend leaving it in place as a reminder to check to make sure the field/property is properly configured. If you want to suppress it, set the following option in the DbContextOptions:

options.ConfigureWarnings(wc => wc.Ignore(RelationalEventId.BoolWithDefaultWarning));

RowVersion/Concurrency Tokens
To set a property as the rowversion datatype, use the IsRowVersion() method. To also set the property as a concurrency token, use the IsConcurrencyToken() method. The combination of these two methods has the same effect as the [Timestamp] data annotation:

modelBuilder.Entity(entity =>
{
...
entity.Property(e => e.TimeStamp)
.IsRowVersion()
.IsConcurrencyToken();
});

Concurrency checking will be covered in the next chapter.

SQL Server Sparse Columns
SQL Server sparse columns are optimized to store null values. EF Core 6 added support for sparse columns with the IsSparse() method in the Fluent API. The following code illustrates setting the fictitious IsRaceCar property to use SQl Server sparse columns:

modelBuilder.Entity(entity =>
{
entity.Property(p => p.IsRaceCare).IsSparse();
});

Computed Columns
Columns can also be set to computed based on the capabilities of the data store. For SQL Server, two of the options are to compute the value based on the value of other fields in the same record or to use a scalar function. For example, to create a computed column on the Inventory table that combines the PetName and Color values to create a property named Display, use the HasComputedColumnSql() function.
First add the new column to the Car class:

public class Car : BaseEntity
{
...
public string Display { get; set; }
}

Then add the Fluent API call to HasComputedColumnSql():

modelBuilder.Entity(entity =>
{
entity.Property(p => p.Display)
.HasComputedColumnSql("[PetName] + ' (' + [Color] + ')'");
});

Introduced in EF Core 5, the computed values can be persisted, so the value is calculated only on row creation or update. While SQL Server supports this, not all data stores do, so check the documentation of your database provider.

modelBuilder.Entity(entity =>
{
entity.Property(p => p.Display)
.HasComputedColumnSql("[PetName] + ' (' + [Color] + ')'", stored:true);

});

The DatabaseGenerated data annotation is often used in conjunction with the Fluent API to increase readability of the code. Here is the updated version of the Display property with the annotation included:

public class Car : BaseEntity
{
...
[DatabaseGenerated(DatabaseGeneratedOption.Computed)] public string Display { get; set; }
}

Check Constraints
Check constraints are a SQL Server feature that define a condition on a row that must be true. For example, in an ecommerce system, a check constraint can be added to make sure the quantity is greater than zero or that the price is greater than the discounted price. Since we don’t have any numeric values in our system, we will make a contrived constraint that prevents using the name “Lemon” in the Makes table.

Add the following to the OnModelCreating() method in the ApplicationDbContext class, which creates the check constraint preventing a Name of “Lemon” in the Makes table:

modelBuilder.Entity()
.HasCheckConstraint(name:"CH_Name", sql:"[Name]<>'Lemon'", buildAction:c => c.HasName(“CK_Check_Name”));

The first parameter gives the constraint a name in the model, the second is the SQL for the constraint, and the final assigns the SQL Server name for the check constraint. Here is the check constraint as
defined in SQL:

ALTER TABLE [dbo].[Makes] WITH CHECK ADD CONSTRAINT [CK_Check_Name] CHECK (([Name]<>'Lemon'))

Now, when a record with the Name of “Lemon” is added to the table, a SQL exception will be thrown.
Execute the following code to see the exception in action:

var context = new ApplicationDbContextFactory().CreateDbContext(null); context.Makes.Add(new Make { Name = "Lemon" });
context.SaveChanges();

This throws the following exception:

The INSERT statement conflicted with the CHECK constraint "CK_Check_Name". The conflict occurred in database "AutoLotSamples", table "dbo.Makes", column 'Name'.

Feel free to revert the migration for the check constraint and remove the migration, as the rest of the book doesn’t use the check constraint. It was added in this section for demonstration purposes.

One-to-Many Relationships
To use the Fluent API to define one-to-many relationships, pick one of the entities to update. Both sides of the navigation chain are set in one block of code.

modelBuilder.Entity(entity =>
{
...
entity.HasOne(d => d.MakeNavigation)
.WithMany(p => p.Cars)
.HasForeignKey(d => d.MakeId)
.OnDelete(DeleteBehavior.ClientSetNull)
.HasConstraintName("FK_Inventory_Makes_MakeId");
});

If you select the principal entity as the base for the navigation property configuration, the code looks like this:

modelBuilder.Entity(entity =>
{
...
entity.HasMany(e=>e.Cars)

.WithOne(c=>c.MakeNavigation)
.HasForeignKey(c=>c.MakeId)
.OnDelete(DeleteBehavior.ClientSetNull)
.HasConstraintName("FK_Inventory_Makes_MakeId");
});

One-to-One Relationships
One-to-one relationships are configured the same way, except that the WithOne() Fluent API method is used instead of WithMany(). Also, a unique index is required on the dependent entity and will be created automatically if one is not defined. The following example explicitly creates the unique index to specify the name. Here is the code for the relationship between the Car and Radio entities using the dependent entity (Radio):

modelBuilder.Entity(entity =>
{
entity.HasIndex(e => e.CarId, "IX_Radios_CarId")
.IsUnique();

entity.HasOne(d => d.CarNavigation)
.WithOne(p => p.RadioNavigation)
.HasForeignKey(d => d.CarId);
});

If the relationship is defined on a principal entity, a unique index will still be added to the dependent entity. Here is the code for the relationship between the Car and Radio entities using the principal entity for the relationship and specifying the name of the index on the dependent entity:

modelBuilder.Entity(entity =>
{
entity.HasIndex(e => e.CarId, "IX_Radios_CarId")
.IsUnique();
});

modelBuilder.Entity(entity =>
{
entity.HasOne(d => d.RadioNavigation)
.WithOne(p => p.CarNavigation)
.HasForeignKey(d => d.CarId);
});

Many-to-Many Relationships
Many-to-many relationships are much more customizable with the Fluent API. The foreign key field names, index names, and cascade behavior can all be set in the statements that define the relationship. It also allows for specifying the pivot table directly, which allows for additional fields to be added and for simplified querying.

Start by adding a CarDriver entity:

//CarDriver.cs
namespace AutoLot.Samples.Models;

[Table("InventoryToDrivers", Schema = "dbo")] public class CarDriver : BaseEntity
{
public int DriverId {get;set;} [ForeignKey(nameof(DriverId))]
public Driver DriverNavigation {get;set;}

[Column("InventoryId")] public int CarId {get;set;} [ForeignKey(nameof(CarId))]
public Car CarNavigation {get;set;}
}

Add a DbSet for the new entity into the ApplicationDbContext: public DbSet CarsToDrivers {get;set;}
Next, update the Car entity to add a navigation property for the new CarDriver entity:

public class Car : BaseEntity
{
...
[InverseProperty(nameof(CarDriver.CarNavigation))]
public IEnumerable CarDrivers { get; set; } = new List();
}

Now, update the Driver entity for the navigation property to the CarDriver entity:

public class Driver : BaseEntity
{
[InverseProperty(nameof(CarDriver.DriverNavigation))]
public IEnumerable CarDrivers { get; set; } = new List();
}

Finally, add in the Fluent API code for the many-to-many relationship:

modelBuilder.Entity()
.HasMany(p => p.Drivers)
.WithMany(p => p.Cars)
.UsingEntity( j => j
.HasOne(cd => cd.DriverNavigation)
.WithMany(d => d.CarDrivers)
.HasForeignKey(nameof(CarDriver.DriverId))
.HasConstraintName("FK_InventoryDriver_Drivers_DriverId")
.OnDelete(DeleteBehavior.Cascade),

j => j
.HasOne(cd => cd.CarNavigation)
.WithMany(c => c.CarDrivers)
.HasForeignKey(nameof(CarDriver.CarId))
.HasConstraintName("FK_InventoryDriver_Inventory_InventoryId")
.OnDelete(DeleteBehavior.ClientCascade), j =>
{
j.HasKey(cd => new { cd.CarId, cd.DriverId });
});

Excluding Entities from Migrations
If an entity is shared between multiple DbContexts, each DbContext will create code in the migration files for creation of or changes to that entity. This causes a problem since the second migration script will fail if the changes are already present in the database. Prior to EF Core 5, the only solution was to manually edit one of the migration files to remove those changes.
In EF Core 5, a DbContext can mark an entity as excluded from migrations, letting the other DbContext become the system of record for that entity. The following code shows an entity being excluded from migrations:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
modelBuilder.Entity().ToTable("Logs", t => t.ExcludeFromMigrations());
}

Using IEntityTypeConfiguration Classes
As you might have surmised at this stage of working with the Fluent API, the OnModelCreating() method can become quite lengthy (and unwieldy) the more complex your model becomes. Introduced in EF Core 6, the IEntityTypeConfiguration interface and the EntityTypeConfiguration attribute allow for moving the Fluent API configuration for an entity into its own class. This makes for a cleaner ApplicationDbContext and supports the separation of concerns design principle.
Start by creating a new directory named Configuration in the Models directory. In this new directory, add a new file named CarConfiguration.cs, make it public, and implement the IEntityTypeConfiguration interface, like this:

namespace AutoLot.Samples.Models.Configuration;

public class CarConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
}
}

Next, move the contents of the configuration for the Car entity from the OnModelCreating() method in the ApplicationDbContext into the Configure() method of the CarConfiguration class. Replace the entity variable with the builder variable so the Configure() method looks like this:

public void Configure(EntityTypeBuilder builder)
{
builder.ToTable("Inventory", "dbo"); builder.HasKey(e => e.Id);
builder.HasIndex(e => e.MakeId, "IX_Inventory_MakeId"); builder.Property(e => e.Color)
.IsRequired()
.HasMaxLength(50)
.HasDefaultValue("Black"); builder.Property(e => e.PetName)
.IsRequired()
.HasMaxLength(50);
builder.Property(e => e.DateBuilt).HasDefaultValueSql("getdate()"); builder.Property(e => e.IsDrivable)
.HasField("_isDrivable")
.HasDefaultValue(true); builder.Property(e => e.TimeStamp)
.IsRowVersion()
.IsConcurrencyToken();
builder.Property(e => e.Display).HasComputedColumnSql("[PetName] + ' (' + [Color] + ')'", stored: true);
builder.HasOne(d => d.MakeNavigation)
.WithMany(p => p.Cars)
.HasForeignKey(d => d.MakeId)
.OnDelete(DeleteBehavior.ClientSetNull)
.HasConstraintName("FK_Inventory_Makes_MakeId");
}

This configuration also works with the fluent many-to-many configuration between Car and Driver. It’s your choice whether to add the configuration into the CarConfiguration class or create a DriverConfiguration class. For this example, move it into the CarConfiguration class at the end of the Configure() method:

public void Configure(EntityTypeBuilder builder)
{
...
builder
.HasMany(p => p.Drivers)
.WithMany(p => p.Cars)
.UsingEntity( j => j
.HasOne(cd => cd.DriverNavigation)
.WithMany(d => d.CarDrivers)
.HasForeignKey(nameof(CarDriver.DriverId))
.HasConstraintName("FK_InventoryDriver_Drivers_DriverId")
.OnDelete(DeleteBehavior.Cascade), j => j
.HasOne(cd => cd.CarNavigation)
.WithMany(c => c.CarDrivers)
.HasForeignKey(nameof(CarDriver.CarId))
.HasConstraintName("FK_InventoryDriver_Inventory_InventoryId")
.OnDelete(DeleteBehavior.ClientCascade),

j =>
{
j.HasKey(cd => new { cd.CarId, cd.DriverId });
});
}

Update the GlobalUsings.cs file to include the new namespace for the configuration classes:

global using AutoLot.Samples.Models.Configuration;

Replace all the code in the OnModelBuilding() method (in the ApplicationDbContext.cs class) that configures the Car class and the Car to Driver many-to-many relationship with the following single line of code:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
...
}

The final step for the Car class is to add the EntityTypeConfiguration attribute:

[Table("Inventory", Schema = "dbo")] [Index(nameof(MakeId), Name = "IX_Inventory_MakeId")] [EntityTypeConfiguration(typeof(CarConfiguration))] public class Car : BaseEntity
{
...
}

Next, repeat the same steps for the Radio Fluent API code. Create a new class named RadioConfiguration, implement the IEntityTypeConfiguration interface, and add the code from the ApplicationDbContext OnModelBuilding() method:

namespace AutoLot.Samples.Models.Configuration;

public class RadioConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.Property(e => e.CarId).HasColumnName("InventoryId"); builder.HasIndex(e => e.CarId, "IX_Radios_CarId").IsUnique(); builder.HasOne(d => d.CarNavigation)
.WithOne(p => p.RadioNavigation)
.HasForeignKey(d => d.CarId);
}
}

Update the OnModelCreating() method in the ApplicationDbContext:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new RadioConfiguration().Configure(modelBuilder.Entity());
}

Finally, add the EntityTypeConfiguration attribute to the Radio class:

[Table("Radios", Schema = "dbo")] [EntityTypeConfiguration(typeof(RadioConfiguration))] public class Radio : BaseEntity
{
...
}

While this didn’t reduce the total number of lines of code, this new feature made the
ApplicationDbContext a lot cleaner.

Conventions, Annotations, and the Fluent API, Oh My!
At this point in the chapter, you might be wondering which of the three options to use to shape your entities and their relationship to each other and the data store. The answer is all three. The conventions are always active (unless you override them with data annotations or the Fluent API). The data annotations can do almost everything the Fluent API methods can do and keep the information in the entity class themselves, which can increase code readability and support. The Fluent API is the most powerful of all three. Whether you use data annotations or the Fluent API, know that data annotations overrule the built-in conventions, and the methods of the Fluent API overrule everything.

Owned Entity Types
There will be time when two or more entities contain the same set of properties. Using a C# class as a property on an entity to define a collection of properties for another entity was first introduced in EF Core version 2.0. When types marked with the [Owned] attribute (or configured with the Fluent API) are added as a property of an entity, EF Core will add all the properties from the [Owned] entity class to the owning entity. This increases the possibility of C# code reuse.
Behind the scenes, EF Core considers this a one-to-one relation. The owned class is the dependent entity, and the owning class is the principal entity. The owned class, even though it is considered an entity, cannot exist without the owning entity. The default column names from the owned type will be formatted as NavigationPropertyName_OwnedEntityPropertyName (e.g., PersonalNavigation_FirstName). The default names can be changed using the Fluent API.
Take this Person class (notice the Owned attribute):

namespace AutoLot.Samples.Models;

[Owned]
public class Person
{
[Required, StringLength(50)]

public string FirstName { get; set; } [Required, StringLength(50)]
public string LastName { get; set; }
}

With this in place, we can replace the FirstName and LastName properties on the Driver class with the new Person class:

namespace AutoLot.Samples.Models;

[Table("Drivers", Schema = "dbo")] public class Driver : BaseEntity
{
public Person PersonInfo {get;set;} = new Person();
public IEnumerable Cars { get; set; } = new List(); [InverseProperty(nameof(CarDriver.DriverNavigation))]
public IEnumerable CarDrivers { get; set; } = new List();
}

By default, the two Person properties are mapped to columns named PersonInfo_FirstName and PersonInfo_LastName. To change this, first add a new file named DriverConfiguration.cs into the Configuration folder, and update the code to the following:

namespace AutoLot.Samples.Models.Configuration;

public class DriverConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
builder.OwnsOne(o => o.PersonInfo, pd =>
{
pd.Property(nameof(Person.FirstName))
.HasColumnName(nameof(Person.FirstName))
.HasColumnType("nvarchar(50)"); pd.Property(nameof(Person.LastName))
.HasColumnName(nameof(Person.LastName))
.HasColumnType("nvarchar(50)");
});
}
}

Update the ApplicationDbContext OnConfiguring() method:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new RadioConfiguration().Configure(modelBuilder.Entity());
new DriverConfiguration().Configure(modelBuilder.Entity());
}

Finally, update the Driver class:

[Table("Drivers", Schema = "dbo")] [EntityTypeConfiguration(typeof(DriverConfiguration))] public class Driver : BaseEntity
{
...
}

The Drivers table is updated like this (note that the nullability of the FirstName and LastName columns doesn’t match the Required data annotations on the Person owned entity):

CREATE TABLE [dbo].[Drivers](
[Id] [INT] IDENTITY(1,1) NOT NULL, [FirstName] [NVARCHAR](50) NULL, [LastName] [NVARCHAR](50) NULL, [TimeStamp] [TIMESTAMP] NULL,
CONSTRAINT [PK_Drivers] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY]] GO

While the Person class has the Required data annotation on both of its properties, the SQL Server columns are both set as NULL. This is due to an issue with how the migration system translates owned entities when they are used with an optional relationship.
To correct this, there are a couple of options. The first is to enable C# null reference types (at the project level or in the classes). This makes the PersonInfo navigation property non-nullable, which EF Core honors, and in turn EF Core then appropriately configures the columns in the owned entity. The other option is to add a Fluent API statement to make the navigation property required.

public class DriverConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
...
builder.Navigation(d=>d.PersonInfo).IsRequired(true);
}
}

This updates the properties of the Person owned type to be set as a not null column in SQL Server:

CREATE TABLE [dbo].[Drivers](
[Id] [INT] IDENTITY(1,1) NOT NULL, [FirstName] [NVARCHAR](50) NOT NULL, [LastName] [NVARCHAR](50) NOT NULL, [TimeStamp] [TIMESTAMP] NULL,

CONSTRAINT [PK_Drivers] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF,
IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] GO

There are four limitations when using owned types:
•You cannot create a DbSet for an owned type.
•You cannot call Entity() with an owned type on ModelBuilder.
•Instances of an owned entity type cannot be shared between multiple owners.
•Owned entity types cannot have inheritance hierarchies.

There are additional options to explore with owned entities, including collections, table splitting, and nesting. These are all beyond the scope of this book. To find out more information, consult the EF Core documentation on owned entities here: https://docs.microsoft.com/en-us/ef/core/modeling/owned- entities.

Query Types
Query types are DbSet collections that are used to represent views, a SQL statement, or tables without a primary key. Prior versions of EF Core used DbQuery for these, but from EF Core 3.x on, the DbQuery type has been retired. Query types are added to the derived DbContext using DbSet properties and are configured as keyless.
Query types are usually used to represent combinations of tables, such as combining the details from the Make and Inventory tables. Take this query, for example:

SELECT m.Id MakeId, m.Name Make,
i.Id CarId, i.IsDrivable, i.Display, i.DateBuilt, i.Color, i.PetName FROM dbo.Makes m
INNER JOIN dbo.Inventory i ON i.MakeId = m.Id

To hold the results of this query, create a new folder named ViewModels, and in that folder create a new class named CarMakeViewModel:
namespace AutoLot.Samples.ViewModels; [Keyless]
public class CarMakeViewModel
{
public int MakeId { get; set; } public string Make { get; set; } public int CarId { get; set; } public bool IsDrivable { get; set; } public string Display { get; set; }
public DateTime DateBuilt { get; set; }

public string Color { get; set; } public string PetName { get; set; }

[NotMapped]
public string FullDetail => $" The {Color} {Make} is named {PetName}"; public override string ToString() => FullDetail;
}

The Keyless attribute instructs EF Core that this entity is a query type and will never be used for updates and is to be excluded from the change tracker when queried. Note the use of the NotMapped attribute to create a display string that combines several properties into a single, human-readable string. Update the ApplicationDbContext to include a DbSet for the view model:

public class ApplicationDbContext : DbContext
{
...

...
}

public DbSet Cars { get; set; } public DbSet Makes { get; set; } public DbSet Radios { get; set; } public DbSet Drivers { get; set; }
public DbSet< CarMakeViewModel> CarMakeViewModels { get; set; }

Update the GlobalUsings.cs file to include the new namespace for the view model and the configuration (which will be created next):

global using AutoLot.Samples.ViewModels;
global using AutoLot.Samples.ViewModels.Configuration;

The remainder of the configuration takes place in the Fluent API. Create a new folder named Configuration under the ViewModels folder, and in that folder create a new class named CarMakeViewModelConfiguration and update the code to the following:

namespace AutoLot.Samples.ViewModels.Configuration;

public class CarMakeViewModelConfiguration : IEntityTypeConfiguration
{
public void Configure(EntityTypeBuilder builder)
{
}
}

Update the CarMakeViewModel class to add the EntityTypeConfiguration attribute:

[Keyless] [EntityTypeConfiguration(typeof(CarMakeViewModelConfiguration))] public class CarMakeViewModel : INonPersisted
{
...
}

Update the OnModelCreating() method to the following:

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
new CarConfiguration().Configure(modelBuilder.Entity());
new RadioConfiguration().Configure(modelBuilder.Entity()); new DriverConfiguration().Configure(modelBuilder.Entity());
new CarMakeViewModelConfiguration().Configure(modelBuilder.Entity());
}

The following example sets the entity as keyless and maps the query type to raw SQL query. The HasNoKey() Fluent API method is not necessary if the Keyless data annotation is on the model, and vice versa, but is shown in this example for completeness):

public void Configure(EntityTypeBuilder builder)
{
builder.HasNoKey().ToSqlQuery(@"
SELECT m.Id MakeId, m.Name Make, i.Id CarId, i.IsDrivable, i.DisplayName, i.DateBuilt, i.Color, i.PetName
FROM dbo.Makes m
INNER JOIN dbo.Inventory i ON i.MakeId = m.Id");
}

Query types can also be mapped to a database view. Presuming there was a view named dbo.
CarMakeView, the configuration would look like this:

builder.HasNoKey().ToView("CarMakeView", "dbo");

■ Note when using EF Core migrations to update your database, query types that are mapped to a view do not get created as tables. Query types that are not mapped to views are created as keyless tables.

If you don’t want the view model mapped to a table in your database and don’t have a view to map, use the following overload of the ToTable() method to exclude the item from migrations:

builder.ToTable( x => x.ExcludeFromMigrations());

The final mechanisms that query types can be used with are the FromSqlRaw() and FromSqlInterpolated() methods. These will be covered in detail in the next chapter, but here is a sneak peek:

var records = context.CarMakeViewModel.FromSqlRaw(
@" SELECT m.Id MakeId, m.Name Make, i.Id CarId, i.IsDrivable, i.Display, i.DateBuilt, i.Color, i.PetName
FROM dbo.Makes m
INNER JOIN dbo.Inventory i ON i.MakeId = m.Id ");

Flexible Query/Table Mapping
EF Core 5 introduced the ability to map the same class to more than one database object. These objects can be tables, views, or functions. For example, CarViewModel from Chapter 20 can be mapped to a view that returns the make name with the Car data and the Inventory table. EF Core will then query from the view and send updates to the table.

modelBuilder.Entity()
.ToTable("Inventory")
.ToView("InventoryWithMakesView");

Query Execution
Data retrieval queries are created with LINQ queries written against the DbSet properties. The LINQ query is changed to the database-specific language (e.g., T-SQL) by the database provider’s LINQ translation engine and executed on the server side. Multirecord (or potential multirecord) LINQ queries are not executed until the query is iterated over (e.g., using a foreach) or bound to a control for display (like a data grid). This deferred execution allows building up queries in code without suffering performance issues from chattiness with the database or retrieving more records than intended.
For example, to get all yellow Car records from the database, execute the following query:

//The factory is not meant to be used like this, but it’s demo code 🙂 var context = new ApplicationDbContextFactory().CreateDbContext(null);
var cars = context.Cars.Where(x=>x.Color == "Yellow");

With deferred execution, that database is not actually queried until the results are iterated over. To have the query execute immediately, use ToList().

var listOfCars = context.Cars.Where(x=>x.Color == "Yellow").ToList();

Since queries aren’t executed until triggered, they can be built up over multiple lines of code. The following code sample executes the same as the previous example:

var query = context.Cars.AsQueryable(); query = query.Where(x=>x.Color == "Yellow"); var moreCars = query.ToList();

Single-record queries (such as when using First()/FirstOrDefault()) execute immediately on calling the action (such as FirstOrDefault()), and create, update, and delete statements are executed immediately when the DbContext.SaveChanges() method is executed.

■ Note The next chapters covers executing CRUD operations in great detail.

Mixed Client-Server Evaluation
Prior versions of EF Core introduced the ability to mix server-side and client-side execution. This meant that a C# function could be used in the middle of a LINQ statement and essentially negate what I described in the previous paragraph. The part up to the C# function would execute on the server side, but then all of the

results (at that point of the query) were brought back on the client side, and then the rest of the query would execute as LINQ to Objects. This ended up causing more problems than it solved, and with the release of EF Core 3.1, this functionality was changed. Now, only the final node of a LINQ statement can execute on the client side.

Tracking vs. NoTracking Queries
When data is read from the database into a DbSet instance with a primary key, the entities (by default) are tracked by the change tracker. This is typically what you want in your application. Any changes to the item can then be persisted to the database merely by calling SaveChanges() on your derived DbContext instance without any additional work on your part. Also, once an instance is tracked by the change tracker, any further calls to the database for that same item (based on the primary key) will result in an update of the item and not a duplication.
However, there might be times when you need to get some data from the database, but you don’t want it to be tracked by the change tracker. The reason might be performance (tracking original and current values for a large set of records can add memory pressure), or maybe you know those records will never be changed by the part of the application that needs the data.
To load data into a DbSet instance without adding the data to the ChangeTracker, add AsNoTracking() into the LINQ statement. This signals EF Core to retrieve the data without adding it into the ChangeTracker. For example, to load a Car record without adding it into the ChangeTracker, execute the following:

var untrackedCar = context.Cars.Where(x=>x.Id ==1).AsNoTracking();

This provides the benefit of not adding the potential memory pressure with a potential drawback: additional calls to retrieve the same Car will create additional copies of the record. At the expense of using more memory and having a slightly slower execution time, the query can be modified to ensure there is only one instance of the unmapped Car.

var untrackedWithIdResolution =
context.Cars.Where(x=>x.Id == 1).AsNoTrackingWithIdentityResolution();

Query types are never tracked since they cannot be updated. The exception to this is when using flexible query/table mapping. In that case, instances are tracked by default so they can be persisted to the target table.

Code First vs. Database First
Whether you are building a new application or adding EF Core into an existing application, you will fall into one of two camps: you have an existing database that you need to work with, or you don’t yet have a database and need to create one.
Code first means that you create and configure your entity classes and the derived DbContext in code and then use migrations to update the database. This is how most greenfield, or new, projects are
developed. The advantage is that as you build your application, your entities evolve based on the needs of your application. The migrations keep the database in sync, so the database design evolves along with your application. This emerging design process is popular with agile development teams, as you build the right parts at the right time.

If you already have a database or prefer to have your database design drive your application, that is referred to as database first. Instead of creating the derived DbContext and all of the entities manually, you scaffold the classes from the database. When the database changes, you need to re-scaffold your classes to keep your code in sync with the database. Any custom code in the entities or the derived DbContext must be placed in partial classes so it doesn’t get overwritten when the classes are re-scaffolded. Fortunately, the scaffolding process creates partial classes just for that reason.
Whichever method you chose, code first or database first, know that it is essentially a commitment. If you are using code first, all changes are made to the entity and context classes, and the database is updated using migrations. If you are working database first, all changes must be made in the database, and then the classes are re-scaffolded. With some effort and planning, you can switch from database first to code first (and vice versa), but you should not be making manual changes to the code and the database at the same time.

The EF Core Global Tool CLI Commands
The dotnet-ef global CLI tool EF Core tooling contains the commands needed to scaffold existing databases into code, to create/remove database migrations, and to operate on a database (update, drop, etc.). Before you can use the dotnet-ef global tooling, it must be installed with the following command:

dotnet tool install --global dotnet-ef --version 6.0.0

■ Note If you have an earlier version of the EF Core command-line tooling installed, you will need to uninstall the older version before installing the latest version. To uninstall the global tool, use
dotnet tool uninstall --global dotnet-ef.

To test the install, open a command prompt and enter the following command:

dotnet ef

If the tooling is successfully installed, you will get the EF Core Unicorn (the team’s mascot) and the list of available commands, like this (the unicorn looks better on the screen):

_/\
---==/ \\
|. \|\
| || | | ) \\\
| _| | _| \_/ | //|\\
| ||_| / \\\/\\

Entity Framework Core .NET Command-line Tools 6.0.0 Usage: dotnet ef [options] [command]
Options:
--version Show version information
-h|--help Show help information
-v|--verbose Show verbose output.
--no-color Don't colorize output.
--prefix-output Prefix output with level.

Commands:
database Commands to manage the database. dbcontext Commands to manage DbContext types. migrations Commands to manage migrations.

Use "dotnet ef [command] --help" for more information about a command.

Table 21-9 describes the three main commands in the EF Core global tool. Each main command has additional subcommands. As with all the .NET commands, each command has a rich help system that can be accessed by entering -h along with the command.

Table 21-9. EF Core Tooling Commands

Command Meaning in Life
Database Commands to manage the database. Subcommands include drop and update.
DbContext Commands to manage the DbContext types. Subcommands include scaffold, list, and
info.
Migrations Commands to manage migrations. Subcommands include add, list, remove, and script.

The EF Core commands execute on .NET project files. The target project needs to reference the EF Core tooling NuGet package Microsoft.EntityFrameworkCore.Design. The commands operate on the project file located in the same directory where the commands are run, or a project file in another directory if referenced through the command-line options.
For the EF Core CLI commands that need an instance of a derived DbContext class (Database and Migrations), if there is only one in the project, that one will be used. If there are more than one, then the DbContext needs to be specified in the command-line options. The derived DbContext class will be instantiated using an instance of a class implementing the IDesignTimeDbContextFactory interface if one can be located. If the tooling cannot find one, the derived DbContext will be instantiated using the parameterless constructor. If neither exists, the command will fail. Note that the using the parameterless constructor (and not the constructor that takes in the DbContextOptions) requires the existence of the OnConfiguring override, which is not considered a good practice to have. The best (and really only) option is to always create an IDesignTimeDbContextFactory for each derived DbContext that you have in your application.
There are common options available for the EF Core commands, shown in Table 21-10. Many of the commands have additional options or arguments.

Table 21-10. EF Core Command Options

Option (Shorthand || Longhand) Meaning in Life
--c || --context The fully qualified derived DbContext class to use. If more than one derived DbContext exists in the project, this is a required option.
-p || --project The project to use (where to place the files). Defaults to the current working directory.
-s || --startup-project
The startup project to use (contains the derived DbContext). Defaults to the current working directory.
-h || --help Displays the help and all of the options.
-v || --verbose Shows verbose output.

To list all the arguments and options for a command, enter dotnet ef -h in a command window, like this:

dotnet ef migrations add -h

■ Note It is important to note that the CLI commands are not C# commands, so the rules of escaping slashes and quotes do not apply.

The Migrations Commands
The migrations commands are used to add, remove, list, and script migrations. As migrations are applied to a base, a record is created in the EFMigrationsHistory table. Table 21-11 describes the commands. The following sections explain the commands in detail.

Table 21-11. EF Core Migrations Commands

Command Meaning in Life
Add Creates a new migration based on the changes from the previous migration
Remove Checks if the last migration in the project has been applied to the database and, if not, deletes the migration file (and its designer) and then rolls back the snapshot class to the previous migration
List Lists all of the migrations for a derived DbContext and their status (applied or pending)
Bundle Creates an executable to update the database.
Script Creates a SQL script for all, one, or a range of migrations

The Add Command
The add command creates a new database migration based on the current object model. The process examines every entity with a DbSet property on the derived DbContext (and every entity that can be reached from those entities using navigation properties) and determines whether there are any changes that need to be applied to the database. If there are changes, the proper code is generated to update the database. You’ll learn more about that shortly.
The Add command requires a name argument, which is used to name the create class and files for the migration. In addition to the common options, the option -o or –output-dir indicates where the migration files should go. The default directory is named Migrations relative to the current path.
Each migration added creates two files that are partials of the same class. Both files start their name with a timestamp and the migration name used as the argument to the add command. The first file is named
_.cs, and the second is named _. Designer.cs. The timestamp is based on when the file was created and will match exactly for both files.
The first file represents the code generated for the database changes in this migration, and the designer file represents the code to create and update the database based on all migrations up to and including this one.

The main file contains two methods, Up() and Down(). The Up() method contains the code to update the database with this migration’s changes, and the Down() method contains the code to roll back this migration’s changes. A partial listing of the initial migration from earlier in this chapter is listed here (all of the migrations used in the previous examples are in the AutoLot.Samples project in the companion code):

public partial class Radio : Migration
{
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.CreateTable( name: "Make",
columns: table => new
{
Id = table.Column(type: "int", nullable: false)
.Annotation("SqlServer:Identity", "1, 1"),
Name = table.Column(type: "nvarchar(max)", nullable: true), TimeStamp = table.Column(type: "varbinary(max)", nullable: true)
},
constraints: table =>
{

...

}

table.PrimaryKey("PK_Make", x => x.Id);
});

migrationBuilder.CreateIndex( name: "IX_Cars_MakeId", table: "Cars",
column: "MakeId");

protected override void Down(MigrationBuilder migrationBuilder)
{
migrationBuilder.DropTable(name: "Cars"); migrationBuilder.DropTable(name: "Make");
}
}

As you can see, the Up() method is creating tables, columns, indexes, etc. The Down() method is dropping the items created. The migrations engine will issue alter, add, and drop statements as necessary to ensure the database matches your model.
The designer file contains two attributes that tie these partials to the filename and the derived
DbContext. The attributes are shown here with a partial list of the design class:

[DbContext(typeof(ApplicationDbContext))] [Migration("20210613000105_Radio")] partial class Radio
{
protected override void BuildTargetModel(ModelBuilder modelBuilder)
{
//omitted for brevity
}
}

The first migration creates an additional file in the target directory named for the derived DbContext in the format of ModelSnapshot.cs. The format of this file is the same as the designer partial and contains the code that is the sum of all migrations. When migrations are added or removed, this file is automatically updated to match the changes.

■ Note It is extremely important that you don’t delete migration files manually. This will result in the ModelSnapshot.cs becoming out of sync with your migrations, essentially breaking them. If you are going to manually delete them, delete them all and start over. To remove a migration, use the remove command, covered shortly.

The Remove Command
The remove command is used to remove migrations from the project and always operates on the last migration (based on the timestamps of the migrations). When removing a migration, EF Core will make sure it hasn’t been applied by checking the EFMigrationsHistory table in the database. If the migration has been applied, the process fails. If the migration hasn’t yet been applied or has been rolled back, the migration is removed, and the model snapshot file is updated.
The remove command doesn’t take any arguments (since it always works on the last migration) and uses the same options as the add command. There is one additional option, the force option (-f || --force).
This will roll back the last migration and then remove it in one step.

The List Command
The list command is used to show all the migrations for a derived DbContext. By default, it will list all migrations and query the database to determine whether they have been applied. If they have not been applied, they will be listed as pending. There is an option to pass in a specific connection string and another option to not connect to the database at all and instead just list the migrations. Table 21-12 shows those options.

Table 21-12. Additional Options for EF Core Migrations List Command

Option (Shorthand ||
Longhand) Meaning in Life
--connection
Connection string to the database. Defaults to the one specified in the instance of
IDesignTimeDbContextFactory or the DbContext’s OnConfiguring method.
--no-connect Instructs the command to skip the database check.

The Bundle Command
The bundle command creates an executable to update the database. The generated executable, built for a target runtime (e.g., Windows, Linux), will apply all contained migrations to the database. Table 21-13 describes the most commonly used arguments with the bundle command.

Table 21-13. Common Arguments for the EF Core Migrations Bundle Command

Argument Meaning in Life
-o | --output The path to the executable file to create.
-f | --force Overwrite existing files.
--self-contained Also bundle the .NET runtime with the executable.
-r | --target-runtime
Target runtime to bundle for. If no runtime is specified, the file will use the runtime of the current machine’s operating system.

The executable will use the connection string from the IDesignTimeDbContextFactory; however, another connection string can be passed into the executable using the --connection flag. If the migrations have already been applied to the target database, they will not be reapplied.
When applying the --self-contained flag, the size of the executable will grow significantly. On my machine with the sample project from this chapter, the regular bundle file is 11MB in size, while the self- contained file is 74MB.

The Script Command
The script command creates a SQL script based on one or more migrations. The command takes two optional arguments representing the migration to start with and the migration to end with. If neither is entered, all migrations are scripted. Table 21-14 describes the arguments.

Table 21-14. Arguments for the EF Core Migrations Script Command

Argument Meaning in Life
The starting migration. Defaults to 0 (zero), the starting migration.
The target migration. Defaults to the last migration.

If no migrations are named, the script created will be the cumulative total of all the migrations. If named migrations are provided, the script will contain the changes between the two migrations (inclusive). Each migration is wrapped in a transaction. If the EFMigrationsHistory table does not exist in the database where the script is executed, it will be created. The table will also be updated to match the migrations that were executed. Some examples are shown here:

//Script all of the migrations dotnet ef migrations script
//script from the beginning to the Many2Many migrations dotnet ef migrations script 0 Many2Many

There are some additional options available, as shown in Table 21-15. The -o option allows you to specify a file for the script (the directory is relative to where the command is executed), and -i creates an idempotent script. This means it contains checks to see whether a migration has already been applied and skips that migration if it has. The –no-transaction option disables the normal transactions that are added to the script.

Table 21-15. Additional Options for the EF Core Migrations Script Command

Option (Shorthand ||
Longhand) Meaning in Life
-o || -output The file to write the resulting script to
-i || --idempotent Generates a script that checks if a migration has already been applied before applying it
--no-transactions Does not wrap each migration in a transaction

The Database Commands
There are two database commands, drop and update. The drop command deletes the database if it exists. The update command updates the database using migrations.

The Drop Command
The drop command drops the database specified by the connection string in the context factory of the OnConfiguring method of DbContext. Using the force option does not ask for confirmation and force closes all connections. See Table 21-16.

Table 21-16. EF Core Database Drop Options

Option (Shorthand || Longhand) Meaning in Life
-f || --force Don’t confirm the drop. Force close all connections.
--dry-run Show which database will be dropped but don’t drop it.

The Database Update Command
The update command takes one argument (the migration name) and the usual options. The command has one additional option, --connection . This allows for using a connection string that isn’t configured in the design-time factory or DbContext.
If the command is executed without a migration name, the command updates the database to the most recent migration, creating the database if necessary. If a migration is named, the database will be updated to that migration. All previous migrations that have not yet been applied will be applied as well. As migrations are applied, their names are stored in the EFMigrationsHistory table.
If the named migration has a timestamp that is earlier than other applied migrations, all later migrations are rolled back. If a 0 (zero) is passed in as the named migration, all migrations are reverted, leaving an empty database (except for the EFMigrationsHistory table).

The DbContext Commands
There are four DbContext commands. Three of them (list, info, script) operate on derived DbContext classes in your project. The scaffold command creates a derived DbContext and entities from an existing database. Table 21-17 shows the available commands.

Table 21-17. The DbContext Commands

Command Meaning in Life
Info Gets information about a DbContext type
List Lists available DbContext types
Optimize Generates a compiled version of the model used by the DbContext
Scaffold Scaffolds a DbContext and entity types for a database
Script Generates SQL script from the DbContext based on the object model, bypassing any migrations

The list and info commands have the usual options available. The list command lists the derived DbContext classes in the target project. The info command provides details about the specified derived DbContext class, including the connection string, provider name, database name, and data source. The script command creates a SQL script that creates your database based on the object model, ignoring any migrations that might be present. The scaffold command is used to reverse engineer an existing database and is covered in the next section.

The DbContext Scaffold Command
The scaffold command creates the C# classes (derived DbContext and entities) complete with data annotations (if requested) and Fluent API commands from an existing database. There are two
required arguments, the database connection string, and the fully qualified provider (e.g., Microsoft. EntityFrameworkCore.SqlServer). Table 21-18 describes the arguments.

Table 21-18. The DbContext Scaffold Arguments

Argument Meaning in Life
Connection The connection string to the database
Provider The EF Core database provider to use (e.g., Microsoft. EntityFrameworkCore.SqlServer)

The options available include selecting specific schemas and tables, the created context class name and namespace, the output directory and namespace of the generated entity classes, and many more. The
standard options are also available. The extended options are listed in Table 21-19, with discussion to follow.

Table 21-19. The DbContext Scaffold Options

Option (Shorthand ||
Longhand) Meaning in Life
-d || --data-annotations Use attributes to configure the model (where possible). If omitted, only the Fluent API is used.
-c || --context The name of the derived DbContext to create.
--context-dir The directory to place the derived DbContext, relative to the project directory. Defaults to database name.
-f || --force Replaces any existing files in the target directory.
-o || --output-dir The directory to put the generated entity classes into. Relative to the project directory.
--schema ... The schemas of the tables to generate entity types for.
-t || --table ... The tables to generate entity types for.
--use-database-names Use the table and column names directly from the database.
-n | --namespaces
The namespace for the generated entity classes. Matches the directory by default.
--context-namespace
The namespace for the generated derived DbContext class. Matches the directory by default.
--no-onconfiguring Does not generate OnConfiguring method.
--no-pluralize Does not use the pluralizer.

The scaffold command has become much more robust with EF Core 6.0. As you can see, there are plenty of options to choose from. If the data annotations (-d) option is selected, EF Core will use data annotations where it can and fill in the differences with the Fluent API. If that option is not selected, the entire configuration (where different than the conventions) is coded in the Fluent API. You can specify the namespace, schema, and location for the generated entities and derived DbContext files. If you do not want to scaffold the entire database, you can select certain schemas and tables. The --no-onconfiguring option eliminates the OnConfiguring() method from the scaffolded class, and the –no-pluralize option turns
off the pluralizer, which turns singular entities (Car) into plural tables (Cars) when creating migrations and turns plural tables into single entities when scaffolding.
New in EF Core 6, database comments on SQL tables and columns are also scaffolded into the entity classes and their properties.

The DbContext Optimize Command
The optimize command optimizes the derived DbContext, performing many of the steps that would normally happen when the derived DbContext is first used. The options available include specifying the directory to place the compiled results as well as what namespace to use. The standard options are also available. The extended options are listed in Table 21-20, with discussion to follow.

Table 21-20. The DbContext Optimize Options

Option (Shorthand || Longhand) Meaning in Life
-o || --output-dir The directory to put the files in. Paths are relative to the project directory.
-n | --namespace The namespace to use. Matches the directory by default.

When the derived DbContext is compiled, the results include a class for each entity in your model, the compiled derived DbContext, and the compiled derived DbContext ModelBuilder. For example, you can compile AutoLot.Samples.ApplicationDbContext using the following command:

dotnet ef dbcontext optimize --output-dir CompiledModels

The compiled files are placed in a directory named CompiledModels. The files are listed here:

ApplicationDbContextModel.cs ApplicationDbContextModelBuilder.cs CarDriverEntityType.cs CarEntityType.cs CarMakeViewModelEntityType.cs DriverEntityType.cs MakeEntityType.cs PersonEntityType.cs RadioEntityType.cs

To use the compiled model, call the UseModel() method in the DbContextOptions, like this:

var optionsBuilder = new DbContextOptionsBuilder();
var connectionString = @"server=.,5433;Database=AutoLotSamples;User Id=sa;Password=P @ssw0rd;Encrypt=False;"; optionsBuilder.UseSqlServer(connectionString).UseModel(ApplicationDbContextModel.Instance); var context = new ApplicationDbContext(optionsBuilder.Options);

Compiling the derived DbContext can significantly improve performance in certain situations, but there are some restrictions:
• Global query filters are not supported.
• Lazy loading proxies are not supported.
• Change tracking proxies are not supported.
• The model must be recompiled any time the model changes.

If these restrictions aren’t an issue for your situation, making use of the DbContext optimization can significantly improve your applications performance.

Summary
This chapter started the journey into Entity Framework Core. This chapter examined EF Core fundamentals, how queries execute, and change tracking. You learned about shaping your model with conventions, data annotations, and the Fluent API. The final section covered the power of the EF Core command-line interface and global tools.

Pro C#10 CHAPTER 20 Data Access with ADO.NET

CHAPTER 20

Data Access with ADO.NET

The .NET platform defines several namespaces that allow you to interact with relational database systems. Collectively speaking, these namespaces are known as ADO.NET. In this chapter, you will learn about the overall role of ADO.NET and the core types and namespaces, and then you will move on to the topic of ADO. NET data providers. The .NET platform supports numerous data providers (both provided as part of the
.NET Framework and available from third-party sources), each of which is optimized to communicate with a specific database management system (e.g., Microsoft SQL Server, Oracle, and MySQL).
After you understand the common functionality provided by various data providers, you will then look at the data provider factory pattern. As you will see, using types within the System.Data namespaces (including System.Data.Common and database provider-specific namespaces like Microsoft.Data.
SqlClient, System.Data.Odbc, and the Windows only database provider namespace System.Data.Oledb), you can build a single code base that can dynamically pick and choose the underlying data provider without the need to recompile or redeploy the application’s code base.
Next, you will learn how to work directly with the SQL Server database provider, creating and opening connections to retrieve data, and then move on to inserting, updating, and deleting data, followed by examining the topic of database transactions. Finally, you will execute SQL Server’s bulk copy feature using ADO.NET to load a list of records into the database.

■ Note This chapter focuses on the raw ADO.NET. Starting with Chapter 21, I cover Entity Framework (EF) Core, Microsoft’s object-relational mapping (ORM) framework. Since EF Core uses ADO.NET for data access under the covers, a solid understanding of how ADO.NET works is vital when troubleshooting data access. There are also scenarios that are not solved by EF Core (such as executing a SQL bulk copy), and you will need to know ADO.NET to solve those issues.

ADO.NET vs. ADO
If you have a background in Microsoft’s previous COM-based data access model (Active Data Objects [ADO]) and are just starting to work with the .NET platform, you need to understand that ADO.NET has little to do with ADO beyond the letters A, D, and O. While it is true that there is some relationship between the two systems (e.g., each has the concept of connection and command objects), some familiar ADO types (e.g., the Recordset) no longer exist. Furthermore, you can find many new types that have no direct equivalent under classic ADO (e.g., the data adapter).

© Andrew Troelsen, Phil Japikse 2022
A. Troelsen and P. Japikse, Pro C# 10 with .NET 6, https://doi.org/10.1007/978-1-4842-7869-7_20

797

Understanding ADO.NET Data Providers
ADO.NET does not provide a single set of objects that communicate with multiple database management systems (DBMSs). Rather, ADO.NET supports multiple data providers, each of which is optimized to interact with a specific DBMS. The first benefit of this approach is that you can program a specific data provider to access any unique features of a particular DBMS. The second benefit is that a specific data provider can connect directly to the underlying engine of the DBMS in question without an intermediate mapping layer standing between the tiers.
Simply put, a data provider is a set of types defined in a given namespace that understand how to communicate with a specific type of data source. Regardless of which data provider you use, each defines a set of class types that provide core functionality. Table 20-1 documents some of the core base classes and the key interfaces they implement.

Table 20-1. The Core Objects of an ADO.NET Data Provider

Base Class Relevant Interfaces Meaning in Life
DbConnection IDbConnection Provides the ability to connect to and disconnect from the data store. Connection objects also provide access to a related transaction object.
DbCommand IDbCommand Represents a SQL query or a stored procedure. Command objects also provide access to the provider’s data reader object.
DbDataReader IDataReader, IDataRecord Provides forward-only, read-only access to data using a server- side cursor.
DbDataAdapter IDataAdapter, IDbDataAdapter Transfers DataSets between the caller and the data store. Data adapters contain a connection and a set of four internal command objects used to select, insert, update, and delete information from the data store.
DbParameter IDataParameter, IDbDataParameter Represents a named parameter within a parameterized query.
DbTransaction IDbTransaction Encapsulates a database transaction.

Although the specific names of these core classes will differ among data providers (e.g., SqlConnection versus OdbcConnection), each class derives from the same base class (DbConnection, in the case of connection objects) that implements identical interfaces (e.g., IDbConnection). Given this, you would be correct to assume that after you learn how to work with one data provider, the remaining providers prove quite straightforward.

■ Note When you refer to a connection object under ADO.NET, you’re actually referring to a specific DbConnection-derived type; there is no class literally named Connection. The same idea holds true for a command object, data adapter object, and so forth. As a naming convention, the objects in a specific data provider are prefixed with the name of the related DBMS (e.g., SqlConnection, SqlCommand, and SqlDataReader).

Figure 20-1 shows the big picture behind ADO.NET data providers. The client assembly can be any type of .NET application: console program, Windows Forms, Windows Presentation Foundation, ASP.NET Core,
.NET code library, and so on.

Figure 20-1. ADO.NET data providers provide access to a given DBMS

A data provider will supply you with other types beyond the objects shown in Figure 20-1; however, these core objects define a common baseline across all data providers.

ADO.NET Data Providers
As with all of .NET, data providers ship as NuGet packages. There are several supported by Microsoft as well as a multitude of third-party providers available. Table 20-2 documents some of the data providers supported by Microsoft.

Table 20-2. Some of the Microsoft-Supported Data Providers

Data Provider Namespace/NuGet Package Name
Microsoft SQL Server Microsoft.Data.SqlClient
ODBC System.Data.Odbc
OLE DB (Windows only) System.Data.OleDb

The Microsoft SQL Server data provider offers direct access to Microsoft SQL Server data stores—and only SQL Server data stores (including SQL Azure). The Microsoft.Data.SqlClient namespace contains the types used by the SQL Server provider.

■ Note While System.Data.SqlClient is still supported, all development effort for interaction with SQL Server (and SQL Azure) is focused on the new Microsoft.Data.SqlClient provider library.

The ODBC provider (System.Data.Odbc) provides access to ODBC connections. The ODBC types defined within the System.Data.Odbc namespace are typically useful only if you need to communicate with a given DBMS for which there is no custom .NET data provider. This is true because ODBC is a widespread model that provides access to several data stores.
The OLE DB data provider, which is composed of the types defined in the System.Data.OleDb namespace, allows you to access data located in any data store that supports the classic COM-based OLE DB protocol. Due to the dependence on COM, this provider will work only on the Windows operating system and should be considered deprecated in the cross-platform world of .NET.

The Types of the System.Data Namespace
Of all the ADO.NET namespaces, System.Data is the lowest common denominator. This namespace contains types that are shared among all ADO.NET data providers, regardless of the underlying
data store. In addition to a number of database-centric exceptions (e.g., NoNullAllowedException, RowNotInTableException, and MissingPrimaryKeyException), System.Data contains types that represent various database primitives (e.g., tables, rows, columns, and constraints), as well as the common interfaces implemented by data provider objects. Table 20-3 lists some of the core types you should be aware of.

Table 20-3. Core Members of the System.Data Namespace

Type Meaning in Life
Constraint Represents a constraint for a given DataColumn object
DataColumn Represents a single column within a DataTable object
DataRelation Represents a parent-child relationship between two DataTable objects
DataRow Represents a single row within a DataTable object
DataSet Represents an in-memory cache of data consisting of any number of interrelated
DataTable objects
DataTable Represents a tabular block of in-memory data
DataTableReader Allows you to treat a DataTable as a firehose cursor (forward-only, read-only data access)
DataView Represents a customized view of a DataTable for sorting, filtering, searching, editing, and navigation
IDataAdapter Defines the core behavior of a data adapter object
IDataParameter Defines the core behavior of a parameter object
IDataReader Defines the core behavior of a data reader object
IDbCommand Defines the core behavior of a command object
IDbDataAdapter Extends IDataAdapter to provide additional functionality of a data adapter object
IDbTransaction Defines the core behavior of a transaction object

Your next task is to examine the core interfaces of System.Data at a high level; this can help you understand the common functionality offered by any data provider. You will also learn specific details throughout this chapter; however, for now it is best to focus on the overall behavior of each interface type.

The Role of the IDbConnection Interface
The IDbConnection type is implemented by a data provider’s connection object. This interface defines a set of members used to configure a connection to a specific data store. It also allows you to obtain the data provider’s transaction object. Here is the formal definition of IDbConnection:

public interface IDbConnection : IDisposable
{
string ConnectionString { get; set; } int ConnectionTimeout { get; }
string Database { get; } ConnectionState State { get; }

IDbTransaction BeginTransaction();
IDbTransaction BeginTransaction(IsolationLevel il); void ChangeDatabase(string databaseName);
void Close();
IDbCommand CreateCommand(); void Open();
void Dispose();
}

The Role of the IDbTransaction Interface
The overloaded BeginTransaction() method defined by IDbConnection provides access to the provider’s transaction object. You can use the members defined by IDbTransaction to interact programmatically with a transactional session and the underlying data store.

public interface IDbTransaction : IDisposable
{
IDbConnection Connection { get; } IsolationLevel IsolationLevel { get; }

void Commit(); void Rollback(); void Dispose();
}

The Role of the IDbCommand Interface
Next up is the IDbCommand interface, which will be implemented by a data provider’s command object. Like other data access object models, command objects allow programmatic manipulation of SQL statements, stored procedures, and parameterized queries. Command objects also provide access to the data provider’s data reader type through the overloaded ExecuteReader() method.

public interface IDbCommand : IDisposable
{
string CommandText { get; set; } int CommandTimeout { get; set; }
CommandType CommandType { get; set; } IDbConnection Connection { get; set; } IDbTransaction Transaction { get; set; } IDataParameterCollection Parameters { get; } UpdateRowSource UpdatedRowSource { get; set; }

void Prepare(); void Cancel();
IDbDataParameter CreateParameter(); int ExecuteNonQuery();
IDataReader ExecuteReader();
IDataReader ExecuteReader(CommandBehavior behavior); object ExecuteScalar();
void Dispose();
}

The Role of the IDbDataParameter and IDataParameter Interfaces
Notice that the Parameters property of IDbCommand returns a strongly typed collection that implements IDataParameterCollection. This interface provides access to a set of IDbDataParameter-compliant class types (e.g., parameter objects).

public interface IDbDataParameter : IDataParameter
{
//Plus members in the IDataParameter interface byte Precision { get; set; }
byte Scale { get; set; } int Size { get; set; }
}

IDbDataParameter extends the IDataParameter interface to obtain the following additional behaviors:

public interface IDataParameter
{
DbType DbType { get; set; } ParameterDirection Direction { get; set; } bool IsNullable { get; }
string ParameterName { get; set; } string SourceColumn { get; set; }
DataRowVersion SourceVersion { get; set; } object Value { get; set; }
}

As you will see, the functionality of the IDbDataParameter and IDataParameter interfaces allows you to represent parameters within a SQL command (including stored procedures) through specific ADO.NET parameter objects, rather than through hard-coded string literals.

The Role of the IDbDataAdapter and IDataAdapter Interfaces
You use data adapters to push and pull DataSets to and from a given data store. The IDbDataAdapter interface defines the following set of properties that you can use to maintain the SQL statements for the related select, insert, update, and delete operations:

public interface IDbDataAdapter : IDataAdapter
{
//Plus members of IDataAdapter IDbCommand SelectCommand { get; set; } IDbCommand InsertCommand { get; set; } IDbCommand UpdateCommand { get; set; } IDbCommand DeleteCommand { get; set; }
}

In addition to these four properties, an ADO.NET data adapter picks up the behavior defined in the base interface, IDataAdapter. This interface defines the key function of a data adapter type: the ability to
transfer DataSets between the caller and underlying data store using the Fill() and Update() methods. The IDataAdapter interface also allows you to map database column names to more user-friendly display names with the TableMappings property.

public interface IDataAdapter
{
MissingMappingAction MissingMappingAction { get; set; } MissingSchemaAction MissingSchemaAction { get; set; } ITableMappingCollection TableMappings { get; }

DataTable[] FillSchema(DataSet dataSet, SchemaType schemaType); int Fill(DataSet dataSet);
IDataParameter[] GetFillParameters(); int Update(DataSet dataSet);
}

The Role of the IDataReader and IDataRecord Interfaces
The next key interface to be aware of is IDataReader, which represents the common behaviors supported by a given data reader object. When you obtain an IDataReader-compatible type from an ADO.NET data provider, you can iterate over the result set in a forward-only, read-only manner.

public interface IDataReader : IDisposable, IDataRecord
{
//Plus members from IDataRecord int Depth { get; }
bool IsClosed { get; }
int RecordsAffected { get; }

void Close();
DataTable GetSchemaTable(); bool NextResult();
bool Read();
Dispose();
}

Finally, IDataReader extends IDataRecord, which defines many members that allow you to extract a strongly typed value from the stream, rather than casting the generic System.Object retrieved from the data reader’s overloaded indexer method. Here is the IDataRecord interface definition:

public interface IDataRecord
{
int FieldCount { get; } object this[ int i ] { get; }
object this[ string name ] { get; } bool GetBoolean(int i);
byte GetByte(int i);
long GetBytes(int i, long fieldOffset, byte[] buffer, int bufferoffset, int length);
char GetChar(int i);
long GetChars(int i, long fieldoffset, char[] buffer, int bufferoffset, int length);
IDataReader GetData(int i); string GetDataTypeName(int i); DateTime GetDateTime(int i); Decimal GetDecimal(int i); double GetDouble(int i);
Type GetFieldType(int i); float GetFloat(int i); Guid GetGuid(int i); short GetInt16(int i); int GetInt32(int i);
long GetInt64(int i); string GetName(int i);
int GetOrdinal(string name); string GetString(int i); object GetValue(int i);
int GetValues(object[] values); bool IsDBNull(int i);
}

■ Note You can use the IDataReader.IsDBNull() method to discover programmatically whether a specified field is set to null before attempting to obtain a value from the data reader (to avoid triggering a runtime exception). Also recall that C# supports nullable data types (see Chapter 4), which are ideal for interacting with data columns that could be null in the database table.

Abstracting Data Providers Using Interfaces
At this point, you should have a better idea of the common functionality found among all .NET data providers. Recall that even though the exact names of the implementing types will differ among data providers, you can program against these types in a similar manner—that is the beauty of interface-based polymorphism. For example, if you define a method that takes an IDbConnection parameter, you can pass in any ADO.NET connection object, like so:

public static void OpenConnection(IDbConnection cn)
{
// Open the incoming connection for the caller.
connection.Open();
}

■ Note Interfaces are not strictly required; you can achieve the same level of abstraction using abstract base classes (such as DbConnection) as parameters or return values. however, using interfaces instead of base classes is the generally accepted best practice.

The same holds true for member return values. Create a new .NET Console application named MyConnectionFactory. Add the following NuGet packages to the project (note that the OleDb package is valid only on Windows):
Microsoft.Data.SqlClient System.Data.Common System.Data.Odbc System.Data.OleDb
Next, add a new file named DataProviderEnum.cs and update the code to the following:

namespace MyConnectionFactory;
//OleDb is Windows only and is not supported in .NET Core/.NET 5+ enum DataProviderEnum
{
SqlServer, #if PC
OleDb, #endif
Odbc, None
}

If you are using a Windows OS on your development machine, update the project file to define the conditional compiler symbol PC.


PC

If you are using Visual Studio, right-click the project, select Properties, and then go to the Build tab to enter the “Conditional compiler symbols” values.
The following code example allows you to obtain a specific connection object based on the value of a custom enumeration. For diagnostic purposes, you simply print the underlying connection object using reflection services.

using System.Data; using System.Data.Odbc; #if PC
using System.Data.OleDb; #endif
using Microsoft.Data.SqlClient; using MyConnectionFactory;

Console.WriteLine(" Very Simple Connection Factory ***\n"); Setup(DataProviderEnum.SqlServer);

if PC

Setup(DataProviderEnum.OleDb); //Not supported on macOS #endif
Setup(DataProviderEnum.Odbc); Setup(DataProviderEnum.None); Console.ReadLine();

void Setup(DataProviderEnum provider)
{
// Get a specific connection.
IDbConnection myConnection = GetConnection(provider);
Console.WriteLine($"Your connection is a {myConnection?.GetType().Name ?? "unrecognized type"}");
// Open, use and close connection...
}

// This method returns a specific connection object
// based on the value of a DataProvider enum. IDbConnection GetConnection(DataProviderEnum dataProvider)
=> dataProvider switch
{
DataProviderEnum.SqlServer => new SqlConnection(), #if PC
//Not supported on macOS
DataProviderEnum.OleDb => new OleDbConnection(), #endif
DataProviderEnum.Odbc => new OdbcConnection(),
_ => null,
};

The benefit of working with the general interfaces of System.Data (or, for that matter, the abstract base classes of System.Data.Common) is that you have a much better chance of building a flexible code base that can evolve over time. For example, today you might be building an application that targets Microsoft SQL Server; however, it is possible your company could switch to a different database. If you build a solution that hard-codes the Microsoft SQL Server–specific types of System.Data.SqlClient, you will need to edit, recompile, and redeploy the code for the new database provider.
At this point, you have authored some (quite simple) ADO.NET code that allows you to create different types of provider-specific connection objects. However, obtaining a connection object is only one aspect of working with ADO.NET. To make a worthwhile data provider factory library, you would also have to account for command objects, data readers, transaction objects, and other data-centric types. Building such a code library would not necessarily be difficult, but it would require a considerable amount of code and time.

Since the release of .NET 2.0, the kind folks in Redmond have built this exact functionality directly into the .NET base class libraries. This functionality has been significantly updated for .NET Core and .NET 5+.
You will examine this formal API in just a moment; however, first you need to create a custom database to use throughout this chapter (and for many chapters to come).

Setting Up SQL Server and Azure Data Studio
As you work through this chapter, you will execute queries against a simple SQL Server test database named AutoLot. In keeping with the automotive theme used throughout this book, this database will contain five interrelated tables (Inventory, Makes, Orders, Customers, and CreditRisks) that contain various bits of data representing information for a fictional automobile sales company. Before getting into the database details, you must set up SQL Server and a SQL Server IDE.

■ Note If you are using a Windows-based development machine and have installed Visual Studio 2022, you also have a special instance of SQL Server Express (called LocalDb) installed, which can be used for all the examples in this book. If you are content to use that version, please skip to the section “Installing a SQL Server IDE.”

Installing SQL Server
For this chapter and many of the remaining chapters in this book, you will need to have access to an instance of SQL Server. If you are using a non-Windows-based development machine and do not have an external instance of SQL Server available, or choose not to use an external SQL Server instance, you can run SQL Server inside a Docker container on your Mac- or Linux-based workstation. Docker also works on Windows machines, so you are welcome to run the examples in this book using Docker regardless of your operating system of choice.

■ Note Containerization is a large topic, and there just isn’t space in this book to get into the deep details of containers or Docker. This book will cover just enough so you can work through the examples.

Installing SQL Server in a Docker Container
Docker Desktop can be downloaded from www.docker.com/get-started. Download and install the appropriate version (Windows, Mac, Linux) for your workstation (you will need a free DockerHub user account). Make sure you select Linux containers when prompted.

■ Note The container choice (Windows or Linux) is the operating system running inside the container, not the operating system of your workstation.

Pulling the Image and Running SQL Server 2019
Containers are based on images, and each image is a layered set that builds up the final product. To get the image needed to run SQL Server 2019 in a container, open a command window and enter the following command:

docker pull mcr.microsoft.com/mssql/server:2019-latest

Once you have the image loaded onto your machine, you need to start SQL Server. To do that, enter the following command (all on one line):

docker run -e "ACCEPT_EULA=Y" -e "SA_PASSWORD=P@ssw0rd" -p 5433:1433 --name AutoLot -h AutoLotHost -d mcr.microsoft.com/mssql/server:2019-latest

The previous command accepts the end user license agreement, sets the password (in real life, you need to use a strong password), sets the port mapping (port 5433 on your machine maps to the default port for SQL Server, which is 1433) in the container, names the container (AutoLot), names the host (AutoLotHost), and finally informs Docker to use the previously downloaded image.

■ Note These are not settings you want to use for real development. For information on changing the SA password and to see a tutorial, go to https://docs.microsoft.com/en-us/sql/linux/quickstart- install-connect-docker?view=sql-server-ver15&pivots=cs1-cmd.

To confirm that it is running, enter the command docker ps -a in your command prompt. You will see output like the following (some columns omitted for brevity):

C:\Users\japik>docker ps -a
CONTAINER ID IMAGE PORTS NAMES
347475cfb823 mcr.microsoft.com/mssql/server:2019-latest 0.0.0.0:5433->1433/tcp AutoLot
To stop the container, enter docker stop 34747, where the numbers 34747 are the first five characters of the container ID. To restart the container, enter docker start 34747, again updating the command with the beginning of your container’s ID.

■ Note You can also use the container’s name (AutoLot in this example) with the Docker CLI commands, for example, docker start AutoLot. Be aware that the Docker commands, regardless of operating system, are case sensitive.

If you want to use the Docker Dashboard, right-click the Docker ship (in your system tray) and select Dashboard, and you should see the image running on port 5433. Hover over the image name with your mouse, and you will see the commands to stop, start, and delete (among others), as shown in Figure 20-2.

Figure 20-2. Docker Dashboard

■ Note To connect to SQL Server with an encrypted connection, there must be a certificate installed on the host. Follow the instructions in the docs to install a certificate in your Docker container and enable secure connections: https://docs.microsoft.com/en-us/sql/linux/sql-server-linux-docker-
container-security?view=sql-server-ver15. For this book, we will use an unencrypted connection to
SQL Server, which should not be used in real development.

Installing SQL Server 2019
A special instance of SQL Server named LocalDb is installed with Visual Studio 2022. If you choose not to use SQL Server Express LocalDB or Docker and you are using a Windows machine, you need to install SQL Server 2019 Developer Edition. SQL Server 2019 Developer Edition is free and can be downloaded from here:

https://www.microsoft.com/en-us/sql-server/sql-server-downloads

If you have another edition, you can use that instance with this book as well; you will just need to change your connection string to match your installed instance.

Installing a SQL Server IDE
Azure Data Studio is a new IDE for use with SQL Server. It is free and cross-platform, so it will work on Windows, Mac, or Linux. It can be downloaded from here:

https://docs.microsoft.com/en-us/sql/azure-data-studio/download-azure-data-studio

■ Note If you are using a Windows machine and prefer to use SQL Server Management Studio (SSMS), you can download the latest copy from here: https://docs.microsoft.com/en-us/sql/ssms/download- sql-server-management-studio-ssms.

Connecting to SQL Server
Once you have Azure Data Studio or SSMS installed, it is time to connect to your database instance. The following sections cover connecting to SQL Server in either a Docker container or using a LocalDb instance. If you are using another instance of SQL Server, please update the connection string used in the following sections accordingly.

Connecting to SQL Server in a Docker Container
To connect to your SQL Server instance running in a Docker container, first make sure it is up and running. Next, click “Create a connection” in Azure Data Studio, as shown in Figure 20-3.

Figure 20-3. Creating a connection in Azure Data Studio

In the Connection Details dialog, enter .,5433 for the Server value. The dot indicates the current host, and 5433 is the port that you indicated when creating the SQL Server instance in the Docker container. The host and the port must be separated by a comma. Enter sa for the username; the password is the same one that you entered when creating the SQL Server instance. The name is optional but allows you to quickly select this connection in subsequent Azure Data Studio sessions. Figure 20-4 shows these connection options.

Figure 20-4. Setting the connection options for Docker SQL Server

Connecting to SQL Server LocalDb
To connect to the Visual Studio–installed version of SQL Server Express LocalDb, update the connection information to match what is shown in Figure 20-5.

Figure 20-5. Setting the connection options for SQL Server LocalDb

When connecting to LocalDb, you can use Windows Authentication, since the instance is running on the same machine as Azure Data Studio and the same security context as the currently logged in user.

Connecting to Any Other SQL Server Instance
If you are connecting to any other SQL Server instance, update the connection properties accordingly.

Restoring the AutoLot Database Backup
Instead of building the database from scratch, you can use either SSMS or Azure Data Studio to restore one of the supplied backups contained in the chapter’s files in the repository. There are two backups supplied: the one named AutoLotWindows.ba is designed for use on a Windows machine (LocalDb, Windows Server, etc.), and the one named AutoLotDocker.ba is designed for use in a Docker container.

■ Note Git by default ignores files with a bak extension. You will need to rename the extension from ba_ to
bak before restoring the database.

Copying the Backup File to Your Container
If you are using SQL Server in a Docker container, you first must copy the backup file to the container. Fortunately, the Docker CLI provides a mechanism for working with a container’s file system. First, create a new directory for the backup using the following command in a command window on your host machine:

docker exec -it AutoLot mkdir var/opt/mssql/backup

The path structure must match the container’s operating system (in this case Ubuntu), even if your host machine is Windows based. Next, copy the backup to your new directory using the following command (updating the location of AutoLotDocker.bak to your local machine’s relative or absolute path):

[Windows]
docker cp .\AutoLotDocker.bak AutoLot:var/opt/mssql/backup

[Non-Windows]
docker cp ./AutoLotDocker.bak AutoLot:var/opt/mssql/backup

Note that the source directory structure matches the host machine (in my example, Windows), while the target is the container name and then the directory path (in the target OS format).

Restoring the Database with SSMS
To restore the database using SSMS, right-click the Databases node in Object Explorer. Select Restore Database. Select Device and click the ellipses. This will open the Select Backup Device dialog.

Restoring the Database to SQL Server (Docker)
Keep “Backup media type” set to File, and then click Add, navigate to the AutoLotDocker.bak file in the container, and click OK. When you are back on the main restore screen, click OK, as shown in Figure 20-6.

Figure 20-6. Restoring the database with SSMS

Restoring the Database to SQL Server (Windows)
Keep “Backup media type” set to File, and then click Add, navigate to AutoLotWindows.bak, and click OK. When you are back on the main restore screen, click OK, as shown in Figure 20-7.

Figure 20-7. Restoring the database with SSMS

Restoring the Database with Azure Data Studio
To restore the database using Azure Data Studio, click View, select the Command Palette (or press Ctrl+Shift+P), and select Restore. Select “Backup file” as the “Restore from” option and then the file you just copied. The target database and related fields will be filled in for you, as shown in Figure 20-8.

Figure 20-8. Restoring the database to Docker using Azure Data Studio

■ Note The process is the same to restore the Windows version of the backup using Azure Data Studio. Simply adjust the filename and paths.

Creating the AutoLot Database
This entire section is devoted to creating the AutoLot database using Azure Data Studio (or SQL Server Management Studio). If you restored the backup, you can skip ahead to the section “The ADO.NET Data Provider Factory Model.”

■ Note All of the script files are located in a folder named Scripts along with this chapter’s code in the Git repository.

Creating the Database
To create the AutoLot database, connect to your database server using Azure Data Studio. Open a new query by selecting File ➤ New Query (or by pressing Ctrl+N) and entering the following command text:

USE [master] GO
CREATE DATABASE [AutoLot] GO
ALTER DATABASE [AutoLot] SET RECOVERY SIMPLE GO

Besides changing the recovery mode to simple, this creates the AutoLot database using the SQL Server defaults. Click Run (or press F5) to create the database.

Creating the Tables
The AutoLot database contains five tables: Inventory, Makes, Customers, Orders, and CreditRisks.

Creating the Inventory Table
With the database created, it is time to create the tables. First up is the Inventory table. Open a new query and enter the following SQL:

USE [AutoLot] GO
CREATE TABLE [dbo].[Inventory](
[Id] [int] IDENTITY(1,1) NOT NULL,
[MakeId] [int] NOT NULL,
[Color] nvarchar NOT NULL, [PetName] nvarchar NOT NULL, [TimeStamp] [timestamp] NULL,
CONSTRAINT [PK_Inventory] PRIMARY KEY CLUSTERED
(
[Id] ASC
) ON [PRIMARY]
) ON [PRIMARY] GO

Click Run (or press F5) to create the table.

■ Note If you are not familiar with the SQL Server TimeStamp data type (that maps to a byte[] in C#), don’t worry about it at this time. Just know that it is used for row-level concurrency checking and will be covered with Entity Framework Core.

Creating the Makes Table
The Inventory table stores a foreign key to the (not yet created) Makes table. Create a new query and enter the following SQL to create the Makes table:

USE [AutoLot] GO
CREATE TABLE [dbo].[Makes](
[Id] [int] IDENTITY(1,1) NOT NULL,
[Name] nvarchar NOT NULL, [TimeStamp] [timestamp] NULL,
CONSTRAINT [PK_Makes] PRIMARY KEY CLUSTERED
(
[Id] ASC
) ON [PRIMARY]
) ON [PRIMARY] GO

Click Run (or press F5) to create the table.

Creating the Customers Table
The Customers table (as the name suggests) will contain a list of customers. Create a new query and enter the following SQL commands:

USE [AutoLot] GO
CREATE TABLE [dbo].[Customers](
[Id] [int] IDENTITY(1,1) NOT NULL,
[FirstName] nvarchar NOT NULL, [LastName] nvarchar NOT NULL, [TimeStamp] [timestamp] NULL,
CONSTRAINT [PK_Customers] PRIMARY KEY CLUSTERED
(
[Id] ASC
) ON [PRIMARY]
) ON [PRIMARY] GO

Click Run (or press F5) to create the Customers table.

Creating the Orders Table
You will use the next table, Orders, to represent the automobile a given customer has ordered. Create a new query, enter the following code, and click Run (or press F5):

USE [AutoLot] GO
CREATE TABLE [dbo].[Orders](
[Id] [int] IDENTITY(1,1) NOT NULL,
[CustomerId] [int] NOT NULL,

[CarId] [int] NOT NULL, [TimeStamp] [timestamp] NULL,
CONSTRAINT [PK_Orders] PRIMARY KEY CLUSTERED
(
[Id] ASC
) ON [PRIMARY]
) ON [PRIMARY] GO

Creating the CreditRisks Table
You will use your final table, CreditRisks, to represent the customers who are considered a credit risk. Create a new query, enter the following code, and click Run (or press F5):

USE [AutoLot] GO
CREATE TABLE [dbo].[CreditRisks]( [Id] [int] IDENTITY(1,1) NOT NULL,
[FirstName] nvarchar NOT NULL, [LastName] nvarchar NOT NULL, [CustomerId] [int] NOT NULL, [TimeStamp] [timestamp] NULL,
CONSTRAINT [PK_CreditRisks] PRIMARY KEY CLUSTERED
(
[Id] ASC
) ON [PRIMARY]
) ON [PRIMARY] GO

Creating the Table Relationships
This section will add the foreign key relationships between the interrelated tables.

Creating the Inventory to Makes Relationship
Open a new query, enter the following SQL, and click Run (or press F5):

USE [AutoLot] GO
CREATE NONCLUSTERED INDEX [IX_Inventory_MakeId] ON [dbo].[Inventory] (
[MakeId] ASC
) ON [PRIMARY] GO
ALTER TABLE [dbo].[Inventory] WITH CHECK ADD CONSTRAINT [FK_Make_Inventory] FOREIGN KEY([MakeId])
REFERENCES [dbo].[Makes] ([Id]) GO
ALTER TABLE [dbo].[Inventory] CHECK CONSTRAINT [FK_Make_Inventory] GO

Creating the Inventory to Orders Relationship
Open a new query, enter the following SQL, and click Run (or press F5):

USE [AutoLot] GO
CREATE NONCLUSTERED INDEX [IX_Orders_CarId] ON [dbo].[Orders] (
[CarId] ASC
) ON [PRIMARY] GO
ALTER TABLE [dbo].[Orders] WITH CHECK ADD CONSTRAINT [FK_Orders_Inventory] FOREIGN KEY([CarId])
REFERENCES [dbo].[Inventory] ([Id]) GO
ALTER TABLE [dbo].[Orders] CHECK CONSTRAINT [FK_Orders_Inventory] GO

Creating the Orders to Customers Relationship
Open a new query, enter the following SQL, and click Run (or press F5):

USE [AutoLot] GO
CREATE UNIQUE NONCLUSTERED INDEX [IX_Orders_CustomerId_CarId] ON [dbo].[Orders] (
[CustomerId] ASC, [CarId] ASC
) ON [PRIMARY] GO
ALTER TABLE [dbo].[Orders] WITH CHECK ADD CONSTRAINT [FK_Orders_Customers] FOREIGN KEY([CustomerId])
REFERENCES [dbo].[Customers] ([Id]) ON DELETE CASCADE
GO
ALTER TABLE [dbo].[Orders] CHECK CONSTRAINT [FK_Orders_Customers] GO

Creating the Customers to CreditRisks Relationship
Open a new query, enter the following SQL, and click Run (or press F5):

USE [AutoLot] GO
CREATE NONCLUSTERED INDEX [IX_CreditRisks_CustomerId] ON [dbo].[CreditRisks] (
[CustomerId] ASC
) ON [PRIMARY] GO

ALTER TABLE [dbo].[CreditRisks] WITH CHECK ADD CONSTRAINT [FK_CreditRisks_Customers] FOREIGN KEY([CustomerId])
REFERENCES [dbo].[Customers] ([Id]) ON DELETE CASCADE
GO
ALTER TABLE [dbo].[CreditRisks] CHECK CONSTRAINT [FK_CreditRisks_Customers] GO

■ Note If you are wondering why there are columns for FirstName and LastName and a relationship to the customer table, it’s merely for demo purposes. I could think up a creative reason for it, but at the end of the day, they are used in later examples in the book.

Creating the GetPetName() Stored Procedure
Later in this chapter, you will learn how to use ADO.NET to invoke stored procedures. As you might already know, stored procedures are code routines stored within a database that do something. Like C# methods, stored procedures can return data or just operate on data without returning anything. You will add a single stored procedure that will return an automobile’s pet name, based on the supplied carId. To do so, create a new query window and enter the following SQL command:

USE [AutoLot] GO
CREATE PROCEDURE [dbo].[GetPetName] @carID int,
@petName nvarchar(50) output AS
SELECT @petName = PetName from dbo.Inventory where Id = @carID GO

Click Run (or press F5) to create the stored procedure.

Adding Test Records
Databases are rather boring without data, and it is a good idea to have scripts that can quickly load test records into the database.

Makes Records
Create a new query and execute the following SQL statements to add records into the Makes table:

USE [AutoLot] GO
SET IDENTITY_INSERT [dbo].[Makes] ON
INSERT INTO [dbo].[Makes] ([Id], [Name]) VALUES (1, N'VW')
INSERT INTO [dbo].[Makes] ([Id], [Name]) VALUES (2, N'Ford')
INSERT INTO [dbo].[Makes] ([Id], [Name]) VALUES (3, N'Saab')
INSERT INTO [dbo].[Makes] ([Id], [Name]) VALUES (4, N'Yugo')

INSERT INTO [dbo].[Makes] ([Id], [Name]) VALUES (5, N'BMW')
INSERT INTO [dbo].[Makes] ([Id], [Name]) VALUES (6, N'Pinto') SET IDENTITY_INSERT [dbo].[Makes] OFF

Inventory Table Records
To add records to your first table, create a new query and execute the following SQL statements to add records into the Inventory table:

USE [AutoLot] GO
SET IDENTITY_INSERT [dbo].[Inventory] ON GO
INSERT INTO [dbo].[Inventory] ([Id], [MakeId], [Color], [PetName]) VALUES (1, 1, N'Black', N'Zippy')
INSERT INTO [dbo].[Inventory] ([Id], [MakeId], [Color], [PetName]) VALUES (2, 2, N'Rust', N'Rusty')
INSERT INTO [dbo].[Inventory] ([Id], [MakeId], [Color], [PetName]) VALUES (3, 3, N'Black', N'Mel')
INSERT INTO [dbo].[Inventory] ([Id], [MakeId], [Color], [PetName]) VALUES (4, 4, N'Yellow', N'Clunker')
INSERT INTO [dbo].[Inventory] ([Id], [MakeId], [Color], [PetName]) VALUES (5, 5, N'Black', N'Bimmer')
INSERT INTO [dbo].[Inventory] ([Id], [MakeId], [Color], [PetName]) VALUES (6, 5, N'Green', N'Hank')
INSERT INTO [dbo].[Inventory] ([Id], [MakeId], [Color], [PetName]) VALUES (7, 5, N'Pink', N'Pinky')
INSERT INTO [dbo].[Inventory] ([Id], [MakeId], [Color], [PetName]) VALUES (8, 6, N'Black', N'Pete')
INSERT INTO [dbo].[Inventory] ([Id], [MakeId], [Color], [PetName]) VALUES (9, 4, N'Brown', N'Brownie')SET IDENTITY_INSERT [dbo].[Inventory] OFF
GO

Customer Records
To add records to the Customers table, create a new query and execute the following SQL statements:

USE [AutoLot] GO
SET IDENTITY_INSERT [dbo].[Customers] ON
INSERT INTO [dbo].[Customers] ([Id], [FirstName], [LastName]) VALUES (1, N'Dave', N'Brenner')
INSERT INTO [dbo].[Customers] ([Id], [FirstName], [LastName]) VALUES (2, N'Matt', N'Walton') INSERT INTO [dbo].[Customers] ([Id], [FirstName], [LastName]) VALUES (3, N'Steve', N'Hagen') INSERT INTO [dbo].[Customers] ([Id], [FirstName], [LastName]) VALUES (4, N'Pat', N'Walton') INSERT INTO [dbo].[Customers] ([Id], [FirstName], [LastName]) VALUES (5, N'Bad', N'Customer')
SET IDENTITY_INSERT [dbo].[Customers] OFF

Order Records
Now add data to your Orders table. Create a new query, enter the following SQL, and click Run (or press F5):

USE [AutoLot] GO
SET IDENTITY_INSERT [dbo].[Orders] ON
INSERT INTO [dbo].[Orders] ([Id], [CustomerId], [CarId]) VALUES (1, 1, 5) INSERT INTO [dbo].[Orders] ([Id], [CustomerId], [CarId]) VALUES (2, 2, 1) INSERT INTO [dbo].[Orders] ([Id], [CustomerId], [CarId]) VALUES (3, 3, 4) INSERT INTO [dbo].[Orders] ([Id], [CustomerId], [CarId]) VALUES (4, 4, 7) SET IDENTITY_INSERT [dbo].[Orders] OFF

CreditRisk Records
The final step is to add data to the CreditRisks table. Create a new query, enter the following SQL, and click Run (or press F5):

USE [AutoLot] GO
SET IDENTITY_INSERT [dbo].[CreditRisks] ON
INSERT INTO [dbo].[CreditRisks] ([Id], [FirstName], [LastName], [CustomerId]) VALUES (1, N'Bad', N'Customer', 5)
SET IDENTITY_INSERT [dbo].[CreditRisks] OFF

With this, the AutoLot database is complete! Of course, this is a far cry from a real-world application database, but it will serve your needs for this chapter and will be added to in the Entity Framework Core chapters. Now that you have a database to test with, you can dive into the details of the ADO.NET data provider factory model.

The ADO.NET Data Provider Factory Model
The .NET data provider factory pattern allows you to build a single code base using generalized data access types. To understand the data provider factory implementation, recall from Table 20-1 that the classes within a data provider all derive from the same base classes defined within the System.Data.Common namespace.
• DbCommand: The abstract base class for all command classes
• DbConnection: The abstract base class for all connection classes
• DbDataAdapter: The abstract base class for all data adapter classes
• DbDataReader: The abstract base class for all data reader classes
• DbParameter: The abstract base class for all parameter classes
• DbTransaction: The abstract base class for all transaction classes

Each of the .NET–compliant data providers contains a class type that derives from System.Data.
Common.DbProviderFactory. This base class defines several methods that retrieve provider-specific data objects. Here are the members of DbProviderFactory:

public abstract class DbProviderFactory
{
..public virtual bool CanCreateDataAdapter { get;};
..public virtual bool CanCreateCommandBuilder { get;}; public virtual DbCommand CreateCommand();
public virtual DbCommandBuilder CreateCommandBuilder(); public virtual DbConnection CreateConnection();
public virtual DbConnectionStringBuilder CreateConnectionStringBuilder();
public virtual DbDataAdapter CreateDataAdapter(); public virtual DbParameter CreateParameter(); public virtual DbDataSourceEnumerator
CreateDataSourceEnumerator();
}

To obtain the DbProviderFactory-derived type for your data provider, each provider provides a static property used to return the correct type. To return the SQL Server version of the DbProviderFactory, use the following code:

// Get the factory for the SQL data provider.
DbProviderFactory sqlFactory = Microsoft.Data.SqlClient.SqlClientFactory.Instance;
To make the program more versatile, you can create a DbProviderFactory factory that returns a specific flavor of a DbProviderFactory based on a setting in the appsettings.json file for the application. You will learn how to do this shortly; for the moment, you can obtain the associated provider-specific data objects (e.g., connections, commands, and data readers) once you have obtained the factory for your data provider.

A Complete Data Provider Factory Example
For a complete example, create a new C# Console Application project (named DataProviderFactory) that prints out the automobile inventory of the AutoLot database. For this initial example, you will hard-code the data access logic directly within the console application (to keep things simple). As you progress through this chapter, you will see better ways to do this.
Add the Microsoft.Extensions.Configuration.Json, System.Data.Common, System.Data.Odbc, System.Data.OleDb, and Microsoft.Data.SqlClient packages to the project. Next, define the PC compiler constant (if you are using a Windows OS).


PC

Next, add a new file named DataProviderEnum.cs and update the code to the following:
namespace DataProviderFactory;
//OleDb is Windows only enum DataProviderEnum
{

SqlServer, #if PC
OleDb, #endif
Odbc
}

Add a new JSON file named appsettings.json to the project and update its contents to the following (update the connection strings based on your specific environment):

{
"ProviderName": "SqlServer",
//"ProviderName": "OleDb",
//"ProviderName": "Odbc", "SqlServer": {
// for localdb use @"Data Source=(localdb)\mssqllocaldb;Integrated Security=true; Initial Catalog=AutoLot"
"ConnectionString": "Data Source=.,5433;User Id=sa;Password=P@ssw0rd;Initial Catalog=Aut oLot;Encrypt=False;"
},
"Odbc": {
// for localdb use @"Driver={ODBC Driver 17 for SQL Server};Server=(localdb)\mssqllocald b;Database=AutoLot;Trusted_Connection=Yes";
"ConnectionString": "Driver={ODBC Driver 17 for SQL Server};Server=localhost,5433; Database=AutoLot;UId=sa;Pwd=P@ssw0rd;Encrypt=False;"
},
"OleDb": {
// if localdb use @"Provider=SQLNCLI11;Data Source=(localdb)\mssqllocaldb;Initial Catalog=AutoLot;Integrated Security=SSPI"),
"ConnectionString": "Provider=SQLNCLI11;Data Source=.,5433;User Id=sa;Password=P@ssw0rd; Initial Catalog=AutoLot;Encrypt=False;"
}
}

■ Note When using SQL Server in a Docker container that does not have a certificate installed, the connection string must be unencrypted, which is why we have the Encrypt=False; setting in the Docker connection strings. For real-world applications, do not use this setting; instead, make sure the container (or your SQL Server instance) has a certificate, and use Encrypt=True; instead.

Configure MSBuild to copy the JSON settings file to the output directory on every build. Update the project file by adding the following:



Always

■ Note The CopyToOutputDirectory is whitespace sensitive. Make sure it is all on one line without any spaces around the word Always.

Now that you have a proper appsettings.json, you can read in the provider and connectionString values using .NET configuration. Start by clearing out all the code in the file and adding the following using statements at the top of the Program.cs file:

using System.Data.Common; using System.Data.Odbc; #if PC
using System.Data.OleDb; #endif
using Microsoft.Data.SqlClient;
using Microsoft.Extensions.Configuration; using DataProviderFactory;

Add the following code to the Program.cs file:

Console.WriteLine(" Fun with Data Provider Factories \n"); var (provider, connectionString) = GetProviderFromConfiguration(); DbProviderFactory factory = GetDbProviderFactory(provider);
// Now get the connection object.
using (DbConnection connection = factory.CreateConnection())
{
Console.WriteLine($"Your connection object is a: {connection.GetType().Name}"); connection.ConnectionString = connectionString;
connection.Open();

// Make command object.
DbCommand command = factory.CreateCommand();
Console.WriteLine($"Your command object is a: {command.GetType().Name}"); command.Connection = connection;
command.CommandText =
"Select i.Id, m.Name From Inventory i inner join Makes m on m.Id = i.MakeId ";

// Print out data with data reader.
using (DbDataReader dataReader = command.ExecuteReader())
{
Console.WriteLine($"Your data reader object is a: {dataReader.GetType().Name}"); Console.WriteLine("\n Current Inventory ");
while (dataReader.Read())
{
Console.WriteLine($"-> Car #{dataReader["Id"]} is a {dataReader["Name"]}.");
}
}
}
Console.ReadLine();

Next, add the following code to the end of the Program.cs file. These methods read the configuration, set the DataProviderEnum to the correct value, get the connection string, and return an instance of the DbProviderFactory:

static DbProviderFactory GetDbProviderFactory(DataProviderEnum provider)
=> provider switch
{
DataProviderEnum.SqlServer => SqlClientFactory.Instance, DataProviderEnum.Odbc => OdbcFactory.Instance,

if PC

DataProviderEnum.OleDb => OleDbFactory.Instance, #endif
_ => null
};

static (DataProviderEnum Provider, string ConnectionString) GetProviderFromConfiguration()
{
IConfiguration config = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", true, true)
.Build();
var providerName = config["ProviderName"]; if (Enum.TryParse
(providerName, out DataProviderEnum provider))
{
return (provider,config[$"{providerName}:ConnectionString"]);
};
throw new Exception("Invalid data provider value supplied.");
}

Notice that, for diagnostic purposes, you use reflection services to print the name of the underlying connection, command, and data reader. If you run this application, you will find the following current data in the Inventory table of the AutoLot database printed to the console:

Fun with Data Provider Factories Your connection object is a: SqlConnection Your command object is a: SqlCommand
Your data reader object is a: SqlDataReader

Current Inventory
-> Car #1 is a VW.
-> Car #2 is a Ford.
-> Car #3 is a Saab.
-> Car #4 is a Yugo.
-> Car #9 is a Yugo.
-> Car #5 is a BMW.
-> Car #6 is a BMW.
-> Car #7 is a BMW.
-> Car #8 is a Pinto.

Now change the settings file to specify a different provider. The code will pick up the related connection string and produce the same output as before, except for the type-specific information.
Of course, based on your experience with ADO.NET, you might be a bit unsure exactly what the connection, command, and data reader objects actually do. Do not sweat the details for the time being (quite a few pages remain in this chapter, after all!). At this point, it is enough to know that you can use the ADO.NET data provider factory model to build a single code base that can consume various data providers in a declarative manner.

A Potential Drawback with the Data Provider Factory Model
Although this is a powerful model, you must make sure that the code base uses only types and methods common to all providers through the members of the abstract base classes. Therefore, when authoring your code base, you are limited to the members exposed by DbConnection, DbCommand, and the other types of the System.Data.Common namespace.
Given this, you might find that this generalized approach prevents you from directly accessing some of the bells and whistles of a particular DBMS. If you must be able to invoke specific members of the underlying provider (e.g., SqlConnection), you can do so using an explicit cast, as in this example:

if (connection is SqlConnection sqlConnection)
{
// Print out which version of SQL Server is used. WriteLine(sqlConnection.ServerVersion);
}

When doing this, however, your code base becomes a bit harder to maintain (and less flexible) because you must add a number of runtime checks. Nevertheless, if you need to build ADO.NET data access libraries in the most flexible way possible, the data provider factory model provides a great mechanism for doing so.

■ Note Entity Framework Core and its support for dependency injection greatly simplifies building data access libraries that need to access disparate data sources.

With this first example behind you, you can now dive into the details of working with ADO.NET.

Diving Deeper into Connections, Commands, and DataReaders
As shown in the previous example, ADO.NET allows you to interact with a database using the connection, command, and data reader objects of your data provider. Now you will create an expanded example to get a deeper understanding of these objects in ADO.NET.
In the previous example demonstrated, you need to perform the following steps when you want to connect to a database and read the records using a data reader object:
1.Allocate, configure, and open your connection object.
2.Allocate and configure a command object, specifying the connection object as a constructor argument or with the Connection property.
3.Call ExecuteReader() on the configured command class.
4.Process each record using the Read() method of the data reader.

To get the ball rolling, create a new Console Application project named AutoLot.DataReader and add the Microsoft.Data.SqlClient package. Here is the complete code within the Program.cs file (analysis will follow):

using Microsoft.Data.SqlClient;

Console.WriteLine(" Fun with Data Readers \n");

// Create and open a connection.
using (SqlConnection connection = new SqlConnection())
{
connection.ConnectionString =
@" Data Source=.,5433;User Id=sa;Password=P@ssw0rd;Initial Catalog=AutoLot;Encryp t=False;";
connection.Open();
// Create a SQL command object. string sql =
@"Select i.id, m.Name as Make, i.Color, i.Petname FROM Inventory i
INNER JOIN Makes m on m.Id = i.MakeId"; SqlCommand myCommand = new SqlCommand(sql, connection);

// Obtain a data reader a la ExecuteReader().
using (SqlDataReader myDataReader = myCommand.ExecuteReader())
{
// Loop over the results. while (myDataReader.Read())
{
Console.WriteLine($"-> Make: {myDataReader["Make"]}, PetName: {myDataReader ["PetName"]}, Color: {myDataReader["Color"]}.");
}
}
}
Console.ReadLine();

Working with Connection Objects
The first step to take when working with a data provider is to establish a session with the data source using the connection object (which, as you recall, derives from DbConnection). .NET connection objects are provided with a formatted connection string; this string contains a number of name-value pairs, separated by semicolons. You use this information to identify the name of the machine you want to connect to, the required security settings, the name of the database on that machine, and other data provider–specific information.
As you can infer from the preceding code, the Initial Catalog name refers to the database you want to establish a session with. The Data Source name identifies the name of the machine that maintains the database. I am using .,5433, which refers to the host machine (the period is the same as using “localhost”), and port 5433, which is the port the Docker container has mapped to the SQL Server port. If you were using a different instance, you would define the property as machinename,port\instance. For example, MYSERVER\ SQLSERVER2019 means MYSERVER is the name of the server the SQL Server is running on, the default port
is being used, and SQLSERVER2019 is the name of the instance. If the machine is local to the development, you can use a period (.) or the token (localhost) for the server name. If the SQL Server instance is the

default instance, the instance name is left off. For example, if you created AutoLot on a Microsoft SQL Server installation set up as the default instance on your local computer, you would use "Data Source=localhost".
Beyond this, you can supply any number of tokens that represent security credentials. If Integrated Security is set to true, current Windows account credentials are used for authentication and authorization.
After you establish your connection string, you can use a call to Open() to establish a connection with the DBMS. In addition to the ConnectionString, Open(), and Close() members, a connection object provides a number of members that let you configure additional settings regarding your connection,
such as timeout settings and transactional information. Table 20-4 lists some (but not all) members of the
DbConnection base class.

Table 20-4. Members of the DbConnection Type

Member Meaning in Life

BeginTransaction() You use this method to begin a database transaction.
ChangeDatabase() You use this method to change the database on an open connection.
ConnectionTimeout This read-only property returns the amount of time to wait while establishing a connection before terminating and generating an error (the default value is provider dependent). If you would like to change the default, specify a Connect Timeout segment in the connection string (e.g., Connect Timeout=30).
Database This read-only property gets the name of the database maintained by the connection object.
DataSource This read-only property gets the location of the database maintained by the connection object.
GetSchema() This method returns a DataTable object that contains schema information from the data source.
State This read-only property gets the current state of the connection, which is represented by the ConnectionState enumeration.

The properties of the DbConnection type are typically read-only in nature and are useful only when you want to obtain the characteristics of a connection at runtime. When you need to override default settings, you must alter the connection string itself. For example, the following connection string sets the connection timeout setting from the default (15 seconds for SQL Server) to 30 seconds:

using(SqlConnection connection = new SqlConnection())
{
connection.ConnectionString =
@" Data Source=.,5433;User Id=sa;Password=P@ssw0rd;Initial Catalog=AutoLot;Encrypt=False
;Connect Timeout=30"; connection.Open();
}

The following code outputs details about the SqlConnection that is passed into it:

static void ShowConnectionStatus(SqlConnection connection)
{
// Show various stats about current connection object. Console.WriteLine(" Info about your connection ");

Console.WriteLine($"Database location: {connection.DataSource}"); Console.WriteLine($"Database name: {connection.Database}"); Console.WriteLine($"Timeout: {connection.ConnectionTimeout}"); Console.WriteLine($"Connection state: {connection.State}");
}

While most of these properties are self-explanatory, the State property is worth a special mention. You can assign this property any value of the ConnectionState enumeration, as shown here:

public enum ConnectionState
{
Broken, Closed, Connecting, Executing, Fetching, Open
}

However, the only valid ConnectionState values are ConnectionState.Open, ConnectionState.
Connecting, and ConnectionState.Closed (the remaining members of this enum are reserved for future use). Also, it is always safe to close a connection, even if the connection state is currently ConnectionState.Closed.

Working with ConnectionStringBuilder Objects
Working with connection strings programmatically can be cumbersome because they are often represented as string literals, which are difficult to maintain and error-prone at best. The .NET-compliant data providers support connection string builder objects, which allow you to establish the name-value pairs using strongly typed properties. Consider the following update to the current code:

var connectionStringBuilder = new SqlConnectionStringBuilder
{
InitialCatalog = "AutoLot", DataSource = ".,5433", UserID = "sa",
Password = "P@ssw0rd", ConnectTimeout = 30, Encrypt=false
};
connection.ConnectionString = connectionStringBuilder.ConnectionString;

In this iteration, you create an instance of SqlConnectionStringBuilder, set the properties accordingly, and obtain the internal string using the ConnectionString property. Also note that you use the default constructor of the type. If you so choose, you can also create an instance of your data provider’s connection string builder object by passing in an existing connection string as a starting point (this can be helpful when you read these values dynamically from an external source). Once you have hydrated the object with the initial string data, you can change specific name-value pairs using the related properties.

Working with Command Objects
Now that you understand better the role of the connection object, the next order of business is to check out how to submit SQL queries to the database in question. The SqlCommand type (which derives from DbCommand) is an OO representation of a SQL query, table name, or stored procedure. You specify the type of command using the CommandType property, which can take any value from the CommandType enum, as shown here:

public enum CommandType
{
StoredProcedure, TableDirect,
Text // Default value.
}

When you create a command object, you can establish the SQL query as a constructor parameter or directly by using the CommandText property. Also, when you create a command object, you need to specify the connection you want to use. Again, you can do so as a constructor parameter or by using the Connection property. Consider this code snippet:

// Create command object via ctor args. string sql =
@"Select i.id, m.Name as Make, i.Color, i.Petname FROM Inventory i
INNER JOIN Makes m on m.Id = i.MakeId"; SqlCommand myCommand = new SqlCommand(sql, connection);
// Create another command object via properties. SqlCommand testCommand = new SqlCommand(); testCommand.Connection = connection; testCommand.CommandText = sql;

Realize that, at this point, you have not actually submitted the SQL query to the AutoLot database but instead prepared the state of the command object for future use. Table 20-5 highlights some additional members of the DbCommand type.

Table 20-5. Members of the DbCommand Type

Member Meaning in Life
CommandTimeout Gets or sets the time to wait while executing the command before terminating the attempt and generating an error. The default is 30 seconds.
Connection Gets or sets the DbConnection used by this instance of the DbCommand.
Parameters Gets the collection of DbParameter objects used for a parameterized query.
Cancel() Cancels the execution of a command.
ExecuteReader() Executes a SQL query and returns the data provider’s DbDataReader object, which provides forward-only, read-only access for the result of the query.
ExecuteNonQuery() Executes a SQL nonquery (e.g., an insert, update, delete, or create table).
(continued)

Table 20-5. (continued)

Member Meaning in Life
ExecuteScalar() A lightweight version of the ExecuteReader() method that was designed specifically for singleton queries (e.g., obtaining a record count).
Prepare() Creates a prepared (or compiled) version of the command on the data source. As you might know, a prepared query executes slightly faster and is useful when you need to execute the same query multiple times (typically with different parameters each time).

Working with Data Readers
After you establish the active connection and SQL command, the next step is to submit the query to the data source. As you might guess, you have a number of ways to do this. The DbDataReader type (which
implements IDataReader) is the simplest and fastest way to obtain information from a data store. Recall that
data readers represent a read-only, forward-only stream of data returned one record at a time. Given this, data readers are useful only when submitting SQL selection statements to the underlying data store.
Data readers are useful when you need to iterate over large amounts of data quickly and you do not need to maintain an in-memory representation. For example, if you request 20,000 records from a table to store in a text file, it would be rather memory intensive to hold this information in a DataSet (because a DataSet holds the entire result of the query in memory at the same time).
A better approach is to create a data reader that spins over each record as rapidly as possible. Be aware, however, that data reader objects (unlike data adapter objects, which you will examine later) maintain an open connection to their data source until you explicitly close the connection.
You obtain data reader objects from the command object using a call to ExecuteReader(). The data reader represents the current record it has read from the database. The data reader has an indexer method (e.g., [] syntax in C#) that allows you to access a column in the current record. You can access the column either by name or by zero-based integer.
The following use of the data reader leverages the Read() method to determine when you have reached the end of your records (using a false return value). For each incoming record that you read from the database, you use the type indexer to print out the make, pet name, and color of each automobile. Also note that you call Close() as soon as you finish processing the records, which frees up the connection object.

...
// Obtain a data reader via ExecuteReader().
using(SqlDataReader myDataReader = myCommand.ExecuteReader())
{
// Loop over the results.
while (myDataReader.Read())
{
WriteLine($"-> Make: { myDataReader["Make"]}, PetName: { myDataReader["PetName"]}, Color: { myDataReader["Color"]}.");
}
}
ReadLine();

In the preceding snippet, you overload the indexer of a data reader object to take either a string (representing the name of the column) or an int (representing the column’s ordinal position). Thus, you can clean up the current reader logic (and avoid hard-coded string names) with the following update (note the use of the FieldCount property):

while (myDataReader.Read())
{
for (int i = 0; i < myDataReader.FieldCount; i++)
{
Console.Write(i != myDataReader.FieldCount - 1
? $"{myDataReader.GetName(i)} = {myDataReader.GetValue(i)}, "
: $"{myDataReader.GetName(i)} = {myDataReader.GetValue(i)} ");
}
Console.WriteLine();
}

If you compile and run your project at this point, you should see a list of all automobiles in the
Inventory table of the AutoLot database.

Fun with Data Readers

Info about your connection Database location: .,5433
Database name: AutoLot Timeout: 30
Connection state: Open

id = 1, Make = VW, Color = Black, Petname = Zippy id = 2, Make = Ford, Color = Rust, Petname = Rusty id = 3, Make = Saab, Color = Black, Petname = Mel
id = 4, Make = Yugo, Color = Yellow, Petname = Clunker id = 5, Make = BMW, Color = Black, Petname = Bimmer
id = 6, Make = BMW, Color = Green, Petname = Hank id = 7, Make = BMW, Color = Pink, Petname = Pinky id = 8, Make = Pinto, Color = Black, Petname = Pete
id = 9, Make = Yugo, Color = Brown, Petname = Brownie

Obtaining Multiple Result Sets Using a Data Reader
Data reader objects can obtain multiple result sets using a single command object. For example, if you want to obtain all rows from the Inventory table, as well as all rows from the Customers table, you can specify both SQL SELECT statements using a semicolon delimiter, like so:

sql += ";Select * from Customers;";

■ Note The semicolon in the beginning is not a typo. When using multiple statements, they must be separated by semicolons. And since the initial statement did not contain one, it is added here at the beginning of the second statement.

After you obtain the data reader, you can iterate over each result set using the NextResult() method. Note that you are always returned the first result set automatically. Thus, if you want to read over the rows of each table, you can build the following iteration construct:

do
{
while (myDataReader.Read())
{
for (int i = 0; i < myDataReader.FieldCount; i++)
{
Console.Write(i != myDataReader.FieldCount - 1
? $"{myDataReader.GetName(i)} = {myDataReader.GetValue(i)}, "
: $"{myDataReader.GetName(i)} = {myDataReader.GetValue(i)} ");
}
Console.WriteLine();
}
Console.WriteLine();
} while (myDataReader.NextResult());

At this point, you should be more aware of the functionality data reader objects bring to the table. Always remember that a data reader can process only SQL Select statements; you cannot use them to modify an existing database table using Insert, Update, or Delete requests. Modifying an existing database requires additional investigation of command objects.

Working with Create, Update, and Delete Queries
The ExecuteReader() method extracts a data reader object that allows you to examine the results of a SQL Select statement using a forward-only, read-only flow of information. However, when you want to submit SQL statements that result in the modification of a given table (or any other nonquery SQL statement, such as creating tables or granting permissions), you call the ExecuteNonQuery() method of your command object. This single method performs inserts, updates, and deletes based on the format of your command text.

■ Note Technically speaking, a nonquery is a SQL statement that does not return a result set. Thus, Select statements are queries, while Insert, Update, and Delete statements are not. Given this, ExecuteNonQuery() returns an int that represents the number of rows affected, not a new set of records.

All the database interaction examples in this chapter so far have only opened connections and used them to retrieve data. This is just one part of working with a database; a data access framework would not be of much use unless it also fully supported Create, Read, Update, and Delete (CRUD) functionality. Next, you will learn how to do this using calls to ExecuteNonQuery().
Begin by creating a new C# Class Library project named AutoLot.Dal (short for AutoLot data access layer), delete the default class file, and add the Microsoft.Data.SqlClient package to the project. Add a new class file named GlobalUsings.cs to the project, and update the file to the following global using statements:

global using System.Data; global using System.Reflection;
global using Microsoft.Data.SqlClient;

Before building the class that will conduct the data operations, we will first create a C# class that represents a record from the Inventory table with its related Make information.

Create the Car and CarViewModel Classes
Modern data access libraries use classes (commonly called models or entities) that are used to represent and transport the data from the database. Additionally, classes can be used to represent a view into the data that combines two or more tables to make the data more meaningful. Entity classes are used to work with the database directory (for update statements), and view model classes are used for displaying the data in a meaningful way. You will see in the next chapter that these concepts are a foundation of object relational mappers (ORMs) like the Entity Framework Core, but for now, you are just going to create one model (for a raw inventory row) and one view model (combining an inventory row with the related data in the Makes table). Add a new folder to your project named Models, and add two new files, named Car.cs and CarViewModel.cs. Update the code to the following:

//Car.cs
namespace AutoLot.Dal.Models; public class Car
{
public int Id { get; set; } public int MakeId { get; set; } public string Color { get; set; }
public string PetName { get; set; } public byte[] TimeStamp {get;set;}
}

//CarViewModel.cs
namespace AutoLot.Dal.Models; public class CarViewModel : Car
{
public string Make { get; set; }
}

These classes will be used shortly, but first, add the AutoLot.Dal.Models namespace into the
GlobalUsings.cs file:

global using System.Data; global using System.Reflection;
global using Microsoft.Data.SqlClient; global using AutoLot.Dal.Models;

Adding the InventoryDal Class
Next, add a new folder named DataOperations. In this new folder, add a new class named InventoryDal.cs and change the class to public. This class will define various members to interact with the Inventory table of the AutoLot database.

Adding Constructors
Create a constructor that takes a string parameter (connectionString) and assigns the value to a class- level variable. Next, create a parameterless constructor that passes a default connection string to the other constructor. This enables the calling code to change the connection string from the default. The relevant code is as follows:

namespace AuoLot.Dal.DataOperations; public class InventoryDal
{
private readonly string _connectionString; public InventoryDal() : this(
@"Data Source=.,5433;User Id=sa;Password=P@ssw0rd;Initial Catalog=AutoLot;Encryp t=False;")
{
}
public InventoryDal(string connectionString)
=> _connectionString = connectionString;
}

Opening and Closing the Connection
Next, add a class-level variable to hold a connection that will be used by the data access code. Also, add two methods, one to open the connection (OpenConnection()) and the other to close the connection (CloseConnection()). In the CloseConnection() method, check the state of the connection, and if it is not closed, then call Close() on the connection. The code listing follows:

private SqlConnection _sqlConnection = null; private void OpenConnection()
{
_sqlConnection = new SqlConnection
{
ConnectionString = _connectionString
};
_sqlConnection.Open();
}
private void CloseConnection()
{
if (_sqlConnection?.State != ConnectionState.Closed)
{
_sqlConnection?.Close();
}
}

For the sake of brevity, most of the methods in the InventoryDal class will not use try/catch blocks to handle possible exceptions, nor will they throw custom exceptions to report various issues with the execution (e.g., a malformed connection string). If you were to build an industrial-strength data access library, you would absolutely want to use structured exception handling techniques (as covered in Chapter 7) to account for any runtime anomalies.

Adding IDisposable
Add the IDisposable interface to the class definition, like this:

public class InventoryDal : IDisposable
{
...
}

Next, implement the disposable pattern, calling Dispose on the SqlConnection object.

bool _disposed = false;
protected virtual void Dispose(bool disposing)
{
if (_disposed)
{
return;
}
if (disposing)
{
_sqlConnection.Dispose();
}
_disposed = true;
}
public void Dispose()
{
Dispose(true); GC.SuppressFinalize(this);
}

Adding the Selection Methods
You start by combining what you already know about Command objects, DataReaders, and generic collections to get the records from the Inventory table. As you saw earlier in this chapter, a data provider’s data reader object allows for a selection of records using a read-only, forward-only mechanism using the Read() method. In this example, the CommandBehavior property on the DataReader is set to automatically close the connection when the reader is closed. The GetAllInventory() method returns a List to represent all the data in the Inventory table.

public List GetAllInventory()
{
OpenConnection();
// This will hold the records.
List inventory = new List();

// Prep command object. string sql =
@"SELECT i.Id, i.Color, i.PetName,m.Name as Make FROM Inventory i
INNER JOIN Makes m on m.Id = i.MakeId"; using SqlCommand command =
new SqlCommand(sql, _sqlConnection)
{
CommandType = CommandType.Text
};
command.CommandType = CommandType.Text; SqlDataReader dataReader =
command.ExecuteReader(CommandBehavior.CloseConnection); while (dataReader.Read())

{
inventory.Add(new CarViewModel
{
Id = (int)dataReader["Id"],
Color = (string)dataReader["Color"], Make = (string)dataReader["Make"], PetName = (string)dataReader["PetName"]
});
}
dataReader.Close(); return inventory;
}

The next selection method gets a single CarViewModel based on the CarId.

public CarViewModel GetCar(int id)
{
OpenConnection(); CarViewModel car = null;
//This should use parameters for security reasons string sql =
$@"SELECT i.Id, i.Color, i.PetName,m.Name as Make FROM Inventory i
INNER JOIN Makes m on m.Id = i.MakeId WHERE i.Id = {id}";
using SqlCommand command =
new SqlCommand(sql, _sqlConnection)
{
CommandType = CommandType.Text
};
SqlDataReader dataReader = command.ExecuteReader(CommandBehavior.CloseConnection);
while (dataReader.Read())
{
car = new CarViewModel
{
Id = (int) dataReader["Id"],
Color = (string) dataReader["Color"], Make = (string) dataReader["Make"], PetName = (string) dataReader["PetName"]
};
}
dataReader.Close(); return car;
}

■ Note It’s generally a bad practice to accept user input into raw SQL statements as is done here. Later in this chapter, this code will be updated to use parameters.

Inserting a New Car
Inserting a new record into the Inventory table is as simple as formatting the SQL Insert statement (based on user input), opening the connection, calling the ExecuteNonQuery() using your command object, and closing the connection. You can see this in action by adding a public method to your InventoryDal type named InsertAuto() that takes three parameters that map to the nonidentity columns of the Inventory table (Color, Make, and PetName). You use these arguments to format a string type to insert the new record. Finally, use your SqlConnection object to execute the SQL statement.

public void InsertAuto(string color, int makeId, string petName)
{
OpenConnection();
// Format and execute SQL statement.
string sql = $"Insert Into Inventory (MakeId, Color, PetName) Values ('{makeId}', '{color}', '{petName}')";
// Execute using our connection.
using (SqlCommand command = new SqlCommand(sql, _sqlConnection))
{
command.CommandType = CommandType.Text; command.ExecuteNonQuery();
}
CloseConnection();
}

This previous method takes three values for Car and works as long as the calling code passes the values in the correct order. A better method uses Car to make a strongly typed method, ensuring all the properties are passed into the method in the correct order.

Create the Strongly Type InsertCar() Method
Add another InsertAuto() method that takes Car as a parameter to your InventoryDal class, as shown here:

public void InsertAuto(Car car)
{
OpenConnection();
// Format and execute SQL statement.
string sql = "Insert Into Inventory (MakeId, Color, PetName) Values " +
$"('{car.MakeId}', '{car.Color}', '{car.PetName}')";

// Execute using our connection.
using (SqlCommand command = new SqlCommand(sql, _sqlConnection))
{
command.CommandType = CommandType.Text; command.ExecuteNonQuery();
}
CloseConnection();
}

Adding the Deletion Logic
Deleting an existing record is as simple as inserting a new record. Unlike when you created the code for InsertAuto(), this time you will learn about an important try/catch scope that handles the possibility of attempting to delete a car that is currently on order for an individual in the Customers table. The default INSERT and UPDATE options for foreign keys default to preventing the deletion of related records in linked tables. When this happens, a SqlException is thrown. A real program would handle that error intelligently; however, in this sample, you are just throwing a new exception. Add the following method to the InventoryDal class type:

public void DeleteCar(int id)
{
OpenConnection();
// Get ID of car to delete, then do so.
string sql = $"Delete from Inventory where Id = '{id}'";
using (SqlCommand command = new SqlCommand(sql, _sqlConnection))
{
try
{
command.CommandType = CommandType.Text; command.ExecuteNonQuery();
}
catch (SqlException ex)
{
Exception error = new Exception("Sorry! That car is on order!", ex); throw error;
}
}
CloseConnection();
}

Adding the Update Logic
When it comes to the act of updating an existing record in the Inventory table, the first thing you must decide is what you want to allow the caller to change, whether it is the car’s color, the pet name, the make, or all of these. One way to give the caller complete flexibility is to define a method that takes a string type to represent any sort of SQL statement, but that is risky at best.
Ideally, you want to have a set of methods that allow the caller to update a record in a variety of ways. However, for this simple data access library, you will define a single method that allows the caller to update the pet name of a given automobile, like so:

public void UpdateCarPetName(int id, string newPetName)
{
OpenConnection();
// Get ID of car to modify the pet name.
string sql = $"Update Inventory Set PetName = '{newPetName}' Where Id = '{id}'"; using (SqlCommand command = new SqlCommand(sql, _sqlConnection))
{
command.ExecuteNonQuery();
}
CloseConnection();
}

Working with Parameterized Command Objects
Currently, the insert, update, and delete logic for the InventoryDal type uses hard-coded string literals for each SQL query. With parameterized queries, SQL parameters are objects, rather than simple blobs of text. Treating SQL queries in a more object-oriented manner helps reduce the number of typos (given strongly typed properties); plus, parameterized queries typically execute much faster than a literal SQL string because they are parsed exactly once (rather than each time the SQL string is assigned to the CommandText property).
Parameterized queries also help protect against SQL injection attacks (a well-known data access security issue).
To support parameterized queries, ADO.NET command objects maintain a collection of individual parameter objects. By default, this collection is empty, but you can insert any number of parameter objects that map to a placeholder parameter in the SQL query. When you want to associate a parameter within a SQL query to a member in the command object’s parameters collection, you can prefix the SQL text parameter with the @ symbol (at least when using Microsoft SQL Server; not all DBMSs support this notation).

Specifying Parameters Using the DbParameter Type
Before you build a parameterized query, you need to familiarize yourself with the DbParameter type (which is the base class to a provider’s specific parameter object). This class maintains a number of properties that allow you to configure the name, size, and data type of the parameter, as well as other characteristics, including the parameter’s direction of travel. Table 20-6 describes some key properties of the DbParameter type.

Table 20-6. Key Members of the DbParameter Type

Property Meaning in Life
DbType Gets or sets the native data type of the parameter, represented as a CLR data type
Direction Gets or sets whether the parameter is input-only, output-only, bidirectional, or a return value parameter
IsNullable Gets or sets whether the parameter accepts null values
ParameterName Gets or sets the name of the DbParameter
Size Gets or sets the maximum parameter size of the data in bytes; this is useful only for textual data
Value Gets or sets the value of the parameter

Now let’s look at how to populate a command object’s collection of DBParameter-compatible objects by reworking the InventoryDal methods to use parameters.

Update the GetCar Method
The original implementation of the GetCar() method used C# string interpolation when building the SQL string to retrieve the car data. To update this method, create an instance of SqlParameter with the appropriate values, as follows:

SqlParameter param = new SqlParameter
{
ParameterName = "@carId", Value = id,

SqlDbType = SqlDbType.Int,
Direction = ParameterDirection.Input
};

The ParameterName value must match the name used in the SQL query (you will update that next), the type must match the database column type, and the direction is dependent on whether the parameter is used to send data into the query (ParameterDirection.Input) or if it is meant to return data from the query (ParameterDirection.Output). Parameters can also be defined as input/output or as return values (e.g., from a stored procedure).
Next, update the SQL string to use the parameter name ("@carId") instead of the C# string interpolation construct ("{id}").

string sql =
@"SELECT i.Id, i.Color, i.PetName,m.Name as Make FROM Inventory i
INNER JOIN Makes m on m.Id = i.MakeId WHERE i.Id = @CarId";

The final update is to add the new parameter to the Parameters collection of the command object.

command.Parameters.Add(param);

Update the DeleteCar Method
Likewise, the original implementation of the DeleteCar() method used C# string interpolation. To update this method, create an instance of SqlParameter with the appropriate values, as follows:

SqlParameter param = new SqlParameter
{
ParameterName = "@carId", Value = id,
SqlDbType = SqlDbType.Int,
Direction = ParameterDirection.Input
};

Next, update the SQL string to use the parameter name ("@carId").

string sql = "Delete from Inventory where Id = @carId";

The final update is to add the new parameter to the Parameters collection of the command object.

command.Parameters.Add(param);

Update the UpdateCarPetName Method
This method requires two parameters, one for the car Id and the other for the new PetName. The first parameter is created just like the two previous examples (with the exception of a different variable name), and the second creates a parameter that maps to the database NVarChar type (the PetName field type from the Inventory table). Notice that a Size value is set. It is important this size matches your database field size so as to not create problems when executing the command.

SqlParameter paramId = new SqlParameter
{
ParameterName = "@carId", Value = id,
SqlDbType = SqlDbType.Int,
Direction = ParameterDirection.Input
};
SqlParameter paramName = new SqlParameter
{
ParameterName = "@petName", Value = newPetName,
SqlDbType = SqlDbType.NVarChar, Size = 50,
Direction = ParameterDirection.Input
};

Next, update the SQL string to use the parameters.

string sql = $"Update Inventory Set PetName = @petName Where Id = @carId";

The final update is to add the new parameters to the Parameters collection of the command object.

command.Parameters.Add(paramId); command.Parameters.Add(paramName);

Update the InsertAuto Method
Add the following version of the InsertAuto() method to leverage parameter objects:

public void InsertAuto(Car car)
{
OpenConnection();
// Note the "placeholders" in the SQL query. string sql = "Insert Into Inventory" +
"(MakeId, Color, PetName) Values" + "(@MakeId, @Color, @PetName)";

// This command will have internal parameters.
using (SqlCommand command = new SqlCommand(sql, _sqlConnection))
{
// Fill params collection.
SqlParameter parameter = new SqlParameter
{
ParameterName = "@MakeId", Value = car.MakeId, SqlDbType = SqlDbType.Int,
Direction = ParameterDirection.Input
};
command.Parameters.Add(parameter); parameter = new SqlParameter

{
ParameterName = "@Color", Value = car.Color,
SqlDbType = SqlDbType. NVarChar, Size = 50,
Direction = ParameterDirection.Input
};
command.Parameters.Add(parameter);

parameter = new SqlParameter
{
ParameterName = "@PetName", Value = car.PetName,
SqlDbType = SqlDbType. NVarChar, Size = 50,
Direction = ParameterDirection.Input
};
command.Parameters.Add(parameter);

command.ExecuteNonQuery(); CloseConnection();
}
}

While building a parameterized query often requires more code, the end result is a more convenient way to tweak SQL statements programmatically, as well as to achieve better overall performance. They also are extremely helpful when you want to trigger a stored procedure.

Executing a Stored Procedure
Recall that a stored procedure is a named block of SQL code stored in the database. You can construct stored procedures so they return a set of rows or scalar data types or do anything else that makes sense (e.g., insert, update, or delete records); you can also have them take any number of optional parameters. The end result is a unit of work that behaves like a typical method, except that it is located on a data store rather than a binary business object. Currently, your AutoLot database contains a single stored procedure named GetPetName.
Now consider the following final method (for now) of the InventoryDal type, which invokes your stored
procedure:

public string LookUpPetName(int carId)
{
OpenConnection(); string carPetName;

// Establish name of stored proc.
using (SqlCommand command = new SqlCommand("GetPetName", _sqlConnection))
{
command.CommandType = CommandType.StoredProcedure;

// Input param.
SqlParameter param = new SqlParameter
{

ParameterName = "@carId", SqlDbType = SqlDbType.Int, Value = carId,
Direction = ParameterDirection.Input
};
command.Parameters.Add(param);

// Output param.
param = new SqlParameter
{
ParameterName = "@petName", SqlDbType = SqlDbType.NVarChar, Size = 50,
Direction = ParameterDirection.Output
};
command.Parameters.Add(param);

// Execute the stored proc. command.ExecuteNonQuery();

// Return output param.
carPetName = (string)command.Parameters["@petName"].Value; CloseConnection();
}
return carPetName;
}

One important aspect of invoking a stored procedure is to keep in mind that a command object can represent a SQL statement (the default) or the name of a stored procedure. When you want to inform a command object that it will be invoking a stored procedure, you pass in the name of the procedure (as a constructor argument or by using the CommandText property) and must set the CommandType property to the value CommandType.StoredProcedure. (If you fail to do this, you will receive a runtime exception because the command object is expecting a SQL statement by default.)
Next, notice that the Direction property of the @petName parameter is set to ParameterDirection.
Output. As before, you add each parameter object to the command object’s parameters collection.
After the stored procedure completes with a call to ExecuteNonQuery(), you can obtain the value of the output parameter by investigating the command object’s parameter collection and casting accordingly.

// Return output param.
carPetName = (string)command.Parameters["@petName"].Value;

At this point, you have an extremely simple data access library that you can use to build a client to display and edit your data. You have not yet examined how to build graphical user interfaces, so next you will test your data library from a new console application.

Creating a Console-Based Client Application
Add a new console application (named AutoLot.Client) to the AutoLot.Dal solution and add a reference to the AutoLot.Dal project. Clear out the generated code in the Program.cs file and add the following using statements to the top of the file:

using AutoLot.Dal.Models;
using AutoLot.Dal.DataOperations; using AutoLot.Dal.BulkImport;

Next, add the following top-level statements to exercise the AutoLot.Dal code:

InventoryDal dal = new InventoryDal(); List list = dal.GetAllInventory();
Console.WriteLine(" All Cars "); Console.WriteLine("Id\tMake\tColor\tPet Name");
foreach (var itm in list)
{
Console.WriteLine($"{itm.Id}\t{itm.Make}\t{itm.Color}\t{itm.PetName}");
}
Console.WriteLine();
CarViewModel car = dal.GetCar(list.OrderBy(x=>x.Color).Select(x => x.Id).First()); Console.WriteLine(" First Car By Color "); Console.WriteLine("CarId\tMake\tColor\tPet Name"); Console.WriteLine($"{car.Id}\t{car.Make}\t{car.Color}\t{car.PetName}");

try
{
//This will fail because of related data in the Orders table dal.DeleteCar(5);
Console.WriteLine("Car deleted.");
}
catch (Exception ex)
{
Console.WriteLine($"An exception occurred: {ex.Message}");
}
dal.InsertAuto(new Car { Color = "Blue", MakeId = 5, PetName = "TowMonster" }); list = dal.GetAllInventory();
var newCar = list.First(x => x.PetName == "TowMonster"); Console.WriteLine(" New Car "); Console.WriteLine("CarId\tMake\tColor\tPet Name");
Console.WriteLine($"{newCar.Id}\t{newCar.Make}\t{newCar.Color}\t{newCar.PetName}"); dal.DeleteCar(newCar.Id);
var petName = dal.LookUpPetName(car.Id);
Console.WriteLine(" New Car "); Console.WriteLine($"Car pet name: {petName}"); Console.Write("Press enter to continue..."); Console.ReadLine();

Understanding Database Transactions
The next tool that we will examine is the use of database transactions. Simply put, a transaction is a set of database operations that succeed or fail as a collective unit. If one of the operations fails, all other operations are rolled back, as if nothing ever happened. As you might imagine, transactions are quite important to ensure that table data is safe, valid, and consistent.

Transactions are important when a database operation involves interacting with multiple tables or multiple stored procedures (or a combination of database atoms). The classic transaction example involves the process of transferring monetary funds between two bank accounts. For example, if you were to transfer $500 from your savings account into your checking account, the following steps should occur in a transactional manner:
1.The bank should remove $500 from your savings account.
2.The bank should add $500 to your checking account.
It would be an extremely bad thing if the money were removed from the savings account but not transferred to the checking account (because of some error on the bank’s part) because then you would be out $500! However, if these steps are wrapped up into a database transaction, the DBMS ensures that all related steps occur as a single unit. If any part of the transaction fails, the entire operation is rolled back to the original state. On the other hand, if all steps succeed, the transaction is committed.

■ Note You might be familiar with the acronym ACID from looking at transactional literature. This represents the four key properties of a prim-and-proper transaction: atomic (all or nothing), consistent (data remains stable throughout the transaction), isolated (transactions do not interfere with other operations), and durable (transactions are saved and logged).

It turns out that the .NET platform supports transactions in a variety of ways. This chapter will look at the transaction object of your ADO.NET data provider (SqlTransaction, in the case of Microsoft.Data. SqlClient).
In addition to the baked-in transactional support within the .NET base class libraries, it is possible to use the SQL language of your database management system. For example, you could author a stored procedure that uses the BEGIN TRANSACTION, ROLLBACK, and COMMIT statements.

Key Members of an ADO.NET Transaction Object
All the transactions we will use implement the IDbTransaction interface. Recall from the beginning of this chapter that IDbTransaction defines a handful of members as follows:

public interface IDbTransaction : IDisposable
{
IDbConnection Connection { get; } IsolationLevel IsolationLevel { get; }

void Commit(); void Rollback();
}

Notice the Connection property, which returns a reference to the connection object that initiated the current transaction (as you will see, you obtain a transaction object from a given connection object). You call the Commit() method when each of your database operations has succeeded. Doing this causes each of the pending changes to be persisted in the data store. Conversely, you can call the Rollback() method in the event of a runtime exception, which informs the DBMS to disregard any pending changes, leaving the original data intact.

■ Note The IsolationLevel property of a transaction object allows you to specify how aggressively a transaction should be guarded against the activities of other parallel transactions. By default, transactions are isolated completely until committed.

Beyond the members defined by the IDbTransaction interface, the SqlTransaction type defines an additional member named Save(), which allows you to define save points. This concept allows you to roll back a failed transaction up until a named point, rather than rolling back the entire transaction. Essentially, when you call Save() using a SqlTransaction object, you can specify a friendly string moniker. When you call Rollback(), you can specify this same moniker as an argument to perform an effective partial rollback. Calling Rollback() with no arguments causes all the pending changes to be rolled back.

Adding a Transaction Method to InventoryDal
Now let’s look at how you work with ADO.NET transactions programmatically. Begin by opening the AutoLot. Dal code library project you created earlier and add a new public method named ProcessCreditRisk() to the InventoryDal class to deal with perceived credit risks. The method will look up a customer, add them to the CreditRisks table, and then update their last name by adding “(Credit Risk)” to the end.

public void ProcessCreditRisk(bool throwEx, int customerId)
{
OpenConnection();
// First, look up current name based on customer ID. string fName;
string lName;
var cmdSelect = new SqlCommand(
"Select * from Customers where Id = @customerId",
_sqlConnection);
SqlParameter paramId = new SqlParameter
{
ParameterName = "@customerId", SqlDbType = SqlDbType.Int, Value = customerId,
Direction = ParameterDirection.Input
};
cmdSelect.Parameters.Add(paramId);
using (var dataReader = cmdSelect.ExecuteReader())
{
if (dataReader.HasRows)
{
dataReader.Read();
fName = (string) dataReader["FirstName"]; lName = (string) dataReader["LastName"];
}
else
{
CloseConnection(); return;
}
}

cmdSelect.Parameters.Clear();
// Create command objects that represent each step of the operation. var cmdUpdate = new SqlCommand(
"Update Customers set LastName = LastName + ' (CreditRisk) ' where Id = @customerId",
_sqlConnection); cmdUpdate.Parameters.Add(paramId); var cmdInsert = new SqlCommand(
"Insert Into CreditRisks (CustomerId,FirstName, LastName) Values( @CustomerId, @ FirstName, @LastName)", _sqlConnection);
SqlParameter parameterId2 = new SqlParameter
{
ParameterName = "@CustomerId", SqlDbType = SqlDbType.Int, Value = customerId,
Direction = ParameterDirection.Input
};
SqlParameter parameterFirstName = new SqlParameter
{
ParameterName = "@FirstName", Value = fName,
SqlDbType = SqlDbType.NVarChar, Size = 50,
Direction = ParameterDirection.Input
};
SqlParameter parameterLastName = new SqlParameter
{
ParameterName = "@LastName", Value = lName,
SqlDbType = SqlDbType.NVarChar, Size = 50,
Direction = ParameterDirection.Input
};

cmdInsert.Parameters.Add(parameterId2); cmdInsert.Parameters.Add(parameterFirstName); cmdInsert.Parameters.Add(parameterLastName);
// We will get this from the connection object. SqlTransaction tx = null;
try
{
tx = _sqlConnection.BeginTransaction();
// Enlist the commands into this transaction. cmdInsert.Transaction = tx; cmdUpdate.Transaction = tx;
// Execute the commands. cmdInsert.ExecuteNonQuery(); cmdUpdate.ExecuteNonQuery();
// Simulate error. if (throwEx)
{
throw new Exception("Sorry! Database error! Tx failed...");

}
// Commit it! tx.Commit();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
// Any error will roll back transaction. Using the new conditional access operator to check for null.
tx?.Rollback();
}
finally
{
CloseConnection();
}
}

Here, you use an incoming bool parameter to represent whether you will throw an arbitrary exception when you attempt to process the offending customer. This allows you to simulate an unforeseen
circumstance that will cause the database transaction to fail. Obviously, you do this here only for illustrative purposes; a true database transaction method would not want to allow the caller to force the logic to fail
on a whim!
Note that you use two SqlCommand objects to represent each step in the transaction you will kick off. After you obtain the customer’s first and last names based on the incoming customerID parameter, you can obtain a valid SqlTransaction object from the connection object using BeginTransaction(). Next, and most importantly, you must enlist each command object by assigning the Transaction property to the transaction object you have just obtained. If you fail to do so, the Insert/Update logic will not be under a transactional context.
After you call ExecuteNonQuery() on each command, you throw an exception if (and only if ) the value of the bool parameter is true. In this case, all pending database operations are rolled back. If you do not throw an exception, both steps will be committed to the database tables once you call Commit().

Testing Your Database Transaction
Select one of the customers you added to the Customers table (e.g., Dave Benner, Id = 1). Next, add a new method to the Program.cs file in the AutoLot.Client project named FlagCustomer().

void FlagCustomer()
{
Console.WriteLine(" Simple Transaction Example \n");

// A simple way to allow the tx to succeed or not. bool throwEx = true;
Console.Write("Do you want to throw an exception (Y or N): "); var userAnswer = Console.ReadLine();
if (string.IsNullOrEmpty(userAnswer) || userAnswer.Equals("N",StringComparison. OrdinalIgnoreCase))
{
throwEx = false;
}
var dal = new InventoryDal();

// Process customer 1 – enter the id for the customer to move. dal.ProcessCreditRisk(throwEx, 1);
Console.WriteLine("Check CreditRisk table for results"); Console.ReadLine();
}

If you were to run your program and elect to throw an exception, you would find that the customer’s last name is not changed in the Customers table because the entire transaction has been rolled back. However,
if you did not throw an exception, you would find that the customer’s last name is updated in the Customers
table and has been added to the CreditRisks table.

Executing Bulk Copies with ADO.NET
In cases where you need to load lots of records into the database, the methods shown so far would be rather inefficient. SQL Server has a feature called bulk copy that is designed specifically for this scenario, and it is wrapped up in ADO.NET with the SqlBulkCopy class. This section of the chapter shows how to do this with ADO.NET.

Exploring the SqlBulkCopy Class
The SqlBulkCopy class has one method, WriteToServer() (and the async version WriteToServerAsync()), that processes a list of records and writes the data to the database more efficiently than writing a series of insert statements and running them with a Command object. The WriteToServer overloads take a DataTable, a DataReader, or an array of DataRows. To keep with the theme of this chapter, you are going to use the DataReader version. For that, you need to create a custom data reader.

Creating a Custom Data Reader
You want your custom data reader to be generic and hold a list of the models that you want to import. Begin by creating a new folder in the AutoLot.Dal project named BulkImport; in the folder, create a new interface class named IMyDataReader.cs that implements IDataReader, and update the code to the following:

namespace AutoLot.Dal.BulkImport;
public interface IMyDataReader : IDataReader
{
List Records { get; set; }
}

Next comes the task of implementing the custom data reader. As you have already seen, data readers have lots of moving parts. The good news for you is that, for SqlBulkCopy, you must implement only a handful of them. Create a new class named MyDataReader.cs, update the class to public and sealed, and implement IMyDataReader. Add a constructor to take in the records and set the property.

public sealed class MyDataReader : IMyDataReader
{
public List Records { get; set; } public MyDataReader(List records)
{
Records = records;

}
}

Have Visual Studio or Visual Studio Code implement all the methods for you (or copy them from the following code listings that follow Table 20-7), and you will have your starting point for the custom data reader. Table 20-7 details the only methods that need to be implemented for this scenario.

Table 20-7. Key Methods of IDataReader for SqlBulkCopy

Method Meaning in Life
Read Gets the next record; returns true if there is another record or returns false if at the end of the list
FieldCount Gets the total number of fields in the data source
GetValue Gets the value of a field based on the ordinal position
GetSchemaTable Gets the schema information for the target table

Starting with the Read() method, return false if the reader is at the end of the list, and return true (and increment a class-level counter) if the reader is not at the end of the list. Add a class-level variable to hold the current index of the List and update the Read() method like this:

public class MyDataReader : IMyDataReader
{
...
private int _currentIndex = -1; public bool Read()
{
if (_currentIndex + 1 >= Records.Count)
{
return false;
}
_currentIndex++; return true;
}
}

Each of the get methods and the FieldCount methods requires an intimate knowledge of the specific model to be loaded. An example of the GetValue() method (using the Car class) is as follows:

public object GetValue(int i)
{
Car currentRecord = Records[_currentIndex] as Car; return i switch
{
0 => currentRecord.Id,
1 => currentRecord.MakeId,
2 => currentRecord.Color,
3 => currentRecord.PetName,
4 => currentRecord.TimeStamp,

_ => string.Empty,
};
}

The database has only four tables, but that means you still have four variations of the data reader. Imagine if you had a real production database with many more tables! You can do better than this using reflection (covered in Chapter 17) and LINQ to Objects (covered in Chapter 13).
Add readonly variables to hold the PropertyInfo values for the model as well as a dictionary that will be used to hold the field position and name for the table in SQL Server. Update the constructor to get the properties of the generic type and initialize the Dictionary. The added code is as follows:

private readonly PropertyInfo[] _propertyInfos;
private readonly Dictionary<int, string> _nameDictionary;

public MyDataReader(List records)
{
Records = records;
_propertyInfos = typeof(T).GetProperties();
_nameDictionary = new Dictionary<int,string>();
}

Next, update the constructor to take a SQLConnection as well as strings for the schema and table names for the table the records are going to be inserted into and add class-level variables for the values.

private readonly SqlConnection _connection; private readonly string _schema;
private readonly string _tableName;
public MyDataReader(List records, SqlConnection connection, string schema, string tableName)
{
Records = records;
_propertyInfos = typeof(T).GetProperties();
_nameDictionary = new Dictionary<int, string>();

_connection = connection;
_schema = schema;
_tableName = tableName;
}

Implement the GetSchemaTable() method next. This retrieves the SQL Server information regarding the target table.

public DataTable GetSchemaTable()
{
using var schemaCommand = new SqlCommand($"SELECT * FROM {_schema}.{tableName}", connection);
using var reader = schemaCommand.ExecuteReader(CommandBehavior.SchemaOnly); return reader.GetSchemaTable();
}

Update the constructor to use the SchemaTable to construct the dictionary that contains the fields of the target table in database order.

public MyDataReader(List records, SqlConnection connection, string schema, string tableName)
{
...
DataTable schemaTable = GetSchemaTable();
for (int x = 0; x<schemaTable?.Rows.Count;x++)
{
DataRow col = schemaTable.Rows[x];
var columnName = col.Field("ColumnName");
_nameDictionary.Add(x,columnName);
}
}

Now, the following methods can be implemented generically, using the reflected information:

public int FieldCount => _propertyInfos.Length; public object GetValue(int i)
=> _propertyInfos
.First(x=>x.Name.Equals(_nameDictionary[i],StringComparison.OrdinalIgnoreCase))
.GetValue(Records[_currentIndex]);

The remainder of the methods that must be present (but not implemented) are listed here for reference:

public string GetName(int i) => throw new NotImplementedException(); public int GetOrdinal(string name) => throw new NotImplementedException();
public string GetDataTypeName(int i) => throw new NotImplementedException(); public Type GetFieldType(int i) => throw new NotImplementedException(); public int GetValues(object[] values) => throw new NotImplementedException(); public bool GetBoolean(int i) => throw new NotImplementedException();
public byte GetByte(int i) => throw new NotImplementedException();
public long GetBytes(int i, long fieldOffset, byte[] buffer, int bufferoffset, int length)
=> throw new NotImplementedException();
public char GetChar(int i) => throw new NotImplementedException();
public long GetChars(int i, long fieldoffset, char[] buffer, int bufferoffset, int length)
=> throw new NotImplementedException();
public Guid GetGuid(int i) => throw new NotImplementedException(); public short GetInt16(int i) => throw new NotImplementedException(); public int GetInt32(int i) => throw new NotImplementedException(); public long GetInt64(int i) => throw new NotImplementedException(); public float GetFloat(int i) => throw new NotImplementedException(); public double GetDouble(int i) => throw new NotImplementedException(); public string GetString(int i) => throw new NotImplementedException(); public decimal GetDecimal(int i) => throw new NotImplementedException();
public DateTime GetDateTime(int i) => throw new NotImplementedException(); public IDataReader GetData(int i) => throw new NotImplementedException(); public bool IsDBNull(int i) => throw new NotImplementedException();
object IDataRecord.this[int i] => throw new NotImplementedException(); object IDataRecord.this[string name] => throw new NotImplementedException();

public void Close() => throw new NotImplementedException();
public DataTable GetSchemaTable() => throw new NotImplementedException(); public bool NextResult() => throw new NotImplementedException();
public int Depth { get; } public bool IsClosed { get; }
public int RecordsAffected { get; }

Executing the Bulk Copy
Add a new public static class named ProcessBulkImport.cs to the BulkImport folder. Add the code to handle opening and closing connections (like the code in the InventoryDal class), as follows:

private const string ConnectionString =
@"Data Source=.,5433;User Id=sa;Password=P@ssw0rd;Initial Catalog=AutoLot;Encrypt=False;"; private static SqlConnection _sqlConnection = null;

private static void OpenConnection()
{
_sqlConnection = new SqlConnection
{
ConnectionString = ConnectionString
};
_sqlConnection.Open();
}

private static void CloseConnection()
{
if (_sqlConnection?.State != ConnectionState.Closed)
{
_sqlConnection?.Close();
}
}

The SqlBulkCopy class requires the name (and schema, if different than dbo) to process the records. After creating a new SqlBulkCopy instance (passing in the connection object), set the
DestinationTableName property. Then, create a new instance of the custom data reader holding the list to be bulk copied, and call WriteToServer(). The ExecuteBulkImport method is shown here:

public static void ExecuteBulkImport(IEnumerable records, string tableName)
{
OpenConnection();
using SqlConnection conn = _sqlConnection; SqlBulkCopy bc = new SqlBulkCopy(conn)
{
DestinationTableName = tableName
};
var dataReader = new MyDataReader(records.ToList(),_sqlConnection, "dbo",tableName); try
{
bc.WriteToServer(dataReader);
}

catch (Exception ex)
{
//Should do something here
}
finally
{
CloseConnection();
}
}

Testing the Bulk Copy
Back in the AutoLot.Client project, add a new method to the Program.cs file named DoBulkCopy(). Create a list of Car objects and pass that (and the name of the table) into the ExecuteBulkImport() method. The rest of the code displays the results of the bulk copy.

void DoBulkCopy()
{
Console.WriteLine(" Do Bulk Copy "); var cars = new List
{
new Car() {Color = "Blue", MakeId = 1, PetName = "MyCar1"}, new Car() {Color = "Red", MakeId = 2, PetName = "MyCar2"}, new Car() {Color = "White", MakeId = 3, PetName = "MyCar3"}, new Car() {Color = "Yellow", MakeId = 4, PetName = "MyCar4"}
};
ProcessBulkImport.ExecuteBulkImport(cars, "Inventory"); InventoryDal dal = new InventoryDal(); List list = dal.GetAllInventory();
Console.WriteLine(" All Cars "); Console.WriteLine("CarId\tMake\tColor\tPet Name");
foreach (var itm in list)
{
Console.WriteLine(
$"{itm.Id}\t{itm.Make}\t{itm.Color}\t{itm.PetName}");
}
Console.WriteLine();
}

While adding four new cars does not show the merits of the work involved in using the SqlBulkCopy class, imagine trying to load thousands of records. I have done this with customers, and the load time has been mere seconds, where looping through each record took hours! As with everything in .NET, this is just another tool to keep in your toolbox to use when it makes the most sense.

Summary
ADO.NET is the native data access technology of the .NET platform. In this chapter, you began by learning the role of data providers, which are essentially concrete implementations of several abstract base classes (in the System.Data.Common namespace) and interface types (in the System.Data namespace). You also saw that it is possible to build a provider-neutral code base using the ADO.NET data provider factory model.
You also learned that you use connection objects, transaction objects, command objects, and data reader objects to select, update, insert, and delete records. Also, recall that command objects support an internal parameter collection, which you can use to add some type safety to your SQL queries; these also prove quite helpful when triggering stored procedures.
Next, you learned how to safeguard your data manipulation code with transactions and wrapped up the chapter with a look at using the SqlBulkCopy class to load large amounts of data into SQL Server using ADO.NET.

Pro C#10 CHAPTER 19 File I/O and Object Serialization

PART VI

File Handling, Object Serialization, and Data Access

CHAPTER 19

File I/O and Object Serialization

When you create desktop applications, the ability to save information between user sessions is commonplace. This chapter examines several I/O-related topics as seen through the eyes of the .NET Framework. The first order of business is to explore the core types defined in the System.IO namespace and learn how to modify a machine’s directory and file structure programmatically. The next task is to explore various ways to read from and write to character-based, binary-based, string-based, and memory-based data stores.
After you learn how to manipulate files and directories using the core I/O types, you will examine the related topic of object serialization. You can use object serialization to persist and retrieve the state of an object to (or from) any System.IO.Stream-derived type.

■ Note To ensure you can run each of the examples in this chapter, start Visual Studio with administrative rights (just right-click the Visual Studio icon and select Run as Administrator). If you do not do so, you may encounter runtime security exceptions when accessing the computer file system.

Exploring the System.IO Namespace
In the framework of .NET Core, the System.IO namespace is the region of the base class libraries devoted to file-based (and memory-based) input and output (I/O) services. Like any namespace, System.IO defines a set of classes, interfaces, enumerations, structures, and delegates, most of which you can find in mscorlib. dll. In addition to the types contained within mscorlib.dll, the System.dll assembly defines additional members of the System.IO namespace.
Many of the types within the System.IO namespace focus on the programmatic manipulation of physical directories and files. However, additional types provide support to read data from and write data to string buffers, as well as raw memory locations. Table 19-1 outlines the core (nonabstract) classes, providing a road map of the functionality in System.IO.

© Andrew Troelsen, Phil Japikse 2022
A. Troelsen and P. Japikse, Pro C# 10 with .NET 6, https://doi.org/10.1007/978-1-4842-7869-7_19

751

Nonabstract I/O
Class Type Meaning in Life
BinaryReader BinaryWriter These classes allow you to store and retrieve primitive data types (integers, Booleans, strings, and whatnot) as a binary value.
BufferedStream This class provides temporary storage for a stream of bytes that you can commit to storage later.
Directory DirectoryInfo You use these classes to manipulate a machine’s directory structure. The Directory type exposes functionality using static members, while the DirectoryInfo type exposes similar functionality from a valid object reference.
DriveInfo This class provides detailed information regarding the drives that a given machine uses.
File FileInfo You use these classes to manipulate a machine’s set of files. The File type exposes functionality using static members, while the FileInfo type exposes similar functionality from a valid object reference.
FileStream This class gives you random file access (e.g., seeking capabilities) with data represented as a stream of bytes.
FileSystemWatcher This class allows you to monitor the modification of external files in a specified directory.
MemoryStream This class provides random access to streamed data stored in memory rather than in a physical file.
Path This class performs operations on System.String types that contain file or directory path information in a platform-neutral manner.
StreamWriter StreamReader You use these classes to store (and retrieve) textual information to (or from) a file. These types do not support random file access.
StringWriter StringReader Like the StreamReader/StreamWriter classes, these classes also work with textual information. However, the underlying storage is a string buffer rather than a physical file.

In addition to these concrete class types, System.IO defines several enumerations, as well as a set of abstract classes (e.g., Stream, TextReader, and TextWriter), that define a shared polymorphic interface to all descendants. You will read about many of these types in this chapter.

The Directory(Info) and File(Info) Types
System.IO provides four classes that allow you to manipulate individual files, as well as interact with a machine’s directory structure. The first two types, Directory and File, expose creation, deletion, copying, and moving operations using various static members. The closely related FileInfo and DirectoryInfo types expose similar functionality as instance-level methods (therefore, you must allocate them with the new keyword). The Directory and File classes directly extend System.Object, while DirectoryInfo and FileInfo derive from the abstract FileSystemInfo type.

FileInfo and DirectoryInfo typically serve as better choices for obtaining full details of a file or directory (e.g., time created or read/write capabilities) because their members tend to return strongly typed objects. In contrast, the Directory and File class members tend to return simple string values rather than strongly typed objects. This is only a guideline, however; in many cases, you can get the same work done using File/FileInfo or Directory/DirectoryInfo.

The Abstract FileSystemInfo Base Class
The DirectoryInfo and FileInfo types receive many behaviors from the abstract FileSystemInfo
base class. For the most part, you use the members of the FileSystemInfo class to discover general characteristics (such as time of creation, various attributes, etc.) about a given file or directory. Table 19-2 lists some core properties of interest.

Table 19-2. FileSystemInfo Properties

Property Meaning in Life
Attributes Gets or sets the attributes associated with the current file that are represented by the FileAttributes enumeration (e.g., is the file or directory read-only, encrypted, hidden, or compressed?)
CreationTime Gets or sets the time of creation for the current file or directory
Exists Determines whether a given file or directory exists
Extension Retrieves a file’s extension
FullName Gets the full path of the directory or file
LastAccessTime Gets or sets the time the current file or directory was last accessed
LastWriteTime Gets or sets the time when the current file or directory was last written to
Name Obtains the name of the current file or directory

FileSystemInfo also defines the Delete() method. This is implemented by derived types to delete a given file or directory from the hard drive. Also, you can call Refresh() prior to obtaining attribute information to ensure that the statistics regarding the current file (or directory) are not outdated.

Working with the DirectoryInfo Type
The first creatable I/O-centric type you will examine is the DirectoryInfo class. This class contains a set of members used for creating, moving, deleting, and enumerating over directories and subdirectories. In addition to the functionality provided by its base class (FileSystemInfo), DirectoryInfo offers the key members detailed in Table 19-3.

Table 19-3. Key Members of the DirectoryInfo Type

Member Meaning in Life
Create() CreateSubdirectory() Creates a directory (or set of subdirectories) when given a path name
Delete() Deletes a directory and all its contents
GetDirectories() Returns an array of DirectoryInfo objects that represent all subdirectories in the current directory
GetFiles() Retrieves an array of FileInfo objects that represent a set of files in the given directory
MoveTo() Moves a directory and its contents to a new path
Parent Retrieves the parent directory of this directory
Root Gets the root portion of a path

You begin working with the DirectoryInfo type by specifying a particular directory path as a constructor parameter. Use the dot (.) notation if you want to obtain access to the current working directory (the directory of the executing application). Here are some examples:

// Bind to the current working directory.
DirectoryInfo dir1 = new DirectoryInfo(".");
// Bind to C:\Windows,
// using a verbatim string.
DirectoryInfo dir2 = new DirectoryInfo(@"C:\Windows");

In the second example, you assume that the path passed into the constructor (C:\Windows) already exists on the physical machine. However, if you attempt to interact with a nonexistent directory, a System.IO
.DirectoryNotFoundException is thrown. Thus, if you specify a directory that is not yet created, you need to call the Create() method before proceeding, like so:

// Bind to a nonexistent directory, then create it. DirectoryInfo dir3 = new DirectoryInfo(@"C:\MyCode\Testing"); dir3.Create();

The path syntax used in the previous example is Windows-centric. If you are developing .NET applications for different platforms, you should use the Path.VolumeSeparatorChar and Path.
DirectorySeparatorChar constructs, which will yield the appropriate characters based on the platform.
Update the previous code to the following:

DirectoryInfo dir3 = new DirectoryInfo(
$@"C{Path.VolumeSeparatorChar}{Path.DirectorySeparatorChar}MyCode{Path.DirectorySeparator Char}Testing");

After you create a DirectoryInfo object, you can investigate the underlying directory contents using any of the properties inherited from FileSystemInfo. To see this in action, create a new Console Application project named DirectoryApp and update your C# file to import System and System.IO. Update your Program.cs file with the following new static method that creates a new DirectoryInfo object mapped to C:\Windows (adjust your path if need be), which displays several interesting statistics:

Console.WriteLine(" Fun with Directory(Info) \n"); ShowWindowsDirectoryInfo();
Console.ReadLine();

static void ShowWindowsDirectoryInfo()
{
// Dump directory information. If you are not on Windows, plug in another directory
DirectoryInfo dir = new DirectoryInfo($@"C{Path.VolumeSeparatorChar}{Path. DirectorySeparatorChar}Windows");
Console.WriteLine(" Directory Info "); Console.WriteLine("FullName: {0}", dir.FullName);
Console.WriteLine("Name: {0}", dir.Name);
Console.WriteLine("Parent: {0}", dir.Parent);
Console.WriteLine("Creation: {0}", dir.CreationTime);
Console.WriteLine("Attributes: {0}", dir.Attributes);
Console.WriteLine("Root: {0}", dir.Root); Console.WriteLine("**\n");
}

While your output might differ, you should see something like the following:

Fun with Directory(Info)
Directory Info FullName: C:\Windows
Name: Windows Parent:
Creation: 3/19/2019 00:37:22 Attributes: Directory
Root: C:\


■Note If you are not on a Windows machine, the output from these samples will show a different directory separator.

Enumerating Files with the DirectoryInfo Type
In addition to obtaining basic details of an existing directory, you can extend the current example to use some methods of the DirectoryInfo type. First, you can leverage the GetFiles() method to obtain
information about all *.jpg files located in the C:\Windows\Web\Wallpaper directory (update this directory
if necessary to one that has images on your machine).

The GetFiles() method returns an array of FileInfo objects, each of which exposes details of a particular file (you will learn the full details of the FileInfo type later in this chapter). Create the following static method in the Program.cs file:

static void DisplayImageFiles()
{
DirectoryInfo dir = new DirectoryInfo($@"C{Path.VolumeSeparatorChar}{Path.DirectorySeparatorChar}Windows{Path. DirectorySeparatorChar}Web{Path.DirectorySeparatorChar}Wallpaper");
// Get all files with a .jpg extension.
FileInfo[] imageFiles =
dir.GetFiles("
.jpg", SearchOption.AllDirectories);

// How many were found?
Console.WriteLine("Found {0} *.jpg files\n", imageFiles.Length);

// Now print out info for each file. foreach (FileInfo f in imageFiles)
{
Console.WriteLine(""); Console.WriteLine("File name: {0}", f.Name); Console.WriteLine("File size: {0}", f.Length); Console.WriteLine("Creation: {0}", f.CreationTime);
Console.WriteLine("Attributes: {0}", f.Attributes); Console.WriteLine("
\n");
}
}

Notice that you specify a search option when you call GetFiles(); you do this to look within all subdirectories of the root. After you run the application, you will see a listing of all files that match the search pattern.

Creating Subdirectories with the DirectoryInfo Type
You can programmatically extend a directory structure using the DirectoryInfo.CreateSubdirectory() method. This method can create a single subdirectory, as well as multiple nested subdirectories, in a single function call. This method illustrates how to do so, extending the directory structure of the application execution directory (denoted with the .) with some custom subdirectories:

static void ModifyAppDirectory()
{
DirectoryInfo dir = new DirectoryInfo(".");

// Create \MyFolder off application directory.
dir.CreateSubdirectory("MyFolder");

// Create \MyFolder2\Data off application directory.
dir.CreateSubdirectory(
$@"MyFolder2{Path.DirectorySeparatorChar}Data");
}

You are not required to capture the return value of the CreateSubdirectory() method, but you should be aware that a DirectoryInfo object representing the newly created item is passed back on successful execution. Consider the following update to the previous method:

static void ModifyAppDirectory()
{
DirectoryInfo dir = new DirectoryInfo(".");

// Create \MyFolder off initial directory.
dir.CreateSubdirectory("MyFolder");

// Capture returned DirectoryInfo object.
DirectoryInfo myDataFolder = dir.CreateSubdirectory(
$@"MyFolder2{Path.DirectorySeparatorChar}Data");

// Prints path to ..\MyFolder2\Data.
Console.WriteLine("New Folder is: {0}", myDataFolder);
}

If you call this method from the top-level statements and examine your Windows directory using Windows Explorer, you will see that the new subdirectories are present and accounted for.

Working with the Directory Type
You have seen the DirectoryInfo type in action; now you are ready to learn about the Directory type.
For the most part, the static members of Directory mimic the functionality provided by the instance-level members defined by DirectoryInfo. Recall, however, that the members of Directory typically return string data rather than strongly typed FileInfo/DirectoryInfo objects.
Now let’s look at some functionality of the Directory type. This final helper function displays the names of all drives mapped to the current computer (using the Directory.GetLogicalDrives() method) and
uses the static Directory.Delete() method to remove the \MyFolder and \MyFolder2\Data subdirectories created previously.

static void FunWithDirectoryType()
{
// List all drives on current computer. string[] drives = Directory.GetLogicalDrives(); Console.WriteLine("Here are your drives:"); foreach (string s in drives)
{
Console.WriteLine("--> {0} ", s);
}

// Delete what was created.
Console.WriteLine("Press Enter to delete directories"); Console.ReadLine();
try
{
Directory.Delete("MyFolder");

// The second parameter specifies whether you
// wish to destroy any subdirectories.
Directory.Delete("MyFolder2", true);
}
catch (IOException e)
{
Console.WriteLine(e.Message);
}
}

Working with the DriveInfo Class Type
The System.IO namespace provides a class named DriveInfo. Like Directory.GetLogicalDrives(), the static DriveInfo.GetDrives() method allows you to discover the names of a machine’s drives. Unlike Directory.GetLogicalDrives(), however, DriveInfo provides numerous other details (e.g., the drive type, available free space, and volume label). Consider the following Program.cs file defined within a new Console Application project named DriveInfoApp:

// Get info regarding all drives.
DriveInfo[] myDrives = DriveInfo.GetDrives();
// Now print drive stats.
foreach(DriveInfo d in myDrives)
{
Console.WriteLine("Name: {0}", d.Name);
Console.WriteLine("Type: {0}", d.DriveType);

// Check to see whether the drive is mounted.
if(d.IsReady)
{
Console.WriteLine("Free space: {0}", d.TotalFreeSpace); Console.WriteLine("Format: {0}", d.DriveFormat);
Console.WriteLine("Label: {0}", d.VolumeLabel);
}
Console.WriteLine();
}
Console.ReadLine();

Here is some possible output:

Fun with DriveInfo Name: C:\
Type: Fixed
Free space: 284131119104 Format: NTFS
Label: OS

Name: M:\ Type: Network
Free space: 4711871942656 Format: NTFS
Label: DigitalMedia

At this point, you have investigated some core behaviors of the Directory, DirectoryInfo, and DriveInfo classes. Next, you will learn how to create, open, close, and destroy the files that populate a given directory.

Working with the FileInfo Class
As shown in the previous DirectoryApp example, the FileInfo class allows you to obtain details regarding existing files on your hard drive (e.g., time created, size, and file attributes) and aids in the creation, copying, moving, and destruction of files. In addition to the set of functionalities inherited by FileSystemInfo, you can find some core members unique to the FileInfo class, which are described in Table 19-4.

Table 19-4. FileInfo Core Members

Member Meaning in Life
AppendText() Creates a StreamWriter object (described later) that appends text to a file
CopyTo() Copies an existing file to a new file
Create() Creates a new file and returns a FileStream object (described later) to interact with the newly created file
CreateText() Creates a StreamWriter object that writes a new text file
Delete() Deletes the file to which a FileInfo instance is bound
Directory Gets an instance of the parent directory
DirectoryName Gets the full path to the parent directory
Length Gets the size of the current file
MoveTo() Moves a specified file to a new location, providing the option to specify a new filename
Name Gets the name of the file
Open() Opens a file with various read/write and sharing privileges
OpenRead() Creates a read-only FileStream object
OpenText() Creates a StreamReader object (described later) that reads from an existing text file
OpenWrite() Creates a write-only FileStream object

Note that a majority of the methods of the FileInfo class return a specific I/O-centric object (e.g., FileStream and StreamWriter) that allows you to begin reading and writing data to (or reading from) the associated file in a variety of formats. You will check out these types in just a moment; however, before you see a working example, you will find it helpful to examine various ways to obtain a file handle using the FileInfo class type.

The FileInfo.Create() Method
The next set of examples are all in a Console Application named SimpleFileIO. One way you can create a file handle is to use the FileInfo.Create() method, like so:

Console.WriteLine(" Simple IO with the File Type \n");
//Change to a folder on your machine that you have read/write access to, or run as administrator
var fileName = $@"C{Path.VolumeSeparatorChar}{Path.DirectorySeparatorChar}temp{Path. DirectorySeparatorChar}Test.dat";
// Make a new file on the C drive. FileInfo f = new FileInfo(fileName); FileStream fs = f.Create();

// Use the FileStream object...

// Close down file stream.
fs.Close();

■ Note These examples might require running Visual Studio as an Administrator, depending on your user permissions and system configuration.

Notice that the FileInfo.Create() method returns a FileStream object, which exposes synchronous and asynchronous write/read operations to/from the underlying file (more details on that in a moment). Be aware that the FileStream object returned by FileInfo.Create() grants full read/write access to all users.
Also notice that after you finish with the current FileStream object, you must ensure you close the handle to release the underlying unmanaged stream resources. Given that FileStream implements
IDisposable, you can use the C# using scope to allow the compiler to generate the teardown logic (see Chapter 8 for details), like so:

var fileName = $@"C{Path.VolumeSeparatorChar}{Path.DirectorySeparatorChar}Test.dat";
...
//wrap the file stream in a using statement
// Defining a using scope for file I/O FileInfo f1 = new FileInfo(fileName); using (FileStream fs1 = f1.Create())
{
// Use the FileStream object...
}
f1.Delete();

■ Note Almost all of these examples in this chapter include using statements. I could have used the new using declaration syntax but chose not to in this rewrite to keep the examples focused on the System.IO components that we are examining.

The FileInfo.Open() Method
You can use the FileInfo.Open() method to open existing files, as well as to create new files with far more precision than you can with FileInfo.Create(). This works because Open() typically takes several parameters to qualify exactly how to iterate the file you want to manipulate. Once the call to Open() completes, you are returned a FileStream object. Consider the following logic:

var fileName = $@"C{Path.VolumeSeparatorChar}{Path.DirectorySeparatorChar}Test.dat";
...
// Make a new file via FileInfo.Open().
FileInfo f2 = new FileInfo(fileName); using(FileStream fs2 = f2.Open(FileMode.OpenOrCreate,
FileAccess.ReadWrite, FileShare.None))
{
// Use the FileStream object...
}
f2.Delete();

This version of the overloaded Open() method requires three parameters. The first parameter of the Open() method specifies the general flavor of the I/O request (e.g., make a new file, open an existing file, and append to a file), which you specify using the FileMode enumeration (see Table 19-5 for details), like so:

public enum FileMode
{
CreateNew, Create, Open,
OpenOrCreate, Truncate, Append
}

Table 19-5. Members of the FileMode Enumeration

Member Meaning in Life
CreateNew Informs the OS to make a new file. If it already exists, an IOException is thrown.
Create Informs the OS to make a new file. If it already exists, it will be overwritten.
Open Opens an existing file. If the file does not exist, a FileNotFoundException is thrown.
OpenOrCreate Opens the file if it exists; otherwise, a new file is created.
Truncate Opens an existing file and truncates the file to 0 bytes in size.
Append Opens a file, moves to the end of the file, and begins write operations (you can use this flag only with a write-only stream). If the file does not exist, a new file is created.

You use the second parameter of the Open() method, a value from the FileAccess enumeration, to determine the read/write behavior of the underlying stream, as follows:

public enum FileAccess
{
Read, Write, ReadWrite
}

Finally, the third parameter of the Open() method, FileShare, specifies how to share the file among other file handlers. Here are the core names:

public enum FileShare
{
None, Read, Write, ReadWrite, Delete,
Inheritable
}

The FileInfo.OpenRead() and FileInfo.OpenWrite() Methods
The FileInfo.Open() method allows you to obtain a file handle in a flexible manner, but the FileInfo class also provides members named OpenRead() and OpenWrite(). As you might imagine, these methods return a properly configured read-only or write-only FileStream object, without the need to supply various enumeration values. Like FileInfo.Create() and FileInfo.Open(), OpenRead() and OpenWrite() return a FileStream object.
Note that the OpenRead() method requires the file to already exist. The following code creates the file and then closes the FileStream so it can be used by the OpenRead() method:

f3.Create().Close();

Here are the full examples:

var fileName = $@"C{Path.VolumeSeparatorChar}{Path.DirectorySeparatorChar}Test.dat";
...
// Get a FileStream object with read-only permissions.
FileInfo f3 = new FileInfo(fileName);
//File must exist before using OpenRead f3.Create().Close();
using(FileStream readOnlyStream = f3.OpenRead())
{
// Use the FileStream object...
}
f3.Delete();

// Now get a FileStream object with write-only permissions.
FileInfo f4 = new FileInfo(fileName); using(FileStream writeOnlyStream = f4.OpenWrite())
{
// Use the FileStream object...
}
f4.Delete();

The FileInfo.OpenText() Method
Another open-centric member of the FileInfo type is OpenText(). Unlike Create(), Open(), OpenRead(), or OpenWrite(), the OpenText() method returns an instance of the StreamReader type, rather than a
FileStream type. Assuming you have a file named boot.ini on your C: drive, the following snippet gives you access to its contents:

var fileName = $@"C{Path.VolumeSeparatorChar}{Path.DirectorySeparatorChar}Test.dat";
...
// Get a StreamReader object.
//If not on a Windows machine, change the file name accordingly
FileInfo f5 = new FileInfo(fileName);
//File must exist before using OpenText f5.Create().Close();
using(StreamReader sreader = f5.OpenText())
{
// Use the StreamReader object...
}
f5.Delete();

As you will see shortly, the StreamReader type provides a way to read character data from the underlying file.

The FileInfo.CreateText() and FileInfo.AppendText() Methods
The final two FileInfo methods of interest at this point are CreateText() and AppendText(). Both return a
StreamWriter object, as shown here:

var fileName = $@"C{Path.VolumeSeparatorChar}{Path.DirectorySeparatorChar}Test.dat";
...
FileInfo f6 = new FileInfo(fileName); using(StreamWriter swriter = f6.CreateText())
{
// Use the StreamWriter object...
}
f6.Delete();
FileInfo f7 = new FileInfo(fileName); using(StreamWriter swriterAppend = f7.AppendText())
{
// Use the StreamWriter object...
}
f7.Delete();

As you might guess, the StreamWriter type provides a way to write character data to the underlying file.

Working with the File Type
The File type uses several static members to provide functionality almost identical to that of the FileInfo type. Like FileInfo, File supplies AppendText(), Create(), CreateText(), Open(), OpenRead(), OpenWrite(), and OpenText() methods. In many cases, you can use the File and FileInfo types interchangeably. Note that OpenText() and OpenRead() require the file to already exist. To see this in action, you can simplify each of the previous FileStream examples by using the File type instead, like so:

var fileName = $@"C{Path.VolumeSeparatorChar}{Path.DirectorySeparatorChar}Test.dat";
...
//Using File instead of FileInfo
using (FileStream fs8 = File.Create(fileName))
{
// Use the FileStream object...
}
File.Delete(fileName);
// Make a new file via FileInfo.Open().
using(FileStream fs9 = File.Open(fileName, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None))
{
// Use the FileStream object...
}
// Get a FileStream object with read-only permissions.
using(FileStream readOnlyStream = File.OpenRead(fileName))
{}
File.Delete(fileName);
// Get a FileStream object with write-only permissions.
using(FileStream writeOnlyStream = File.OpenWrite(fileName))
{}
// Get a StreamReader object.
using(StreamReader sreader = File.OpenText(fileName))
{}
File.Delete(fileName);
// Get some StreamWriters.
using(StreamWriter swriter = File.CreateText(fileName))
{}
File.Delete(fileName);

using(StreamWriter swriterAppend = File.AppendText(fileName))
{}
File.Delete(fileName);

Additional File-centric Members
The File type also supports a few members, shown in Table 19-6, which can greatly simplify the processes of reading and writing textual data.

Table 19-6. Methods of the File Type

Method Meaning in Life
ReadAllBytes() Opens the specified file, returns the binary data as an array of bytes, and then closes the file
ReadAllLines() Opens a specified file, returns the character data as an array of strings, and then closes the file
ReadAllText() Opens a specified file, returns the character data as a System.String, and then closes the file
WriteAllBytes() Opens the specified file, writes out the byte array, and then closes the file
WriteAllLines() Opens a specified file, writes out an array of strings, and then closes the file
WriteAllText() Opens a specified file, writes the character data from a specified string, and then closes the file

You can use these methods of the File type to read and write batches of data in only a few lines of code. Even better, each of these members automatically closes the underlying file handle. For example, the following code persists the string data into a new file on the C: drive (and reads it into memory) with minimal fuss:

Console.WriteLine(" Simple I/O with the File Type \n"); string[] myTasks = {
"Fix bathroom sink", "Call Dave", "Call Mom and Dad", "Play Xbox One"};

// Write out all data to file on C drive.
File.WriteAllLines(@"tasks.txt", myTasks);

// Read it all back and print out.
foreach (string task in File.ReadAllLines(@"tasks.txt"))
{
Console.WriteLine("TODO: {0}", task);
}
Console.ReadLine(); File.Delete("tasks.txt");

The lesson here is that when you want to obtain a file handle quickly, the File type will save you some keystrokes. However, one benefit of creating a FileInfo object first is that you can investigate the file using the members of the abstract FileSystemInfo base class.

The Abstract Stream Class
At this point, you have seen many ways to obtain FileStream, StreamReader, and StreamWriter objects, but you have yet to read data from or write data to a file using these types. To understand how to do this, you will need to familiarize yourself with the concept of a stream. In the world of I/O manipulation, a stream represents a chunk of data flowing between a source and a destination. Streams provide a common way to interact with a sequence of bytes, regardless of what kind of device (e.g., file, network connection, or printer) stores or displays the bytes in question.

The abstract System.IO.Stream class defines several members that provide support for synchronous and asynchronous interactions with the storage medium (e.g., an underlying file or memory location).

■ Note The concept of a stream is not limited to file I/O. To be sure, the .neT libraries provide stream access to networks, memory locations, and other stream-centric abstractions.

Again, Stream descendants represent data as a raw stream of bytes; therefore, working directly with raw streams can be quite cryptic. Some Stream-derived types support seeking, which refers to the process of obtaining and adjusting the current position in the stream. Table 19-7 helps you understand the functionality provided by the Stream class by describing its core members.

Table 19-7. Abstract Stream Members

Member Meaning in Life
CanRead CanWrite CanSeek Determines whether the current stream supports reading, seeking, and/or writing.
Close() Closes the current stream and releases any resources (such as sockets and file handles) associated with the current stream. Internally, this method is aliased to the Dispose() method; therefore, closing a stream is functionally equivalent to disposing a stream.
Flush() Updates the underlying data source or repository with the current state of the buffer and then clears the buffer. If a stream does not implement a buffer, this method does nothing.
Length Returns the length of the stream in bytes.
Position Determines the position in the current stream.
Read() ReadByte() ReadAsync() Reads a sequence of bytes (or a single byte) from the current stream and advances the current position in the stream by the number of bytes read.
Seek() Sets the position in the current stream.
SetLength() Sets the length of the current stream.
Write() WriteByte() WriteAsync() Writes a sequence of bytes (or a single byte) to the current stream and advances the current position in this stream by the number of bytes written.

Working with FileStreams
The FileStream class provides an implementation for the abstract Stream members in a manner appropriate for file-based streaming. It is a primitive stream; it can read or write only a single byte or an array of bytes. However, you will not often need to interact directly with the members of the FileStream type.
Instead, you will probably use various stream wrappers, which make it easier to work with textual data or
.NET types. Nevertheless, you will find it helpful to experiment with the synchronous read/write capabilities of the FileStream type.

Assume you have a new Console Application project named FileStreamApp (and verify that the System.
Text namespace is imported into your initial C# code file). Your goal is to write a simple text message to a new file named myMessage.dat. However, given that FileStream can operate only on raw bytes, you will be required to encode the System.String type into a corresponding byte array. Fortunately, the System.Text namespace defines a type named Encoding that provides members that encode and decode strings to (or from) an array of bytes.
Once encoded, the byte array is persisted to file with the FileStream.Write() method. To read the bytes back into memory, you must reset the internal position of the stream (using the Position property) and call the ReadByte() method. Finally, you display the raw byte array and the decoded string to the console. Here is the complete code:

using System.Text;

// Don't forget to import the System.Text namespaces.
Console.WriteLine(" Fun with FileStreams \n");

// Obtain a FileStream object.
using(FileStream fStream = File.Open("myMessage.dat", FileMode.Create))
{
// Encode a string as an array of bytes.
string msg = "Hello!";
byte[] msgAsByteArray = Encoding.Default.GetBytes(msg);

// Write byte[] to file.
fStream.Write(msgAsByteArray, 0, msgAsByteArray.Length);

// Reset internal position of stream.
fStream.Position = 0;

// Read the types from file and display to console. Console.Write("Your message as an array of bytes: "); byte[] bytesFromFile = new byte[msgAsByteArray.Length]; for (int i = 0; i < msgAsByteArray.Length; i++)
{
bytesFromFile[i] = (byte)fStream.ReadByte(); Console.Write(bytesFromFile[i]);
}

// Display decoded messages.
Console.Write("\nDecoded Message: "); Console.WriteLine(Encoding.Default.GetString(bytesFromFile)); Console.ReadLine();
}
File.Delete("myMessage.dat");

This example populates the file with data, but it also punctuates the major downfall of working directly with the FileStream type: it demands to operate on raw bytes. Other Stream-derived types operate in
a similar manner. For example, if you want to write a sequence of bytes to a region of memory, you can allocate a MemoryStream.
As mentioned previously, the System.IO namespace provides several reader and writer types that encapsulate the details of working with Stream-derived types.

Working with StreamWriters and StreamReaders
The StreamWriter and StreamReader classes are useful whenever you need to read or write character-based data (e.g., strings). Both work by default with Unicode characters; however, you can change this by supplying a properly configured System.Text.Encoding object reference. To keep things simple, assume that the default Unicode encoding fits the bill.
StreamReader derives from an abstract type named TextReader, as does the related StringReader type (discussed later in this chapter). The TextReader base class provides a limited set of functionalities to each of these descendants; specifically, it provides the ability to read and peek into a character stream.
The StreamWriter type (as well as StringWriter, which you will examine later in this chapter) derives from an abstract base class named TextWriter. This class defines members that allow derived types to write textual data to a given character stream.
To aid in your understanding of the core writing capabilities of the StreamWriter and StringWriter
classes, Table 19-8 describes the core members of the abstract TextWriter base class.

Table 19-8. Core Members of TextWriter

Member Meaning in Life
Close() This method closes the writer and frees any associated resources. In the process, the buffer is automatically flushed (again, this member is functionally equivalent to calling the Dispose() method).
Flush() This method clears all buffers for the current writer and causes any buffered data to be written to the underlying device; however, it does not close the writer.
NewLine This property indicates the newline constant for the derived writer class. The default line terminator for the Windows OS is a carriage return, followed by a line feed (\r\n).
Write() WriteAsync() This overloaded method writes data to the text stream without a newline constant.
WriteLine() WriteLineAsync() This overloaded method writes data to the text stream with a newline constant.

■ Note The last two members of the TextWriter class probably look familiar to you. If you recall, the System. Console type has Write() and WriteLine() members that push textual data to the standard output device. In fact, the Console.In property wraps a TextReader, and the Console.Out property wraps a TextWriter.

The derived StreamWriter class provides an appropriate implementation for the Write(), Close(), and Flush() methods, and it defines the additional AutoFlush property. When set to true, this property forces StreamWriter to flush all data every time you perform a write operation. Be aware that you can gain better performance by setting AutoFlush to false, provided you always call Close() when you finish writing with a StreamWriter.

Writing to a Text File
To see the StreamWriter type in action, create a new Console Application project named StreamWriterReaderApp. The following code creates a new file named reminders.txt in the current execution folder, using the File.CreateText() method. Using the obtained StreamWriter object, you can add some textual data to the new file.

Console.WriteLine(" Fun with StreamWriter / StreamReader \n");

// Get a StreamWriter and write string data.
using(StreamWriter writer = File.CreateText("reminders.txt"))
{
writer.WriteLine("Don't forget Mother's Day this year..."); writer.WriteLine("Don't forget Father's Day this year..."); writer.WriteLine("Don't forget these numbers:");
for(int i = 0; i < 10; i++)
{
writer.Write(i + " ");
}

// Insert a new line.
writer.Write(writer.NewLine);
}
Console.WriteLine("Created file and wrote some thoughts..."); Console.ReadLine();
//File.Delete("reminders.txt");

After you run this program, you can examine the contents of this new file. You will find this file in the root directory of your project (Visual Studio Code) or under the bin\Debug\net6.0 folder (Visual Studio) because you did not specify an absolute path at the time you called CreateText() and the file location defaults to the current execution directory of the assembly.

Reading from a Text File
Next, you will learn to read data from a file programmatically by using the corresponding StreamReader type. Recall that this class derives from the abstract TextReader, which offers the functionality described in Table 19-9.

Table 19-9. TextReader Core Members

Member Meaning in Life
Peek() Returns the next available character (expressed as an integer) without changing the position of the reader. A value of -1 indicates you are at the end of the stream.
Read() ReadAsync() Reads data from an input stream.
ReadBlock() ReadBlockAsync() Reads a specified maximum number of characters from the current stream and writes the data to a buffer, beginning at a specified index.
ReadLine() ReadLineAsync() Reads a line of characters from the current stream and returns the data as a string (a null string indicates EOF).
ReadToEnd() ReadToEndAsync() Reads all characters from the current position to the end of the stream and returns them as a single string.

If you now extend the current sample application to use a StreamReader, you can read in the textual data from the reminders.txt file, as shown here:

Console.WriteLine(" Fun with StreamWriter / StreamReader \n");
...
// Now read data from file.
Console.WriteLine("Here are your thoughts:\n"); using(StreamReader sr = File.OpenText("reminders.txt"))
{
string input = null;
while ((input = sr.ReadLine()) != null)
{
Console.WriteLine (input);
}
}
Console.ReadLine();

After you run the program, you will see the character data in reminders.txt displayed to the console.

Directly Creating StreamWriter/StreamReader Types
One of the confusing aspects of working with the types within System.IO is that you can often achieve an identical result using different approaches. For example, you have already seen that you can use the CreateText() method to obtain a StreamWriter with the File or FileInfo type. It so happens that you
can work with StreamWriters and StreamReaders another way: by creating them directly. For example, you
could retrofit the current application as follows:

Console.WriteLine(" Fun with StreamWriter / StreamReader \n");

// Get a StreamWriter and write string data.
using(StreamWriter writer = new StreamWriter("reminders.txt"))
{
...
}

// Now read data from file.
using(StreamReader sr = new StreamReader("reminders.txt"))
{
...
}

Although it can be a bit confusing to see so many seemingly identical approaches to file I/O, keep in mind that the result is greater flexibility. In any case, you are now ready to examine the role of the
StringWriter and StringReader classes, given that you have seen how to move character data to and from a given file using the StreamWriter and StreamReader types.

Working with StringWriters and StringReaders
You can use the StringWriter and StringReader types to treat textual information as a stream of in- memory characters. This can prove helpful when you would like to append character-based information to an underlying buffer. The following Console Application project (named StringReaderWriterApp) illustrates this by writing a block of string data to a StringWriter object, rather than to a file on the local hard drive (do not forget to import System.Text):

using System.Text;

Console.WriteLine(" Fun with StringWriter / StringReader \n");

// Create a StringWriter and emit character data to memory.
using(StringWriter strWriter = new StringWriter())
{
strWriter.WriteLine("Don't forget Mother's Day this year...");
// Get a copy of the contents (stored in a string) and dump
// to console.
Console.WriteLine("Contents of StringWriter:\n{0}", strWriter);
}
Console.ReadLine();

StringWriter and StreamWriter both derive from the same base class (TextWriter), so the writing logic is similar. However, given the nature of StringWriter, you should also be aware that this class allows you to use the following GetStringBuilder() method to extract a System.Text.StringBuilder object:

using (StringWriter strWriter = new StringWriter())
{
strWriter.WriteLine("Don't forget Mother's Day this year..."); Console.WriteLine("Contents of StringWriter:\n{0}", strWriter);

// Get the internal StringBuilder.
StringBuilder sb = strWriter.GetStringBuilder(); sb.Insert(0, "Hey!! ");
Console.WriteLine("-> {0}", sb.ToString());
sb.Remove(0, "Hey!! ".Length);
Console.WriteLine("-> {0}", sb.ToString());
}

When you want to read from a stream of character data, you can use the corresponding StringReader type, which (as you would expect) functions identically to the related StreamReader class. In fact, the StringReader class does nothing more than override the inherited members to read from a block of character data, rather than from a file, as shown here:

using (StringWriter strWriter = new StringWriter())
{
strWriter.WriteLine("Don't forget Mother's Day this year..."); Console.WriteLine("Contents of StringWriter:\n{0}", strWriter);

// Read data from the StringWriter.
using (StringReader strReader = new StringReader(strWriter.ToString()))
{
string input = null;
while ((input = strReader.ReadLine()) != null)
{
Console.WriteLine(input);
}
}
}

Working with BinaryWriters and BinaryReaders
The final writer/reader sets you will examine in this section are BinaryReader and BinaryWriter. Both derive directly from System.Object. These types allow you to read and write discrete data types to an underlying stream in a compact binary format. The BinaryWriter class defines a highly overloaded Write() method to place a data type in the underlying stream. In addition to the Write() member, BinaryWriter provides additional members that allow you to get or set the Stream-derived type; it also offers support for random access to the data (see Table 19-10).

Table 19-10. BinaryWriter Core Members

Member Meaning in Life
BaseStream This read-only property provides access to the underlying stream used with the BinaryWriter
object.
Close() This method closes the binary stream.
Flush() This method flushes the binary stream.
Seek() This method sets the position in the current stream.
Write() This method writes a value to the current stream.

The BinaryReader class complements the functionality offered by BinaryWriter with the members described in Table 19-11.

Table 19-11. BinaryReader Core Members

Member Meaning in Life
BaseStream This read-only property provides access to the underlying stream used with the BinaryReader
object.
Close() This method closes the binary reader.
PeekChar() This method returns the next available character without advancing the position in the stream.
Read() This method reads a given set of bytes or characters and stores them in the incoming array.
ReadXXXX() The BinaryReader class defines numerous read methods that grab the next type from the stream (e.g., ReadBoolean(), ReadByte(), and ReadInt32()).

The following example (a Console Application project named BinaryWriterReader with using System.
IO) writes some data types to a new *.dat file:

Console.WriteLine(" Fun with Binary Writers / Readers \n");

// Open a binary writer for a file.
FileInfo f = new FileInfo("BinFile.dat"); using(BinaryWriter bw = new BinaryWriter(f.OpenWrite()))
{
// Print out the type of BaseStream.
// (System.IO.FileStream in this case).
Console.WriteLine("Base stream is: {0}", bw.BaseStream);

// Create some data to save in the file.
double aDouble = 1234.67; int anInt = 34567;
string aString = "A, B, C";

// Write the data. bw.Write(aDouble); bw.Write(anInt); bw.Write(aString);
}
Console.WriteLine("Done!"); Console.ReadLine();

Notice how the FileStream object returned from FileInfo.OpenWrite() is passed to the constructor of the BinaryWriter type. Using this technique makes it easy to layer in a stream before writing out the data. Note that the constructor of BinaryWriter takes any Stream-derived type (e.g., FileStream, MemoryStream, or BufferedStream). Thus, writing binary data to memory instead is as simple as supplying a valid MemoryStream object.
To read the data out of the BinFile.dat file, the BinaryReader type provides several options. Here, you call various read-centric members to pluck each chunk of data from the file stream:

...
FileInfo f = new FileInfo("BinFile.dat");
...
// Read the binary data from the stream.
using(BinaryReader br = new BinaryReader(f.OpenRead()))
{
Console.WriteLine(br.ReadDouble()); Console.WriteLine(br.ReadInt32()); Console.WriteLine(br.ReadString());
}
Console.ReadLine();

Watching Files Programmatically
Now that you have a better handle on the use of various readers and writers, you will look at the role of the FileSystemWatcher class. This type can be quite helpful when you want to monitor (or “watch”) files on your system programmatically. Specifically, you can instruct the FileSystemWatcher type to monitor files for any of the actions specified by the System.IO.NotifyFilters enumeration.

public enum NotifyFilters
{
Attributes, CreationTime, DirectoryName, FileName, LastAccess, LastWrite, Security, Size
}

To begin working with the FileSystemWatcher type, you need to set the Path property to specify the name (and location) of the directory that contains the files you want to monitor, as well as the Filter property that defines the file extensions of the files you want to monitor.
At this point, you may choose to handle the Changed, Created, and Deleted events, all of which work in conjunction with the FileSystemEventHandler delegate. This delegate can call any method matching the following pattern:

// The FileSystemEventHandler delegate must point
// to methods matching the following signature.
void MyNotificationHandler(object source, FileSystemEventArgs e)

You can also handle the Renamed event using the RenamedEventHandler delegate type, which can call methods that match the following signature:

// The RenamedEventHandler delegate must point
// to methods matching the following signature.
void MyRenamedHandler(object source, RenamedEventArgs e)

Next, let’s look at the process of watching a file. The following Console Application project (named MyDirectoryWatcher and with a using for System.IO) monitors the *.txt files in the bin\debug\net6.0 directory and prints messages when files are created, deleted, modified, or renamed:

Console.WriteLine(" The Amazing File Watcher App \n");
// Establish the path to the directory to watch. FileSystemWatcher watcher = new FileSystemWatcher(); try
{
watcher.Path = @".";
}
catch(ArgumentException ex)
{
Console.WriteLine(ex.Message); return;
}

// Set up the things to be on the lookout for.
watcher.NotifyFilter = NotifyFilters.LastAccess
| NotifyFilters.LastWrite
| NotifyFilters.FileName
| NotifyFilters.DirectoryName;

// Only watch text files.
watcher.Filter = "*.txt";

// Add event handlers.
// Specify what is done when a file is changed, created, or deleted.
watcher.Changed += (s, e) =>
Console.WriteLine($"File: {e.FullPath} {e.ChangeType}!"); watcher.Created += (s, e) =>
Console.WriteLine($"File: {e.FullPath} {e.ChangeType}!"); watcher.Deleted += (s, e) =>
Console.WriteLine($"File: {e.FullPath} {e.ChangeType}!");
// Specify what is done when a file is renamed.
watcher.Renamed += (s, e) =>
Console.WriteLine($"File: {e.OldFullPath} renamed to {e.FullPath}");
// Begin watching the directory.
watcher.EnableRaisingEvents = true;

// Wait for the user to quit the program.
Console.WriteLine(@"Press 'q' to quit app.");
// Raise some events.
using (var sw = File.CreateText("Test.txt"))
{
sw.Write("This is some text");
}
File.Move("Test.txt","Test2.txt"); File.Delete("Test2.txt");

while(Console.Read()!='q');

When you run this program, the last lines will create, change, rename, and then delete a text file, raising the events along the way. You can also navigate to the project directory (Visual Studio Code) or the bin\debug\net6.0 directory (Visual Studio) and play with files (with the *.txt extension) and raise additional events.

The Amazing File Watcher App Press 'q' to quit app.
File: .\Test.txt Created! File: .\Test.txt Changed!
File: .\Test.txt renamed to .\Test2.txt File: .\Test2.txt Deleted!

That wraps up this chapter’s look at fundamental I/O operations within the .NET platform. While you will certainly use these techniques in many of your applications, you might also find that object serialization services can greatly simplify how you persist large amounts of data.

Understanding Object Serialization
The term serialization describes the process of persisting (and possibly transferring) the state of an object into a stream (e.g., file stream or memory stream). The persisted data sequence contains all the necessary information you need to reconstruct (or deserialize) the public state of the object for use later. Using this technology makes it trivial to save vast amounts of data. In many cases, saving application data using serialization services results in less code than using the readers/writers you find in the System.IO namespace.
For example, assume you want to create a GUI-based desktop application that provides a way for end users to save their preferences (e.g., window color and font size). To do this, you might define a class named UserPrefs that encapsulates 20 or so pieces of field data. Now, if you were to use a System.IO.BinaryWriter type, you would need to save each field of the UserPrefs object manually. Likewise, if you were to load the data from a file back into memory, you would need to use a System.IO.BinaryReader and (once again) manually read in each value to reconfigure a new UserPrefs object.
This is all doable, but you can save yourself time by using either eXtensible Markup Language (XML) or JavaScript Object Notation (JSON) serialization. Each of these formats enables representing the public state of an object in a single block of text that is usable across platforms and programming languages. Doing this means that you can persist the entire public state of the object with only a few lines of code.

■ Note The BinaryFormatter type, covered in previous editions of this book, is a high security risk, and you should stop using it immediately (http://aka.ms/binaryformatter). More secure alternatives include using BinaryReaders/BinaryWriters in conjunction with XMl or jSOn serialization.

.NET object serialization makes it easy to persist objects; however, the processes used behind the scenes are quite sophisticated. For example, when an object is persisted to a stream, all associated public data (e.g., base class data and contained objects) is automatically serialized as well. Therefore, if you attempt to persist a derived class, all public data up the chain of inheritance comes along for the ride. As you will see, you use an object graph to represent a set of interrelated objects.
Finally, understand that you can persist an object graph into any System.IO.Stream-derived type. All that matters is that the sequence of data correctly represents the state of objects within the graph.

The Role of Object Graphs
As mentioned previously, the .NET Runtime will account for all related objects to ensure that public data is persisted correctly when an object is serialized. This set of related objects is referred to as an object graph. Object graphs provide a simple way to document how a set of items refer to each other. Object graphs are not denoting OOP is-a or has-a relationships. Rather, you can read the arrows in an object diagram as “requires” or “depends on.”
Each object in an object graph is assigned a unique numerical value. Keep in mind that the numbers assigned to the members in an object graph are arbitrary and have no real meaning to the outside world. Once you assign all objects a numerical value, the object graph can record each object’s set of dependencies.
For example, assume you have created a set of classes that model some automobiles (of course). You have a base class named Car, which has-a Radio. Another class named JamesBondCar extends the Car base type. Figure 19-1 shows a possible object graph that models these relationships.

Figure 19-1. A simple object graph

When reading object graphs, you can use the phrase depends on or refers to when connecting the arrows. Thus, in Figure 19-1, you can see that the Car refers to the Radio class (given the has-a relationship). JamesBondCar refers to Car (given the is-a relationship), as well as to Radio (it inherits this protected member variable).
Of course, the CLR does not paint pictures in memory to represent a graph of related objects. Rather, the relationship documented in Figure 19-1 is represented by a mathematical formula that looks something like this:

[Car 3, ref 2], [Radio 2], [JamesBondCar 1, ref 3, ref 2]

If you parse this formula, you can see that object 3 (the Car) has a dependency on object 2 (the Radio). Object 2, the Radio, is a lone wolf and requires nobody. Finally, object 1 (the JamesBondCar) has a dependency on object 3, as well as object 2. In any case, when you serialize or deserialize an instance of JamesBondCar, the object graph ensures that the Radio and Car types also participate in the process.
The beautiful thing about the serialization process is that the graph representing the relationships among your objects is established automatically behind the scenes. As you will see later in this chapter, however, you can become more involved in the construction of a given object graph by customizing the serialization process using attributes.

Creating the Sample Types and Top-Level Statements
Create a new Console Application project named SimpleSerialize. Update the Program.cs file to the following:

global using System.Text.Json;
global using System.Text.Json.Serialization; global using System.Xml;
global using System.Xml.Serialization; using SimpleSerialize;
Console.WriteLine(" Fun with Object Serialization \n");

Next, add a new class named Radio.cs, and update the code to the following:
namespace SimpleSerialize; public class Radio
{
public bool HasTweeters; public bool HasSubWoofers;
public List StationPresets; public string RadioId = "XF-552RR6"; public override string ToString()
{
var presets = string.Join(",", StationPresets.Select(i => i.ToString()).ToList()); return $"HasTweeters:{HasTweeters} HasSubWoofers:{HasSubWoofers} Station Presets:{presets}";
}
}

Next, add a class named Car.cs, and update the code to match this listing:

namespace SimpleSerialize; public class Car
{
public Radio TheRadio = new Radio(); public bool IsHatchBack;
public override string ToString()
=> $"IsHatchback:{IsHatchBack} Radio:{TheRadio.ToString()}";
}

Next, add another class named JamesBondCar.cs and use the following code for this class:

namespace SimpleSerialize; public class JamesBondCar : Car
{
public bool CanFly; public bool CanSubmerge;
public override string ToString()
=> $"CanFly:{CanFly}, CanSubmerge:{CanSubmerge} {base.ToString()}";
}

The final class, Person.cs, is shown here:
namespace SimpleSerialize; public class Person
{
// A public field.
public bool IsAlive = true;
// A private field.
private int PersonAge = 21;
// Public property/private data. private string _fName = string.Empty;

public string FirstName
{
get { return _fName; } set { _fName = value; }
}
public override string ToString() =>
$"IsAlive:{IsAlive} FirstName:{FirstName} Age:{PersonAge} ";
}

Finally, add the following code to the Program.cs file in the starter code (preserving the using
statements added earlier):

Console.WriteLine(" Fun with Object Serialization \n"); var theRadio = new Radio
{
StationPresets = new() { 89.3, 105.1, 97.1 }, HasTweeters = true
};
// Make a JamesBondCar and set state. JamesBondCar jbc = new()
{
CanFly = true, CanSubmerge = false, TheRadio = new()
{
StationPresets = new() { 89.3, 105.1, 97.1 }, HasTweeters = true
}
};

List myCars = new()
{
new JamesBondCar { CanFly = true, CanSubmerge = true, TheRadio = theRadio }, new JamesBondCar { CanFly = true, CanSubmerge = false, TheRadio = theRadio }, new JamesBondCar { CanFly = false, CanSubmerge = true, TheRadio = theRadio }, new JamesBondCar { CanFly = false, CanSubmerge = false, TheRadio = theRadio },
};

Person p = new Person
{
FirstName = "James", IsAlive = true
};

Now you are all set up to explore XML and JSON serialization.

Extensible Markup Language (XML)
One of the original goals of XML was to represent an object (or set of objects) in human- and machine- readable format. An XML document is a single file that contains the item(s) being serialized. To conform to the standard (and be usable by software systems that support XML), the document opens with an

XML declaration defining the version and optionally the encoding. The next line is the root element and contains the XML namespaces. All the data is contained between the opening and closing tags for the root element.
For example, the Person class can be represented in XML as shown in the following sample. You can see the XML declaration and the root element (Person), as well as additional markup for the properties. Optionally, properties can be represented using attributes.

<?xml version="1.0"?>


true
James

If you have a list of objects, such as a list of JamesBondCar objects, the structure is the same. In the following example, the root element is not a JamesBondCar, but an array of JamesBondCar. Then each JamesBondCar in the array is contained within the root element. The following example shows the attribute syntax for the CanFly and CanSubmerge properties:

<?xml version="1.0"?>




false
false
XF-552RR6

89.3
105.1
97.1


false



false
false
XF-552RR6

89.3
105.1
97.1


false

Serializing and Deserializing with the XmlSerializer
The System.Xml namespace provides the System.Xml.Serialization.XmlSerializer. You can use this formatter to persist the public state of a given object as pure XML. Note that the XmlSerializer requires you to declare the type that will be serialized (or deserialized).

Controlling the Generated XML Data
If you have a background in XML technologies, you know that it is often critical to ensure the data within an XML document conforms to a set of rules that establishes the validity of the data. Understand that a valid XML document does not have anything to do with the syntactic well-being of the XML elements (e.g., all opening elements must have a closing element). Rather, valid documents conform to agreed-upon formatting rules (e.g., field X must be expressed as an attribute and not a subelement), which are typically defined by an XML schema or document-type definition (DTD) file.
By default, XmlSerializer serializes all public fields/properties as XML elements, rather than as XML attributes. If you want to control how the XmlSerializer generates the resulting XML document, you
can decorate types with any number of additional .NET attributes from the System.Xml.Serialization namespace. Table 19-12 documents some (but not all) of the .NET attributes that influence how XML data is encoded to a stream.

Table 19-12. Select Attributes of the System.Xml.Serialization Namespace

.NET Attribute Meaning in Life
[XmlAttribute] You can use this .NET attribute on a public field or property in a class to tell
XmlSerializer to serialize the data as an XML attribute (rather than as a subelement).
[XmlElement] The field or property will be serialized as an XML element named as you so choose.
[XmlEnum] This attribute provides the element name of an enumeration member.
[XmlRoot] This attribute controls how the root element will be constructed (namespace and element name).
[XmlText] The property or field will be serialized as XML text (i.e., the content between the start tag and the end tag of the root element).
[XmlType] This attribute provides the name and namespace of the XML type.

Of course, you can use many other .NET attributes to control how the XmlSerializer generates the resulting XML document. For full details, look up the System.Xml.Serialization namespace in the .NET documentation.

■ Note The XmlSerializer demands that all serialized types in the object graph support a default constructor (so be sure to add it back if you define custom constructors).

Serializing Objects Using the XmlSerializer
Consider the following local function added to your Program.cs file:

static void SaveAsXmlFormat(T objGraph, string fileName)
{
//Must declare type in the constructor of the XmlSerializer XmlSerializer xmlFormat = new XmlSerializer(typeof(T)); using (Stream fStream = new FileStream(fileName,
FileMode.Create, FileAccess.Write, FileShare.None))
{
xmlFormat.Serialize(fStream, objGraph);
}
}

Add the following code to your top-level statements:

SaveAsXmlFormat(jbc, "CarData.xml"); Console.WriteLine("=> Saved car in XML format!");

SaveAsXmlFormat(p, "PersonData.xml"); Console.WriteLine("=> Saved person in XML format!");

If you were to look within the newly generated CarData.xml file, you would find the XML data shown here:

<?xml version="1.0"?>



true
false

89.3
105.1
97.1

XF-552RR6

false
true
false

If you want to specify a custom XML namespace that qualifies the JamesBondCar and encodes the canFly and canSubmerge values as XML attributes instead of elements, you can do so by modifying the C# definition of JamesBondCar like this:

[Serializable, XmlRoot(Namespace = "http://www.MyCompany.com")] public class JamesBondCar : Car
{
[XmlAttribute]

public bool CanFly;
[XmlAttribute]
public bool CanSubmerge;
...
}

This yields the following XML document (note the opening element):

<?xml version="1.0"""?>
<JamesBondCar xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema"
CanFly="true" CanSubmerge="false" xmlns="http://www.MyCompany.com">
...

Next, examine the following PersonData.xml file:

<?xml version="1.0"?>


true
James

Notice how the PersonAge property is not serialized into the XML. This confirms that XML serialization serializes only public properties and fields.

Serializing Collections of Objects
Serializing collections works in the same manner. Add the following to your top-level statements:

SaveAsXmlFormat(myCars,"CarCollection.xml"); Console.WriteLine("=> Saved list of cars!");

The generated XML matches the example shown at the beginning of this section, with the
ArrayOfJamesBondCar as the root element.

Deserializing Objects and Collections of Objects
XML deserialization is literally the opposite of serializing objects (and collections of objects). Consider the following local function to deserialize XML back into an object graph. Notice that, once again, the type to be deserialized must be passed into the constructor for the XmlSerializer:

static T ReadAsXmlFormat(string fileName)
{
// Create a typed instance of the XmlSerializer XmlSerializer xmlFormat = new XmlSerializer(typeof(T));
using (Stream fStream = new FileStream(fileName, FileMode.Open))
{
T obj = default;

obj = (T)xmlFormat.Deserialize(fStream); return obj;
}
}

Add the following code to the top-level statements to reconstitute your XML back into objects (or list of objects):

JamesBondCar savedCar = ReadAsXmlFormat("CarData.xml"); Console.WriteLine("Original Car:\t {0}",jbc.ToString()); Console.WriteLine("Read Car:\t {0}",savedCar.ToString());

List savedCars = ReadAsXmlFormat<List>("CarCollection.xml");

XML serialization is used not only for storing and retrieving data, as shown in these examples, but also for sending data between systems, especially systems developed with differing technology stacks. All modern programming languages (and many database providers) have built-in support for XML.

JavaScript Object Notation (JSON) Serialization
While XML serialization is still widely used, it has largely taken a backseat to systems using JSON to share, persist, and/or load data. JSON, like XML, is a textual representation of an object (or object graph) that is cross-platform compatible and adheres to an open standard. Systems built with a wide range of languages and tooling have built-in support for JSON.
Objects in JSON documents are designated using name-value pairs for the properties enclosed in curly braces ({}). For example, a Person instance when serialized to JSON produces the following document:

{
"firstName": "James", "isAlive": true
}

Notice some of the key differences between the JSON and the XML representation of the same instance from the previous section. There isn’t a declaration or a root name, just the properties of the serialized object. This results in a much smaller amount of text, making it a more efficient format.
The lack of the class name (Person) in the JSON provides additional flexibility. The sender (in this case us) might call the class Person, while the receiver might call the class Human. As long as the properties match, the JSON will be properly applied to the object.
Lists of objects are stored as JavaScript arrays using square brackets ([]). The following is a list containing two JamesBondCar objects:

[
{
"CanFly": true, "CanSubmerge": true, "TheRadio": {
"StationPresets": ["89.3", "105.1", "97.1"],
"HasTweeters": true,

"HasSubWoofers": false, "RadioId": "XF-552RR6"
},
"IsHatchBack": false
},
{
"CanFly": true, "CanSubmerge": false, "TheRadio": {
"StationPresets": ["89.3", "105.1", "97.1"],
"HasTweeters": true, "HasSubWoofers": false, "RadioId": "XF-552RR6"
},
"IsHatchBack": false
}
]

Notice that the entire file is opened and closed with a square bracket, and then each object in the array is opened and closed with a curly brace. The radio presets are also a list, so they are serialized as an array
of values.

Serializing and Deserializing with System.Text.Json
The System.Text.Json namespace provides the System.Text.Json.JsonSerializer. You can use this formatter to persist the public state of a given object as JSON.

Controlling the Generated JSON Data
By default, JsonSerializer serializes all public properties as JSON name-value pairs using the same name (and casing) of the object’s property names. You can control many aspects of the serialization process with attributes; some of the more commonly used attributes are listed in Table 19-13.

Table 19-13. Select Attributes of the System.Text.Json.Serialization Namespace

.NET Attribute Meaning in Life
[JsonIgnore] The property will be ignored.
[JsonInclude] The member will be included.
[JsonPropertyName] This specifies the property name to be used when serializing/deserializing a member. This is commonly used to resolve character casing issues.
[JsonConstructor] This indicates the constructor that should be used when deserializing JSON back into an object graph.

Serializing Objects Using the JsonSerializer
The JsonSerializer contains static Serialize methods used to convert .NET objects (including object graphs) into a string representation of the public properties. The data is represented as name-value pairs in JavaScript Object Notation. Consider the following local function added to your Program.cs file:

static void SaveAsJsonFormat(T objGraph, string fileName)
{
File.WriteAllText(fileName, System.Text.Json.JsonSerializer.Serialize(objGraph));
}

Add the following code to your top-level statements:

SaveAsJsonFormat(jbc, "CarData.json"); Console.WriteLine("=> Saved car in JSON format!");

SaveAsJsonFormat(p, "PersonData.json"); Console.WriteLine("=> Saved person in JSON format!");

When you examine the created JSON files, you might be surprised to see that the CarData.json file is empty (except for a pair of braces) and the PersonData.json file contains only the Firstname value. This is because the JsonSerializer only writes public properties by default, and not public fields. You will correct this in the next section.

Including Fields
To include public fields into the generated JSON, you have two options. The other method is to use the JsonSerializerOptions class to instruct the JsonSerializer to include all fields. The second is to update your classes by adding the [JsonInclude] attribute to each public field that should be included in the JSON output. Note that the first method (using the JsonSerializationOptions) will include all public fields in the object graph. To exclude certain public fields using this technique, you must use the JsonExclude attribute on them to be excluded.
Update the SaveAsJsonFormat method to the following:

static void SaveAsJsonFormat(T objGraph, string fileName)
{
var options = new JsonSerializerOptions
{
IncludeFields = true,
};
File.WriteAllText(fileName, System.Text.Json.JsonSerializer.Serialize(objGraph, options));
}

Instead of using the JsonSerializerOptions, you can achieve the same result by updating all public fields in the sample classes to the following (note that you can leave the Xml attributes in the classes and they will not interfere with the JsonSerializer):

//Radio.cs
public class Radio
{
[JsonInclude]

public bool HasTweeters;
[JsonInclude]
public bool HasSubWoofers;
[JsonInclude]
public List StationPresets;
[JsonInclude]
public string RadioId = "XF-552RR6";
...
}

//Car.cs
public class Car
{
[JsonInclude]
public Radio TheRadio = new Radio();
[JsonInclude]
public bool IsHatchBack;
...
}

//JamesBondCar.cs
public class JamesBondCar : Car
{
[XmlAttribute] [JsonInclude] public bool CanFly; [XmlAttribute] [JsonInclude]
public bool CanSubmerge;
...
}

//Person.cs
public class Person
{
// A public field.
[JsonInclude]
public bool IsAlive = true;
...
}

Now when you run the code using either method, all public properties and fields are written to the file. However, when you examine the contents, you will see that the JSON is written minified. Minified is a
format where all insignificant whitespace and line breaks are removed. This is the default format largely due to JSON’s wide use of RESTful services and reduces the size of the data packet when sending information between services over HTTP/HTTPS.

■ Note The field handling for serializing jSOn is the same as deserializing jSOn. If you chose to set the option to include fields when serializing jSOn, you must also include that option when deserializing jSOn.

Pretty-Print the JSON
In addition to the option to include public fields, the JsonSerializer can be instructed to write the JSON indented (and human readable). Update your method to the following:

static void SaveAsJsonFormat(T objGraph, string fileName)
{
var options = new JsonSerializerOptions
{
IncludeFields = true,
WriteIndented = true
};
File.WriteAllText(fileName, System.Text.Json.JsonSerializer.Serialize(objGraph, options));
}

Now examine the CarData.json file; the output is much more readable.

{
"canFly": true, "canSubmerge": false, "theRadio": {
"stationPresets": [ "89.3",
"105.1",
"97.1"
],
"hasTweeters": true, "hasSubWoofers": false, "radioId": "XF-552RR6"
},
"isHatchBack": false
}

PascalCase or camelCase JSON
Pascal casing is a format that uses the first character capitalized and every significant part of a name capitalized as well. Camel casing, on the other hand, sets the first character to lowercase (like the word camelCase in the title of this section), and then every significant part of the name starts with a capital. Take the previous JSON listing. canSubmerge is an example of camel casing. The Pascal case version of the previous example is CanSubmerge.
Why does this matter? It matters because most of the popular languages are case sensitive (like C#). That means CanSubmerge and canSubmerge are two different items. As you have seen throughout this book, the generally accepted standard for naming public things in C# (classes, public properties, functions, etc.) is to use Pascal casing. However, most of the JavaScript frameworks prefer to use camel casing. This can be problematic when using .NET and C# to pass JSON data to/from non-C#/.NET RESTful services.

Fortunately, the JsonSerializer is customizable to handle most situations, including casing differences. If no naming policy is specified, the JsonSerializer will use camel casing when serializing and deserializing JSON. To change the serialization process to use Pascal casing, you need to set PropertyNamingPolicy to null, as follows:

static void SaveAsJsonFormat(T objGraph, string fileName)
{
JsonSerializerOptions options = new()
{
PropertyNamingPolicy = null, IncludeFields = true, WriteIndented = true,
};
File.WriteAllText(fileName, System.Text.Json.JsonSerializer.Serialize(objGraph, options));
}

Now, when you execute the calling code, the JSON produced is all Pascal cased.

{
"CanFly": true, "CanSubmerge": false, "TheRadio": {
"StationPresets": [ "89.3",
"105.1",
"97.1"
],
"HasTweeters": true, "HasSubWoofers": false, "RadioId": "XF-552RR6"
},
"IsHatchBack": false
}

When reading JSON, C# is (by default) case sensitive. The casing setting of the PropertyNamingPolicy is used during Deserialization. If the property is not set, the default (camel casing) is used. By setting the PropertyNamingPolicy to Pascal case, then all incoming JSON is expected to be in Pascal case. If the casing does not match, the deserialization process (covered soon) fails.
There is a third option when deserializing JSON, and that is casing indifference. By setting the PropertyNameCaseInsensitive option to true, then C# will deserialize canSubmerge as well as CanSubmerge. Here is the code to set the option:

JsonSerializerOptions options = new()
{
PropertyNameCaseInsensitive = true, IncludeFields = true
};

Ignoring Circular References with JsonSerializer (New 10)
Introduced in .NET 6/C# 10, the System.Text.Json.JsonSerializer supports ignoring circular references when serializing an object graph. This is done by setting the ReferenceHandler to ReferenceHandler.
IgnoreCycles in the JsonSerializerOptions. Here is the code to set the serializer to ignore the circular references:

JsonSerializerOptions options = new()
{
ReferenceHandler = ReferenceHandler.IgnoreCycles
};

Table 19-14 lists the available values in the ReferenceHandler enum.

Table 19-14. ReferenceHandler Enum Values

Enum Value Meaning in Life
IgnoreCycles Circular references are not serialized, and the reference loop is replaced with a null.
Preserve Metadata properties will be honored when deserializing JSON objects and arrays into reference types and written when serializing reference types. This is necessary to create round-trippable JSON from objects that contain cycles or duplicate references.

Number Handling with JsonSerializer
The default handling of numbers is Strict, meaning numbers will be serialized as numbers (without quotes) and deserialized as numbers (without quotes). The JsonSerializerOptions has a NumberHandling property that controls reading and writing numbers. Table 19-15 lists the available values in the JsonNumberHandling enum.

Table 19-15. JsonNumberHandling Enum Values

Enum Value Meaning in Life
Strict (0) Numbers are read from numbers and written as numbers. Quotes are not allowed nor are they generated.
AllowReadingFromString (1) Numbers can be read from number or string tokens.
WriteAsString (2) Numbers are written as JSON strings (with quotes).
AllowNamedFloatingPointLiterals (4) The Nan, Infinity, and -Infinity string tokens can be read, and Single and Double values will be written as their corresponding JSON string representations.

The enum has a flags attribute, which allows a bitwise combination of its values. For example, if you want to read strings (and numbers) and write numbers as strings, you use the following option setting:

JsonSerializerOptions options = new()
{
...
NumberHandling = JsonNumberHandling.AllowReadingFromString & JsonNumberHandling.WriteAsString

};

With this change, the JSON created for the Car class is as follows:

{
"canFly": true, "canSubmerge": false, "theRadio": {
"hasTweeters": true, "hasSubWoofers": false, "stationPresets": [
"89.3",
"105.1",
"97.1"
],
"radioId": "XF-552RR6"
},
"isHatchBack": false
}

JSON Property Ordering (New 10)
With the release of .NET 6/C# 10, the JsonPropertyOrder attribute controls property ordering during serialization. The smaller the number (including negative values), the earlier the property is in the resulting JSON. Properties without an order are assigned a default order of zero. Update the Person class to the following:
namespace SimpleSerialize; public class Person
{
[JsonPropertyOrder(1)] public bool IsAlive = true;

private int PersonAge = 21;

private string _fName = string.Empty; [JsonPropertyOrder(-1)]
public string FirstName
{
get { return _fName; } set { _fName = value; }
}
public override string ToString() => $"IsAlive:{IsAlive} FirstName:{FirstName} Age:{PersonAge} ";
}

With that change, the properties are serialized in the order of FirstName (-1) and then IsAlive (1).
PersonAge doesn’t get serialized because it’s private. If it were made public, it would get the default order of zero and be placed between the other two properties.

Support for IAsyncEnumerable (New 10)
With the release of .NET 6/C# 10, the System.Text.Json.JsonSerializer now has support for serializing and deserializing async streams.

Streaming Serialization
To demonstrate streaming serialization, start by adding a new method that will return an
IAsyncEnumerable:

static async IAsyncEnumerable PrintNumbers(int n)
{
for (int i = 0; i < n; i++)
{
yield return i;
}
}

Next, create a Stream from the Console and serialize the IAsyncEnumerable returned from the
PrintNumbers() function.

async static void SerializeAsync()
{
Console.WriteLine("Async Serialization");
using Stream stream = Console.OpenStandardOutput(); var data = new { Data = PrintNumbers(3) };
await JsonSerializer.SerializeAsync(stream, data); Console.WriteLine();
}

Streaming Deserialization
There is a new API to support streaming deserialization, DeserializeAsyncEnumerable(). To demonstrate the use of this, add a new method that will create a new MemoryStream and then deserialize from the stream:

async static void DeserializeAsync()
{
Console.WriteLine("Async Deserialization");
var stream = new MemoryStream(System.Text.Encoding.UTF8.GetBytes("[0,1,2,3,4]")); await foreach (int item in JsonSerializer.DeserializeAsyncEnumerable(stream))
{
Console.Write(item);
}
Console.WriteLine();
}

Potential Performance Issues Using JsonSerializerOptions
When using JsonSerializerOption, it is best to create a single instance and reuse it throughout your application. With that in mind, update your top-level statements and JSON methods to the following:

JsonSerializerOptions options = new()
{
PropertyNameCaseInsensitive = true,
//PropertyNamingPolicy = JsonNamingPolicy.CamelCase, PropertyNamingPolicy = null, //Pascal casing IncludeFields = true,
WriteIndented = true,
NumberHandling = JsonNumberHandling.AllowReadingFromString | JsonNumberHandling. WriteAsString
};
SaveAsJsonFormat(options, jbc, "CarData.json"); Console.WriteLine("=> Saved car in JSON format!");

SaveAsJsonFormat(options, p, "PersonData.json"); Console.WriteLine("=> Saved person in JSON format!");

static void SaveAsJsonFormat(JsonSerializerOptions options, T objGraph, string fileName)
=> File.WriteAllText(fileName, System.Text.Json.JsonSerializer.Serialize(objGraph, options));

Web Defaults for JsonSerializer
When building web applications, you can use a specialized constructor to set the following properties:

PropertyNameCaseInsensitive = true, PropertyNamingPolicy = JsonNamingPolicy.CamelCase,
NumberHandling = JsonNumberHandling.AllowReadingFromString

You can still set additional properties or override the defaults through object initialization, like this:

JsonSerializerOptions options = new(JsonSerializerDefaults.Web)
{
WriteIndented = true,
ReferenceHandler = ReferenceHandler.IgnoreCycles
};

General Defaults for JsonSerializer
If you want to start off with more general settings, passing JsonSerializerDefaults.General into the constructor will set the following properties:

PropertyNameCaseInsensitive = false, PropertyNamingPolicy = null, //Pascal Casing NumberHandling = JsonNumberHandling.Strict

Like the web version, you can still set additional properties or override the defaults through object initialization, like this:

JsonSerializerOptions options = new(JsonSerializerDefaults.General)
{
WriteIndented = true,
ReferenceHandler = ReferenceHandler.IgnoreCycles, PropertyNameCaseInsensitive = true
};

Serializing Collections of Objects
Serializing a collection of objects into JSON is handled the same way as a single object. Add the following line to the top-level statements:

SaveAsJsonFormat(options, myCars, "CarCollection.json");

Deserializing Objects and Collections of Objects
JSON deserialization is the opposite of serialization. The following function will deserialize JSON into the type specified using the generic version of the method:

static T ReadAsJsonFormat(JsonSerializerOptions options, string fileName) => System.Text.Json.JsonSerializer.Deserialize(File.ReadAllText(fileName), options);

Add the following code to the top-level statements to reconstitute your XML back into objects (or list of objects):

JamesBondCar savedJsonCar = ReadAsJsonFormat(options, "CarData.json"); Console.WriteLine("Read Car: {0}", savedJsonCar.ToString());

List savedJsonCars = ReadAsJsonFormat<List>(options, "CarCollection.json");
Console.WriteLine("Read Car: {0}", savedJsonCar.ToString());

Note that the type being created during the deserializing process can be a single object or a generic collection.

JsonConverters
You can take additional control of the serialization/deserialization process by adding in custom converters. Custom converters inherit from JsonConverter (where T is the type that the converter operates on) and override the base Read() and Write() methods. Here are the abstract base methods to overwrite:

public abstract T? Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options);
public abstract void Write(Utf8JsonWriter writer, T value, JsonSerializerOptions options);

One common scenario is converting null values in your object to an empty string in the JSON and back to a null in your object. To demonstrate this, add a new file named JsonStringNullToEmptyConverter.cs, make the class public, inherit from JsonConverter, and implement the abstract members. Here is the initial code:

namespace SimpleSerialize
{
public class JsonStringNullToEmptyConverter : JsonConverter
{
public override string Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
}

public override void Write(Utf8JsonWriter writer, string value, JsonSerializerOptions options)
{
}
}
}

In the Read method, use the Utf8JsonReader instance to read the string value for the node, and if it is
null or an empty string, then return null. Otherwise, return the value read:

public override string Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
var value = reader.GetString(); if (string.IsNullOrEmpty(value))
{
return null;
}
return value;
}

In the Write method, use the Utf8JsonWriter to write an empty string if the value is null:

public override void Write(Utf8JsonWriter writer, string value, JsonSerializerOptions options)
{
value ??= string.Empty; writer.WriteStringValue(value);
}

The final step for the custom converter is to force the serializer to process null values. By default, null values are not sent through the conversion process to improve performance. However, in this scenario, we want the null values to be processed, so override the base property HandleNull and set it to true instead of the default false value.

public override bool HandleNull => true;

The final step is to add the custom converter into the serialization options. Create a new method named
HandleNullStrings() and add the following code:

static void HandleNullStrings()
{
Console.WriteLine("Handling Null Strings"); var options = new JsonSerializerOptions
{
PropertyNameCaseInsensitive = true, PropertyNamingPolicy = null, IncludeFields = true,
WriteIndented = true,
Converters = { new JsonStringNullToEmptyConverter() },
};
//Create a new object with a null string var radio = new Radio
{
HasSubWoofers = true, HasTweeters = true, RadioId = null
};
//serialize the object to JSON
var json = JsonSerializer.Serialize(radio, options); Console.WriteLine(json);
}

When you call this method from the top-level statements, you can see that the RadioId property is indeed an empty string in the JSON instead of a null. The StationPresets values are still null in the JSON because the custom converter acts only on string types, not on List types.

{
"StationPresets": null, "HasTweeters": true, "HasSubWoofers": true, "RadioId": null
}

Summary
You began this chapter by examining the use of the Directory(Info) and File(Info) types. As you learned, these classes allow you to manipulate a physical file or directory on your hard drive. Next, you examined several classes derived from the abstract Stream class. Given that Stream-derived types operate on a raw stream of bytes, the System.IO namespace provides numerous reader/writer types (e.g., StreamWriter, StringWriter, and BinaryWriter) that simplify the process. Along the way, you also checked out the functionality provided by DriveType, learned how to monitor files using the FileSystemWatcher type, and saw how to interact with streams in an asynchronous manner.
This chapter also introduced you to the topic of object serialization services. As you have seen, the
.NET platform uses an object graph to account for the full set of related objects that you want to persist to a stream. You then worked with XML and JSON serialization and deserialization.

Pro C#10 CHAPTER 18 Understanding CIL and the Role of Dynamic Assemblies

CHAPTER 18

Understanding CIL and the Role of Dynamic Assemblies

When you are building a full-scale .NET application, you will most certainly use C# (or a similar managed language such as Visual Basic), given its inherent productivity and ease of use. However, as you learned in the beginning of this book, the role of a managed compiler is to translate *.cs code files into terms of CIL code, type metadata, and an assembly manifest. As it turns out, CIL is a full-fledged .NET programming language, with its own syntax, semantics, and compiler (ilasm.exe).
In this chapter, you will be given a tour of .NET’s mother tongue. Here, you will understand the distinction between a CIL directive, CIL attribute, and CIL opcode. You will then learn about the role of round-trip engineering of a .NET Core assembly and various CIL programming tools. The remainder of the chapter will then walk you through the basics of defining namespaces, types, and members using the grammar of CIL. The chapter will wrap up with an examination of the role of the System.Reflection.Emit namespace and explain how it is possible to construct an assembly (with CIL instructions) dynamically at runtime.
Of course, few programmers will ever need to work with raw CIL code on a day-to-day basis. Therefore, I will start this chapter by examining a few reasons why getting to know the syntax and semantics of this low- level .NET language might be worth your while.

Motivations for Learning the Grammar of CIL
When you build a .NET assembly using your managed language of choice (C#, VB, F#, etc.), the associated compiler translates your source code into terms of CIL. Like any programming language, CIL provides numerous structural and implementation-centric tokens. Given that CIL is just another .NET programming language, it should come as no surprise that it is possible to build your .NET assemblies directly using CIL and the CIL compiler (ilasm.exe).

© Andrew Troelsen, Phil Japikse 2022
A. Troelsen and P. Japikse, Pro C# 10 with .NET 6, https://doi.org/10.1007/978-1-4842-7869-7_18

713

■ Note As covered in Chapter 1, neither ildasm.exe nor ilasm.exe ships with the .NET Runtime.
There are two options for getting these tools. The first is to compile the .NET Runtime from the source located at https://github.com/dotnet/runtime. The second, and easier, method is to pull down the desired version from www.nuget.org. The URL for ILDasm on NuGet is https://www.nuget.org/packages/Microsoft. NETCore.ILDAsm/, and for ILAsm.exe it is https://www.nuget.org/packages/Microsoft.NETCore. ILAsm/. Make sure to select the correct version (for this book you need version 6.0.0 or greater). Add the ILDasm and ILAsm NuGet packages to your project with the following commands: dotnet add package Microsoft. NETCore.ILDAsm --version 6.0.0dotnet add package Microsoft.NETCore.ILAsm --version 6.0.0
This does not actually add ILDasm.exe or ILAsm.exe into your project but places them in your package folder (on Windows): %userprofile%.nuget\packages\microsoft.netcore.ilasm\6.0.0\runtimes\ native\%userprofile%.nuget\packages\microsoft.netcore.ildasm\6.0.0\runtimes\native\
I have also included the 6.0.0 version of both programs in this book’s GitHub repo.

Now while it is true that few (if any!) programmers would choose to build an entire .NET application directly with CIL, CIL is still an extremely interesting intellectual pursuit. Simply put, the more you understand the grammar of CIL, the better able you are to move into the realm of advanced .NET development. By way of some concrete examples, individuals who possess an understanding of CIL are capable of the following:
•Disassembling an existing .NET assembly, editing the CIL code, and recompiling the updated code base into a modified .NET binary. For example, there are some scenarios where you might need to modify the CIL to interoperate with some advanced COM features.
•Building dynamic assemblies using the System.Reflection.Emit namespace. This API allows you to generate an in-memory .NET assembly, which can optionally be persisted to disk. This is a useful technique for the tool builders of the world who need to generate assemblies on the fly.
•Understanding aspects of the Common Type System (CTS) that are not supported by higher-level managed languages but do exist at the level of CIL. To be sure, CIL is the only .NET language that allows you to access every aspect of the CTS. For example, using raw CIL, you can define global-level members and fields (which are not permissible in C#).
Again, to be perfectly clear, if you choose not to concern yourself with the details of CIL code, you are still able to gain mastery of C# and the .NET base class libraries. In many ways, knowledge of CIL is analogous to a C (and C++) programmer’s understanding of assembly language. Those who know the ins and outs of the low-level “goo” can create rather advanced solutions for the task at hand and gain a
deeper understanding of the underlying programming (and runtime) environment. So, if you are up for the challenge, let’s begin to examine the details of CIL.

■ Note This chapter is not intended to be a comprehensive treatment of the syntax and semantics of CIL. We are really just skimming the surface. If you want (or need) to get deeper into CIL, consult the documentation.

Examining CIL Directives, Attributes, and Opcodes
When you begin to investigate low-level languages such as CIL, you are guaranteed to find new (and often intimidating sounding) names for familiar concepts. For example, the previous chapters showed CIL examples that contained the following set of items:

{new, public, this, base, get, set, explicit, unsafe, enum, operator, partial}

After reading the chapters leading up to this in this text, you understand them to be keywords of the C# language. However, if you look more closely at the members of this set, you might be able to see that while each item is indeed a C# keyword, it has radically different semantics. For example, the enum keyword defines a System.Enum-derived type, while the this and base keywords allow you to reference the current object or the object’s parent class, respectively. The unsafe keyword is used to establish a block of code that cannot be directly monitored by the CLR, while the operator keyword allows you to build a hidden (specially named) method that will be called when you apply a specific C# operator (such as the plus sign).
In stark contrast to a higher-level language such as C#, CIL does not just simply define a general set of keywords per se. Rather, the token set understood by the CIL compiler is subdivided into the following three broad categories based on semantics:
•CIL directives
•CIL attributes
•CIL operation codes (opcodes)
Each category of CIL token is expressed using a particular syntax, and the tokens are combined to build a valid .NET assembly.

The Role of CIL Directives
First up, there is a set of well-known CIL tokens that are used to describe the overall structure of a .NET assembly. These tokens are called directives. CIL directives are used to inform the CIL compiler how to define the namespaces(s), type(s), and member(s) that will populate an assembly.
Directives are represented syntactically using a single dot (.) prefix (e.g., .namespace, .class,
.publickeytoken, .method, .assembly, etc.). Thus, if your *.il file (the conventional extension for a file containing CIL code) has a single .namespace directive and three .class directives, the CIL compiler will generate an assembly that defines a single .NET Core namespace containing three .NET class types.

The Role of CIL Attributes
In many cases, CIL directives in and of themselves are not descriptive enough to fully express the definition of a given .NET type or type member. Given this fact, many CIL directives can be further specified with various CIL attributes to qualify how a directive should be processed. For example, the .class directive can be adorned with the public attribute (to establish the type visibility), the extends attribute (to explicitly specify the type’s base class), and the implements attribute (to list the set of interfaces supported by
the type).

■ Note Don’t confuse a .NET attribute (see Chapter 17) with that of a CIL attribute, which are two very different concepts.

The Role of CIL Opcodes
Once a .NET assembly, namespace, and type set have been defined in terms of CIL using various directives and related attributes, the final remaining task is to provide the type’s implementation logic. This is a job for operation codes, or simply opcodes. In the tradition of other low-level languages, many CIL opcodes tend to be cryptic and completely unpronounceable by us mere humans. For example, if you need to load a string variable into memory, you do not use a friendly opcode named LoadString but rather ldstr.
Now, to be fair, some CIL opcodes do map quite naturally to their C# counterparts (e.g., box, unbox, throw, and sizeof). As you will see, the opcodes of CIL are always used within the scope of a member’s
implementation, and unlike CIL directives, they are never written with a dot prefix.

The CIL Opcode/CIL Mnemonic Distinction
As just explained, opcodes such as ldstr are used to implement the members of a given type. However, tokens such as ldstr are CIL mnemonics for the actual binary CIL opcodes. To clarify the distinction, assume you have authored the following method in C# in a .NET Console Application named FirstSamples:

int Add(int x, int y)
{
return x + y;
}

The act of adding two numbers is expressed in terms of the CIL opcode 0X58. In a similar vein, subtracting two numbers is expressed using the opcode 0X59, and the act of allocating a new object on the managed heap is achieved using the 0X73 opcode. Given this reality, understand that the “CIL code” processed by a JIT compiler is nothing more than blobs of binary data.
Thankfully, for each binary opcode of CIL, there is a corresponding mnemonic. For example, the add mnemonic can be used rather than 0X58, sub rather than 0X59, and newobj rather than 0X73. Given this opcode/mnemonic distinction, realize that CIL decompilers such as ildasm.exe translate an assembly’s binary opcodes into their corresponding CIL mnemonics. For example, here would be the CIL presented by ildasm.exe for the previous C# Add() method (your exact output may differ based on your version of
.NET Core):

.method /06000002/ assembly hidebysig static int32 '<

$>g Add|0_0'(int32 x, int32 y) cil managed
// SIG: 00 02 08 08 08
{
// Method begins at RVA 0x2060
// Code size 9 (0x9)
.maxstack 2
.locals /11000001/ init (int32 V_0)
IL_0000: / 00 | / nop
IL_0001: / 02 | / ldarg.0
IL_0002: / 03 | / ldarg.1
IL_0003: / 58 | / add
IL_0004: / 0A | / stloc.0
IL_0005: / 2B | 00 / br.s IL_0007
IL_0007: / 06 | / ldloc.0
IL_0008: / 2A | / ret
} // end of method '$'::'<
$>g Add|0_0'

Unless you are building some extremely low-level .NET software (such as a custom managed compiler), you will never need to concern yourself with the literal numeric binary opcodes of CIL. For all practical purposes, when .NET programmers speak about “CIL opcodes,” they are referring to the set of friendly string token mnemonics (as I have done within this text and will do for the remainder of this chapter) rather than the underlying numerical values.

Pushing and Popping: The Stack-Based Nature of CIL
Higher-level .NET languages (such as C#) attempt to hide low-level CIL grunge from view as much as possible. One aspect of .NET development that is particularly well hidden is that CIL is a stack-based programming language. Recall from the examination of the collection namespaces (see Chapter 10) that the Stack class can be used to push a value onto a stack as well as pop the topmost value off the stack for use. Of course, CIL developers do not use an object of type Stack to load and unload the values to be evaluated; however, the same pushing and popping mindset still applies.
Formally speaking, the entity used to hold a set of values to be evaluated is termed the virtual execution stack. As you will see, CIL provides several opcodes that are used to push a value onto the stack; this process is termed loading. As well, CIL defines additional opcodes that transfer the topmost value on the stack into memory (such as a local variable) using a process termed storing.
In the world of CIL, it is impossible to access a point of data directly, including locally defined variables, incoming method arguments, or field data of a type. Rather, you are required to explicitly load the item onto the stack, only to then pop it off for later use (keep this point in mind, as it will help explain why a given block of CIL code can look a bit redundant).

■ Note Recall that CIL is not directly executed but compiled on demand. During the compilation of CIL code, many of these implementation redundancies are optimized away. furthermore, if you enable the code
optimization option for your current project (using the build tab of the Visual studio project properties window or adding true into the main property group of the project file), the compiler will also remove various CIL redundancies.

To understand how CIL leverages a stack-based processing model, consider a simple C# method, PrintMessage(), which takes no arguments and returns void. Within the implementation of this method, you will simply print the value of a local string variable to the standard output stream, like so:

void PrintMessage()
{
string myMessage = "Hello."; Console.WriteLine(myMessage);
}

If you were to examine how the C# compiler translates this method in terms of CIL, you would first find that the PrintMessage() method defines a storage slot for a local variable using the .locals directive. The local string is then loaded and stored in this local variable using the ldstr (load string) and stloc.0 opcodes (which can be read as “store the current value in a local variable at storage slot zero”).

The value (again, at index 0) is then loaded into memory using the ldloc.0 (“load the local argument at index 0”) opcode for use by the System.Console.WriteLine() method invocation (specified using the call opcode). Finally, the function returns via the ret opcode. Here is the (annotated) CIL code for the PrintMessage() method (note that I have removed the nop opcodes from this listing, for brevity):

.method assembly hidebysig static void
'<

$>g PrintMessage|0_1'() cil managed
{
// Method begins at RVA 0x2064
// Code size 13 (0xd)
.maxstack 1
// Define a local string variable (at index 0).
.locals init (string V_0)
// Load a string onto the stack with the value "Hello."
IL_0000: ldstr "Hello."
// Store string value on the stack in the local variable.
IL_0005: stloc.0
// Load the value at index 0.
IL_0006: ldloc.0
// Call method with current value.
IL_0007: call void System.Console::WriteLine(string) IL_000c: ret
} // end of method '$'::'<
$>g PrintMessage|0_1'

■ Note As you can see, CIL supports code comments using the double-slash syntax (as well as the /.../
syntax, for that matter). As in C#, code comments are completely ignored by the CIL compiler.

Now that you have the basics of CIL directives, attributes, and opcodes, let’s see a practical use of CIL programming, beginning with the topic of round-trip engineering.

Understanding Round-Trip Engineering
You are aware of how to use ildasm.exe to view the CIL code generated by the C# compiler (see Chapter 1). Once you have the CIL code at your disposal, you are free to edit and recompile the code base using the CIL compiler, ilasm.exe.
Formally speaking, this technique is termed round-trip engineering, and it can be useful under select circumstances, such as the following:
•You need to modify an assembly for which you no longer have the source code.
•You are working with a less-than-perfect .NET language compiler that has emitted ineffective (or flat-out incorrect) CIL code, and you want to modify the code base.
•You are constructing a COM interoperability library and want to account for some COM IDL attributes that have been lost during the conversion process (such as the COM [helpstring] attribute).

To illustrate the process of round-tripping, begin by creating a new C# .NET Core Console application named RoundTrip. Update the Program.cs file to the following:

// A simple C# console app. Console.WriteLine("Hello CIL code!"); Console.ReadLine();

Compile your program using the .NET Core CLI.dotnet build

■ Note Recall from Chapter 1 that all .NET Core assemblies (class libraries or console apps) by default are compiled to assemblies that have a *.dll extension and are executed using dotnet.exe. New in .NET Core
3.0 (and newer), the dotnet.exe file is copied into the output directory and renamed to match the assembly name. so, while it looks like your project was compiled to RoundTrip.exe, it was compiled to RoundTrip. dll with dotnet.exe copied to RoundTrip.exe along with the required command-line arguments needed to execute Roundtrip.dll. If you publish as a single file (covered in Chapter 16), then RoundTrip.exe contains even more than just your code.

Next execute ildasm.exe against RoundTrip.dll using the following command (executed from the solution folder level):

ildasm /all /METADATA /out=.\RoundTrip\RoundTrip.il .\RoundTrip\bin\Debug\net6.0\ RoundTrip.dll

The previous command output just about everything contained in the assembly, including the file headers, hex commands as comments, all metadata, and much more. If you want a more concise file to work with when examining IL code, you can drop the /all and /METADATA options. However, for these examples, you need all of the extra information.

■Note ildasm.exe will also generate a *.res file when dumping the contents of an assembly to file. These resource files can be ignored (and deleted) throughout this chapter, as you will not be using them. This file contains some low-level CLR security information (among other things).

Now you can view RoundTrip.il using Visual Studio, Visual Studio Code, or your text editor of choice.
First, notice that the *.il file opens by declaring each externally referenced assembly that the current assembly is compiled against. If your class library used additional types within other referenced assemblies (beyond System.Runtime and System.Console), you would find additional .assembly extern directives.

.assembly extern System.Runtime
{
.publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )
.ver 6:0:0:0
}
.assembly extern System.Console
{
.publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )
.ver 6:0:0:0
}

Next, you find the formal definition of your RoundTrip.dll assembly, described using various CIL directives (such as .module, .imagebase, etc.).

.assembly RoundTrip
{
...
.hash algorithm 0x00008004
.ver 1:0:0:0
}
.module RoundTrip.dll
.imagebase 0x00400000
.file alignment 0x00000200
.stackreserve 0x00100000
.subsystem 0x0003 // WINDOWS_CUI
.corflags 0x00000001 // ILONLY

After documenting the externally referenced assemblies and defining the current assembly, you find a definition of the Program type, created from the top-level statements. Note that the .class directive has
various attributes (many of which are optional) such as extends, shown here, which marks the base class of
the type:

.class private abstract auto ansi sealed beforefieldinit '$' extends [System.Runtime]System.Object
{ ... }

The bulk of the CIL code represents the implementation of the class’s default constructor and the autogenerated Main() method, both of which are defined (in part) with the .method directive. Once the members have been defined using the correct directives and attributes, they are implemented using various opcodes.

.method private hidebysig static void '

$'(string[] args) cil managed
{
.entrypoint
// Code size 18 (0x12)
.maxstack 8
IL_0000: ldstr "Hello CIL code!"
IL_0005: call void [System.Console]System.Console::WriteLine(string) IL_000a: nop
IL_000b: call string [System.Console]System.Console::ReadLine() IL_0010: pop
IL_0011: ret
} // end of method '$'::'
$'

It is critical to understand that when interacting with .NET Core types (such as System.Console) in CIL, you will always need to use the type’s fully qualified name. Furthermore, the type’s fully qualified name must always be prefixed with the friendly name of the defining assembly (in square brackets), as in the following two lines from the generated Main() method:

IL_0005: call void [System.Console]System.Console::WriteLine(string)
IL_000b: call string [System.Console]System.Console::ReadLine()

The Role of CIL Code Labels
One thing you certainly have noticed is that each line of implementation code is prefixed with a token of the form IL_XXX: (e.g., IL_0000:, IL_0001:, etc.). These tokens are called code labels and may be named in any manner you choose (provided they are not duplicated within the same member scope). When you dump an assembly to file using ildasm.exe, it will automatically generate code labels that follow an IL_XXX: naming convention. However, you may change them to reflect a more descriptive marker. Here is an example:
.method private hidebysig static void Main(string[] args) cil managed
{
.entrypoint
.maxstack 8
Load_String: ldstr "Hello CIL code!"
PrintToConsole: call void [System.Console]System.Console::WriteLine(string) Nothing_2: nop
WaitFor_KeyPress: call string [System.Console]System.Console::ReadLine() RemoveValueFromStack: pop
Leave_Function: ret
}

The truth of the matter is that most code labels are completely optional. The only time code labels are truly mandatory is when you are authoring CIL code that makes use of various branching or looping
constructs, as you specify where to direct the flow of logic via these code labels. For the current example, you can remove these autogenerated labels altogether with no ill effect, like so:
.method private hidebysig static void Main(string[] args) cil managed
{
.entrypoint
.maxstack 8
ldstr "Hello CIL code!"
call void [System.Console]System.Console::WriteLine(string) nop
call string [System.Console]System.Console::ReadLine() pop
ret
}

Interacting with CIL: Modifying an .il File
Now that you have a better understanding of how a basic CIL file is composed, let’s complete the round- tripping experiment. The goal here is quite simple: change the message that is output to the console. You can do more, such as add assembly references or create new classes and methods, but we will keep it simple.
To make the change, you need to alter the current implementation of the top-level statements, created as the

$ method. Locate this method within the .il file and change the message to “Hello from altered CIL Code!”
In effect, you have just updated the CIL code to correspond to the following C# class definition:

static void Main(string[] args)
{
Console.WriteLine("Hello from altered CIL code!"); Console.ReadLine();
}

There are two ways to create compile .NET assemblies using an .il file. Using the IL project type provides more flexibility but is a bit more involved. The second simply uses ILASM.EXE to create a .dll file from the IL file. We will explore using ILASM.EXE first.

Compiling CIL Code with ILASM.EXE
Start by creating a new directory on your machine (in the samples on GitHub, I named the new directory RoundTrip2). In this directory, copy in the updated RoundTrip.il file. Also copy the RoundTrip. runtimeconfig.json file from the RoundTrip\bin\Debug.net6.0 folder. This file is needed for executables created using ILASM.EXE to configure the target framework moniker and the target framework. For reference, the contents of the file are listed here:

{
"runtimeOptions": { "tfm": "net6.0", "framework": {
"name": "Microsoft.NETCore.App", "version": "6.0.0-preview.3.21201.4"
}
}
}

Finally, compile the assembly with the following command from the RoundTrip2 directory (update the path to ILASM.EXE as necessary):

....\ilasm /DLL RoundTrip.il /X64

To execute the program, use the CLI, like this:

dotnet RoundTrip.dll

Sure enough, you will see the updated message displaying in the console window.

Compiling CIL Code with Microsoft.NET.Sdk.il Projects
As you just saw, compiling IL with ILASM.EXE is a bit limited. A much more powerful way is to create a project that uses the Microsoft.NET.Sdk.IL project type. Unfortunately, at the time of this writing, this project type is not included in the standard project templates, so manual intervention is required. Begin by creating a new directory named RoundTrip3 and copying the modified RoundTrip.il file into the new directory.

■ Note At the time of this writing, Visual studio does directly support the *.ilproj project type. While there are some extensions in the marketplace, I can’t recommend for or against using them. Visual studio Code supports all of the code in this section.

In this directory, create a global.json file. The global.json file applies to the current directory and all subdirectories below the file. It is used to define which SDK version you will use when running .NET Core CLI commands. Update the files to the following:

{
"msbuild-sdks": { "Microsoft.NET.Sdk.IL": "6.0.0"
}
}

The next step is to create the project file. Create a file named RoundTrip.ilproj and update it to the following:



Exe
net6.0
6.0.0
false

Finally, copy in your updated RoundTrip.il file into the directory. Compile the assembly using the
.NET Core CLI:

dotnet build

You will find the resulting files in the usual bin\debug\net6.0 folder. At this point, you can run your new application by executing RoundTrip.exe, just as if you built it using a standard C# console application template.
In addition to a better resulting experience, the IL project can take advantage of producing single-file assemblies, as was covered in Chapter 16. Update the project file to the following:



Exe
net6.0
6.0.0-preview.3.21201.4
false
true
true
win-x64
true
true

Now you can publish as a stand-alone file just like a C# project. Use the publish command to see this in action:

dotnet publish -r win-x64 -p:PublishSingleFile=true -c release -o singlefile --self- contained true

While the output of this simple example is not all that spectacular, it does illustrate one practical use of programming in CIL: round-tripping.

Understanding CIL Directives and Attributes
Now that you have seen how to convert .NET Core assemblies into IL and compile IL into assemblies, you can get down to the business of checking out the syntax and semantics of CIL itself. The next sections will walk you through the process of authoring a custom namespace containing a set of types. However, to keep things simple, these types will not contain any implementation logic for their members (yet). After you understand how to create empty types, you can then turn your attention to the process of defining “real” members using CIL opcodes.

Specifying Externally Referenced Assemblies in CIL
In a new directory named CILTypes, copy the global.json file from the previous example. Create a new project file named CILTypes.ilproj, and update it to the following:



net6.0
6.0.0
false

Next, create a new file named CILTypes.il using your editor of choice. The first task a CIL project will require is to list the set of external assemblies used by the current assembly. For this example, you will only use types found within System.Runtime.dll. To do so, the .assembly directive will be qualified using the external attribute. When you are referencing a strongly named assembly, such as System.Runtime.dll, you will want to specify the .publickeytoken and .ver directives as well, like so:

.assembly extern System.Runtime
{
.publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )
.ver 6:0:0:0
}
.assembly extern System.Runtime.Extensions
{
.publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )
.ver 6:0:0:0
}
.assembly extern mscorlib

{
.publickeytoken = (B7 7A 5C 56 19 34 E0 89)
.ver 6:0:0:0
}

Defining the Current Assembly in CIL
The next order of business is to define the assembly you are interested in building using the .assembly directive. At the simplest level, an assembly can be defined by specifying the friendly name of the binary, like so:

// Our assembly.
.assembly CILTypes { }

While this indeed defines a new .NET Core assembly, you will typically place additional directives within the scope of the assembly declaration. For this example, update your assembly definition to include a version number of 1.0.0.0 using the .ver directive (note that each numerical identifier is separated by colons, not the C#-centric dot notation), as follows:

// Our assembly.
.assembly CILTypes
{
.ver 1:0:0:0
}

Given that the CILTypes assembly is a single-file assembly, you will finish up the assembly definition using the following single .module directive, which marks the official name of your .NET binary, CILTypes.dll:

.assembly CILTypes
{
.ver 1:0:0:0
}
// The module of our single-file assembly.
.module CILTypes.dll

In addition to .assembly and .module are CIL directives that further qualify the overall structure of the
.NET binary you are composing. Table 18-1 lists a few of the more common assembly-level directives.

Table 18-1. Additional Assembly-Centric Directives

Directive Meaning in Life
.mresources If your assembly uses internal resources (such as bitmaps or string tables), this directive is used to identify the name of the file that contains the resources to be embedded.
.subsystem This CIL directive is used to establish the preferred UI that the assembly wants to execute within. For example, a value of 2 signifies that the assembly should run within a GUI application, whereas a value of 3 denotes a console executable.

Defining Namespaces in CIL
Now that you have defined the look and feel of your assembly (and the required external references), you can create a .NET Core namespace (MyNamespace) using the .namespace directive, like so:

// Our assembly has a single namespace.
.namespace MyNamespace {}

Like C#, CIL namespace definitions can be nested within further namespaces. There is no need to define a root namespace here; however, for the sake of argument, assume you want to create the following root namespace named MyCompany:

.namespace MyCompany
{
.namespace MyNamespace {}
}

Like C#, CIL allows you to define a nested namespace as follows:

// Defining a nested namespace.
.namespace MyCompany.MyNamespace {}

Defining Class Types in CIL
Empty namespaces are not remarkably interesting, so let’s now check out the process of defining a class type using CIL. Not surprisingly, the .class directive is used to define a new class. However, this simple directive can be adorned with numerous additional attributes, to further qualify the nature of the type. To illustrate, add a public class to your namespace named MyBaseClass. As in C#, if you do not specify an explicit base class, your type will automatically be derived from System.Object.

.namespace MyNamespace
{
// System.Object base class assumed.
.class public MyBaseClass {}
}

When you are building a class type that derives from any class other than System.Object, you use the extends attribute. Whenever you need to reference a type defined within the same assembly, CIL demands that you also use the fully qualified name (however, if the base type is within the same assembly, you can omit the assembly’s friendly name prefix). Therefore, the following attempt to extend MyBaseClass results in a compiler error:

// This will not compile!
.namespace MyNamespace
{
.class public MyBaseClass {}

.class public MyDerivedClass
extends MyBaseClass {}
}

To correctly define the parent class of MyDerivedClass, you must specify the full name of MyBaseClass
as follows:

// Better!
.namespace MyNamespace
{
.class public MyBaseClass {}

.class public MyDerivedClass
extends MyNamespace.MyBaseClass {}
}

In addition to the public and extends attributes, a CIL class definition may take numerous additional qualifiers that control the type’s visibility, field layout, and so on. Table 18-2 illustrates some (but not all) of the attributes that may be used in conjunction with the .class directive.

Table 18-2. Various Attributes Used in Conjunction with the .class Directive

Attributes Meaning in Life
public, private, nested assembly, nested famandassem, nested family, nested famorassem, nested public, nested private CIL defines various attributes that are used to specify the visibility of a given type. As you can see, raw CIL offers numerous
possibilities other than those offered by C#. Refer to ECMA 335 for details if you are interested.
abstract, sealed These two attributes may be tacked onto a .class directive to define an abstract class or sealed class, respectively.
auto, sequential, explicit These attributes are used to instruct the CLR how to lay out field data in memory. For class types, the default layout flag (auto) is appropriate. Changing this default can be helpful if you need to use P/Invoke to call into unmanaged C code.
extends, implements These attributes allow you to define the base class of a type (via
extends) or implement an interface on a type (via implements).

Defining and Implementing Interfaces in CIL
As odd as it might seem, interface types are defined in CIL using the .class directive. However, when the .class directive is adorned with the interface attribute, the type is realized as a CTS interface type.
Once an interface has been defined, it may be bound to a class or structure type using the CIL implements
attribute, like so:

.namespace MyNamespace
{
// An interface definition.
.class public interface IMyInterface {}

// A simple base class.
.class public MyBaseClass {}

// MyDerivedClass now implements IMyInterface,

// and extends MyBaseClass.
.class public MyDerivedClass extends MyNamespace.MyBaseClass
implements MyNamespace.IMyInterface {}
}

■ Note The extends clause must precede the implements clause. As well, the implements clause can incorporate a comma-separated list of interfaces.

As you recall from Chapter 10, interfaces can function as the base interface to other interface types to build interface hierarchies. However, contrary to what you might be thinking, the extends attribute cannot be used to derive interface A from interface B. The extends attribute is used only to qualify a type’s base class. When you want to extend an interface, you will use the implements attribute yet again. Here is an example:

// Extending interfaces in terms of CIL.
.class public interface IMyInterface {}

.class public interface IMyOtherInterface implements MyNamespace.IMyInterface {}

Defining Structures in CIL
The .class directive can be used to define a CTS structure if the type extends System.ValueType. As well, the .class directive must be qualified with the sealed attribute (given that structures can never be a base structure to other value types). If you attempt to do otherwise, ilasm.exe will issue a compiler error.

// A structure definition is always sealed.
.class public sealed MyStruct
extends [System.Runtime]System.ValueType{}

Do be aware that CIL provides a shorthand notation to define a structure type. If you use the value attribute, the new type will derive the type from [System.Runtime]System.ValueType automatically. Therefore, you could define MyStruct as follows:

// Shorthand notation for declaring a structure.
.class public sealed value MyStruct{}

Defining Enums in CIL
.NET Core enumerations (as you recall) derive from System.Enum, which is a System.ValueType (and therefore must also be sealed). When you want to define an enum in terms of CIL, simply extend [System. Runtime]System.Enum, like so:

// An enum.
.class public sealed MyEnum
extends [System.Runtime]System.Enum{}

Like a structure definition, enumerations can be defined with a shorthand notation using the enum
attribute. Here is an example:

// Enum shorthand.
.class public sealed enum MyEnum{}

You will see how to specify the name-value pairs of an enumeration in just a moment.

Defining Generics in CIL
Generic types also have a specific representation in the syntax of CIL. Recall from Chapter 10 that a given generic type or generic member may have one or more type parameters. For example, the List type has a single
type parameter, while Dictionary<TKey, TValue> has two. In terms of CIL, the number of type parameters is specified using a backward-leaning single tick (`), followed by a numerical value representing the number of type parameters. Like C#, the actual value of the type parameters is encased within angled brackets.

■ Note on Us keyboards, you can usually find the ` character on the key above the Tab key (and to the left of the 1 key).

For example, assume you want to create a List variable, where T is of type System.Int32. In C#, you would type the following:
void SomeMethod()
{
List myInts = new List();
}
In CIL, you would author the following (which could appear in any CIL method scope):

// In C#: List myInts = new List();
newobj instance void class [System.Collections] System.Collections.Generic.List1::.ctor() Notice that this generic class is defined as List1, as List has a single type parameter.
However, if you needed to define a Dictionary<string, int> type, you would do so as follows:

// In C#: Dictionary<string, int> d = new Dictionary<string, int>();

newobj instance void class [System.Collections] System.Collections.Generic.Dictionary`2<string,int32>
::.ctor()

As another example, if you have a generic type that uses another generic type as a type parameter, you would author CIL code such as the following:

// In C#: List<List> myInts = new List<List>();
newobj instance void class [mscorlib]
System.Collections.Generic.List1<class [System.Collections] System.Collections.Generic.List1>
::.ctor()

Compiling the CILTypes.il File
Even though you have not yet added any members or implementation code to the types you have defined, you are able to compile this *.il file into a .NET Core DLL assembly (which you must do, as you have not specified a Main() method). Open a command prompt and enter the following command:

dotnet build

After you have done so, you can now open your compiled assembly into ildasm.exe to verify the creation of each type. To understand how to populate a type with content, you first need to examine the fundamental data types of CIL.

.NET Base Class Library, C#, and CIL Data Type Mappings
Table 18-3 illustrates how a .NET base class type maps to the corresponding C# keyword and how each C# keyword maps into raw CIL. As well, Table 18-3 documents the shorthand constant notations used for each CIL type. As you will see in just a moment, these constants are often referenced by numerous CIL opcodes.

Table 18-3. Mapping .NET Base Class Types to C# Keywords and C# Keywords to CIL

.NET Core Base Class Type C# Keyword CIL Representation CIL Constant Notation
System.SByte sbyte int8 I1
System.Byte byte unsigned int8 U1
System.Int16 short int16 I2
System.UInt16 ushort unsigned int16 U2
System.Int32 int int32 I4
System.UInt32 uint unsigned int32 U4
System.Int64 long int64 I8
System.UInt64 ulong unsigned int64 U8
System.Char char char CHAR
System.Single float float32 R4
System.Double double float64 R8
System.Boolean bool bool BOOLEAN
System.String string string N/A
System.Object object object N/A
System.Void void void VOID

■ Note The System.IntPtr and System.UIntPtr types map to native int and native unsigned int
(many CoM interoperability and p/Invoke scenarios use these extensively).

Defining Type Members in CIL
As you are already aware, .NET types may support various members. Enumerations have some set of name- value pairs. Structures and classes may have constructors, fields, methods, properties, static members, and so on. Over the course of this book’s first 18 chapters, you have already seen partial CIL definitions for the items previously mentioned, but nevertheless, here is a quick recap of how various members map to CIL primitives.

Defining Field Data in CIL
Enumerations, structures, and classes can all support field data. In each case, the .field directive will be used. For example, let’s breathe some life into the skeleton MyEnum enumeration and define the following three name-value pairs (note the values are specified within parentheses):

.class public sealed enum MyEnum
{
.field public static literal valuetype MyNamespace.MyEnum A = int32(0)
.field public static literal valuetype MyNamespace.MyEnum B = int32(1)
.field public static literal valuetype MyNamespace.MyEnum C = int32(2)
}

Fields that reside within the scope of a .NET Core System.Enum-derived type are qualified using the static and literal attributes. As you would guess, these attributes set up the field data to be a fixed value accessible from the type itself (e.g., MyEnum.A).

■ Note The values assigned to an enum value may also be in hexadecimal with a 0x prefix.

Of course, when you want to define a point of field data within a class or structure, you are not limited to a point of public static literal data. For example, you could update MyBaseClass to support two points of private, instance-level field data, set to default values.

.class public MyBaseClass
{
.field private string stringField = "hello!"
.field private int32 intField = int32(42)
}

As in C#, class field data will automatically be initialized to an appropriate default value. If you want to allow the object user to supply custom values at the time of creation for each of these points of private field data, you (of course) need to create custom constructors.

Defining Type Constructors in CIL
The CTS supports both instance-level and class-level (static) constructors. In terms of CIL, instance-level constructors are represented using the .ctor token, while a static-level constructor is expressed via .cctor (class constructor). Both CIL tokens must be qualified using the rtspecialname (return type special name) and specialname attributes. Simply put, these attributes are used to identify a specific CIL token that can be treated in unique ways by a given .NET language. For example, in C#, constructors do not define a return type; however, in terms of CIL, the return value of a constructor is indeed void.

.class public MyBaseClass
{
.field private string stringField
.field private int32 intField

.method public hidebysig specialname rtspecialname instance void .ctor(string s, int32 i) cil managed
{
// TODO: Add implementation code...
}
}

Note that the .ctor directive has been qualified with the instance attribute (as it is not a static constructor). The cil managed attributes denote that the scope of this method contains CIL code, rather than unmanaged code, which may be used during platform invocation requests.

Defining Properties in CIL
Properties and methods also have specific CIL representations. By way of an example, if MyBaseClass were updated to support a public property named TheString, you would author the following CIL (note again the use of the specialname attribute):

.class public MyBaseClass
{
...
.method public hidebysig specialname
instance string get_TheString() cil managed
{
// TODO: Add implementation code...
}

.method public hidebysig specialname
instance void set_TheString(string 'value') cil managed
{
// TODO: Add implementation code...
}

.property instance string TheString()
{
.get instance string MyNamespace.MyBaseClass::get_TheString()
.set instance void

MyNamespace.MyBaseClass::set_TheString(string)
}
}

In terms of CIL, a property maps to a pair of methods that take get and set prefixes. The .property directive makes use of the related .get and .set directives to map property syntax to the correct “specially named” methods.

■ Note Notice that the incoming parameter to the set method of a property is placed in single quotation marks, which represents the name of the token to use on the right side of the assignment operator within the method scope.

Defining Member Parameters
In a nutshell, specifying arguments in CIL is (more or less) identical to doing so in C#. For example, each argument is defined by specifying its data type, followed by the parameter name. Furthermore, like C#, CIL provides a way to define input, output, and pass-by-reference parameters. As well, CIL allows you to define a parameter array argument (aka the C# params keyword), as well as optional parameters.
To illustrate the process of defining parameters in raw CIL, assume you want to build a method that takes an int32 (by value), an int32 (by reference), an [mscorlib]System.Collection.ArrayList, and a single output parameter (of type int32). In terms of C#, this method would look something like the following:

public static void MyMethod(int inputInt,
ref int refInt, ArrayList ar, out int outputInt)
{
outputInt = 0; // Just to satisfy the C# compiler...
}

If you were to map this method into CIL terms, you would find that C# reference parameters are marked with an ampersand (&) suffixed to the parameter’s underlying data type (int32&).
Output parameters also use the & suffix, but they are further qualified using the CIL [out] token. Also notice that if the parameter is a reference type (in this case, the [mscorlib]System.Collections.ArrayList type), the class token is prefixed to the data type (not to be confused with the .class directive!).

.method public hidebysig static void MyMethod(int32 inputInt, int32& refInt,
class [System.Runtime.Extensions]System.Collections.ArrayList ar, [out] int32& outputInt) cil managed
{
...
}

Examining CIL Opcodes
The final aspect of CIL code you will examine in this chapter has to do with the role of various operational codes (opcodes). Recall that an opcode is simply a CIL token used to build the implementation logic for a given member. The complete set of CIL opcodes (which is large) can be grouped into the following broad categories:
• Opcodes that control program flow
• Opcodes that evaluate expressions
• Opcodes that access values in memory (via parameters, local variables, etc.)
To provide some insight to the world of member implementation via CIL, Table 18-4 defines some of the more useful opcodes that are related to member implementation logic, grouped by related functionality.

Table 18-4. Various Implementation-Specific CIL Opcodes

Opcodes Meaning in Life
add, sub, mul, div, rem These CIL opcodes allow you to add, subtract, multiply, and divide two values (rem returns the remainder of a division operation).
and, or, not, xor These CIL opcodes allow you to perform bit-wise operations on two values.
ceq, cgt, clt These CIL opcodes allow you to compare two values on the stack in various manners. Here are some examples:
ceq: Compare for equality cgt: Compare for greater than clt: Compare for less than
box, unbox These CIL opcodes are used to convert between reference types and value types.
Ret This CIL opcode is used to exit a method and return a value to the caller (if necessary).
beq, bgt, ble, blt, switch These CIL opcodes (in addition to many other related opcodes) are used to control branching logic within a method. Here are some examples:
beq: Break to code label if equal
bgt: Break to code label if greater than
ble: Break to code label if less than or equal to
blt: Break to code label if less than
All the branch-centric opcodes require that you specify a CIL code label to jump to if the result of the test is true.
Call This CIL opcode is used to call a member on a given type.
newarr, newobj These CIL opcodes allow you to allocate a new array or new object type into memory (respectively).

The next broad category of CIL opcodes (a subset of which is shown in Table 18-5) is used to load (push) arguments onto the virtual execution stack. Note how these load-specific opcodes take a ld (load) prefix.

Table 18-5. The Primary Stack-Centric Opcodes of CIL

Opcode Meaning in Life
ldarg (with numerous variations) Loads a method’s argument onto the stack. In addition to the general ldarg (which works in conjunction with a given index that identifies the argument), there are numerous other variations. For example, ldarg opcodes that have a numerical suffix (ldarg.0) hard-code which argument to load. As well, variations of the ldarg opcode allow you to hard-code the data type using the CIL constant notation shown in Table 18-4 (ldarg_I4, for an int32), as well as the data type and value (ldarg_I4_5, to load an int32 with the value of 5).
ldc (with numerous variations) Loads a constant value onto the stack.
ldfld (with numerous variations) Loads the value of an instance-level field onto the stack.
ldloc (with numerous variations) Loads the value of a local variable onto the stack.
Ldobj Obtains all the values gathered by a heap-based object and places them on the stack.
Ldstr Loads a string value onto the stack.

In addition to the set of load-specific opcodes, CIL provides numerous opcodes that explicitly pop the topmost value off the stack. As shown over the first few examples in this chapter, popping a value off the stack typically involves storing the value into temporary local storage for further use (such as a parameter for an upcoming method invocation). Given this, note how many opcodes that pop the current value off the virtual execution stack take an st (store) prefix. Table 18-6 hits the highlights.

Table 18-6. Various Pop-Centric Opcodes

Opcode Meaning in Life
Pop Removes the value currently on top of the evaluation stack but does not bother to store the value
Starg Stores the value on top of the stack into the method argument at a specified index
stloc (with numerous variations) Pops the current value from the top of the evaluation stack and stores it in a local variable list at a specified index
Stobj Copies a value of a specified type from the evaluation stack into a supplied memory address
Stsfld Replaces the value of a static field with a value from the evaluation stack

Do be aware that various CIL opcodes will implicitly pop values off the stack to perform the task at hand. For example, if you are attempting to subtract two numbers using the sub opcode, it should be clear that sub will have to pop off the next two available values before it can perform the calculation. Once the calculation is complete, the result of the value (surprise, surprise) is pushed onto the stack once again.

The .maxstack Directive
When you write method implementations using raw CIL, you need to be mindful of a special directive named .maxstack. As its name suggests, .maxstack establishes the maximum number of variables that may be pushed onto the stack at any given time during the execution of the method. The good news is that the
.maxstack directive has a default value (8), which should be safe for a vast majority of methods you might be authoring. However, if you want to be explicit, you can manually calculate the number of local variables on the stack and define this value explicitly, like so:

.method public hidebysig instance void Speak() cil managed
{
// During the scope of this method, exactly
// 1 value (the string literal) is on the stack.
.maxstack 1
ldstr "Hello there..."
call void [mscorlib]System.Console::WriteLine(string) ret
}

Declaring Local Variables in CIL
Let’s first check out how to declare a local variable. Assume you want to build a method in CIL named MyLocalVariables() that takes no arguments and returns void. Within the method, you want to define three local variables of types System.String, System.Int32, and System.Object. In C#, this member would appear as follows (recall that locally scoped variables do not receive a default value and should be set to an initial state before further use):

public static void MyLocalVariables()
{
string myStr = "CIL code is fun!"; int myInt = 33;
object myObj = new object();
}

If you were to construct MyLocalVariables() directly in CIL, you could author the following:

.method public hidebysig static void MyLocalVariables() cil managed
{
.maxstack 8
// Define three local variables.
.locals init (string myStr, int32 myInt, object myObj)
// Load a string onto the virtual execution stack.
ldstr "CIL code is fun!"

// Pop off current value and store in local variable [0].
stloc.0

// Load a constant of type "i4"
// (shorthand for int32) set to the value 33.
ldc.i4.s 33
// Pop off current value and store in local variable [1].
stloc.1

// Create a new object and place on stack.
newobj instance void [mscorlib]System.Object::.ctor()
// Pop off current value and store in local variable [2].
stloc.2 ret
}

The first step taken to allocate local variables in raw CIL is to use the .locals directive, which is paired with the init attribute. Each variable is identified by its data type and an optional variable name. After the local variables have been defined, you load a value onto the stack (using the various load-centric opcodes) and store the value within the local variable (using the various storage-centric opcodes).

Mapping Parameters to Local Variables in CIL
You have already seen how to declare local variables in raw CIL using the .locals init directive; however, you have yet to see exactly how to map incoming parameters to local methods. Consider the following static C# method:

public static int Add(int a, int b)
{
return a + b;
}

This innocent-looking method has a lot to say in terms of CIL. First, the incoming arguments (a and b) must be pushed onto the virtual execution stack using the ldarg (load argument) opcode. Next, the add opcode will be used to pop the next two values off the stack and find the summation and store the value on the stack yet again. Finally, this sum is popped off the stack and returned to the caller via the ret opcode. If you were to disassemble this C# method using ildasm.exe, you would find numerous additional tokens injected by the build process, but the crux of the CIL code is quite simple.

.method public hidebysig static int32 Add(int32 a, int32 b) cil managed
{
.maxstack 2
ldarg.0 // Load "a" onto the stack. ldarg.1 // Load "b" onto the stack. add // Add both values.
ret
}

The Hidden this Reference
Notice that the two incoming arguments (a and b) are referenced within the CIL code using their indexed position (index 0 and index 1), given that the virtual execution stack begins indexing at position 0.
One thing to be mindful of when you are examining or authoring CIL code is that every nonstatic method that takes incoming arguments automatically receives an implicit additional parameter, which is a reference to the current object (like the C# this keyword). Given this, if the Add() method were defined as nonstatic, like so:

// No longer static!
public int Add(int a, int b)
{
return a + b;
}

Then the incoming a and b arguments are loaded using ldarg.1 and ldarg.2 (rather than the expected ldarg.0 and ldarg.1 opcodes). Again, the reason is that slot 0 contains the implicit this reference. Consider the following pseudocode:

// This is JUST pseudocode!
.method public hidebysig static int32 AddTwoIntParams( MyClass_HiddenThisPointer this, int32 a, int32 b) cil managed
{
ldarg.0 // Load MyClass_HiddenThisPointer onto the stack. ldarg.1 // Load "a" onto the stack.
ldarg.2 // Load "b" onto the stack.
...
}

Representing Iteration Constructs in CIL
Iteration constructs in the C# programming language are represented using the for, foreach, while, and do
keywords, each of which has a specific representation in CIL. Consider the following classic for loop:

public static void CountToTen()
{
for(int i = 0; i < 10; i++)
{
}
}

Now, as you may recall, the br opcodes (br, blt, etc.) are used to control a break in flow when some condition has been met. In this example, you have set up a condition in which the for loop should break out of its cycle when the local variable i is equal to or greater than the value of 10. With each pass, the value of 1 is added to i, at which point the test condition is yet again evaluated.
Also recall that when you use any of the CIL branching opcodes, you will need to define a specific code label (or two) that marks the location to jump to when the condition is indeed true. Given these points, ponder the following (edited) CIL code generated via ildasm.exe (including the autogenerated code labels):

.method public hidebysig static void CountToTen() cil managed
{
.maxstack 2

.locals init (int32 V_0, bool V_1)
IL_0000: ldc.i4.0 // Load this value onto the stack. IL_0001: stloc.0 // Store this value at index "0". IL_0002: br.s IL_0007 // Jump to IL_0008.
IL_0003: ldloc.0 // Load value of variable at index 0. IL_0004: ldc.i4.1 // Load the value "1" on the stack. IL_0005: add // Add current value on the stack at index 0. IL_0006: stloc.0
IL_0007: ldloc.0 // Load value at index "0".
IL_0008: ldc.i4.s 10 // Load value of "10" onto the stack. IL_0009: clt // check less than value on the stack IL_000a: stloc.1 // Store result at index "1"
IL_000b: ldloc.1 // Load value at index "1"
IL_000c: brtrue.s IL_0003 // if true jump back to IL_0003 IL_000d: ret
}

In a nutshell, this CIL code begins by defining the local int32 and loading it onto the stack. At this point, you jump back and forth between code labels IL_0008 and IL_0004, each time bumping the value of i by 1 and testing to see whether i is still less than the value 10. If so, you exit the method.

The Final Word on CIL
Now that you see the process for creating an executable from an *.IL file, you are probably thinking “that is an awful lot of work” and then wondering “what’s the benefit?” For the vast majority, you will never create a .NET Core executable from IL. However, being able to understand IL can be helpful if you are trying to dig into an assembly that you do not have the source code for.
There are also commercial projects that can take a .NET assembly and reverse engineer it into source code. If you have ever used one of these tools, now you know how they work!

Understanding Dynamic Assemblies
To be sure, the process of building a complex .NET application in CIL would be quite the labor of love. On the one hand, CIL is an extremely expressive programming language that allows you to interact with all the programming constructs allowed by the CTS. On the other hand, authoring raw CIL is tedious, error- prone, and painful. While it is true that knowledge is power, you might indeed wonder just how important it is to commit the laws of CIL syntax to memory. The answer is “it depends.” To be sure, most of your .NET programming endeavors will not require you to view, edit, or author CIL code. However, with the CIL
primer behind you, you are now ready to investigate the world of dynamic assemblies (as opposed to static assemblies) and the role of the System.Reflection.Emit namespace.
The first question you may have is “What exactly is the difference between static and dynamic assemblies?” By definition, static assemblies are .NET binaries loaded directly from disk storage, meaning they are located somewhere in a physical file (or possibly a set of files in the case of a multifile assembly) at the time the CLR requests them. As you might guess, every time you compile your C# source code, you end up with a static assembly.
A dynamic assembly, on the other hand, is created in memory, on the fly, using the types provided by the System.Reflection.Emit namespace. The System.Reflection.Emit namespace makes it possible to create an assembly and its modules, type definitions, and CIL implementation logic at runtime. After
you have done so, you are then free to save your in-memory binary to disk. This, of course, results in a new

static assembly. To be sure, the process of building a dynamic assembly using the System.Reflection.Emit
namespace does require some level of understanding regarding the nature of CIL opcodes.
Although creating dynamic assemblies is an advanced (and uncommon) programming task, they can be useful under various circumstances. Here is an example:
• You are building a .NET programming tool that needs to generate assemblies on demand based on user input.
• You are building a program that needs to generate proxies to remote types on the fly, based on the obtained metadata.
• You want to load a static assembly and dynamically insert new types into the binary image.
Let’s check out the types within System.Reflection.Emit.

Exploring the System.Reflection.Emit Namespace
Creating a dynamic assembly requires you to have some familiarity with CIL opcodes, but the types of the System.Reflection.Emit namespace hide the complexity of CIL as much as possible. For example,
rather than specifying the necessary CIL directives and attributes to define a class type, you can simply use the TypeBuilder class. Likewise, if you want to define a new instance-level constructor, you have no need to emit the specialname, rtspecialname, or .ctor token; rather, you can use the ConstructorBuilder.
Table 18-7 documents the key members of the System.Reflection.Emit namespace.

Table 18-7. Select Members of the System.Reflection.Emit Namespace

Members Meaning in Life

AssemblyBuilder Used to create an assembly (.dll or .exe) at runtime. .exes must
call the ModuleBuilder.SetEntryPoint() method to set the method that is the entry point to the module. If no entry point is specified, a
.dll will be generated.
ModuleBuilder Used to define the set of modules within the current assembly.
EnumBuilder Used to create a .NET enumeration type.
TypeBuilder May be used to create classes, interfaces, structures, and delegates within a module at runtime.

MethodBuilder LocalBuilder PropertyBuilder FieldBuilder ConstructorBuilder CustomAttributeBuilder ParameterBuilder EventBuilder

Used to create type members (such as methods, local variables, properties, constructors, and attributes) at runtime.

ILGenerator Emits CIL opcodes into a given type member.
OpCodes Provides numerous fields that map to CIL opcodes. This type is used in conjunction with the various members of System.Reflection. Emit.ILGenerator.

In general, the types of the System.Reflection.Emit namespace allow you to represent raw CIL tokens programmatically during the construction of your dynamic assembly. You will see many of these members in the example that follows; however, the ILGenerator type is worth checking out straightaway.

The Role of the System.Reflection.Emit.ILGenerator
As its name implies, the ILGenerator type’s role is to inject CIL opcodes into a given type member. However, you cannot directly create ILGenerator objects, as this type has no public constructors; rather, you receive an ILGenerator type by calling specific methods of the builder-centric types (such as the MethodBuilder and ConstructorBuilder types). Here is an example:

// Obtain an ILGenerator from a ConstructorBuilder
// object named "myCtorBuilder".
ConstructorBuilder myCtorBuilder = helloWorldClass.DefineConstructor( MethodAttributes.Public,
CallingConventions.Standard, constructorArgs);
ILGenerator myCILGen = myCtorBuilder.GetILGenerator();

Once you have an ILGenerator in your hands, you are then able to emit the raw CIL opcodes using any number of methods. Table 18-8 documents some (but not all) methods of ILGenerator.

Table 18-8. Various Methods of ILGenerator

Method Meaning in Life
BeginCatchBlock() Begins a catch block
BeginExceptionBlock() Begins an exception scope for an exception
BeginFinallyBlock() Begins a finally block
BeginScope() Begins a lexical scope
DeclareLocal() Declares a local variable
DefineLabel() Declares a new label
Emit() Is overloaded numerous times to allow you to emit CIL opcodes
EmitCall() Pushes a call or callvirt opcode into the CIL stream
EmitWriteLine() Emits a call to Console.WriteLine() with different types of values
EndExceptionBlock() Ends an exception block
EndScope() Ends a lexical scope
ThrowException() Emits an instruction to throw an exception
UsingNamespace() Specifies the namespace to be used in evaluating locals and watches for the current active lexical scope

The key method of ILGenerator is Emit(), which works in conjunction with the System.Reflection. Emit.OpCodes class type. As mentioned earlier in this chapter, this type exposes a good number of read-only fields that map to raw CIL opcodes. The full set of these members is documented within online help, and you will see various examples in the pages that follow.

Emitting a Dynamic Assembly
To illustrate the process of defining a .NET Core assembly at runtime, let’s walk through the process of creating a single-file dynamic assembly. Within this assembly is a class named HelloWorld. The HelloWorld class supports a default constructor and a custom constructor that is used to assign the value of a private member variable (theMessage) of type string. In addition, HelloWorld supports a public instance method named SayHello(), which prints a greeting to the standard I/O stream, and another instance method named GetMsg(), which returns the internal private string. In effect, you are going to programmatically generate the following class type:

// This class will be created at runtime
// using System.Reflection.Emit.
public class HelloWorld
{
private string theMessage; HelloWorld() {}
HelloWorld(string s) {theMessage = s;}

public string GetMsg() {return theMessage;} public void SayHello()
{
System.Console.WriteLine("Hello from the HelloWorld class!");
}
}

Assume you have created a new Console Application project named DynamicAsmBuilder and you add the System.Reflection.Emit NuGet package. Next, import the System.Reflection and System. Reflection.Emit namespaces. Define a static method named CreateMyAsm() in the Program.cs file. This single method oversees the following:
• Defining the characteristics of the dynamic assembly (name, version, etc.)
• Implementing the HelloClass type
• Returning the AssemblyBuilder to the calling method Here is the complete code, with analysis to follow:
static AssemblyBuilder CreateMyAsm()
{
// Establish general assembly characteristics.
AssemblyName assemblyName = new AssemblyName
{
Name = "MyAssembly",
Version = new Version("1.0.0.0")
};

// Create new assembly.
var builder = AssemblyBuilder.DefineDynamicAssembly( assemblyName,AssemblyBuilderAccess.Run);

// Define the name of the module.
ModuleBuilder module =

builder.DefineDynamicModule("MyAssembly");
// Define a public class named "HelloWorld".
TypeBuilder helloWorldClass = module.DefineType("MyAssembly.HelloWorld", TypeAttributes.Public);

// Define a private String variable named "theMessage".
FieldBuilder msgField = helloWorldClass.DefineField( "theMessage",
Type.GetType("System.String"), attributes: FieldAttributes.Private);

// Create the custom ctor.
Type[] constructorArgs = new Type[1]; constructorArgs[0] = typeof(string); ConstructorBuilder constructor =
helloWorldClass.DefineConstructor( MethodAttributes.Public, CallingConventions.Standard, constructorArgs);
ILGenerator constructorIl = constructor.GetILGenerator(); constructorIl.Emit(OpCodes.Ldarg_0);
Type objectClass = typeof(object); ConstructorInfo superConstructor =
objectClass.GetConstructor(new Type[0]); constructorIl.Emit(OpCodes.Call, superConstructor); constructorIl.Emit(OpCodes.Ldarg_0); constructorIl.Emit(OpCodes.Ldarg_1); constructorIl.Emit(OpCodes.Stfld, msgField); constructorIl.Emit(OpCodes.Ret);

// Create the default constructor.
helloWorldClass.DefineDefaultConstructor( MethodAttributes.Public);
// Now create the GetMsg() method.
MethodBuilder getMsgMethod = helloWorldClass.DefineMethod( "GetMsg",
MethodAttributes.Public, typeof(string),
null);
ILGenerator methodIl = getMsgMethod.GetILGenerator(); methodIl.Emit(OpCodes.Ldarg_0); methodIl.Emit(OpCodes.Ldfld, msgField); methodIl.Emit(OpCodes.Ret);

// Create the SayHello method.
MethodBuilder sayHiMethod = helloWorldClass.DefineMethod( "SayHello", MethodAttributes.Public, null, null);
methodIl = sayHiMethod.GetILGenerator(); methodIl.EmitWriteLine("Hello from the HelloWorld class!"); methodIl.Emit(OpCodes.Ret);

// "Bake" the class HelloWorld.
// (Baking is the formal term for emitting the type.) helloWorldClass.CreateType();

return builder;
}

Emitting the Assembly and Module Set
The method body begins by establishing the minimal set of characteristics about your assembly, using the AssemblyName and Version types (defined in the System.Reflection namespace). Next, you obtain an AssemblyBuilder type via the static AssemblyBuilder.DefineDynamicAssembly() method.
When calling DefineDynamicAssembly(), you must specify the access mode of the assembly you want to define, the most common values of which are shown in Table 18-9.

Table 18-9. Common Values of the AssemblyBuilderAccess Enumeration

Value Meaning in Life
RunAndCollect The assembly will be immediately unloaded, and its memory is reclaimed once it is no longer accessible.
Run This represents that a dynamic assembly can be executed in memory but not saved to disk.

The next task is to define the module set (and its name) for your new assembly. Once the DefineDynamicModule() method has returned, you are provided with a reference to a valid ModuleBuilder type.

// Create new assembly.
var builder = AssemblyBuilder.DefineDynamicAssembly( assemblyName,AssemblyBuilderAccess.Run);

The Role of the ModuleBuilder TypeC
ModuleBuilder is the key type used during the development of dynamic assemblies. As you would expect, ModuleBuilder supports several members that allow you to define the set of types contained within a given module (classes, interfaces, structures, etc.) as well as the set of embedded resources (string tables, images, etc.) contained within. Table 18-10 describes two of the creation-centric methods. (Do note that each method will return to you a related type that represents the type you want to construct.)

Table 18-10. Select Members of the ModuleBuilder Type

Method Meaning in Life
DefineEnum() Used to emit a .NET enum definition
DefineType() Constructs a TypeBuilder, which allows you to define value types, interfaces, and class types (including delegates)

The key member of the ModuleBuilder class to be aware of is DefineType(). In addition to specifying the name of the type (via a simple string), you will also use the System.Reflection.TypeAttributes enum to describe the format of the type itself. Table 18-11 lists some (but not all) of the key members of the TypeAttributes enumeration.

Table 18-11. Select Members of the TypeAttributes Enumeration

Member Meaning in Life
Abstract Specifies that the type is abstract
Class Specifies that the type is a class
Interface Specifies that the type is an interface
NestedAssembly Specifies that the class is nested with assembly visibility and is thus accessible only by methods within its assembly
NestedFamANDAssem Specifies that the class is nested with assembly and family visibility and is thus accessible only by methods lying in the intersection of its family and assembly
NestedFamily Specifies that the class is nested with family visibility and is thus accessible only by methods within its own type and any subtypes
NestedFamORAssem Specifies that the class is nested with family or assembly visibility and is thus accessible only by methods lying in the union of its family and assembly
NestedPrivate Specifies that the class is nested with private visibility
NestedPublic Specifies that the class is nested with public visibility
NotPublic Specifies that the class is not public
Public Specifies that the class is public
Sealed Specifies that the class is concrete and cannot be extended
Serializable Specifies that the class can be serialized

Emitting the HelloClass Type and the String Member Variable
Now that you have a better understanding of the role of the ModuleBuilder.CreateType() method, let’s examine how you can emit the public HelloWorld class type and the private string variable.

// Define a public class named "HelloWorld".
TypeBuilder helloWorldClass = module.DefineType("MyAssembly.HelloWorld", TypeAttributes.Public);

// Define a private String variable named "theMessage".
FieldBuilder msgField = helloWorldClass.DefineField( "theMessage",
Type.GetType("System.String"), attributes: FieldAttributes.Private);

Notice how the TypeBuilder.DefineField() method provides access to a FieldBuilder type. The TypeBuilder class also defines other methods that provide access to other “builder” types. For example, DefineConstructor() returns a ConstructorBuilder, DefineProperty() returns a PropertyBuilder, and so forth.

Emitting the Constructors
As mentioned earlier, the TypeBuilder.DefineConstructor() method can be used to define a constructor for the current type. However, when it comes to implementing the constructor of HelloClass, you need to inject raw CIL code into the constructor body, which is responsible for assigning the incoming parameter to the internal private string. To obtain an ILGenerator type, you call the GetILGenerator() method from the respective “builder” type you have reference to (in this case, the ConstructorBuilder type).
The Emit() method of the ILGenerator class is the entity in charge of placing CIL into a member implementation. Emit() itself makes frequent use of the OpCodes class type, which exposes the opcode set of CIL using read-only fields. For example, OpCodes.Ret signals the return of a method call, OpCodes.Stfld makes an assignment to a member variable, and OpCodes.Call is used to call a given method (in this case, the base class constructor). That said, ponder the following constructor logic:

// Create the custom ctor taking single string arg. Type[] constructorArgs = new Type[1]; constructorArgs[0] = typeof(string); ConstructorBuilder constructor =
helloWorldClass.DefineConstructor( MethodAttributes.Public, CallingConventions.Standard, constructorArgs);
//Emit the necessary CIL into the ctor
ILGenerator constructorIl = constructor.GetILGenerator(); constructorIl.Emit(OpCodes.Ldarg_0);
Type objectClass = typeof(object); ConstructorInfo superConstructor =
objectClass.GetConstructor(new Type[0]); constructorIl.Emit(OpCodes.Call, superConstructor);
//Load this pointer onto the stack constructorIl.Emit(OpCodes.Ldarg_0); constructorIl.Emit(OpCodes.Ldarg_1);
//Load argument on virtual stack and store in msdField constructorIl.Emit(OpCodes.Stfld, msgField); constructorIl.Emit(OpCodes.Ret);

Now, as you are aware, as soon as you define a custom constructor for a type, the default constructor is silently removed. To redefine the no-argument constructor, simply call the DefineDefaultConstructor() method of the TypeBuilder type as follows:

// Create the default ctor.
helloWorldClass.DefineDefaultConstructor( MethodAttributes.Public);

Emitting the SayHello() Method
Finally, let’s examine the process of emitting the SayHello() method. The first task is to obtain a MethodBuilder type from the helloWorldClass variable. After you do this, you define the method and obtain the underlying ILGenerator to inject the CIL instructions, like so:

// Create the SayHello method.
MethodBuilder sayHiMethod = helloWorldClass.DefineMethod( "SayHello", MethodAttributes.Public, null, null);
methodIl = sayHiMethod.GetILGenerator();

//Write to the console
methodIl.EmitWriteLine("Hello from the HelloWorld class!"); methodIl.Emit(OpCodes.Ret);

Here you have established a public method (MethodAttributes.Public) that takes no parameters and returns nothing (marked by the null entries contained in the DefineMethod() call). Also note the EmitWriteLine() call. This helper member of the ILGenerator class automatically writes a line to the standard output with minimal fuss and bother.

Using the Dynamically Generated Assembly
Now that you have the logic in place to create your assembly, all that is needed is to execute the generated code. The logic in the calling code calls the CreateMyAsm() method, getting a reference to the created AssemblyBuilder.
Next, you will exercise some late binding (see Chapter 17) to create an instance of the HelloWorld class and interact with its members. Update your top-level statements as follows:

using System.Reflection; using System.Reflection.Emit;

Console.WriteLine(" The Amazing Dynamic Assembly Builder App ");
// Create the assembly builder using our helper f(x). AssemblyBuilder builder = CreateMyAsm();

// Get the HelloWorld type.
Type hello = builder.GetType("MyAssembly.HelloWorld");

// Create HelloWorld instance and call the correct ctor. Console.Write("-> Enter message to pass HelloWorld class: "); string msg = Console.ReadLine();
object[] ctorArgs = new object[1]; ctorArgs[0] = msg;
object obj = Activator.CreateInstance(hello, ctorArgs);

// Call SayHello and show returned string. Console.WriteLine("-> Calling SayHello() via late binding."); MethodInfo mi = hello.GetMethod("SayHello");

mi.Invoke(obj, null);

// Invoke method.
mi = hello.GetMethod("GetMsg"); Console.WriteLine(mi.Invoke(obj, null));

In effect, you have just created a .NET Core assembly that is able to create and execute .NET Core assemblies at runtime! That wraps up the examination of CIL and the role of dynamic assemblies. I hope this chapter has deepened your understanding of the .NET Core type system, the syntax and semantics of CIL, and how the C# compiler processes your code at compile time.

Summary
This chapter provided an overview of the syntax and semantics of CIL. Unlike higher-level managed languages such as C#, CIL does not simply define a set of keywords but provides directives (used to define the structure of an assembly and its types), attributes (which further qualify a given directive), and opcodes (which are used to implement type members).
You were introduced to a few CIL-centric programming tools and learned how to alter the contents of a .NET assembly with new CIL instructions using round-trip engineering. After this point, you spent time learning how to establish the current (and referenced) assembly, namespaces, types, and members. I
wrapped up with a simple example of building a .NET code library and executable using little more than CIL, command-line tools, and a bit of elbow grease.
Finally, you took an introductory look at the process of creating a dynamic assembly. Using the System. Reflection.Emit namespace, it is possible to define a .NET Core assembly in memory at runtime. As you have seen firsthand, using this API requires you to know the semantics of CIL code in some detail. While the need to build dynamic assemblies is certainly not a common task for most .NET Core applications, it can be useful for those of you who need to build support tools and other programming utilities.

Pro C#10 CHAPTER 17 Type Reflection, Late Binding, Attribute, and Dynamic Types

CHAPTER 17

Type Reflection, Late Binding, Attribute, and Dynamic Types

As shown in Chapter 16, assemblies are the basic unit of deployment in the .NET universe. Using the integrated Object Browser of Visual Studio (and numerous other IDEs), you can examine the types within a project’s referenced set of assemblies. Furthermore, external tools such as ildasm.exe allow you to peek into the underlying CIL code, type metadata, and assembly manifest for a given .NET binary. In addition to this design-time investigation of .NET assemblies, you are also able to programmatically obtain this same information using the System.Reflection namespace. To this end, the first task of this chapter is to define the role of reflection and the necessity of .NET metadata.
The next sections of the chapter examine several closely related topics, which hinge upon reflection services. For example, you will learn how a .NET client may employ dynamic loading and late binding to activate types it has no compile-time knowledge of. You will also learn how to insert custom metadata into your .NET assemblies using system-supplied and custom attributes. To put all of these (seemingly esoteric) topics into perspective, the chapter closes by demonstrating how to build several “snap-in objects” that you can plug into an extendable console application.
In this chapter, you will also be introduced to the C# dynamic keyword and understand how loosely typed calls are mapped to the correct in-memory object using the Dynamic Language Runtime (DLR). After you understand the services provided by the DLR, you will see examples of using dynamic types to streamline how you can perform late-bound method calls (via reflection services) and to easily communicate with legacy COM libraries.

■ Note Don’t confuse the C# dynamic keyword with the concept of a dynamic assembly (see Chapter 18). While you could use the dynamic keyword when building a dynamic assembly, these are ultimately two independent concepts.

The Necessity of Type Metadata
The ability to fully describe types (classes, interfaces, structures, enumerations, and delegates) using metadata is a key element of the .NET platform. Many .NET technologies, such as object serialization, require the ability to discover the format of types at runtime. Furthermore, cross-language interoperability, numerous compiler services, and an IDE’s IntelliSense capabilities all rely on a concrete description of type.

© Andrew Troelsen, Phil Japikse 2022
A. Troelsen and P. Japikse, Pro C# 10 with .NET 6, https://doi.org/10.1007/978-1-4842-7869-7_17

661

Recall that the ildasm.exe utility allows you to view an assembly’s type metadata. In the generated CarLibrary.il file (from Chapter 16), navigate to the METAINFO section to see all the CarLibrary’s metadata. A small snippet is included here:

// ==== M E T A I N F O ===

// ===========================================================
// ScopeName : CarLibrary.dll
// MVID : {DF92DBD2-2C47-4CF8-B25E-7319F1351625}
// ===========================================================
// Global functions
//
//
// Global fields
//
//
// Global MemberRefs
//
//
// TypeDef #1
//
// TypDefName: CarLibrary.Car
// Flags : [Public] [AutoLayout] [Class] [Abstract] [AnsiClass] [BeforeFieldInit]
// Extends : [TypeRef] System.Object
// Field #1
//
// Field Name: k BackingField
// Flags : [Private]
// CallCnvntn : [FIELD]
// Field type : String
...

As you can see, the .NET type metadata is verbose (the actual binary format is much more compact). In fact, if I were to list the entire metadata description representing the CarLibrary.dll assembly, it would span several pages. Given that this act would be a woeful waste of paper, let’s just glimpse into some key metadata descriptions of the CarLibrary.dll assembly.

■ Note Don’t be too concerned with the exact syntax of every piece of .NET metadata in the next few sections. The bigger point to absorb is that .NET metadata is very descriptive and lists each internally defined (and externally referenced) type found within a given code base.

Viewing (Partial) Metadata for the EngineStateEnum Enumeration
Each type defined within the current assembly is documented using a TypeDef #n token (where TypeDef is short for type definition). If the type being described uses a type defined within a separate .NET assembly, the referenced type is documented using a TypeRef #n token (where TypeRef is short for type reference).
A TypeRef token is a pointer (if you will) to the referenced type’s full metadata definition in an external assembly. In a nutshell, .NET metadata is a set of tables that clearly mark all type definitions (TypeDefs) and referenced types (TypeRefs), all of which can be examined using ildasm.exe.

As far as CarLibrary.dll goes, one TypeDef is the metadata description of the CarLibrary.
EngineStateEnum enumeration (your number may differ; TypeDef numbering is based on the order in which the C# compiler processes the file).

// TypeDef #2
//
// TypDefName: CarLibrary.EngineStateEnum
// Flags : [Public] [AutoLayout] [Class] [Sealed] [AnsiClass]
// Extends : [TypeRef] System.Enum
// Field #1
//
// Field Name: value
// Flags : [Public] [SpecialName] [RTSpecialName]
// CallCnvntn: [FIELD]
// Field type: I4
//
// Field #2
//
// Field Name: EngineAlive
// Flags : [Public] [Static] [Literal] [HasDefault]
// DefltValue: (I4) 0
// CallCnvntn: [FIELD]
// Field type: ValueClass CarLibrary.EngineStateEnum
//
// Field #3 (04000007)
//
// Field Name: EngineDead (04000007)
// Flags : [Public] [Static] [Literal] [HasDefault] (00008056)
// DefltValue: (I4) 1
// CallCnvntn: [FIELD]
// Field type: ValueClass CarLibrary.EngineStateEnum
...

Here, the TypDefName token is used to establish the name of the given type, which in this case is the custom CarLibrary.EngineStateEnum enum. The Extends metadata token is used to document the base type of a given .NET type (in this case, the referenced type, System.Enum). Each field of an enumeration is marked using the Field #n token.

■ Note While they look like typos, TypDefName does not have the e and DefltValue does not have the au
one would expect.

Viewing (Partial) Metadata for the Car Type
Here is a partial dump of the Car class that illustrates the following:
• How fields are defined in terms of .NET metadata
• How methods are documented via .NET metadata
• How an automatic property is represented in .NET metadata

// TypeDef #1
//
// TypDefName: CarLibrary.Car
// Flags : [Public] [AutoLayout] [Class] [Abstract] [AnsiClass] [BeforeFieldInit]
// Extends : [TypeRef] System.Object
// Field #1
//
// Field Name: k BackingField
// Flags : [Private]
// CallCnvntn: [FIELD]
// Field type: String
...
// Method #1
//
// MethodName: get_PetName
// Flags : [Public] [HideBySig] [ReuseSlot] [SpecialName]
// RVA : 0x00002050
// ImplFlags : [IL] [Managed]
// CallCnvntn: [DEFAULT]
// hasThis
// ReturnType: String
// No arguments.
...
// Method #2
//
// MethodName: set_PetName
// Flags : [Public] [HideBySig] [ReuseSlot] [SpecialName]
// RVA : 0x00002058
// ImplFlags : [IL] [Managed]
// CallCnvntn: [DEFAULT]
// hasThis
// ReturnType: Void
// 1 Arguments
// Argument #1: String
// 1 Parameters
// (1) ParamToken : Name : value flags: [none]
...

// Property #1
//
// Prop.Name : PetName
// Flags : [none]
// CallCnvntn: [PROPERTY]
// hasThis
// ReturnType: String
// No arguments.
// DefltValue:
// Setter : set_PetName
// Getter : get_PetName
// 0 Others
...

First, note that the Car class metadata marks the type’s base class (System.Object) and includes various flags that describe how this type was constructed (e.g., [Public], [Abstract], and whatnot). Methods (such as the Car’s constructor) are described by their parameters, return value, and name.
Note how an automatic property results in a compiler-generated private backing field (which was named k BackingField) and two compiler-generated methods (in the case of a read-write property) named, in this example, get_PetName() and set_PetName(). Finally, the actual property is mapped to the internal get/set methods using the .NET metadata Getter/Setter tokens.

Examining a TypeRef
Recall that an assembly’s metadata will describe not only the set of internal types (Car, EngineStateEnum, etc.) but also any external types the internal types reference. For example, given that CarLibrary.dll has defined two enumerations, you find a TypeRef block for the System.Enum type, as follows:

// TypeRef #19
//
// Token: 0x01000013
// ResolutionScope: 0x23000001
// TypeRefName: System.Enum

Documenting the Defining Assembly
The CarLibrary.il file also allows you to view the .NET metadata that describes the assembly itself using the Assembly token. The following is a partial dump of the manifest of CarLibrary.dll. Note that the version matches the assembly version set for the library.

// Assembly
//
// Token: 0x20000001
// Name : CarLibrary
// Public Key :
// Hash Algorithm : 0x00008004
// Version: 1.0.0.1
// Major Version: 0x00000001
// Minor Version: 0x00000000
// Build Number: 0x00000000
// Revision Number: 0x00000001
// Locale:
// Flags : [none] (00000000)

Documenting Referenced Assemblies
In addition to the Assembly token and the set of TypeDef and TypeRef blocks, .NET metadata also makes use of AssemblyRef #n tokens to document each external assembly. Given that each .NET assembly references the System.Runtime base class library assembly, you will find an AssemblyRef for the System.Runtime assembly, as shown in the following code:

// AssemblyRef #1
//
// Token: 0x23000001
// Public Key or Token: b0 3f 5f 7f 11 d5 0a 3a
// Name: System.Runtime
// Version: 6.0.0.0
// Major Version: 0x00000006
// Minor Version: 0x00000000
// Build Number: 0x00000000
// Revision Number: 0x00000000
// Locale:
// HashValue Blob:
// Flags: [none] (00000000)

Documenting String Literals
The final point of interest regarding .NET metadata is the fact that every string literal in your code base is documented under the User Strings token.

// User Strings
//
// 70000001 : (23) L"CarLibrary Version 2.0!"
// 70000031 : (13) L"Quiet time..."
// 7000004d : (11) L"Jamming {0}"
// 70000065 : (32) L"Eek! Your engine block exploded!"
// 700000a7 : (34) L"Ramming speed! Faster is better..."

■ Note as illustrated in the previous metadata listing, always be aware that all strings are clearly documented in the assembly metadata. This could have huge security consequences if you were to use string literals to represent passwords, credit card numbers, or other sensitive information.

The next question on your mind may be (in the best-case scenario) “How can I leverage this information in my applications?” or (in the worst-case scenario) “Why should I care about metadata?” To address both points of view, allow me to introduce .NET reflection services. Be aware that the usefulness of the topics presented over the pages that follow may be a bit of a head-scratcher until this chapter’s endgame. So, hang tight.

■ Note you will also find a number of CustomAttribute tokens displayed by the METAINFO section, which documents the attributes applied within the code base. you will learn about the role of .NET attributes later in this chapter.

Understanding Reflection
In the .NET universe, reflection is the process of runtime type discovery. Using reflection services, you can programmatically obtain the same metadata information generated by ildasm.exe using a friendly object model. For example, through reflection, you can obtain a list of all types contained within a given .dll or
.exe assembly, including the methods, fields, properties, and events defined by a given type. You can also dynamically discover the set of interfaces supported by a given type, the parameters of a method, and other related details (base classes, namespace information, manifest data, etc.).
Like any namespace, System.Reflection (which is defined in System.Runtime.dll) contains several related types. Table 17-1 lists some of the core items you should be familiar with.

Table 17-1. A Sampling of Members of the System.Reflection Namespace

Type Meaning in Life
Assembly This abstract class contains members that allow you to load, investigate, and manipulate an assembly.
AssemblyName This class allows you to discover numerous details behind an assembly’s identity (version information, culture information, etc.).
EventInfo This abstract class holds information for a given event.
FieldInfo This abstract class holds information for a given field.
MemberInfo This is the abstract base class that defines common behaviors for the EventInfo, FieldInfo, MethodInfo, and PropertyInfo types.
MethodInfo This abstract class contains information for a given method.
Module This abstract class allows you to access a given module within a multifile assembly.
ParameterInfo This class holds information for a given parameter.
PropertyInfo This abstract class holds information for a given property.

To understand how to leverage the System.Reflection namespace to programmatically read .NET metadata, you need to first come to terms with the System.Type class.

The System.Type Class
The System.Type class defines members that can be used to examine a type’s metadata, a great number of which return types from the System.Reflection namespace. For example, Type.GetMethods() returns an array of MethodInfo objects, Type.GetFields() returns an array of FieldInfo objects, and so on. The complete set of members exposed by System.Type is quite expansive; however, Table 17-2 offers a partial snapshot of the members supported by System.Type (see the .NET documentation for full details).

Table 17-2. Select Members of System.Type

Member Meaning in Life
IsAbstract IsArray IsClass IsCOMObject IsEnum
IsGenericTypeDefinition IsGenericParameter IsInterface
IsPrimitive IsNestedPrivate IsNestedPublic IsSealed IsValueType These properties (among others) allow you to discover a number of basic traits about the Type you are referring to (e.g., if it is an abstract entity, an array, a nested class, etc.).
GetConstructors() GetEvents() GetFields() GetInterfaces() GetMembers() GetMethods() GetNestedTypes() GetProperties() These methods (among others) allow you to obtain an array representing the items (interface, method, property, etc.) you are interested in. Each method returns a related array (e.g., GetFields() returns a FieldInfo array, GetMethods() returns a MethodInfo array, etc.). Be aware that each of these methods has a singular form (e.g., GetMethod(), GetProperty(), etc.) that allows you to retrieve a specific item by name, rather than an array of all related items.
FindMembers() This method returns a MemberInfo array based on search criteria.
GetType() This static method returns a Type instance given a string name.
InvokeMember() This method allows “late binding” for a given item. You’ll learn about late binding later in this chapter.

Obtaining a Type Reference Using System.Object.GetType()
You can obtain an instance of the Type class in a variety of ways. However, the one thing you cannot do is directly create a Type object using the new keyword, as Type is an abstract class. Recall that System.
Object defines a method named GetType(), which returns an instance of the Type class that represents the metadata for the current object.

// Obtain type information using a SportsCar instance. SportsCar sc = new SportsCar();
Type t = sc.GetType();

Obviously, this approach will work only if you have compile-time knowledge of the type you want to reflect over (SportsCar in this case) and currently have an instance of the type in memory. Given this
restriction, it should make sense that tools such as ildasm.exe do not obtain type information by directly calling System.Object.GetType() for each type since that ildasm.exe was not compiled against your custom assemblies.

Obtaining a Type Reference Using typeof()
The next way to obtain type information is using the C# typeof operator, like so:

// Get the type using typeof. Type t = typeof(SportsCar);

Unlike System.Object.GetType(), the typeof operator is helpful, in that you do not need to first create an object instance to extract type information. However, your code base must still have compile- time knowledge of the type you are interested in examining, as typeof expects the strongly typed name of the type.

Obtaining a Type Reference Using System.Type.GetType()
To obtain type information in a more flexible manner, you may call the static GetType() member of the System.Type class and specify the fully qualified string name of the type you are interested in examining. Using this approach, you do not need to have compile-time knowledge of the type you are extracting metadata from, given that Type.GetType() takes an instance of the omnipresent System.String.

■ Note When i say you do not need compile-time knowledge when calling Type.GetType(), i am referring to the fact that this method can take any string value whatsoever (rather than a strongly typed variable). of course, you would still need to know the name of the type in a “stringified” format!

The Type.GetType() method has been overloaded to allow you to specify two Boolean parameters, one of which controls whether an exception should be thrown if the type cannot be found, and the other of which establishes the case sensitivity of the string. To illustrate, ponder the following:

// Obtain type information using the static Type.GetType() method
// (don't throw an exception if SportsCar cannot be found and ignore case).
Type t = Type.GetType("CarLibrary.SportsCar", false, true);

In the previous example, notice that the string you are passing into GetType() makes no mention of the assembly containing the type. In this case, the assumption is that the type is defined within the currently executing assembly. However, when you want to obtain metadata for a type within an external assembly, the string parameter is formatted using the type’s fully qualified name, followed by a comma, followed by the friendly name (the assembly name without any version information) of the assembly containing the type, like so:

// Obtain type information for a type within an external assembly. Type t = Type.GetType("CarLibrary.SportsCar, CarLibrary");

As well, do know that the string passed into Type.GetType() may specify a plus token (+) to denote a nested type. Assume you want to obtain type information for an enumeration (SpyOptions) nested within a class named JamesBondCar. To do so, you would write the following:

// Obtain type information for a nested enumeration
// within the current assembly.
Type t = Type.GetType("CarLibrary.JamesBondCar+SpyOptions");

Building a Custom Metadata Viewer
To illustrate the basic process of reflection (and the usefulness of System.Type), let’s create a Console Application project named MyTypeViewer. This program will display details of the methods, properties, fields, and supported interfaces (in addition to some other points of interest) for any type within System. Runtime.dll (recall all .NET applications have automatic access to this framework class library) or a type within MyTypeViewer itself. Once the application has been created, be sure to import the System. Reflection namespace.

// Need to import this namespace to do any reflection! using System.Reflection;

Reflecting on Methods
Several static methods will be added to the Program.cs file, each of which takes a single System.Type parameter and returns void. First you have ListMethods(), which (as you might guess) prints the name of each method defined by the incoming type. Notice how Type.GetMethods() returns an array of System. Reflection.MethodInfo objects, which can be enumerated with a standard foreach loop, as follows:

// Display method names of type. static void ListMethods(Type t)
{
Console.WriteLine(" Methods "); MethodInfo[] mi = t.GetMethods(); foreach(MethodInfo m in mi)
{
Console.WriteLine("->{0}", m.Name);
}
Console.WriteLine();
}

Here, you are simply printing the name of the method using the MethodInfo.Name property. As you might guess, MethodInfo has many additional members that allow you to determine whether the method is static, virtual, generic, or abstract. As well, the MethodInfo type allows you to obtain the method’s return value and parameter set. You’ll spruce up the implementation of ListMethods() in just a bit.
If you wanted, you could also build a fitting LINQ query to enumerate the names of each method.
Recall from Chapter 13, LINQ to Objects allows you to build strongly typed queries that can be applied to in-memory object collections. As a good rule of thumb, whenever you find blocks of looping or decision programming logic, you could make use of a related LINQ query. For example, you could rewrite the previous method with LINQ like this:

static void ListMethods(Type t)
{
Console.WriteLine(" Methods ");
var methodNames = from n in t.GetMethods() orderby n.Name select n.Name;
// Using LINQ extensions:
// var methodNames = t.GetMethods().OrderBy(m=>m.Name).Select(m=>m.Name); foreach (var name in methodNames)
{

Console.WriteLine("->{0}", name);
}
Console.WriteLine();
}

Reflecting on Fields and Properties
The implementation of ListFields() is similar. The only notable difference is the call to Type.GetFields() and the resulting FieldInfo array. Again, to keep things simple, you are printing out only the name of each field using a LINQ query.

// Display field names of type. static void ListFields(Type t)
{
Console.WriteLine(" Fields ");
// var fieldNames = from f in t.GetFields() orderby f.Name select f.Name; var fieldNames = t.GetFields().OrderBy(m=>m.Name).Select(x=>x.Name);
foreach (var name in fieldNames)
{
Console.WriteLine("->{0}", name);
}
Console.WriteLine();
}

The logic to display a type’s properties is similar.

// Display property names of type. static void ListProps(Type t)
{
Console.WriteLine(" Properties ");
var propNames = from p in t.GetProperties() orderby p.Name select p.Name;
//var propNames = t.GetProperties().Select(p=>p.Name); foreach (var name in propNames)
{
Console.WriteLine("->{0}", name);
}
Console.WriteLine();
}

Reflecting on Implemented Interfaces
Next, you will author a method named ListInterfaces() that will print the names of any interfaces supported on the incoming type. The only point of interest here is that the call to GetInterfaces() returns an array of System.Types! This should make sense given that interfaces are, indeed, types.

// Display implemented interfaces. static void ListInterfaces(Type t)
{
Console.WriteLine(" Interfaces ");

var ifaces = from i in t.GetInterfaces() orderby i.Name select i;
//var ifaces = t.GetInterfaces().OrderBy(i=>i.Name); foreach(Type i in ifaces)
{
Console.WriteLine("->{0}", i.Name);
}
}

■ Note Be aware that a majority of the “get” methods of System.Type (GetMethods(), GetInterfaces(), etc.) have been overloaded to allow you to specify values from the BindingFlags enumeration. This provides a greater level of control on exactly what should be searched for (e.g., only static members, only public members, include private members, etc.). Consult the documentation for details.

Displaying Various Odds and Ends
Last but not least, you have one final helper method that will simply display various statistics (indicating whether the type is generic, what the base class is, whether the type is sealed, etc.) regarding the incoming type.

// Just for good measure.
static void ListVariousStats(Type t)
{
Console.WriteLine(" Various Statistics "); Console.WriteLine("Base class is: {0}", t.BaseType); Console.WriteLine("Is type abstract? {0}", t.IsAbstract); Console.WriteLine("Is type sealed? {0}", t.IsSealed); Console.WriteLine("Is type generic? {0}", t.IsGenericTypeDefinition); Console.WriteLine("Is type a class type? {0}", t.IsClass); Console.WriteLine();
}

Adding the Top-Level Statements
The top-level statements of the Program.cs file prompts the user for the fully qualified name of a type. Once you obtain this string data, you pass it into the Type.GetType() method and send the extracted System.Type into each of your helper methods. This process repeats until the user presses Q to terminate the application.

Console.WriteLine(" Welcome to MyTypeViewer "); string typeName = "";

do
{
Console.WriteLine("\nEnter a type name to evaluate"); Console.Write("or enter Q to quit: ");

// Get name of type.
typeName = Console.ReadLine();

// Does user want to quit?
if (typeName.Equals("Q",StringComparison.OrdinalIgnoreCase))
{
break;
}

// Try to display type. try
{
Type t = Type.GetType(typeName); Console.WriteLine(""); ListVariousStats(t); ListFields(t);
ListProps(t);
ListMethods(t);
ListInterfaces(t);
}
catch
{
Console.WriteLine("Sorry, can't find type");
}
} while (true);

At this point, MyTypeViewer.exe is ready to take a test-drive. For example, run your application and enter the following fully qualified names (be aware that Type.GetType() requires case-sensitive string names):
•System.Int32
•System.Collections.ArrayList
•System.Threading.Thread
•System.Void
•System.Math
For example, here is some partial output when specifying System.Math:

Welcome to MyTypeViewer Enter a type name to evaluate
or enter Q to quit: System.Math

Various Statistics Base class is: System.Object Is type abstract? True
Is type sealed? True Is type generic? False
Is type a class type? True

Fields
->E
->PI
->Tau

Properties

Methods
->Abs
->Abs
...
->Acos
->Asin
->Atan
->Atan2
->Ceiling
->Cos
...

Notice the repeated listing for Abs. This is because there will be at least one overload for the Abs()
method. The code will be expanded to show the parameters and return types shortly.

Reflecting on Static Types
If you enter System.Console in the previous method, an exception will be thrown in the first helper method because the value for t will be null. Static types cannot be loaded using the Type.GetType(typeName) method. Instead, you must use another mechanism, the typeof function from System.Type. Update the program to handle the System.Console special case like this:

Type t = Type.GetType(typeName);
if (t == null && typeName.Equals("System.Console", StringComparison.OrdinalIgnoreCase))
{
t = typeof(System.Console);
}

Reflecting on Generic Types
When you call Type.GetType() to obtain metadata descriptions of generic types, you must make use of a special syntax involving a “backtick” character (`) followed by a numerical value that represents the number of type parameters the type supports. For example, if you want to print out the metadata description of System.Collections.Generic.List, you need to pass the following string into your application:

System.Collections.Generic.List`1

Here, you are using the numerical value of 1, given that List has only one type parameter. However, if you want to reflect over Dictionary<TKey, TValue>, supply the value 2, like so:

System.Collections.Generic.Dictionary`2

Reflecting on Method Parameters and Return Values
So far, so good! Next, we will make a minor enhancement to the current application. Specifically, you will update the ListMethods() helper function to list not only the name of a given method but also the return type and incoming parameter types. The MethodInfo type provides the ReturnType property and GetParameters()

method for these tasks. In the following modified code, notice that you are building a string that contains the type and name of each parameter using a nested foreach loop (without the use of LINQ):

static void ListMethods(Type t)
{
Console.WriteLine(" Methods ");
MethodInfo[] mi = t.GetMethods().OrderBy(m=>m.Name).ToArray(); foreach (MethodInfo m in mi)
{
// Get return type.
string retVal = m.ReturnType.FullName; string paramInfo = "( ";
// Get params.
foreach (ParameterInfo pi in m.GetParameters())
{
paramInfo += string.Format("{0} {1} ", pi.ParameterType, pi.Name);
}
paramInfo += " )";

// Now display the basic method sig.
Console.WriteLine("->{0} {1} {2}", retVal, m.Name, paramInfo);
}
Console.WriteLine();
}

If you now run this updated application, you will find that the methods of a given type are much more detailed and the mystery of the repeated methods is solved. If you enter System.Math into the program, both of the Abs() methods (and all other methods) will display the return type and the parameter(s).

Methods
->System.Double Abs ( System.Double value )
->System.Single Abs ( System.Single value )

The current implementation of ListMethods() is helpful, in that you can directly investigate each parameter and method return type using the System.Reflection object model. As an extreme shortcut, be aware that all of the XXXInfo types (MethodInfo, PropertyInfo, EventInfo, etc.) have overridden ToString() to display the signature of the item requested. Thus, you could also implement ListMethods()
as follows (once again using LINQ, where you simply select all MethodInfo objects, rather than only the Name
values):

static void ListMethods(Type t)
{
Console.WriteLine(" Methods ");
var methods = t.GetMethods().OrderBy(m=>m.Name); foreach (var m in methods)
{
Console.WriteLine("->{0}", m);
}
Console.WriteLine();
}

Interesting stuff, huh? Clearly, the System.Reflection namespace and System.Type class allow you to reflect over many other aspects of a type beyond what MyTypeViewer is currently displaying. As you would hope, you can obtain a type’s events, get the list of any generic parameters for a given member, and glean dozens of other details.
Nevertheless, at this point you have created a (somewhat capable) object browser. The major limitation with this specific example is that you have no way to reflect beyond the current assembly (MyTypeViewer)
or assemblies in the base class libraries that are always referenced. This begs the question “How can I build applications that can load (and reflect over) assemblies not referenced at compile time?” Glad you asked.

Dynamically Loading Assemblies
There will be times when you need to load assemblies on the fly programmatically, even if there is no record of said assembly in the manifest. Formally speaking, the act of loading external assemblies on demand is known as a dynamic load.
System.Reflection defines a class named Assembly. Using this class, you can dynamically load an assembly, as well as discover properties about the assembly itself. In essence, the Assembly class provides methods that allow you to programmatically load assemblies off disk.
To illustrate dynamic loading, create a new Console Application project named ExternalAssemblyReflector. Your task is to construct code that prompts for the name of an assembly (minus any extension) to load dynamically. You will pass the Assembly reference into a helper method named DisplayTypes(), which will simply print the names of each class, interface, structure, enumeration, and delegate it contains. The code is refreshingly simple.

using System.Reflection;

Console.WriteLine(" External Assembly Viewer "); string asmName = "";
Assembly asm = null; do
{
Console.WriteLine("\nEnter an assembly to evaluate"); Console.Write("or enter Q to quit: ");
// Get name of assembly. asmName = Console.ReadLine();
// Does user want to quit?
if (asmName.Equals("Q",StringComparison.OrdinalIgnoreCase))
{
break;
}

// Try to load assembly. try
{
asm = Assembly.LoadFrom(asmName); DisplayTypesInAsm(asm);
}
catch
{
Console.WriteLine("Sorry, can't find assembly.");
}

} while (true);

static void DisplayTypesInAsm(Assembly asm)
{
Console.WriteLine("\n Types in Assembly "); Console.WriteLine("->{0}", asm.FullName);
Type[] types = asm.GetTypes(); foreach (Type t in types)
{
Console.WriteLine("Type: {0}", t);
}
Console.WriteLine("");
}

If you want to reflect over CarLibrary.dll, you will need to copy the CarLibrary.dll binary (from the previous chapter) to the project directory (if using Visual Studio Code) or to the \bin\Debug\net6.0 directory (if using Visual Studio) of the ExternalAssemblyReflector application to run this program. Enter CarLibrary (the extension is optional) when prompted, and the output will look like this:

External Assembly Viewer Enter an assembly to evaluate
or enter Q to quit: CarLibrary

Types in Assembly
->CarLibrary, Version=1.0.0.1, Culture=neutral, PublicKeyToken=null Type: CarLibrary.Car
Type: CarLibrary.EngineStateEnum Type: CarLibrary.MiniVan
Type: CarLibrary.MusicMediaEnum Type: CarLibrary.MyInternalClass Type: CarLibrary.SportsCar

The LoadFrom method can also take an absolute path to the assembly you want to view (e.g., C:\MyApp\ MyAsm.dll). With this method, you can pass in a full path to your Console Application project. Thus, if CarLibrary.dll was located under C:\MyCode, you could enter C:\MyCode\CarLibrary (the extension is still optional).

Reflecting on Framework Assemblies
The Assembly.Load() method has several overloads. One variation allows you to specify a culture value (for localized assemblies), as well as a version number and public key token value (for framework assemblies).
Collectively speaking, the set of items identifying an assembly is termed the display name. The format of a display name is a comma-delimited string of name-value pairs that begins with the friendly name of the assembly, followed by optional qualifiers (that may appear in any order). Here is the template to follow (optional items appear in parentheses):

Name (,Version = major.minor.build.revision) (,Culture = culture token) (,PublicKeyToken= public key token)

When you are crafting a display name, the convention PublicKeyToken=null indicates that binding and matching against a nonstrongly named assembly is required. Additionally, Culture="" indicates matching against the default culture of the target machine. Here is an example:

// Load version 1.0.0.1 of CarLibrary using the default culture. Assembly a =
Assembly.Load("CarLibrary, Version=1.0.0.1, PublicKeyToken=null, Culture=\"\"");
// The quotes must be escaped with back slashes in C#

Also be aware that the System.Reflection namespace supplies the AssemblyName type, which allows you to represent the preceding string information in a handy object variable. Typically, this class is used in conjunction with System.Version, which is a wrapper around an assembly’s version number. Once you have established the display name, it can then be passed into the overloaded Assembly.Load() method, like so:

// Make use of AssemblyName to define the display name. AssemblyName asmName;
asmName = new AssemblyName(); asmName.Name = "CarLibrary"; Version v = new Version("1.0.0.1"); asmName.Version = v;
Assembly a = Assembly.Load(asmName);

To load a .NET Framework assembly (not .NET), the Assembly.Load() parameter should specify a PublicKeyToken value. With .NET, it’s not required, due to the diminished use of strong naming. For
example, assume you have a new Console Application project named FrameworkAssemblyViewer that has a reference to the Microsoft.EntityFrameworkCore package. As a reminder, this can all be done with the
.NET command-line interface (CLI).

dotnet new console -lang c# -n FrameworkAssemblyViewer -o .\FrameworkAssemblyViewer -f net6.0 dotnet sln .\Chapter17_AllProjects.sln add .\FrameworkAssemblyViewer
dotnet add .\FrameworkAssemblyViewer package Microsoft.EntityFrameworkCore -v 6.0.0

Recall that when you reference another assembly, a copy of that assembly is copied into the output directory of the referencing project. Build the project using the CLI.

dotnet build

With the project created, EntityFrameworkCode referenced, and the project built, you can now load it and inspect it. Given that the number of types in this assembly is quite large, the following application prints out only the names of public enums, using a simple LINQ query:

using System.Reflection;

Console.WriteLine(" The Framework Assembly Reflector App \n");

// Load Microsoft.EntityFrameworkCore.dll var displayName =
"Microsoft.EntityFrameworkCore, Version=6.0.0.0, Culture=neutral, PublicKeyToken=a db9793829ddae60";
Assembly asm = Assembly.Load(displayName); DisplayInfo(asm); Console.WriteLine("Done!");

Console.ReadLine();

static void DisplayInfo(Assembly a)
{
AssemblyName asmNameInfo = a.GetName(); Console.WriteLine(" Info about Assembly "); Console.WriteLine($"Asm Name: {asmNameInfo.Name}"); Console.WriteLine($"Asm Version: {asmNameInfo.Version}");
Console.WriteLine($"Asm Culture: {asmNameInfo.CultureInfo.DisplayName}");

Console.WriteLine("\nHere are the public enums:");
// Use a LINQ query to find the public enums.
var publicEnums = a.GetTypes().Where(p=>p.IsEnum && p.IsPublic); foreach (var pe in publicEnums)
{
Console.WriteLine(pe);
}
}

At this point, you should understand how to use some of the core members of the System.Reflection namespace to discover metadata at runtime. Of course, you likely will not need to build custom object browsers often (if ever). However, reflection services are the foundation for many common programming activities, including late binding.

Understanding Late Binding
Simply put, late binding is a technique in which you can create an instance of a given type and invoke its members at runtime without having hard-coded compile-time knowledge of its existence. When you are building an application that binds late to a type in an external assembly, you have no reason to set a reference to the assembly; therefore, the caller’s manifest has no direct listing of the assembly.
At first glance, it is not easy to see the value of late binding. It is true that if you can “bind early” to an object (e.g., add an assembly reference and allocate the type using the C# new keyword), you should opt to do so. For one reason, early binding allows you to determine errors at compile time, rather than at runtime. Nevertheless, late binding does have a critical role in any extendable application you may be building. You will have a chance to build such an “extendable” program later in this chapter, in the section “Building an Extendable Application.” Until then, let’s examine the role of the Activator class.

The System.Activator Class
The System.Activator class is the key to the .NET late-binding process. For the next example, you are interested only in the Activator.CreateInstance() method, which is used to create an instance of a type through late binding. This method has been overloaded numerous times to provide a good deal of flexibility. The simplest variation of the CreateInstance() member takes a valid Type object that describes the entity you want to allocate into memory on the fly.
Create a new Console Application project named LateBindingApp and update the Program.cs file as follows:

using System.Reflection;

// This program will load an external library,
// and create an object using late binding.

Console.WriteLine(" Fun with Late Binding ");
// Try to load a local copy of CarLibrary.
Assembly a = null; try
{
a = Assembly.LoadFrom("CarLibrary");
}
catch(FileNotFoundException ex)
{
Console.WriteLine(ex.Message); return;
}
if(a != null)
{
CreateUsingLateBinding(a);
}
Console.ReadLine();

static void CreateUsingLateBinding(Assembly asm)
{
try
{
// Get metadata for the Minivan type.
Type miniVan = asm.GetType("CarLibrary.MiniVan");

// Create a Minivan instance on the fly.
object obj = Activator.CreateInstance(miniVan); Console.WriteLine("Created a {0} using late binding!", obj);
}
catch(Exception ex)
{
Console.WriteLine(ex.Message);
}
}

Now, before you run this application, you will need to manually place a copy of CarLibrary.dll into the project file folder (or bin\Debug\net6.0 folder if you are using Visual Studio) of this new application.

■ Note Don’t add a reference to CarLibrary.dll for this example! The whole point of late binding is that you are trying to create an object that is not known at compile time.

Notice that the Activator.CreateInstance() method returns a System.Object rather than a strongly typed MiniVan. Therefore, if you apply the dot operator on the obj variable, you will fail to see any members of the MiniVan class. At first glance, you might assume you can remedy this problem with an explicit cast, like so:

// Cast to get access to the members of MiniVan?
// Nope! Compiler error!
object obj = (MiniVan)Activator.CreateInstance(minivan);

However, because your program has not added a reference to CarLibrary.dll, you cannot use the C# using keyword to import the CarLibrary namespace, and therefore, you cannot use a MiniVan type during the casting operation! Remember that the whole point of late binding is to create instances of objects for which there is no compile-time knowledge. Given this, how can you invoke the underlying methods of the MiniVan object stored in the System.Object reference? The answer, of course, is by using reflection.

Invoking Methods with No Parameters
Assume you want to invoke the TurboBoost() method of the MiniVan. As you recall, this method will set the state of the engine to “dead” and display an informational message. The first step is to obtain a MethodInfo object for the TurboBoost() method using Type.GetMethod(). From the resulting MethodInfo, you are then able to call MiniVan.TurboBoost using Invoke(). MethodInfo.Invoke() requires you to send in all parameters that are to be given to the method represented by MethodInfo. These parameters are represented by an array of System.Object types (as the parameters for a given method could be any number of various entities).
Given that TurboBoost() does not require any parameters, you can simply pass null (meaning “this method has no parameters”). Update your CreateUsingLateBinding() method as follows (updates in bold):

static void CreateUsingLateBinding(Assembly asm)
{
try
{
// Get metadata for the Minivan type.
Type miniVan = asm.GetType("CarLibrary.MiniVan");

// Create the Minivan on the fly.
object obj = Activator.CreateInstance(miniVan); Console.WriteLine($"Created a {obj} using late binding!");

// Get info for TurboBoost.
MethodInfo mi = miniVan.GetMethod("TurboBoost");

// Invoke method ('null' for no parameters). mi.Invoke(obj, null);
}
catch(Exception ex)
{
Console.WriteLine(ex.Message);
}
}

At this point, you will see the message in the console that your engine exploded.

Invoking Methods with Parameters
When you want to use late binding to invoke a method requiring parameters, you should package up the arguments as a loosely typed array of objects. The version of the Car class that has a radio and has the following method:

public void TurnOnRadio(bool musicOn, MusicMediaEnum mm)
=> MessageBox.Show(musicOn ? $"Jamming {mm}" : "Quiet time...");

This method takes two parameters: a Boolean representing if the automobile’s music system should be turned on or off and an enum representing the type of music player. Recall this enum was structured as so:

public enum MusicMediaEnum
{
musicCd, // 0
musicTape, // 1
musicRadio, // 2
musicMp3 // 3
}

Here is a new method of the Program.cs file, which invokes TurnOnRadio(). Notice that you are using the underlying numerical values of the MusicMediaEnum enumeration to specify a “radio” media player.

static void InvokeMethodWithArgsUsingLateBinding(Assembly asm)
{
try
{
// First, get a metadata description of the sports car. Type sport = asm.GetType("CarLibrary.SportsCar");

// Now, create the sports car.
object obj = Activator.CreateInstance(sport);
// Invoke TurnOnRadio() with arguments. MethodInfo mi = sport.GetMethod("TurnOnRadio"); mi.Invoke(obj, new object[] { true, 2 });
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}

Ideally, at this point, you can see the relationships among reflection, dynamic loading, and late binding. To be sure, the reflection API provides many additional features beyond what has been covered here, but you should be in good shape to dig into more details if you are interested.
Again, you still might wonder exactly when you should use these techniques in your own applications.
The extendable app later in this chapter should shed light on this issue; however, the next topic under investigation is the role of .NET attributes.

Understanding the Role of .NET Attributes
As illustrated at the beginning of this chapter, one role of a .NET compiler is to generate metadata descriptions for all defined and referenced types. In addition to this standard metadata contained within any assembly, the .NET platform provides a way for programmers to embed additional metadata into
an assembly using attributes. In a nutshell, attributes are nothing more than code annotations that can be applied to a given type (class, interface, structure, etc.), member (property, method, etc.), assembly, or module.
.NET attributes are class types that extend the abstract System.Attribute base class. As you explore the .NET namespaces, you will find many predefined attributes that you are able to use in your applications. Furthermore, you are free to build custom attributes to further qualify the behavior of your types by creating

a new type deriving from Attribute. The .NET base class library provides attributes in various namespaces. Table 17-3 gives a snapshot of some—but by absolutely no means all—predefined attributes.

Table 17-3. A Tiny Sampling of Predefined Attributes

Attribute Meaning in Life

[CLSCompliant] Enforces the annotated item to conform to the rules of the Common Language Specification (CLS). Recall that CLS-compliant types are guaranteed to be used seamlessly across all .NET programming languages.
[DllImport] Allows .NET code to make calls to any unmanaged C- or C++-based code library, including the API of the underlying operating system.
[Obsolete] Marks a deprecated type or member. If other programmers attempt to use such an item, they will receive a compiler warning describing the error of their ways.

Understand that when you apply attributes in your code, the embedded metadata is essentially useless until another piece of software explicitly reflects over the information. If this is not the case, the blurb of metadata embedded within the assembly is ignored and completely harmless.

Attribute Consumers
As you would guess, the .NET Framework ships with numerous utilities that are indeed on the lookout for various attributes. The C# compiler (csc.exe) itself has been preprogrammed to discover the presence
of various attributes during the compilation cycle. For example, if the C# compiler encounters the [CLSCompliant] attribute, it will automatically check the attributed item to ensure it is exposing only CLS- compliant constructs. By way of another example, if the C# compiler discovers an item attributed with the [Obsolete] attribute, it will display a compiler warning.
In addition to development tools, numerous methods in the .NET base class libraries are preprogrammed to reflect over specific attributes. Chapter 19 introduces XML and JSON serialization, both of which use attributes to control the serialization process.
Finally, you are free to build applications that are programmed to reflect over your own custom attributes, as well as any attribute in the .NET base class libraries. By doing so, you are essentially able to create a set of “keywords” that are understood by a specific set of assemblies.

Applying Attributes in C#
To illustrate the process of applying attributes in C#, create a new Console Application project named ApplyingAttributes and add a reference to the System.Text.Json NuGet package. Update the Program.cs file to include the following global using statements:

global using System.Text.Json.Serialization; global using System.Xml.Serialization;

Assume you want to build a class named Motorcycle that can be persisted to JSON format. If you have a field that should not be exported to JSON, you can apply the [JsonIgnore] attribute.

namespace ApplyingAttributes; public class Motorcycle
{

[JsonIgnore]
public float weightOfCurrentPassengers;
// These fields are still serializable. public bool hasRadioSystem;
public bool hasHeadSet; public bool hasSissyBar;
}

■ Note an attribute applies to the “very next” item.

At this point, do not concern yourself with the actual process of object serialization (again, Chapter 19 examines the details). Just notice that when you want to apply an attribute, the name of the attribute is sandwiched between square brackets.
As you might guess, a single item can be attributed with multiple attributes. Assume you have a legacy C# class type (HorseAndBuggy) that was attributed to have a custom XML namespace. The code base has changed over time, and the class is now considered obsolete for current development. Rather than deleting the class definition from your code base (and risk breaking existing software), you can mark the class with the [Obsolete] attribute. To apply multiple attributes to a single item, simply use a comma-delimited list, like so:

namespace ApplyingAttributes;
[XmlRoot(Namespace = "http://www.MyCompany.com"), Obsolete("Use another vehicle!")] public class HorseAndBuggy
{
// ...
}

As an alternative, you can also apply multiple attributes on a single item by stacking each attribute as follows:

[XmlRoot(Namespace = "http://www.MyCompany.com")] [Obsolete("Use another vehicle!")]
public class HorseAndBuggy
{
// ...
}

C# Attribute Shorthand Notation
If you were consulting the .NET documentation, you might have noticed that the actual class name of the [Obsolete] attribute is ObsoleteAttribute, not Obsolete. As a naming convention, all .NET attributes (including custom attributes you may create yourself) are suffixed with the Attribute token. However, to simplify the process of applying attributes, the C# language does not require you to type in the Attribute suffix. Given this, the following iteration of the HorseAndBuggy type is identical to the previous example (it just involves a few more keystrokes):

[XmlRootAttribute(Namespace = "http://www.MyCompany.com")] [ObsoleteAttribute("Use another vehicle!")]
public class HorseAndBuggy

{
// ...
}

Be aware that this is a courtesy provided by C#. Not all .NET languages support this shorthand attribute syntax.

Specifying Constructor Parameters for Attributes
Notice that the [Obsolete] attribute can accept what appears to be a constructor parameter. If you view the formal definition of the [Obsolete] attribute, you will find that this class indeed provides a constructor receiving a System.String.

public sealed class ObsoleteAttribute : Attribute
{
public ObsoleteAttribute();
public ObsoleteAttribute(string? message);
public ObsoleteAttribute(string? message, bool error); public string? Message { get; }
public bool IsError { get; }
public string DiagnosticId { get; set; } public string UrlFormat { get; set; }
}

Understand that when you supply constructor parameters to an attribute, the attribute is not allocated into memory until the parameters are reflected upon by another type or an external tool. The string data defined at the attribute level is simply stored within the assembly as a blurb of metadata.

The Obsolete Attribute in Action
Now that HorseAndBuggy has been marked as obsolete, if you were to allocate an instance of this type:

using ApplyingAttributes;

Console.WriteLine("Hello World!"); HorseAndBuggy mule = new HorseAndBuggy();

you would find that a compiler warning is issued. The warning is specifically CS0618, and the message includes the information passed into the attribute.

'HorseAndBuggy' is obsolete: 'Use another vehicle!'

Visual Studio and Visual Studio Code also help with IntelliSense, which gets their information through reflection. Figure 17-1 shows the results of the Obsolete attribute in Visual Studio through IntelliSense, and Figure 17-2 shows the more detailed messaging in the Visual Studio code editor. Note that both use the term deprecated instead of obsolete.

Figure 17-1. Attributes in action in Visual Studio

Figure 17-2. Hovering over Obsolete types in the Visual Studio editor window

Figure 17-3 and Figure 17-4 show the results of the Obsolete attribute in Visual Studio Code.

Figure 17-3. Attributes in action in Visual Studio Code

Figure 17-4. Hovering over Obsolete types in the Visual Studio Code editor

Ideally, at this point, you should understand the following key points regarding .NET attributes:
•Attributes are classes that derive from System.Attribute.
•Attributes result in embedded metadata.
•Attributes are basically useless until another agent (including IDEs) reflects upon them.
•Attributes are applied in C# using square brackets.

Next up, let’s examine how you can build your own custom attributes and a piece of custom software that reflects over the embedded metadata.

Building Custom Attributes
The first step in building a custom attribute is to create a new class deriving from System.Attribute. Keeping in step with the automobile theme used throughout this book, assume you have created a new C# Class Library project named AttributedCarLibrary.
This assembly will define a handful of vehicles, each of which is described using a custom attribute named VehicleDescriptionAttribute, as follows:

namespace AttributedCarLibrary;
// A custom attribute.
public sealed class VehicleDescriptionAttribute :Attribute
{
public string Description { get; set; }

public VehicleDescriptionAttribute(string description)
=> Description = description;
public VehicleDescriptionAttribute(){ }
}

As you can see, VehicleDescriptionAttribute maintains a piece of string data using an automatic property (Description). Beyond the fact that this class derived from System.Attribute, there is nothing unique to this class definition.

■ Note for security reasons, it is considered a .NET best practice to design all custom attributes as sealed. in fact, both Visual studio and Visual studio Code provide a code snippet named Attribute that will scaffold a new System.Attribute-derived class into your code window. you can expand any snippet by typing its name and pressing the Tab key.

Applying Custom Attributes
Given that VehicleDescriptionAttribute is derived from System.Attribute, you are now able to annotate your vehicles as you see fit. For testing purposes, add the following classes to your new class library:

//Motorcycle.cs
namespace AttributedCarLibrary;
// Assign description using a "named property." [VehicleDescription(Description = "My rocking Harley")] public class Motorcycle
{
}

//HorseAndBuggy.cs
namespace AttributedCarLibrary; [Obsolete ("Use another vehicle!")]

[VehicleDescription("The old gray mare, she ain't what she used to be...")] public class HorseAndBuggy
{
}

//Winnebago.cs
namespace AttributedCarLibrary;
[VehicleDescription("A very long, slow, but feature-rich auto")] public class Winnebago
{
}

Named Property Syntax
Notice that the description of Motorcycle is assigned a description using a new bit of attribute syntax termed a named property. In the constructor of the first [VehicleDescription] attribute, you set the underlying string data by using the Description property. If this attribute is reflected upon by an external agent, the value is fed into the Description property (named property syntax is legal only if the attribute supplies a writable .NET property).
In contrast, the HorseAndBuggy and Winnebago types are not using named property syntax and are simply passing the string data via the custom constructor. In any case, once you compile the
AttributedCarLibrary assembly, you can use ildasm.exe to view the injected metadata descriptions for your type. For example, the following shows the embedded description of the Winnebago class:

// CustomAttribute #1
//
// CustomAttribute Type: 06000005
// CustomAttributeName: AttributedCarLibrary.VehicleDescriptionAttribute :: instance void
.ctor(class System.String)
// Length: 45
// Value : 01 00 28 41 20 76 65 72 79 20 6c 6f 6e 67 2c 20 > (A very long, <
// : 73 6c 6f 77 2c 20 62 75 74 20 66 65 61 74 75 72 >slow, but feature<
// : 65 2d 72 69 63 68 20 61 75 74 6f 00 00 >e-rich auto <
// ctor args: ("A very long, slow, but feature-rich auto")

Restricting Attribute Usage
By default, custom attributes can be applied to just about any aspect of your code (methods, classes, properties, etc.). Thus, if it made sense to do so, you could use VehicleDescription to qualify methods, properties, or fields (among other things).

[VehicleDescription("A very long, slow, but feature-rich auto")] public class Winnebago
{
[VehicleDescription("My rocking CD player")] public void PlayMusic(bool On)
{
...
}
}

In some cases, this is exactly the behavior you require. Other times, however, you may want to build a custom attribute that can be applied only to select code elements. If you want to constrain the scope of a custom attribute, you will need to apply the [AttributeUsage] attribute on the definition of your custom attribute. The [AttributeUsage] attribute allows you to supply any combination of values (via an OR operation) from the AttributeTargets enumeration, like so:

// This enumeration defines the possible targets of an attribute. public enum AttributeTargets
{
All, Assembly, Class, Constructor,
Delegate, Enum, Event, Field, GenericParameter, Interface, Method, Module, Parameter,
Property, ReturnValue, Struct
}

Furthermore, [AttributeUsage] also allows you to optionally set a named property (AllowMultiple) that specifies whether the attribute can be applied more than once on the same item (the default is false). As well, [AttributeUsage] allows you to establish whether the attribute should be inherited by derived classes using the Inherited named property (the default is true).
To establish that the [VehicleDescription] attribute can be applied only once on a class or structure, you can update the VehicleDescriptionAttribute definition as follows:

// This time, we are using the AttributeUsage attribute
// to annotate our custom attribute.
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Struct, Inherited = false)] public sealed class VehicleDescriptionAttribute : System.Attribute
{
...
}

With this, if a developer attempted to apply the [VehicleDescription] attribute on anything other than a class or structure, they are issued a compile-time error.

Assembly-Level Attributes
It is also possible to apply attributes on all types within a given assembly using the [assembly:] tag. For example, assume you want to ensure that every public member of every public type defined within your assembly is CLS compliant. To do so, simply add the following assembly-level attribute at the top of any C# source code file, like this (outside of any namespace declarations):

[assembly: CLSCompliant] namespace AttributedCarLibrary;
...

■ Note all assembly-level or module-level attributes must be listed outside the scope of any namespace scope.

Using a Separate File for Assembly Attributes
Another approach is to add a new file to your project named similar to AssemblyAttributes.cs (any name will work, but that name conveys the purpose of the file) and place your assembly-level attributes in that file. In the .NET Framework, it was recommended to use AssemblyInfo.cs; however, with .NET (Core), that file can’t be used. Placing the attributes in a separate file will make it easier for other developers to find the attributes applied to the project.

■ Note There are two significant changes in .NET. The first is that the AssemblyInfo.cs file is now autogenerated from the project properties and is not meant for developer use. The second (and related) change is that many of the prior assembly-level attributes (Version, Company, etc.) have been replaced with properties in the project file.

Create a new file named AssemblyAttributes.cs and update it to match the following:

// List "using" statements first.

// Now list any assembly- or module-level attributes.
// Enforce CLS compliance for all public types in this
// assembly.
[assembly: CLSCompliant(true)]

If you now add a bit of code that falls outside the CLS specification (such as an exposed point of unsigned data), you will be issued a compiler warning.

// Ulong types don't jibe with the CLS. public class Winnebago
{
public ulong notCompliant;
}

Using the Project File for Assembly Attributes
As shown in Chapter 16 with InternalsVisibleToAttribute, assembly attributes can also be added to the project file. There is a catch, in that only single-string parameter attributes can be used this way. This is true for the properties that can be set on the Package tab in the project properties.

■ Note at the time of this writing, there is an active discussion on the msBuild github repo regarding adding capability for nonstring parameters support. This would enable the CLSCompliant attribute to be added using the project file instead of a *.cs file.

Go ahead and set some of the properties (such as Authors, Description) by right-clicking the project in Solution Explorer, selecting Properties, and then clicking Package. Also, add InternalsVisibleToAttribute as you did in Chapter 16. Your project file will now look something like this:


net6.0
enable
disable
Philip Japikse
Apress
This is a simple car library with attributes

After you compile your project, navigate to the \obj\Debug\net6.0 directory, and look for the AttributedCarLibrary.AssemblyInfo.cs file. Open that, and you will see those properties as attributes (unfortunately, not very readable in this format).

using System;
using System.Reflection;

[assembly: System.Runtime.CompilerServices.InternalsVisibleToAttribute("CSharpCarClient")] [assembly: System.Reflection.AssemblyCompanyAttribute("Philip Japikse")]
[assembly: System.Reflection.AssemblyConfigurationAttribute("Debug")]
[assembly: System.Reflection.AssemblyDescriptionAttribute("This is a sample car library with attributes")]
[assembly: System.Reflection.AssemblyFileVersionAttribute("1.0.0.0")] [assembly: System.Reflection.AssemblyInformationalVersionAttribute("1.0.0")] [assembly: System.Reflection.AssemblyProductAttribute("AttributedCarLibrary")] [assembly: System.Reflection.AssemblyTitleAttribute("AttributedCarLibrary")] [assembly: System.Reflection.AssemblyVersionAttribute("1.0.0.0")]

As a final closing remark on assembly attributes, you can turn off the generation of the AssemblyInfo. cs class if you want to manage the process yourself.

Reflecting on Attributes Using Early Binding
Remember that an attribute is quite useless until another piece of software reflects over its values. Once a given attribute has been discovered, that piece of software can take whatever course of action necessary. Now, like any application, this “other piece of software” could discover the presence of a custom attribute using either early binding or late binding. If you want to make use of early binding, you’ll require the client application to have a compile-time definition of the attribute in question (VehicleDescriptionAttribute, in this case). Given that the AttributedCarLibrary assembly has defined this custom attribute as a public class, early binding is the best option.

To illustrate the process of reflecting on custom attributes, add a new C# Console Application project named VehicleDescriptionAttributeReader to the solution. Next, add a reference to the AttributedCarLibrary project. Using the CLI, execute these commands (each command must be on its own line):

dotnet new console -lang c# -n VehicleDescriptionAttributeReader -o .\ VehicleDescriptionAttributeReader -f net6.0
dotnet sln .\Chapter17_AllProjects.sln add .\VehicleDescriptionAttributeReader dotnet add VehicleDescriptionAttributeReader reference .\AttributedCarLibrary

Update the Program.cs file with the following code:

using AttributedCarLibrary;

Console.WriteLine(" Value of VehicleDescriptionAttribute \n"); ReflectOnAttributesUsingEarlyBinding();
Console.ReadLine();

static void ReflectOnAttributesUsingEarlyBinding()
{
// Get a Type representing the Winnebago. Type t = typeof(Winnebago);

// Get all attributes on the Winnebago.
object[] customAtts = t.GetCustomAttributes(false);

// Print the description.
foreach (VehicleDescriptionAttribute v in customAtts)
{
Console.WriteLine("-> {0}\n", v.Description);
}
}

The Type.GetCustomAttributes() method returns an object array that represents all the attributes applied to the member represented by the Type (the Boolean parameter controls whether the search should extend up the inheritance chain). Once you have obtained the list of attributes, iterate over each VehicleDescriptionAttribute class, and print out the value obtained by the Description property.

Reflecting on Attributes Using Late Binding
The previous example used early binding to print out the vehicle description data for the Winnebago type. This was possible because the VehicleDescriptionAttribute class type was defined as a public member in the AttributedCarLibrary assembly. It is also possible to make use of dynamic loading and late binding to reflect over attributes.
Add a new project called VehicleDescriptionAttributeReaderLateBinding to the solution, set it as the startup project, and copy AttributedCarLibrary.dll to the project’s folder (or \bin\Debug\net6.0 if using Visual Studio). Now, update your Program.cs file as follows:

using System.Reflection;

Console.WriteLine(" Value of VehicleDescriptionAttribute \n");

ReflectAttributesUsingLateBinding(); Console.ReadLine();

static void ReflectAttributesUsingLateBinding()
{
try
{
// Load the local copy of AttributedCarLibrary.
Assembly asm = Assembly.LoadFrom("AttributedCarLibrary");

// Get type info of VehicleDescriptionAttribute.
Type vehicleDesc = asm.GetType("AttributedCarLibrary.VehicleDescriptionAttribute");

// Get type info of the Description property.
PropertyInfo propDesc = vehicleDesc.GetProperty("Description");

// Get all types in the assembly.
Type[] types = asm.GetTypes();

// Iterate over each type and obtain any VehicleDescriptionAttributes.
foreach (Type t in types)
{
object[] objs = t.GetCustomAttributes(vehicleDesc, false);

// Iterate over each VehicleDescriptionAttribute and print
// the description using late binding.
foreach (object o in objs)
{
Console.WriteLine("-> {0}: {1}\n", t.Name, propDesc.GetValue(o, null));
}
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}

If you were able to follow along with the examples in this chapter, this code should be (more or less) self-explanatory. The only point of interest is the use of the PropertyInfo.GetValue() method, which is used to trigger the property’s accessor. Here is the output of the current example:

Value of VehicleDescriptionAttribute
-> Motorcycle: My rocking Harley

-> HorseAndBuggy: The old gray mare, she ain't what she used to be...

-> Winnebago: A very long, slow, but feature-rich auto

Putting Reflection, Late Binding, and Custom Attributes in Perspective
Even though you have seen numerous examples of these techniques in action, you may still be wondering when to make use of reflection, dynamic loading, late binding, and custom attributes in your programs. To be sure, these topics can seem a bit on the academic side of programming (which may or may not be a bad thing, depending on your point of view). To help map these topics to a real-world situation, you need a solid example. Assume for the moment that you are on a programming team that is building an application with the following requirement:
The product must be extendable using additional third-party tools.
What exactly is meant by extendable? Well, consider the Visual Studio or Visual Studio Code IDEs. When this application was developed, various “hooks” were inserted into the code base to allow other software vendors to “snap” (or plug in) custom modules into the IDE. Obviously, the Visual Studio/Visual Studio Code development teams had no way to set references to external .NET assemblies that had not been developed yet (thus, no early binding), so how exactly would an application provide the required hooks? Here is one possible way to solve this problem:
1.First, an extendable application must provide some input mechanism to allow the user to specify the module to plug in (such as a dialog box or command-line flag). This requires dynamic loading.
2.Second, an extendable application must be able to determine whether the module supports the correct functionality (such as a set of required interfaces) to be plugged into the environment. This requires reflection.
3.Finally, an extendable application must obtain a reference to the required infrastructure (such as a set of interface types) and invoke the members to trigger the underlying functionality. This may require late binding.
Simply put, if the extendable application has been preprogrammed to query for specific interfaces, it is able to determine at runtime whether the type can be activated. Once this verification test has been passed, the type in question may support additional interfaces that provide a polymorphic fabric to their functionality. This is the exact approach taken by the Visual Studio team and, despite what you might be thinking, is not at all difficult!

Building an Extendable Application
In the sections that follow, I will take you through an example that illustrates the process of building an application that can be augmented by the functionality of external assemblies. To serve as a road map, the extendable application entails the following assemblies:
•CommonSnappableTypes.dll: This assembly contains type definitions that will be used by each snap-in object and will be directly referenced by the Windows Forms application.
•CSharpSnapIn.dll: A snap-in written in C#, which leverages the types of
CommonSnappableTypes.dll.
•VBSnapIn.dll: A snap-in written in Visual Basic, which leverages the types of
CommonSnappableTypes.dll.
•MyExtendableApp.exe: A console application that may be extended by the functionality of each snap-in.

This application will use dynamic loading, reflection, and late binding to dynamically gain the functionality of assemblies it has no prior knowledge of.

■ Note you might be thinking to yourself, “my boss has never asked me to build a console application,” and you are probably correct! line-of-business applications built with C# usually fall into the category of smart client (Winforms or Wpf), web applications/rEsTful services (asp.NET Core), or headless processes (azure functions, Windows services, etc.). We are using console applications to focus on the specific concepts of the example, in this case dynamic loading, reflection, and late binding. later in this book, you will explore “real” user-facing applications using Wpf and asp.NET Core.

Building the Multiproject ExtendableApp Solution
Up to this point in this book, most of the applications have been stand-alone projects, with a few exceptions (like the previous one). This was done to keep the examples simple and focused. However, in real-world development, you typically work with multiple projects together in a solution.

Creating the Solution and Projects with the CLI
To get started using the CLI, enter the following commands to create a new solution, the class libraries and console application, and the project references:

dotnet new sln -n Chapter17_ExtendableApp

dotnet new classlib -lang c# -n CommonSnappableTypes -o .\CommonSnappableTypes -f net6.0 dotnet sln .\Chapter17_ExtendableApp.sln add .\CommonSnappableTypes

dotnet new classlib -lang c# -n CSharpSnapIn -o .\CSharpSnapIn -f net6.0 dotnet sln .\Chapter17_ExtendableApp.sln add .\CSharpSnapIn
dotnet add CSharpSnapin reference CommonSnappableTypes

dotnet new classlib -lang vb -n VBSnapIn -o .\VBSnapIn -f net6.0 dotnet sln .\Chapter17_ExtendableApp.sln add .\VBSnapIn
dotnet add VBSnapIn reference CommonSnappableTypes

dotnet new console -lang c# -n MyExtendableApp -o .\MyExtendableApp -f net6.0 dotnet sln .\Chapter17_ExtendableApp.sln add .\MyExtendableApp
dotnet add MyExtendableApp reference CommonSnappableTypes

Adding PostBuild Events into the Project Files
When projects are built (either from Visual Studio or from the command line), there are events that can be hooked into. For example, we want to copy the two snap-in assemblies into the console app project directory (when debugging with dotnet run) and the console app output directory (when debugging with Visual Studio) after every successful build. To do this, we are going to take advantage of several built-in macros.

If you are using Visual Studio, copy this block of markup into the CSharpSnapIn.csproj and VBSnapIn.vbproj files, which copies the compiled assembly into the MyExtendableApp output directory (MyExtendableApp\bin\debug\net6.0):



If you are using Visual Studio Code, copy this block of markup into the CSharpSnapIn.csproj and
VBSnapIn.vbproj files, which copies the compiled assembly into the MyExtendableApp project directory:



Now when each project is built, its assembly is copied into the MyExtendableApp’s target directory as well.

Creating the Solution and Projects with Visual Studio
Recall that, by default, Visual Studio names the solution the same as the first project created in that solution. However, you can easily change the name of the solution, as shown in Figure 17-5.

Figure 17-5. Creating the CommonSnappableTypes project and the ExtendableApp solution

To create the ExtendableApp solution, start by selecting File ➤ New Project to load the New Project dialog. Select Class Library and enter the name CommonSnappableTypes. Before you click OK, enter the solution name ExtendableApp, as shown in Figure 17-5.
To add another project into the solution, right-click the solution name (ExtendableApp) in Solution Explorer (or click File ➤ Add ➤ New Project) and select Add ➤ New Project. When adding another project into an existing solution, the Add New Project dialog box is a little different now; the solution options are no longer there, so you will see just the project information (name and location). Name the Class Library project CSharpSnapIn and click Create.
Next, add a reference to the CommonSnappableTypes project from the CSharpSnapIn project. To do this in Visual Studio, right-click the CSharpSnapIn project and select Add ➤ Project Reference. In the
Reference Manager dialog, select Projects ➤ Solution from the left (if not already selected) and select the box
next to CommonSnappableTypes.
Repeat the process for a new Visual Basic class library (VBSnapIn) that references the CommonSnappableTypes project.
The final project to add is a .NET Console app named MyExtendableApp. Add a reference to the CommonSnappableTypes project and set the console app as the startup project for the solution. To do this, right-click the MyExtendableApp project in Solution Explorer and select Set as StartUp Project.

■ Note if you right-click the Extendableapp solution instead of one of the projects, the context menu option displayed is set startup projects. in addition to having just one project execute when you click run, you can set up multiple projects to execute. This will be demonstrated in later chapters.

Setting Project Build Dependencies
When Visual Studio is given the command to run a solution, the startup projects and all referenced projects are built if any changes are detected; however, any unreferenced projects are not built. This can be changed by setting project dependencies. To do this, right-click the solution in Solution Explorer, select Project Build Order, and, in the resulting dialog, select the Dependencies tab and change the project to MyExtendableApp.
Notice that the CommonSnappableTypes project is already selected and the check box is disabled. This is because it is referenced directly. Select the CSharpSnapIn and VBSnapIn project check boxes as well, as shown in Figure 17-6.

Figure 17-6. Accessing the Project Build Order context menu

Now, each time the MyExtendableApp project is built, the CSharpSnapIn and VBSnapIn projects build as well.

Adding PostBuild Events
Open the project properties for the CSharpSnapIn (right-click Solution Explorer and select Properties) and navigate to the Build Events page (C#). Click the Edit Post-build button and then click Macros>>. Here you can see the macros available for use, and they all refer to paths and/or filenames. The advantage of using these macros in build events is that they are machine independent and work on the relative paths. For example, I am working in a directory called c-sharp-wf\code\chapter17. You might be (and probably are) using a different directory. By using the macros, MSBuild will always use the correct path relative to the
*.csproj files.
In the PostBuild box, enter the following (over two lines):

copy $(TargetPath) $(SolutionDir)MyExtendableApp\$(OutDir)$(TargetFileName) /Y copy $(TargetPath) $(SolutionDir)MyExtendableApp\$(TargetFileName) /Y

Do the same for the VBSnapIn project, except the property page is called Compile, and from there you can click the Build Events button.
After you have added these post-build event commands, each assembly will be copied into the MyExtendableApp’s project and output directories each time they are compiled.

Building CommonSnappableTypes.dll
In the CommonSnappableTypes project, delete the default Class1.cs file, add a new interface file named
IAppFunctionality.cs, and update the file to the following:

namespace CommonSnappableTypes; public interface IAppFunctionality
{
void DoIt();
}

Add a class file named CompanyInfoAttribute.cs and update it to the following:

namespace CommonSnappableTypes; [AttributeUsage(AttributeTargets.Class)]
public sealed class CompanyInfoAttribute : System.Attribute
{
public string CompanyName { get; set; } public string CompanyUrl { get; set; }
}

The IAppFunctionality interface provides a polymorphic interface for all snap-ins that can be consumed by the extendable application. Given that this example is purely illustrative, you supply a single method named DoIt().
The CompanyInfoAttribute type is a custom attribute that can be applied on any class type that wants to be snapped into the container. As you can tell by the definition of this class, [CompanyInfo] allows the developer of the snap-in to provide some basic details about the component’s point of origin.

Building the C# Snap-In
In the CSharpSnapIn project, delete the Class1.cs file and add a new file named CSharpModule.cs. Update the code to match the following:
using CommonSnappableTypes; namespace CSharpSnapIn;
[CompanyInfo(CompanyName = "FooBar", CompanyUrl = "www.FooBar.com")] public class CSharpModule : IAppFunctionality
{
void IAppFunctionality.DoIt()
{
Console.WriteLine("You have just used the C# snap-in!");
}
}

Notice that I chose to make use of explicit interface implementation (see Chapter 8) when supporting the IAppFunctionality interface. This is not required; however, the idea is that the only part of the system that needs to directly interact with this interface type is the hosting application. By explicitly implementing this interface, the DoIt() method is not directly exposed from the CSharpModule type.

Building the Visual Basic Snap-In
Moving on to the VBSnapIn project, delete the Class1.vb file and add a new file named VBSnapIn.vb. The code is (again) intentionally simple.

Imports CommonSnappableTypes

<CompanyInfo(CompanyName:="Chucky's Software", CompanyUrl:="www.ChuckySoft.com")> Public Class VBSnapIn
Implements IAppFunctionality

Public Sub DoIt() Implements CommonSnappableTypes.IAppFunctionality.DoIt Console.WriteLine("You have just used the VB snap in!")
End Sub End Class

Notice that applying attributes in the syntax of Visual Basic requires angle brackets (< >) rather than square brackets ([ ]). Also notice that the Implements keyword is used to implement interface types on a given class or structure.

Adding the Code for the ExtendableApp
The final project to update is the C# console application (MyExtendableApp). After adding the MyExtendableApp console application to the solution and setting it as the startup project, add a reference to the CommonSnappableTypes project but not the CSharpSnapIn.dll or VBSnapIn.dll project.
Begin by updating the using statements at the top of the Program.cs class to the following:

using System.Reflection; using CommonSnappableTypes;

The LoadExternalModule() method performs the following tasks:
• Dynamically loads the selected assembly into memory
• Determines whether the assembly contains any types implementing
IAppFunctionality
• Creates the type using late binding
If a type implementing IAppFunctionality is found, the DoIt() method is called and then sent to the
DisplayCompanyData() method to output additional information from the reflected type.

static void LoadExternalModule(string assemblyName)
{
Assembly theSnapInAsm = null; try
{
// Dynamically load the selected assembly. theSnapInAsm = Assembly.LoadFrom(assemblyName);
}
catch (Exception ex)
{

Console.WriteLine($"An error occurred loading the snapin: {ex.Message}"); return;
}

// Get all IAppFunctionality compatible classes in assembly. var theClassTypes = theSnapInAsm
.GetTypes()
.Where(t => t.IsClass && (t.GetInterface("IAppFunctionality") != null))
.ToList();
if (!theClassTypes.Any())
{
Console.WriteLine("Nothing implements IAppFunctionality!");
}

// Now, create the object and call DoIt() method. foreach (Type t in theClassTypes)
{
// Use late binding to create the type.
IAppFunctionality itfApp = (IAppFunctionality) theSnapInAsm.CreateInstance (t.FullName, true);
itfApp?.DoIt();
// Show company info.
DisplayCompanyData(t);
}
}

The final task is to display the metadata provided by the [CompanyInfo] attribute. Create the
DisplayCompanyData() method as follows. Notice this method takes a single System.Type parameter.

static void DisplayCompanyData(Type t)
{
// Get [CompanyInfo] data. var compInfo = t
.GetCustomAttributes(false)
.Where(ci => (ci is CompanyInfoAttribute));
// Show data.
foreach (CompanyInfoAttribute c in compInfo)
{
Console.WriteLine($"More info about {c.CompanyName} can be found at {c.CompanyUrl}");
}
}

Finally, update the top-level statements to the following:

Console.WriteLine(" Welcome to MyTypeViewer "); string typeName = "";
do
{
Console.WriteLine("\nEnter a snapin to load"); Console.Write("or enter Q to quit: ");

// Get name of type.
typeName = Console.ReadLine();

// Does user want to quit?
if (typeName.Equals("Q", StringComparison.OrdinalIgnoreCase))
{
break;
}
// Try to display type. try
{
LoadExternalModule(typeName);
}
catch (Exception ex)
{
Console.WriteLine("Sorry, can't find snapin");
}
}
while (true);

That wraps up the example application. When you run the application, type in either VBSnapIn or CSharpSnapIn and see the program in action. Note that while C# is case sensitive, when using reflection, case doesn’t matter. Both CSharpSnapIn and csharpsnapin work the equally as well.
This concludes our look at late binding. Next up is exploring dynamics in C#.

The Role of the C# dynamic Keyword
In Chapter 3, you learned about the var keyword, which allows you to define local variables in such a way that the underlying data type is determined at compile time, based on the initial assignment (recall that this is termed implicit typing). Once this initial assignment has been made, you have a strongly typed variable, and any attempt to assign an incompatible value will result in a compiler error.
To begin your investigation into the C# dynamic keyword, create a new Console Application project named DynamicKeyword. Now, add the following method in your Program.cs file, and verify that the final code statement will indeed trigger a compile-time error if uncommented:

static void ImplicitlyTypedVariable()
{
// a is of type List. var a = new List {90};
// This would be a compile-time error!
// a = "Hello";
}

As you saw in Chapter 13, implicit typing is useful with LINQ, as many LINQ queries return enumerations of anonymous classes (via projections) that you cannot directly declare in your C# code. However, even in such cases, the implicitly typed variable is, in fact, strongly typed. In the previous example, the variable a is strongly typed to be List.
On a related note, as you learned in Chapter 6, System.Object is the topmost parent class in the .NET Core Framework and can represent anything at all. If you declare a variable of type object, you have a strongly typed piece of data; however, what it points to in memory can differ based on your assignment of

the reference. To gain access to the members the object reference is pointing to in memory, you need to perform an explicit cast.
Assume you have a simple class named Person that defines two automatic properties (FirstName and
LastName) both encapsulating a string:
//Person.cs
namespace DynamicKeyword; class Person
{
public string FirstName { get; set; } = ""; public string LastName { get; set; } = "";
}

Now, observe the following code:

static void UseObjectVariable()
{
// Create a new instance of the Person class
// and assign it to a variable of type System.Object
object o = new Person() { FirstName = "Mike", LastName = "Larson" };

// Must cast object as Person to gain access
// to the Person properties.
Console.WriteLine("Person's first name is {0}", ((Person)o).FirstName);
}

Now, back to the dynamic keyword. From a high level, you can consider the dynamic keyword a specialized form of System.Object, in that any value can be assigned to a dynamic data type. At first glance, this can appear horribly confusing, as it appears you now have three ways to define data whose underlying type is not directly indicated in your code base. For example, this method

static void PrintThreeStrings()
{
var s1 = "Greetings"; object s2 = "From";
dynamic s3 = "Minneapolis";

Console.WriteLine("s1 is of type: {0}", s1.GetType()); Console.WriteLine("s2 is of type: {0}", s2.GetType()); Console.WriteLine("s3 is of type: {0}", s3.GetType());
}

would print out the following if invoked from your top-level statements:

s1 is of type: System.String s2 is of type: System.String s3 is of type: System.String

What makes a dynamic variable vastly different from a variable declared implicitly or via a System.
Object reference is that it is not strongly typed. Said another way, dynamic data is not statically typed. As far as the C# compiler is concerned, a data point declared with the dynamic keyword can be assigned any initial

value at all and can be reassigned to any new (and possibly unrelated) value during its lifetime. Consider the following method and the resulting output:

static void ChangeDynamicDataType()
{
// Declare a single dynamic data point
// named "t".
dynamic t = "Hello!";
Console.WriteLine("t is of type: {0}", t.GetType());

t = false;
Console.WriteLine("t is of type: {0}", t.GetType());

t = new List();
Console.WriteLine("t is of type: {0}", t.GetType());
}

t is of type: System.String t is of type: System.Boolean
t is of type: System.Collections.Generic.List`1[System.Int32]

At this point in your investigation, do be aware that the previous code would compile and execute identically if you were to declare the t variable as a System.Object. However, as you will soon see, the dynamic keyword offers many additional features.

Calling Members on Dynamically Declared Data
Given that a dynamic variable can take on the identity of any type on the fly (just like a variable of type System.Object), the next question on your mind might be about calling members on the dynamic variable (properties, methods, indexers, register with events, etc.). Syntactically speaking, it will look no different. Just apply the dot operator to the dynamic data variable, specify a public member, and supply any arguments (if required).
However (and this is a very big “however”), the validity of the members you specify will not be checked by the compiler! Remember, unlike a variable defined as a System.Object, dynamic data is not statically typed. It is not until runtime that you will know whether the dynamic data you invoked supports a specified member, whether you passed in the correct parameters, whether you spelled the member correctly, and so on. Thus, as strange as it might seem, the following method compiles perfectly:

static void InvokeMembersOnDynamicData()
{
dynamic textData1 = "Hello"; Console.WriteLine(textData1.ToUpper());

// You would expect compiler errors here!
// But they compile just fine. Console.WriteLine(textData1.toupper()); Console.WriteLine(textData1.Foo(10, "ee", DateTime.Now));
}

Notice the second call to WriteLine() attempts to call a method named toupper() on the dynamic data point (note the incorrect casing—it should be ToUpper()). As you can see, textData1 is of type string, and therefore, you know it does not have a method of this name in all lowercase letters. Furthermore, string certainly does not have a method named Foo() that takes int, string, and DateTime objects!
Nevertheless, the C# compiler is satisfied. However, if you invoke this method from within Main(), you will get runtime errors like the following output:

Unhandled Exception: Microsoft.CSharp.RuntimeBinder.RuntimeBinderException: 'string' does not contain a definition for 'toupper'

Another obvious distinction between calling members on dynamic data and strongly typed data is that when you apply the dot operator to a piece of dynamic data, you will not see the expected Visual
Studio IntelliSense. The IDE will allow you to enter any member name you could dream up. This means you need to be extremely careful when you are typing C# code on such data points. Any misspelling
or incorrect capitalization of a member will throw a runtime error, specifically an instance of the
RuntimeBinderException class.
The RuntimeBinderException represents an error that will be thrown if you attempt to invoke a member on a dynamic data type that does not actually exist (as in the case of the toupper() and Foo() methods). This same error will be raised if you specify the wrong parameter data to a member that does exist.
Because dynamic data is so volatile, whenever you are invoking members on a variable declared with the C# dynamic keyword, you could wrap the calls within a proper try/catch block and handle the error in a graceful manner, like so:

static void InvokeMembersOnDynamicData()
{
dynamic textData1 = "Hello";

try
{
Console.WriteLine(textData1.ToUpper()); Console.WriteLine(textData1.toupper()); Console.WriteLine(textData1.Foo(10, "ee", DateTime.Now));
}
catch (Microsoft.CSharp.RuntimeBinder.RuntimeBinderException ex)
{
Console.WriteLine(ex.Message);
}
}

If you call this method again, you will find the call to ToUpper() (note the capital T and U) works correctly; however, you then find the error data displayed to the console.

HELLO
'string' does not contain a definition for 'toupper'

Of course, the process of wrapping all dynamic method invocations in a try/catch block is rather tedious. If you watch your spelling and parameter passing, this is not required. However, catching exceptions is handy when you might not know in advance if a member will be present on the target type.

The Scope of the dynamic Keyword
Recall that implicitly typed data (declared with the var keyword) is possible only for local variables in a member scope. The var keyword can never be used as a return value, a parameter, or a member of a class/ structure. This is not the case with the dynamic keyword, however. Consider the following class definition:

namespace DynamicKeyword; class VeryDynamicClass
{
// A dynamic field.
private static dynamic _myDynamicField;

// A dynamic property.
public dynamic DynamicProperty { get; set; }

// A dynamic return type and a dynamic parameter type.
public dynamic DynamicMethod(dynamic dynamicParam)
{
// A dynamic local variable.
dynamic dynamicLocalVar = "Local variable"; int myInt = 10;
if (dynamicParam is int)
{
return dynamicLocalVar;
}
else
{
return myInt;
}
}
}

You could now invoke the public members as expected; however, as you are operating on dynamic methods and properties, you cannot be completely sure what the data type will be! While the
VeryDynamicClass definition might not be useful in a real-world application, it does illustrate the scope of where you can apply this C# keyword.

Limitations of the dynamic Keyword
While a great many things can be defined using the dynamic keyword, there are some limitations regarding its usage. While they are not showstoppers, do know that a dynamic data item cannot make use of lambda expressions or C# anonymous methods when calling a method. For example, the following code will always result in errors, even if the target method does indeed take a delegate parameter that takes a string value and returns void:

dynamic a = GetDynamicObject();

// Error! Methods on dynamic data can't use lambdas! a.Method(arg => Console.WriteLine(arg));

To circumvent this restriction, you will need to work with the underlying delegate directly, using the techniques described in Chapter 12. Another limitation is that a dynamic point of data cannot understand any extension methods (see Chapter 11). Unfortunately, this would also include any of the extension methods that come from the LINQ APIs. Therefore, a variable declared with the dynamic keyword has limited use within LINQ to Objects and other LINQ technologies.

dynamic a = GetDynamicObject();
// Error! Dynamic data can't find the Select() extension method! var data = from d in a select d;

Practical Uses of the dynamic Keyword
Given that dynamic data is not strongly typed, not checked at compile time, has no ability to trigger IntelliSense, and cannot be the target of a LINQ query, you are absolutely correct to assume that using the dynamic keyword just for the sake of doing so is a poor programming practice.
However, in a few circumstances, the dynamic keyword can radically reduce the amount of code you
need to author by hand. Specifically, if you are building a .NET application that makes heavy use of late binding (via reflection), the dynamic keyword can save you typing time. As well, if you are building a .NET application that needs to communicate with legacy COM libraries (such as Microsoft Office products), you can greatly simplify your code base via the dynamic keyword. By way of a final example, web applications built using ASP.NET Core frequently use the ViewBag type, which can also be accessed in a simplified manner using the dynamic keyword.

■Note Com interaction is strictly a Windows paradigm and removes the cross-platform capabilities of your application.

Like any “shortcut,” you need to weigh the pros and cons. The use of the dynamic keyword is a trade- off between brevity of code and type safety. While C# is a strongly typed language at its core, you can opt in (or opt out) of dynamic behaviors on a call-by-call basis. Always remember that you never need to use the dynamic keyword. You could always get to the same end result by authoring alternative code by hand (and typically much more of it).

The Role of the Dynamic Language Runtime
Now that you better understand what “dynamic data” is about, let’s learn how it is processed. Since the release of .NET 4.0, the Common Language Runtime (CLR) was supplemented with a complementary runtime environment named the Dynamic Language Runtime. The concept of a “dynamic runtime” is certainly not new. In fact, many programming languages such as JavaScript, LISP, Ruby, and Python have used it for years. In a nutshell, a dynamic runtime allows a dynamic language the ability to discover types completely at runtime with no compile-time checks.

■Note While a lot of the Dlr was ported to .NET Core (starting with 3.0), feature parity between the Dlr in
.NET 6 and .NET 4.8 hasn’t been achieved.

If you have a background in strongly typed languages (including C#, without dynamic types), the notion of such a runtime might seem undesirable. After all, you typically want to receive compile-time errors, not

runtime errors, wherever possible. Nevertheless, dynamic languages/runtimes do provide some interesting features, including the following:
•An extremely flexible code base. You can refactor code without making numerous changes to data types.
•A simple way to interoperate with diverse object types built in different platforms and programming languages.
•A way to add or remove members to a type, in memory, at runtime.
One role of the DLR is to enable various dynamic languages to run with the .NET Runtime and give them a way to interoperate with other .NET code. These languages live in a dynamic universe, where type is discovered solely at runtime. And yet, these languages have access to the richness of the .NET base class
libraries. Even better, their code bases can interoperate with C# (or vice versa), thanks to the inclusion of the
dynamic keyword.

■ Note This chapter will not address how the Dlr can be used to integrate with dynamic languages.

The Role of Expression Trees
The DLR makes use of expression trees to capture the meaning of a dynamic call in neutral terms. For example, take the following C# code:

dynamic d = GetSomeData(); d.SuperMethod(12);

In this example, the DLR will automatically build an expression tree that says, in effect, “Call the method named SuperMethod on object d, passing in the number 12 as an argument.” This information (formally termed the payload) is then passed to the correct runtime binder, which again could be the C# dynamic binder or even (as explained shortly) legacy COM objects.
From here, the request is mapped into the required call structure for the target object. The nice thing about these expression trees (beyond that you do not need to manually create them) is that this allows you to write a fixed C# code statement and not worry about what the underlying target actually is.

Dynamic Runtime Lookup of Expression Trees
As explained, the DLR will pass the expression trees to a target object; however, this dispatching will be influenced by a few factors. If the dynamic data type is pointing in memory to a COM object, the expression tree is sent to a low-level COM interface named IDispatch. As you might know, this interface was COM’s way of incorporating its own set of dynamic services. COM objects, however, can be used in a .NET application without the use of the DLR or C# dynamic keyword. Doing so, however (as you will see), tends to result in much more complex C# coding.
If the dynamic data is not pointing to a COM object, the expression tree may be passed to an object implementing the IDynamicObject interface. This interface is used behind the scenes to allow a language, such as IronRuby, to take a DLR expression tree and map it to Ruby specifics.
Finally, if the dynamic data is pointing to an object that is not a COM object and does not implement IDynamicObject, the object is a normal, everyday .NET object. In this case, the expression tree is dispatched to the C# runtime binder for processing. The process of mapping the expression tree to .NET specifics involves reflection services.

After the expression tree has been processed by a given binder, the dynamic data will be resolved to the real in-memory data type, after which the correct method is called with any necessary parameters. Now, let’s look at a few practical uses of the DLR, beginning with the simplification of late-bound .NET calls.

Simplifying Late-Bound Calls Using Dynamic Types
One instance where you might decide to use the dynamic keyword is when you are working with reflection services, specifically when making late-bound method calls. Earlier in this chapter, you saw a few examples of when this type of method call can be useful, most commonly when you are building some type of extensible application. At that time, you learned how to use the Activator.CreateInstance() method to create an object, for which you have no compile-time knowledge of (beyond its display name). You can then make use of the types of the System.Reflection namespace to invoke members via late binding. Recall the earlier example:

static void CreateUsingLateBinding(Assembly asm)
{
try
{
// Get metadata for the Minivan type.
Type miniVan = asm.GetType("CarLibrary.MiniVan");

// Create the Minivan on the fly.
object obj = Activator.CreateInstance(miniVan);

// Get info for TurboBoost.
MethodInfo mi = miniVan.GetMethod("TurboBoost");

// Invoke method ("null" for no parameters). mi.Invoke(obj, null);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}

While this code works as expected, you might agree it is a bit clunky. You must manually make use of the MethodInfo class, manually query the metadata, and so forth. The following is a version of this same method, now using the C# dynamic keyword and the DLR:

static void InvokeMethodWithDynamicKeyword(Assembly asm)
{
try
{
// Get metadata for the Minivan type.
Type miniVan = asm.GetType("CarLibrary.MiniVan");

// Create the Minivan on the fly and call method! dynamic obj = Activator.CreateInstance(miniVan); obj.TurboBoost();
}

catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}

By declaring the obj variable using the dynamic keyword, the heavy lifting of reflection is done on your behalf, courtesy of the DRL.

Leveraging the dynamic Keyword to Pass Arguments
The usefulness of the DLR becomes even more obvious when you need to make late-bound calls on methods that take parameters. When you use “longhand” reflection calls, arguments need to be packaged up as an array of objects, which are passed to the Invoke() method of MethodInfo.
To illustrate using a fresh example, begin by creating a new C# Console Application project named LateBindingWithDynamic. Next, add a Class Library project named MathLibrary. Rename the initial Class1.cs file of the MathLibrary project to SimpleMath.cs, and implement the class like so:

namespace MathLibrary; public class SimpleMath
{
public int Add(int x, int y)
{
return x + y;
}
}

If you are using Visual Studio, update the MathLibrary.csproj file with the following (to copy the compiled assembly to the LateBindingWithDynamic target directory):



If you are using Visual Studio Code, update the MathLibrary.csproj file with the following (to copy the compiled assembly to the LateBindingWithDynamic project directory):



Now, back in the LateBindingWithDynamic project and the Program.cs file, update the using
statements to the following:

using System.Reflection;
using Microsoft.CSharp.RuntimeBinder;

Next, add the following method to the Program.cs file, which invokes the Add() method using typical reflection API calls:

static void AddWithReflection()
{
Assembly asm = Assembly.LoadFrom("MathLibrary"); try
{
// Get metadata for the SimpleMath type.
Type math = asm.GetType("MathLibrary.SimpleMath");

// Create a SimpleMath on the fly.
object obj = Activator.CreateInstance(math);

// Get info for Add.
MethodInfo mi = math.GetMethod("Add");

// Invoke method (with parameters).
object[] args = { 10, 70 };
Console.WriteLine("Result is: {0}", mi.Invoke(obj, args));
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}

Now, consider the simplification of the previous logic with the dynamic keyword, via the following new method:

static void AddWithDynamic()
{
Assembly asm = Assembly.LoadFrom("MathLibrary");

try
{
// Get metadata for the SimpleMath type.
Type math = asm.GetType("MathLibrary.SimpleMath");

// Create a SimpleMath on the fly.
dynamic obj = Activator.CreateInstance(math);

// Note how easily we can now call Add().
Console.WriteLine("Result is: {0}", obj.Add(10, 70));
}
catch (RuntimeBinderException ex)
{
Console.WriteLine(ex.Message);
}
}

Not too shabby! If you call both methods, you will see identical output. However, when using the dynamic keyword, you saved yourself quite a bit of work. With dynamically defined data, you no longer need to manually package up arguments as an array of objects, query the assembly metadata, or set other such details. If you are building an application that makes heavy use of dynamic loading/late binding, I am sure you can see how these code savings would add up over time.

Summary
Reflection is an interesting aspect of a robust object-oriented environment. In the world of .NET, the keys to reflection services revolve around the System.Type class and the System.Reflection namespace. As you have seen, reflection is the process of placing a type under the magnifying glass at runtime to understand the who, what, where, when, why, and how of a given item.
Late binding is the process of creating an instance of a type and invoking its members without prior knowledge of the specific names of said members. Late binding is often a direct result of dynamic loading, which allows you to load a .NET assembly into memory programmatically. As shown during this chapter’s extendable application example, this is a powerful technique used by tool builders as well as tool consumers.
This chapter also examined the role of attribute-based programming. When you adorn your types with attributes, the result is the augmentation of the underlying assembly metadata.
The dynamic keyword allows you to define data whose identity is not known until runtime. When processed by the Dynamic Language Runtime, the automatically created “expression tree” will be passed to the correct dynamic language binder, where the payload will be unpackaged and sent to the correct object member.
Using dynamic data and the DLR, complex C# programming tasks can be radically simplified, especially the act of incorporating COM libraries into your .NET applications.
While these features can certainly simplify your code, always remember that dynamic data makes your C# code much less type-safe and open to runtime errors. Be sure you weigh the pros and cons of using dynamic data in your C# projects, and test accordingly!

Pro C#10 CHAPTER 16 Building and Configuring Class Libraries

CHAPTER 16

Building and Configuring Class Libraries

For most of the examples so far in this book, you have created “stand-alone” executable applications, in which all the programming logic was packaged within a single assembly (.dll) and executed using
dotnet.exe (or a copy of dotnet.exe named after the assembly). These assemblies were using little more
than the .NET base class libraries. While some simple .NET programs may be constructed using nothing more than the base class libraries, chances are it will be commonplace for you (or your teammates) to isolate reusable programming logic into custom class libraries (
.dll files) that can be shared among applications.
In this chapter, you’ll start by learning the details of partitioning types into .NET namespaces. After this, you will take a deep look at class libraries in .NET, the difference between a .NET and .NET Standard, application configuration, publishing .NET console applications, and packaging your libraries into reusable NuGet packages.

Defining Custom Namespaces (Updated 10.0)
Before diving into the aspects of library deployment and configuration, the first task is to learn the details of packaging your custom types into .NET namespaces. Up to this point in the text, you’ve been building
small test programs that leverage existing namespaces in the .NET universe (System, in particular). However, when you build larger applications with many types, it can be helpful to group your related types into custom namespaces. In C#, this is accomplished using the namespace keyword. Explicitly defining custom namespaces is even more important when creating shared assemblies, as other developers will need to reference the library and import your custom namespaces to use your types. Custom namespaces also prevent name collisions by segregating your custom classes from other custom classes that might have the same name.
To investigate the issues firsthand, begin by creating a new .NET Console Application project named CustomNamespaces. Now, assume you are developing a collection of geometric classes named Square, Circle, and Hexagon. Given their similarities, you would like to group them into a unique namespace called CustomNamespaces.MyShapes within the CustomNamespaces.exe assembly.

■ Guidance While you are free to use any name you choose for your namespaces, the naming convention is typically similar to CompanyName.ProductName.AssemblyName.Path.

© Andrew Troelsen, Phil Japikse 2022
A. Troelsen and P. Japikse, Pro C# 10 with .NET 6, https://doi.org/10.1007/978-1-4842-7869-7_16

625

While the C# compiler has no problems with a single C# code file containing multiple types,
this can be problematic when working in a team environment. If you are working on the Circle class and your co-worker needs to work on the Hexagon class, you would have to take turns working in the monolithic file or face difficult-to-resolve (well, at least time-consuming) merge conflicts.
A better approach is to place each class in its own file with a namespace definition. To ensure each type is packaged into the same logical group, simply wrap the given class definitions in the same namespace scope. The following example uses the pre–C# 10 namespace syntax, where each namespace declaration wrapped its contents with an opening and closing curly brace, like so:

// Circle.cs
namespace CustomNamespaces.MyShapes
{
// Circle class
public class Circle { / Interesting methods... / }
}
// Hexagon.cs
namespace CustomNamespaces.MyShapes
{
// Hexagon class
public class Hexagon { / More interesting methods... / }
}
// Square.cs
namespace CustomNamespaces.MyShapes
{
// Square class
public class Square { / Even more interesting methods.../}
}

One of the updates in C# 10 is the addition of file-scoped namespaces. This eliminates the need for opening and closing curly braces wrapping the contents. Simply declare the namespace, and everything that comes after that namespace declaration is included in the namespace. The following code sample produces the same result as the example with the namespaces wrapping their contents:

// Circle.cs
namespace CustomNamespaces.MyShapes;
// Circle class
public class Circle { / Interesting methods... / }
// Hexagon.cs
namespace CustomNamespaces.MyShapes;
// Hexagon class
public class Hexagon { / More interesting methods... / }
// Square.cs
namespace CustomNamespaces.MyShapes;
// Square class
public class Square { / Even more interesting methods.../}

This change by the C# team (along with global using statements and implicit global using statements) has eliminated a lot of the boilerplate code that was necessary in versions of C# prior to C# 10. In fact, as I updated the book for this version, I was able to remove on average two pages per chapter, just by removing the (now) unnecessary using statements and curly braces!

Notice how the CustomNamespaces.MyShapes namespace acts as the conceptual “container” of these classes. When another namespace (such as CustomNamespaces) wants to use types in a separate namespace, you use the using keyword, just as you would when using namespaces of the .NET base class libraries, as follows:

// Make use of types defined the MyShapes namespace. using CustomNamespaces.MyShapes;

Hexagon h = new Hexagon(); Circle c = new Circle(); Square s = new Square();

For this example, the assumption is that the C# files that define the CustomNamespaces.MyShapes namespace are part of the same Console Application project; in other words, all the files are compiled into a single assembly. If you defined the CustomNamespaces.MyShapes namespace within an external assembly, you would also need to add a reference to that library before you could compile successfully. You’ll learn all the details of building applications that use external libraries during this chapter.

Resolving Name Clashes with Fully Qualified Names
Technically speaking, you are not required to use the C# using keyword when referring to types defined in external namespaces. You could use the fully qualified name of the type, which, as you may recall from Chapter 1, is the type’s name prefixed with the defining namespace. Here’s an example:

// Note we are not importing CustomNamespaces.MyShapes anymore! CustomNamespaces.MyShapes.Hexagon h = new CustomNamespaces.MyShapes.Hexagon(); CustomNamespaces.MyShapes.Circle c = new CustomNamespaces.MyShapes.Circle(); CustomNamespaces.MyShapes.Square s = new CustomNamespaces.MyShapes.Square();

Typically, there is no need to use a fully qualified name. Not only does it require a greater number of keystrokes, it also makes no difference whatsoever in terms of code size or execution speed. In fact, in CIL code, types are always defined with the fully qualified name. In this light, the C# using keyword is simply a typing time-saver.
However, fully qualified names can be helpful (and sometimes necessary) to avoid potential name clashes when using multiple namespaces that contain identically named types. Assume you have a new namespace termed CustomNamespaces.My3DShapes, which defines the following three classes, capable of rendering a shape in stunning 3D:

// Another shape-centric namespace.
//Circle.cs
namespace CustomNamespaces.My3DShapes;
// 3D Circle class. public class Circle { }
//Hexagon.cs
namespace CustomNamespaces.My3DShapes;
// 3D Hexagon class. public class Hexagon { }
//Square.cs
namespace CustomNamespaces.My3DShapes;
// 3D Square class. public class Square { }

If you update the top-level statements as shown next, you are issued several compile-time errors, because both namespaces define identically named classes:

// Ambiguities abound!
using CustomNamespaces.MyShapes; using CustomNamespaces.My3DShapes;

// Which namespace do I reference?
Hexagon h = new Hexagon(); // Compiler error! Circle c = new Circle(); // Compiler error! Square s = new Square(); // Compiler error!

The ambiguity can be resolved using the type’s fully qualified name, like so:

// We have now resolved the ambiguity.
CustomNamespaces.My3DShapes.Hexagon h = new CustomNamespaces.My3DShapes.Hexagon(); CustomNamespaces.My3DShapes.Circle c = new CustomNamespaces.My3DShapes.Circle(); CustomNamespaces.MyShapes.Square s = new CustomNamespaces.MyShapes.Square();

Resolving Name Clashes with Aliases
The C# using keyword also lets you create an alias for a type’s fully qualified name. When you do so, you define a token that is substituted for the type’s full name at compile time. Defining aliases provides a second way to resolve name clashes. Here’s an example:

using MyShapes; using My3DShapes;

// Resolve the ambiguity for a type using a custom alias. using The3DHexagon = My3DShapes.Hexagon;

// This is really creating a My3DShapes.Hexagon class. The3DHexagon h2 = new The3DHexagon();
...

There is another (more commonly used) using syntax that lets you create an alias for a namespace instead of a type. For example, you could alias the CustomNamespaces.My3DShapes namespace, and create an instance of the 3D Hexagon as follows:

using ThreeD = CustomNamespaces.My3DShapes; ThreeD.Hexagon h2 = new ThreeD.Hexagon();

■Note Be aware that overuse of C# aliases for types can result in a confusing code base. if other programmers on your team are unaware of your custom aliases for types, they could have difficulty locating the real types in the project(s).

Creating Nested Namespaces
When organizing your types, you are free to define namespaces within other namespaces. The base class libraries do so in numerous places to provide deeper levels of type organization. For example, the IO namespace is nested within System to yield System.IO. In fact, you already created nested namespaces in the previous example. The multipart namespaces (CustomNamespaces.MyShapes and CustomNamespaces. My3DShapes) are nested under the root namespace, CustomNamespaces.
As you have seen already throughout this book, the .NET project templates add the initial code for console applications in a file named Program.cs. This file contains a namespace named after the project and a single class, Program. This base namespace is referred to as the root namespace. In our current example, the root namespace created by the .NET template is CustomNamespaces. To nest the MyShapes and My3DShapes namespaces inside the root namespace, there are three options. The first is to simply nest the namespace keyword, like this (using pre-C# 10 syntax):

namespace CustomNamespaces
{
namespace MyShapes
{
// Circle class public class Circle
{
/ Interesting methods... /
}
}
}

The second option, using C# 10 (and later), uses a file-scoped namespace followed by a block-scoped namespace for the nested namespace:

namespace CustomNamespaces; namespace MyShapes
{
// Circle class public class Circle
{
/ Interesting methods... /
}
}

The third option (and more commonly used) is to use “dot notation” in the namespace definition, as we did with the previous class examples:

namespace CustomNamespaces.MyShapes;
// Circle class public class Circle
{
/ Interesting methods... /
}

Namespaces do not have to directly contain any types. This allows developers to use namespaces to provide a further level of scope.

■Guidance a common practice is to group files in a namespace by directory. Where a file lives in the directory structure has no impact on namespaces. however, it does make the namespace structure clearer to other developers. therefore, many developers and code linting tools expect the namespaces to match the folder structures.

Change the Root Namespace Using Visual Studio 2022
As mentioned, when you create a new C# project using Visual Studio (or the .NET CLI), the name of your application’s root namespace will be identical to the project name. From this point on, when you use Visual Studio to insert new code files using the Project ➤ Add New Item menu selection, types will automatically be wrapped within the root namespace and have directory path appended. If you want to change the name of the root namespace, simply access the “Default namespace” option using the Application/General tab of the project’s properties window (see Figure 16-1).

Figure 16-1. Configuring the default/root namespace

■Note the Visual studio property pages still refer to the root namespace as the Default namespace. You will see next why i refer to it as the root namespace.

Change the Root Namespace Using the Project File
If not using Visual Studio (or even with Visual Studio), you can also configure the root namespace by updating the project (*.csproj) file. With .NET projects, editing the project file in Visual Studio is as easy as double-clicking the project file in Solution Explorer (or right-clicking the project file in Solution Explorer

and selecting “Edit project file”). Once the file is open, update the main PropertyGroup by adding the
RootNamespace node, like this:


Exe
net6.0
enable
disable
CustomNamespaces2

So far, so good. Now that you have seen some details regarding how to package your custom types into well-organized namespaces, let’s quickly review the benefits and format of the .NET assembly. After this, you will delve into the details of creating, deploying, and configuring your custom class libraries.

The Role of .NET Assemblies
.NET applications are constructed by piecing together any number of assemblies. Simply put, an assembly is a versioned, self-describing binary file hosted by the .NET Runtime. Now, despite that .NET assemblies have the same file extensions (.exe or .dll) as previous Windows binaries, they have little in common under the hood with those files. Before unpacking that last statement, let’s consider some of the benefits provided by the assembly format.

Assemblies Promote Code Reuse
As you have built your Console Application projects over the previous chapters, it might have seemed that all the applications’ functionality was contained within the executable assembly you were constructing. Your applications were leveraging numerous types contained within the always-accessible .NET base class libraries.
As you might know, a code library (also termed a class library) is a .dll that contains types intended to be used by external applications. When you are creating executable assemblies, you will no doubt be leveraging numerous system-supplied and custom code libraries as you create your application. Do be aware, however, that a code library need not take a .dll file extension. It is perfectly possible (although certainly not common) for an executable assembly to use types defined within an external executable file. In this light, a referenced *.exe can also be considered a code library.
Regardless of how a code library is packaged, the .NET platform allows you to reuse types in a language-independent manner. For example, you could create a code library in C# and reuse that library in any other .NET programming language. It is possible not only to allocate types across languages but also to derive from them. A base class defined in C# could be extended by a class authored in Visual Basic.
Interfaces defined in F# can be implemented by structures defined in C# and so forth. The point is that when you begin to break apart a single monolithic executable into numerous .NET assemblies, you achieve a language-neutral form of code reuse.

Assemblies Establish a Type Boundary
Recall that a type’s fully qualified name is composed by prefixing the type’s namespace (e.g., System) to its name (e.g., Console). Strictly speaking, however, the assembly in which a type resides further
establishes a type’s identity. For example, if you have two uniquely named assemblies (say, MyCars.dll and YourCars.dll) that both define a namespace (CarLibrary) containing a class named SportsCar, they are considered unique types in the .NET universe.

Assemblies Are Versionable Units
.NET assemblies are assigned a four-part numerical version number of the form ...

. (If you do not explicitly provide a version number, the assembly is automatically assigned a version of 1.0.0.0, given the default .NET project settings.) This number allows multiple versions of the same assembly to coexist in harmony on a single machine.

Assemblies Are Self-Describing
Assemblies are regarded as self-describing, in part because they record in the assembly’s manifest every external assembly they must be able to access to function correctly. Recall from Chapter 1 that a manifest is a blob of metadata that describes the assembly itself (name, version, required external assemblies, etc.).
In addition to manifest data, an assembly contains metadata that describes the composition (member names, implemented interfaces, base classes, constructors, etc.) of every contained type. Because an assembly is documented in such detail, the .NET Runtime does not consult the Windows system registry to resolve its location (quite the radical departure from Microsoft’s legacy COM programming model). This separation from the registry is one of the factors that enables .NET applications to run on other operating systems besides Windows as well as supporting multiple versions of .NET on the same machine.
As you will discover during this chapter, the .NET Runtime makes use of an entirely new scheme to resolve the location of external code libraries.

Understanding the Format of a .NET Assembly
Now that you’ve learned about several benefits provided by the .NET assembly, let’s shift gears and get a better idea of how an assembly is composed under the hood. Structurally speaking, a .NET assembly (*.dll or *.exe) consists of the following elements:
•An operating system (e.g., Windows) file header
•A CLR file header
•CIL code
•Type metadata
•An assembly manifest
•Optional embedded resources
While the first two elements (the operating system and CLR headers) are blocks of data you can typically always ignore, they do deserve some brief consideration. Here’s an overview of each element.

Installing the C++ Profiling Tools
The next few sections use a utility call dumpbin.exe, and it ships with the C++ profiling tools. To install them, type C++ profiling tools in the quick search bar, and click the prompt to install the tools, as shown in Figure 16-2.

Figure 16-2. Installing the C++ profiling tools from Quick Launch

This will bring up the Visual Studio installer with the tools selected. Alternatively, you can launch the Visual Studio installer yourself and select the components shown in Figure 16-3.

Figure 16-3. Installing the C++ profiling tools

The Operating System (Windows) File Header
The operating system file header establishes the fact that the assembly can be loaded and manipulated by the target operating system (in the following example, Windows). This header data also identifies the kind of application (console-based, GUI-based, or *.dll code library) to be hosted by the operating system.

Open the CarLibrary.dll file (in the book’s repo or created later in this chapter) using the dumpbin.exe
utility (via the developer command prompt) with the /headers flag as so:
dumpbin /headers CarLibrary.dll
This displays the assembly’s operating system header information (shown in the following when built for Windows). Here is the (partial) Windows header information for CarLibrary.dll:

Dump of file carlibrary.dll PE signature found
File Type: DLL

FILE HEADER VALUES
14C machine (x86)
3 number of sections 877429B3 time date stamp
0 file pointer to symbol table
0 number of symbols
E0 size of optional header 2022 characteristics
Executable
Application can handle large (>2GB) addresses DLL
...

Now, remember that most .NET programmers will never need to concern themselves with the format of the header data embedded in a .NET assembly. Unless you happen to be building a new .NET language compiler (where you would care about such information), you are free to remain blissfully unaware of the grimy details of the header data. Do be aware, however, that this information is used under the covers when the operating system loads the binary image into memory.

The CLR File Header
The CLR header is a block of data that all .NET assemblies must support (and do support, courtesy of the C# compiler) to be hosted by the .NET Runtime. In a nutshell, this header defines numerous flags that enable the runtime to understand the layout of the managed file. For example, flags exist that identify the location of the metadata and resources within the file, the version of the runtime the assembly was built against, the value of the (optional) public key, and so forth. Execute dumpbin.exe again with the /clrheader flag.

dumpbin /clrheader CarLibrary.dll

You are presented with the internal CLR header information for a given .NET assembly, as shown here:

Dump of file CarLibrary.dll File Type: DLL

clr Header:

48 cb
2.05 runtime version

2158 [ B7C] RVA [size] of MetaData Directory
1 flags
IL Only
0 entry point token
0 [ 0] RVA [size] of Resources Directory
0 [ 0] RVA [size] of StrongNameSignature Directory
0 [ 0] RVA [size] of CodeManagerTable Directory
0 [ 0] RVA [size] of VTableFixups Directory
0 [ 0] RVA [size] of ExportAddressTableJumps Directory
0 [ 0] RVA [size] of ManagedNativeHeader Directory Summary
2000 .reloc
2000 .rsrc
2000 .text

Again, as a .NET developer, you will not need to concern yourself with the gory details of an assembly’s CLR header information. Just understand that every .NET assembly contains this data, which is used behind the scenes by the .NET Runtime as the image data loads into memory. Now turn your attention to some information that is much more useful in your day-to-day programming tasks.

CIL Code, Type Metadata, and the Assembly Manifest
At its core, an assembly contains CIL code, which, as you recall, is a platform- and CPU-agnostic intermediate language. At runtime, the internal CIL is compiled on the fly using a just-in-time (JIT) compiler, according to platform- and CPU-specific instructions. Given this design, .NET assemblies can indeed execute on a variety of architectures, devices, and operating systems. (Although you can live a happy and productive life without understanding the details of the CIL programming language, Chapter 18 offers an introduction to the syntax and semantics of CIL.)
An assembly also contains metadata that completely describes the format of the contained types, as well as the format of external types referenced by this assembly. The .NET Runtime uses this metadata to resolve the location of types (and their members) within the binary, lay out types in memory, and facilitate remote method invocations. You’ll check out the details of the .NET metadata format in Chapter 17 during your examination of reflection services.
An assembly must also contain an associated manifest (also referred to as assembly metadata). The manifest documents each module within the assembly, establishes the version of the assembly, and documents any external assemblies referenced by the current assembly. As you will see over the course of this chapter, the CLR makes extensive use of an assembly’s manifest during the process of locating external assembly references.

Optional Assembly Resources
Finally, a .NET assembly may contain any number of embedded resources, such as application icons, image files, sound clips, or string tables. In fact, the .NET platform supports satellite assemblies that contain nothing but localized resources. This can be useful if you want to partition your resources based on a specific culture (English, German, etc.) for the purposes of building international software. The topic of building satellite assemblies is outside the scope of this text. Consult the .NET documentation for information on satellite assemblies and localization if you are interested.

Class Libraries vs. Console Applications
So far in this book, the examples were almost exclusively .NET Console applications. If you are reading this book as a current .NET Framework developer, these are like .NET console applications, with the main difference being the configuration process (to be covered later) and, of course, that they run on .NET Core. Console applications have a single-entry point (either a specified Main() method or top-level statements), can interact with the console, and can be launched directly from the operating system. Another difference
between .NET Framework and .NET console applications is that console applications in .NET are launched using the .NET Application Host (dotnet.exe).
Class libraries, on the other hand, don’t have an entry point and therefore cannot be launched directly. They are used to encapsulate logic, custom types, and so on, and they are referenced by other class libraries and/or console applications. In other words, class libraries are used to contain the things talked about in “The Role of .NET Assemblies” section.

.NET Standard vs. .NET (Core) Class Libraries
.NET (including .NET Core/.NET 5/.NET 6) class libraries run on .NET Core, and .NET Framework class libraries run on the .NET Framework. While it’s pretty straightforward, there is a problem with this. Assume you have a large .NET Framework code base in your organization, with (potentially) years of development behind you and your team. There is probably a significant amount of shared code leveraged by the applications you and your team have built over the years. Perhaps it’s centralized logging, reporting, or domain-specific functionality.
Now you (and your organization) want to move to the new .NET for all new development. What about all that shared code? The effort to rewrite all of your legacy code into .NET 6 assemblies could be significant, and until all of your applications were moved to .NET 6, you would have to support two versions (one in
.NET Framework and one in .NET 6). That would bring productivity to a screeching halt.
Fortunately, the builders of .NET (Core) thought through this scenario. .NET Standard is a new type of class library project that was introduced with .NET Core 1.0 and can be referenced by .NET Framework as well as .NET (Core) applications. Before you get your hopes up, though, there is a catch with .NET 5 and
.NET 6. More on that shortly.
Each .NET Standard version defines a common set of APIs that must be supported by all .NET versions (.NET, .NET Core, Xamarin, etc.) to conform to the standard. For example, if you were to build a class library as a .NET Standard 2.0 project, it can be referenced by .NET 4.61+ and .NET Core 2.0+ (plus various versions of Xamarin, Mono, Universal Windows Platform, and Unity).
This means you could move the code from your .NET Framework class libraries into .NET Standard 2.0 class libraries, and they can be shared by .NET (Core) and .NET Framework applications. That’s much better than supporting duplicate copies of the same code, one for each framework.
Now for the catch. Each .NET Standard version represents the lowest common denominator for the frameworks that it supports. That means the lower the version, the less that you can do in your class library. While .NET Framework and .NET 6 projects can reference a .NET Standard 2.0 library, you cannot use a significant number of C# 8.0 features in a .NET Standard 2.0 library, and you can’t use any new features from C# 9.0 or later. You must use .NET Standard 2.1 for full C# 8.0+ support. And .NET 4.8 (the latest/last version of the original .NET Framework) only goes up to .NET Standard 2.0.
It’s still a good mechanism for leveraging existing code in newer applications up to and including .NET Core 3.1, but not a silver bullet. With the unification of the frameworks (.NET, Xamarin, Mon, etc.) with .NET 6, the usefulness of .NET Standard is slowly fading into the past.

Configuring Applications with Configuration Files
While it is possible to keep all the information needed by your .NET application in the source code, being able to change certain values at runtime is vitally important in most applications of significance. One of the more common options is to use one or more configuration files shipped (or deployed) along with your application’s executable.
The .NET Framework relied mostly on XML files named app.config (or web.config for ASP.NET applications) for configuration. While XML-based configuration files can still be used, the most common method for configuring .NET applications is with JavaScript Object Notation (JSON) files..

■Note if you are not familiar with Json, it is a name-value pair format where each object is enclosed in curly braces. Values can also be objects using the same name-value pair format. Chapter 20 covers working with Json files in depth.

To illustrate the process, create a new .NET Console application named FunWithConfiguration and add the following package reference to your project:

dotnet new console -lang c# -n FunWithConfiguration -o .\FunWithConfiguration -f net6.0 dotnet add FunWithConfiguration package Microsoft.Extensions.Configuration
dotnet add FunWithConfiguration package Microsoft.Extensions.Configuration.Binder dotnet add FunWithConfiguration package Microsoft.Extensions.Configuration.Json

This adds a reference for configuration subsystem, the JSON file–based .NET configuration subsystem, and the binding extensions for configuration into your project. Start by adding a new JSON file into your project named appsettings.json. Update the project file to make sure the file is always copied to the output directory when the project is built.



Always

Finally, update the appsettings.json file to match the following:

{
"CarName": "Suzy"
}

The final step to adding the configuration into your app is to read in the configuration file and get the
CarName value. Update the Program.cs file to the following:
using Microsoft.Extensions.Configuration; IConfiguration config = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", true, true)
.Build();

The new configuration system starts with a ConfigurationBuilder. The path that the configuration build will start to look for the files being added is set with the SetBasePath() method. Then the configuration file is added with the AddJsonFile() method, which takes three parameters. The first parameter is the path and name of the file. Since this file is in the same location as the base path, there isn’t any path information in the string, just the filename. The second parameter sets whether the file is optional (true) or required (false), and the final parameter determines if the configuration should use a file watcher to look for any changes in the file (true) or to ignore any changes during runtime (false). The final step
is to build the configuration into an instance of IConfiguration using the Build() method. This instance provides access to all the configured values.

■ Note file watchers are covered in Chapter 20.

Once you have an instance of IConfiguration, you can get the values from the configuration files much the same way as calling the ConfigurationManager in .NET 4.8. Add the following to the bottom of the Main() method, and when you run the app, you will see the value written to the console:

Console.WriteLine($"My car's name is {config["CarName"]}");

If the request name doesn’t exist in the configuration, the result will be null. The following code still runs without an exception; it just doesn’t display a name in the first line and displays True in the second line:

Console.WriteLine($"My car's name is {config["CarName2"]}"); Console.WriteLine($"CarName2 is null? {config["CarName2"] == null}");

There is also a GetValue() method (and its generic version GetValue()) that can retrieve primitive values from the configuration. The following example shows both of these methods to get the CarName:

Console.WriteLine($"My car's name is {config.GetValue(typeof(string),"CarName")}"); Console.WriteLine($"My car's name is {config.GetValue("CarName")}");

These methods return the default value (e.g., null for reference types, 0 for numeric types) if the name requested doesn’t exist. The following code returns 0 for the CarName2 property:

Console.WriteLine($"My car's name is {config.GetValue("CarName2")}");

These methods will throw an exception if the value found for the name can’t be intrinsically cast to the requested datatype. For example, trying to cast the CarName property to an int will throw an InvalidOperationException as shown by the following code:

try
{
Console.WriteLine($"My car's name is {config.GetValue("CarName")}");
}
catch (InvalidOperationException ex)
{
Console.WriteLine($"An exception occurred: {ex.Message}");
}

■ Note the GetValue() method is designed to work with primitive types. for complex types, use the
Bind() or Get()/Get() methods, covered in the “Working with objects” section.

Multiple Configuration Files
More than one configuration file can be added into the configuration system. When more than one file is used, the configuration properties are additive, unless any names of the name-value pairs conflict. When there is a name conflict, the last one in wins. To see this in action, add another file named appsettings. development.json, and set the project to always copy it to the output directory.



Always

Update the JSON to the following:

{
"CarName": "Logan"
}

Now update the code that creates the instance of the IConfiguration interface to the following (update in bold):

IConfiguration config = new ConfigurationBuilder()
.SetBasePath(Directory.GetCurrentDirectory())
.AddJsonFile("appsettings.json", true, true)
.AddJsonFile("appsettings.development.json", true, true)
.Build();

When you run this program, you will see the name of the car is indeed Logan, and not Suzy.

Working with Objects (Updated 10.0)
Our sample JSON file so far is extremely simple with a single name-value pair. In real-world projects, application configuration is usually more complex than a single property. Update the appsettings. development.json file to the following, which adds a new Car object to the JSON:

{
"CarName": "Suzy",
"Car": {
"Make":"Honda",
"Color": "Blue", "PetName":"Dad's Taxi"
}
}

To access multilevel JSON values, the key used for searching is the hierarchy of the JSON, with each level separated by colons (:). For example, the key for the Car object’s Make property is Car:Make. Update the top-level statements to the following to get all the Car properties and display them:

Console.Write($"My car object is a {config["Car:Color"]} "); Console.WriteLine($"{config["Car:Make"]} named {config["Car:PetName"]}");

Instead of traversing the hierarchy of names, entire sections can be retrieved using the GetSection() method. Once you have the section, you can then get the values from the section using the simple name format, as shown in the following example:

IConfigurationSection section = config.GetSection("Car"); Console.Write($"My car object is a {section["Color"]} "); Console.WriteLine($"{section["Make"]} named {section["PetName"]}");

As a final note for working with objects, you can use the Bind() method to bind configuration values to an existing instance of an object or the Get() method to create a new object instance. These are similar to the GetValue() method but work with nonprimitive types. To get started, create a simple Car class at the end of the top-level statements:

public class Car
{
public string Make {get;set;} public string Color {get;set;} public string PetName { get; set; }
}

Next, create a new instance of a Car class, and then call Bind() on the section passing in the Car instance:

var c = new Car(); section.Bind(c);
Console.Write($"My car object is a {c.Color} "); Console.WriteLine($"{c.Make} named {c.PetName}");

If the section isn’t configured, the Bind() method will not update the instance but will leave all of the properties as they existed prior to the call to Bind(). The following will leave the Color set to Red and the remaining properties null:

var notFoundCar = new Car { Color = "Red"}; config.GetSection("Car2").Bind(notFoundCar); Console.Write($"My car object is a {notFoundCar.Color} ");
Console.WriteLine($"{notFoundCar.Make} named {notFoundCar.PetName}");

The Get() method creates a new instance of the specified type from a section of the configuration. The non-generic version of the method returns an object type, so the return value must be cast to the specific type before being used. Here is an example using the Get() method to create an instance of the Car class from the Car section of the configuration:

var carFromGet = config.GetSection(nameof(Car)).Get(typeof(Car)) as Car; Console.Write($"My car object (using Get()) is a {carFromGet.Color} "); Console.WriteLine($"{carFromGet.Make} named {carFromGet.PetName}");

If the named section isn’t found, the Get() method returns null:

var notFoundCarFromGet = config.GetSection("Car2").Get(typeof(Car)); Console.WriteLine($"The not found car is null? {notFoundCarFromGet == null}");

The generic version returns an instance of the specified type without having to perform a cast. If the section isn’t found, the method returns null.

//Returns a Car instance
var carFromGet2 = config.GetSection(nameof(Car)).Get(); Console.Write($"My car object (using Get()) is a {carFromGet.Color} "); Console.WriteLine($"{carFromGet.Make} named {carFromGet.PetName}");

//Returns null
var notFoundCarFromGet2 = config.GetSection("Car2").Get(); Console.WriteLine($"The not found car is null? {notFoundCarFromGet2 == null}");

The Bind() and Get()/Get() methods use reflection (covered in the next chapter) to match the names of the public properties on the class to the names in the configuration section in a case-insensitive manner. For example, if you update appsettings.development.json to the following (notice the casing change of the petName property), the previous code will still work:

{
"CarName": "Suzy", "Car": {
"Make":"Honda",
"Color": "Blue",
"petName":"Dad's Taxi"
}
}

If a property in the configuration doesn’t exist in the class (or the name is spelled differently), then that particular configuration value (by default) is ignored. If you update the JSON to the following, the Make and Color properties are populated, but the PetName property on the Car object isn’t:

{
"CarName": "Suzy", "Car": {
"Make":"Honda",
"Color": "Blue",
"PetNameForCar":"Dad's Taxi"
}
}

The Bind(), Get() and Get() methods can optionally take an Action to further refine the process of updating (Bind()) or instantiating (Get()/Get()) class instances. The BinderOptions class is listed here:

public class BinderOptions
{
public bool BindNonPublicProperties { get; set; } //Defaults to false public bool ErrorOnUnknownConfiguration { get; set; } //Defaults to false
}

If the ErrorOnUnknownConfiguration option is set to true, then an InvalidOperationException will be thrown if the configuration contains a name that doesn’t exist on the model. With the renamed configuration value (PetNameForCar), the following call throws the exception listed in the code sample:

try
{
_ = config.GetSection(nameof(Car)).Get(t=>t.ErrorOnUnknownConfiguration=true);
}
catch (InvalidOperationException ex)
{
Console.WriteLine($"An exception occurred: {ex.Message}");
}
/*
Error message: 'ErrorOnUnknownConfiguration' was set on the provided BinderOptions, but the following properties were not found on the instance of Car: 'PetNameForCar'
*/

The other option allows for binding non-public properties. By default, both properties are false. If non- public properties should be bound from the configuration, set the BindNonPublicProperties, like this:

var carFromGet3 = config.GetSection(nameof(Car)).Get(t=>t.BindNonPublicPropert ies=true);

New in C# 10, the GetRequiredSection() method will throw an exception if the section isn’t configured. For example, the following code will throw an exception since there isn’t a Car2 section in the configuration:

try
{
config.GetRequiredSection("Car2").Bind(notFoundCar);
}
catch (InvalidOperationException ex)
{
Console.WriteLine($"An exception occurred: {ex.Message}");
}

Additional Configuration Options
In addition to using file-based configuration, there are options to use environment variables, Azure Key Vault, command-line arguments, and many more. Many of these are used intrinsically in ASP.NET Core, as you will see later in this book. You can also reference the .NET documentation for more information on using other methods for configuring applications.

Building and Consuming a .NET Class Library
To begin exploring the world of .NET class libraries, you’ll first create a *.dll assembly (named CarLibrary) that contains a small set of public types. If you are working in Visual Studio, name the solution file something meaningful (different than CarLibrary). You will add two projects into the solution later in this section. If you are working in Visual Studio Code, you don’t need to have a solution, although most developers find it useful to group related projects into a single solution.
As a reminder, you can create and manage solutions and projects through the .NET Command Line Interface (CLI). Use the following command to create the solution and class library:

dotnet new sln -n Chapter16_AllProjects
dotnet new classlib -lang c# -n CarLibrary -o .\CarLibrary -f net6.0 dotnet sln .\Chapter16_AllProjects.sln add .\CarLibrary

The first command creates an empty solution file named Chapter16_AllProjects (-n) in the current directory. The next command creates a new .NET 6.0 (-f) class library named CarLibrary (-n) in the subdirectory named CarLibrary (-o). The output (-o) location is optional. If left off, the project will be created in a subdirectory with the same name as the project name. The final command adds the new project to the solution.

■ Note the .net Cli has a good help system. to get help for any command, add -h to the command. for example, to see all templates, type dotnet new -h. to get more information about creating a class library, type dotnet new classlib -h.

Now that you have created your project and solution, you can open it in Visual Studio (or Visual Studio Code) to begin building the classes. After opening the solution, delete the autogenerated file Class1.cs.
The design of your automobile library begins with the EngineStateEnum and MusicMediaEnum enums. Add two files to your project named MusicMediaEnum.cs and EngineStateEnum.cs and add the following code to each file, respectively:

//MusicMediaEnum.cs namespace CarLibrary;
// Which type of music player does this car have? public enum MusicMediaEnum
{
MusicCd, MusicTape, MusicRadio, MusicMp3
}
//EngineStateEnum.cs namespace CarLibrary;
// Represents the state of the engine. public enum EngineStateEnum
{
EngineAlive, EngineDead
}

Next, insert a new C# class file into your project, named Car.cs, that will hold an abstract base class named Car. This class defines various state data via automatic property syntax. This class also has a single abstract method named TurboBoost(), which uses the EngineStateEnum enumeration to represent the current condition of the car’s engine. Update the file to the following code:

namespace CarLibrary;
// The abstract base class in the hierarchy. public abstract class Car
{
public string PetName {get; set;} public int CurrentSpeed {get; set;} public int MaxSpeed {get; set;}

protected EngineStateEnum State = EngineStateEnum.EngineAlive; public EngineStateEnum EngineState => State;
public abstract void TurboBoost();

protected Car(){}
protected Car(string name, int maxSpeed, int currentSpeed)
{
PetName = name;
MaxSpeed = maxSpeed;
CurrentSpeed = currentSpeed;
}
}

Now assume you have two direct descendants of the Car type named MiniVan and SportsCar. Each overrides the abstract TurboBoost() method by displaying an appropriate message via console message. Insert two new C# class files into your project, named MiniVan.cs and SportsCar.cs, respectively. Update the code in each file with the relevant code.

//SportsCar.cs namespace CarLibrary;
public class SportsCar : Car
{
public SportsCar(){ } public SportsCar(
string name, int maxSpeed, int currentSpeed)
: base (name, maxSpeed, currentSpeed){ }

public override void TurboBoost()
{
Console.WriteLine("Ramming speed! Faster is better...");
}
}

//MiniVan.cs namespace CarLibrary;
public class MiniVan : Car
{
public MiniVan(){ }

public MiniVan(
string name, int maxSpeed, int currentSpeed)
: base (name, maxSpeed, currentSpeed){ }

public override void TurboBoost()
{
// Minivans have poor turbo capabilities! State = EngineStateEnum.EngineDead;
Console.WriteLine("Eek! Your engine block exploded!");
}
}

Exploring the Manifest
Before using CarLibrary.dll from a client application, let’s check out how the code library is composed under the hood. Assuming you have compiled this project, run ildasm.exe against the compiled assembly. If you don’t have ildasm.exe (covered earlier in this book), it is also located in the Chapter 16 directory of this book’s repository.

ildasm /METADATA /out=CarLibrary.il .\CarLibrary\bin\Debug\net6.0\CarLibrary.dll

The Manifest section of the disassembled results starts with //Metadata version: 4.0.30319. Immediately following is the list of all external assemblies required by the class library, as shown here:

// Metadata version: v4.0.30319
.assembly extern System.Runtime
{
.publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )
.ver 6:0:0:0
}
.assembly extern System.Console
{
.publickeytoken = (B0 3F 5F 7F 11 D5 0A 3A )
.ver 6:0:0:0
}

Each .assembly extern block is qualified by the .publickeytoken and .ver directives. The
.publickeytoken instruction is present only if the assembly has been configured with a strong name. The
.ver token defines the numerical version identifier of the referenced assembly.

■ Note prior versions of the .net framework relied heavily on strong naming, which involved using a public/ private key combination. this was required on Windows for an assembly to be added into the global assembly Cache, but its need has severely diminished with the advent of .net Core.

After the external references, you will find a number of .custom tokens that identify assembly-level attributes (some system generated, but also copyright information, company name, assembly version, etc.). Here is a (very) partial listing of this chunk of manifest data:

.assembly CarLibrary
{
...
.custom instance void ... TargetFrameworkAttribute ...
.custom instance void ... AssemblyCompanyAttribute ...
.custom instance void ... AssemblyConfigurationAttribute ...
.custom instance void ... AssemblyFileVersionAttribute ...
.custom instance void ... AssemblyProductAttribute ...
.custom instance void ... AssemblyTitleAttribute ...

These settings can be set either using the Visual Studio property pages or editing the project file and adding in the correct elements. To edit the package properties in Visual Studio 2022, right-click the project in Solution Explorer, select Properties, and navigate to the Package menu in the left rail of the window. This brings up the dialog shown in Figure 16-4. For the sake of brevity, Figure 16-4 is only part of the dialog.

Figure 16-4. Editing assembly information using Visual Studio 2022’s Properties window

■ Note there are three different version fields in the package screen. the assembly version and the file version use the same schema, which is based on semantic versioning (https://semver.org). the first number is the major build version, the second is the minor build version, and the third is the patch number. the fourth number is usually used to indicate a build number. the package version number should adhere to semantic versioning, with just the {major}.{minor}.{patch} placeholders used. semantic versioning allows for an alphanumeric extension to the version, which is separated by a dash instead of a period (e.g., 1.0.0-rc). this denotes noncomplete versions, such as betas and release candidates.

a package version sets the nuget package version (nuget packaging is covered in more detail later in this chapter). the assembly version is used by .net during build and runtime to locate, link, and load assemblies. the file version is used only by Windows explorer, and it not used by .net.

Another way to add the metadata to your assembly is directly in the *.csproj project file. The following update to the main PropertyGroup in the project file does the same thing as filling in the form shown in Figure 16-4. Notice that the package version is simply called Version in the project file.


net6.0
enable
disable
Copyright 2021
Phil Japikse
Apress
Pro C# 10.0
CarLibrary
This is an awesome library for cars.
1.0.0.1
1.0.0.2
1.0.3

■ Note the rest of the entries from figure 16-4 (and the project file listing) are used when generating nuget packages from your assembly. this is covered later in the chapter.

Exploring the CIL
Recall that an assembly does not contain platform-specific instructions; rather, it contains platform-agnostic Common Intermediate Language (CIL) instructions. When the .NET Runtime loads an assembly into memory, the underlying CIL is compiled (using the JIT compiler) into instructions that can be understood by the target platform. For example, the TurboBoost() method of the SportsCar class is represented by the following CIL:

.method public hidebysig virtual
instance void TurboBoost() cil managed
{
.maxstack 8 IL_0000: nop
IL_0001: ldstr "Ramming speed! Faster is better..."
IL_0006: call void [System.Console]System.Console::WriteLine(string) IL_000b: nop
IL_000c: ret
}
// end of method SportsCar::TurboBoost

As with the other CIL examples in this book, most .NET developers don’t need to be deeply concerned with the details. Chapter 18 provides more details on its syntax and semantics, which can be helpful when you are building more complex applications that require advanced services, such as runtime construction of assemblies.

Exploring the Type Metadata
Before you build some applications that use your custom .NET library, examine the metadata for the types within the CarLibrary.dll assembly. For an example, here is the TypeDef for the EngineStateEnum:

TypeDef #2

TypDefName: CarLibrary.EngineStateEnum
Flags : [Public] [AutoLayout] [Class] [Sealed] [AnsiClass] Extends : [TypeRef] System.Enum
Field #1

Field Name: value
Flags : [Public] [SpecialName] [RTSpecialName] CallCnvntn: [FIELD]
Field type: I4 Field #2
Field Name: EngineAlive
Flags : [Public] [Static] [Literal] [HasDefault] DefltValue: (I4) 0
CallCnvntn: [FIELD]
Field type: ValueClass CarLibrary.EngineStateEnum Field #3
Field Name: EngineDead
Flags : [Public] [Static] [Literal] [HasDefault] DefltValue: (I4) 1
CallCnvntn: [FIELD]
Field type: ValueClass CarLibrary.EngineStateEnum

As explained in the next chapter, an assembly’s metadata is an important element of the .NET platform and serves as the backbone for numerous technologies (object serialization, late binding, extendable applications, etc.). In any case, now that you have looked inside the CarLibrary.dll assembly, you can build some client applications that use your types.

Building a C# Client Application
Because each of the CarLibrary project types has been declared using the public keyword, other .NET applications are able to use them as well. Recall that you may also define types using the C# internal keyword (in fact, this is the default C# access mode for classes). Internal types can be used only by the assembly in which they are defined. External clients can neither see nor create types marked with the internal keyword.

■ Note the exception to the internal rule is when an assembly explicitly allows access to another assembly using the InternalsVisibleTo attribute, covered shortly.

To use your library’s functionality, create a new C# Console Application project named CSharpCarClient in the same solution as CarLibrary. You can do this using Visual Studio (right-click the solution and select Add ➤ New Project) or using the command line (three lines, each executed separately).

dotnet new console -lang c# -n CSharpCarClient -o .\CSharpCarClient -f net6.0 dotnet add CSharpCarClient reference CarLibrary
dotnet sln .\Chapter16_AppRojects.sln add .\CSharpCarClient

The previous commands created the console application, added a project reference to the CarLibrary project for the new project, and added it to your solution.

■ Note the add reference command creates a project reference. this is convenient for development, as CsharpCarClient will always be using the latest version of Carlibrary. You can also reference an assembly directly. direct references are created by referencing the compiled class library.

If you still have the solution open in Visual Studio, you’ll notice that the new project shows up in Solution Explorer without any intervention on your part.
The final change to make is to right-click CSharpCarClient in Solution Explorer and select “Set as Startup Project.” If you are not using Visual Studio, you can run the new project by executing dotnet run in the project directory.

■ Note You can set the project reference in Visual studio as well by right-clicking the CsharpCarClient project in solution explorer, selecting add ➤ reference, and selecting the Carlibrary project from the project’s node.

At this point, you can build your client application to use the external types. Update the Program.cs file as follows:

// Don't forget to import the CarLibrary namespace! using CarLibrary;

Console.WriteLine("***** C# CarLibrary Client App *****");
// Make a sports car.
SportsCar viper = new SportsCar("Viper", 240, 40); viper.TurboBoost();

// Make a minivan.
MiniVan mv = new MiniVan(); mv.TurboBoost();

Console.WriteLine("Done. Press any key to terminate"); Console.ReadLine();

This code looks just like the code of the other applications developed thus far in the book. The only point of interest is that the C# client application is now using types defined within a separate custom library. Run your program and verify that you see the display of messages.
You might be wondering exactly what happened when you referenced the CarLibrary project. When a project reference is made, the solution build order is adjusted so that dependent projects (CarLibrary in this example) build first, and then the output from that build is copied into the output directory of the parent project (CSharpCarLibrary). The compiled client library references the compiled class library. When the client project is rebuilt, so is the dependent library, and the new version is once again copied to the target folder.

■ Note if you are using Visual studio, you can click the show all files button in solution explorer, and you can see all the output files and verify that the compiled Carlibrary is there. if you are using Visual studio Code, navigate to the bin/debug/net6.0 directory in the explorer tab.

When a direct reference instead of a project reference is made, the compiled library is also copied to the output directory of the client library, but at the time the reference is made. Without the project reference in place, the projects can be built independently of each other, and the files could become out of sync. In short, if you are developing dependent libraries (as is usually the case with real software projects), it is best to reference the project and not the project output.

Building a Visual Basic Client Application
Recall that the .NET platform allows developers to share compiled code across programming languages. To illustrate the language-agnostic attitude of the .NET platform, let’s create another Console Application project (VisualBasicCarClient), this time using Visual Basic (note that each of commands is a one-line command).

dotnet new console -lang vb -n VisualBasicCarClient -o .\VisualBasicCarClient -f net6.0 dotnet add VisualBasicCarClient reference CarLibrary
dotnet sln .\Chapter16_AllProjects.sln add VisualBasicCarClient

Like C#, Visual Basic allows you to list each namespace used within the current file. However, Visual Basic uses the Imports keyword rather than the C# using keyword, so add the following Imports statement within the Program.vb code file:

Imports CarLibrary
Module Program Sub Main() End Sub
End Module

Notice that the Main() method is defined within a Visual Basic module type. In a nutshell, modules are a Visual Basic notation for defining a class that can contain only static methods (much like a C# static
class). In any case, to exercise the MiniVan and SportsCar types using the syntax of Visual Basic, update your
Main() method as follows:

Sub Main()
Console.WriteLine("***** VB CarLibrary Client App *****")
' Local variables are declared using the Dim keyword.

Dim myMiniVan As New MiniVan() myMiniVan.TurboBoost()

Dim mySportsCar As New SportsCar() mySportsCar.TurboBoost() Console.ReadLine()
End Sub

When you compile and run your application (make sure to set VisualBasicCarClient as the startup project if you are using Visual Studio), you will once again find a series of message boxes displayed.
Furthermore, this new client application has its own local copy of CarLibrary.dll located under the bin\ Debug\net6.0 folder.

Cross-Language Inheritance in Action
An enticing aspect of .NET development is the notion of cross-language inheritance. To illustrate, let’s create a new Visual Basic class that derives from SportsCar (which was authored using C#). First, add a new class file named PerformanceCar.vb to your current Visual Basic application. Update the initial class definition by deriving from the SportsCar type using the Inherits keyword. Then, override the abstract TurboBoost() method using the Overrides keyword, like so:

Imports CarLibrary
' This VB class is deriving from the C# SportsCar.
Public Class PerformanceCar Inherits SportsCar
Public Overrides Sub TurboBoost()
Console.WriteLine("Zero to 60 in a cool 4.8 seconds...") End Sub
End Class

To test this new class type, update the module’s Main() method as follows:

Sub Main()
...
Dim dreamCar As New PerformanceCar()

' Use Inherited property. dreamCar.PetName = "Hank" dreamCar.TurboBoost() Console.ReadLine()
End Sub

Notice that the dreamCar object can invoke any public member (such as the PetName property) found up the chain of inheritance, although the base class was defined in a completely different language and in a completely different assembly! The ability to extend classes across assembly boundaries in a language- independent manner is a natural aspect of the .NET development cycle. This makes it easy to use compiled code written by individuals who would rather not build their shared code with C#.

Exposing internal Types to Other Assemblies
As mentioned earlier, internal classes are visible only to other objects in the assembly where they are defined. The exception to this is when visibility is explicitly granted to another project.
Begin by adding a new class named MyInternalClass to the CarLibrary project, and update the code to the following:

namespace CarLibrary;
internal class MyInternalClass
{
}

■ Note Why expose internal types at all? this is usually done for unit and integration testing. developers want to be able to test their code but not necessarily expose it beyond the borders of the assembly.

Using an Assembly Attribute
Chapter 17 will cover attributes in depth, but for now open the Car.cs class in the CarLibrary project, and add the following attribute and using statement:

using System.Runtime.CompilerServices; [assembly:InternalsVisibleTo("CSharpCarClient")] namespace CarLibrary;

The InternalsVisibleTo attribute takes the name of the project that can see into the class that has the attribute set. Note that other projects cannot “ask” for this permission; it must be granted by the project holding the internal types.

■ Note previous versions of the .net framework allowed you to place assembly-level attributes into the AssemblyInfo.cs class, which still exists in .net but is autogenerated and not meant for developer consumption.

Now you can update the CSharpCarClient project by adding the following code to the top-level statements:

var internalClassInstance = new MyInternalClass();

This works perfectly. Now try to do the same thing in the VisualBasicCarClient Main method.

'Will not compile
'Dim internalClassInstance = New MyInternalClass()

Because the VisualBasicCarClient library was not granted permission to see the internals, the previous line of code will not compile.

Using the Project File
Another way to accomplish the same thing is to use the updated capabilities in the .NET project file. Comment out the attribute you just added and open the project file for CarLibrary. Add the following ItemGroup in the project file:



<_Parameter1>CSharpCarClient

This accomplishes the same thing as using the attribute on a class and, in my opinion, is a better solution, since other developers will see it right in the project file instead of having to know where to look throughout the project.

NuGet and .NET Core
NuGet is the package manager for the .NET Framework and .NET (Core). It is a mechanism to share software in a format that .NET applications understand and is the default mechanism for loading .NET and its related framework pieces (ASP.NET Core, EF Core, etc.). Many organizations package their standard assemblies for cross-cutting concerns (like logging and error reporting) into NuGet packages for consumption into their line-of-business applications.

Packaging Assemblies with NuGet
To see this in action, we will turn the CarLibrary into a NuGet package and then reference it from the two client applications.
The NuGet Package properties can be accessed from the project’s property pages. Right-click the CarLibrary project and select Properties. Navigate to the Package page and see the values that we entered before to customize the assembly. There are additional properties that can be set for the NuGet package (i.e., license agreement acceptance and project information such as URL and repository location).

■ Note all of the values in the Visual studio package page ui can be entered into the project file manually, but you need to know the keywords. it helps to use Visual studio at least once to fill everything out, and then you can edit the project file by hand. You can also find all the allowable properties in the .net documentation.

For this example, we don’t need to set any additional properties except to select the “Generate NuGet package on build” check box or update the project file with the following:


net6.0
enable
disable
Copyright 2021
Phil Japikse
Apress

Pro C# 10.0
CarLibrary
This is an awesome library for cars.
1.0.0.1
1.0.0.2
1.0.0.3
true

This will cause the package to be rebuilt every time the software is built. By default, the package will be created in the bin/Debug or bin/Release folder, depending on which configuration is selected.
Packages can also be created from the command line, and the CLI provides more options than Visual Studio. For example, to build the package and place it in a directory called Publish, enter the following commands (in the CarLibrary project directory). The first command builds the assembly, and the second packages up the NuGet package.

dotnet build -c Release
dotnet pack -o .\Publish -c Release

The CarLibrary.1.0.3.nupkg file is now in the Publish directory. To see its contents, open the file with any zip utility (such as 7-Zip), and you can see the entire content, which includes the assembly, but also additional metadata.

Referencing NuGet Packages
You might be wondering where the packages that were added in the previous examples came from. The location of NuGet packages is controlled by an XML-based file named NuGet.Config. On Windows, this file is in the %appdata%\NuGet directory. This is the main file. Open it, and you will see several package sources.



The previous file listing shows two sources. The first points to NuGet.org, which is the largest NuGet package repository in the world. The second is on your local drive and is used by Visual Studio as a cache of packages.
The important item to note is that NuGet.Config files are additive by default. To add additional sources without changing the list for the entire system, you can add additional NuGet.Config files. Each file is valid for the directory that it’s placed in as well as any subdirectory. Add a new file named NuGet.Config into the solution directory, and update the contents to this:


You can also reset the list of packages by adding into the node, like this:




■ Note if you are using Visual studio, you will have to restart the ide before the updated nuget.config
settings take effect.

Remove the project references from the CSharpCarClient and VisualBasicCarClient projects, and then add package references like this (from the solution directory):

dotnet add CSharpCarClient package CarLibrary dotnet add VisualBasicCarClient package CarLibrary

Once the references are set, build the solution and look at the output in the target directories (bin\ Debug\new6.0), and you will see the CarLibrary.dll in the directory and not the CarLibrary.nupkg file. This is because .NET unpacks the contents and adds in the assemblies contained as direct references.
Now set one of the clients as the startup project and run the application, and it works exactly like before. Next, update the version number of CarLibrary to 1.0.0.4, and repackage it. In the Publish directory,
there are now two CarLibrary NuGet packages. If you rerun the add package commands, the project will be updated to use the new version. If the older version was preferred, the add package command allows for adding version numbers for a specific package.

Publishing Console Applications (Updated .NET 5/6)
Now that you have your C# CarClient application (and its related CarLibrary assembly), how do you get it out to your users? Packaging up your application and its related dependencies is referred to as publishing. Publishing .NET Framework applications required the framework to be installed on the target machine, and then it was a simple copy of the application executable and related files to run your application on another machine.
As you might expect, .NET applications can also be published in a similar manner, which is referred to as a framework-dependent deployment. However, .NET applications can also be published as a self- contained application, which doesn’t require .NET to be installed at all!
When publishing applications as self-contained, you must specify the target runtime identifier (RID) in the project file or through the command-line options. The runtime identifier is used to package your application for a specific operating system. When a runtime identifier is specified, the publish process defaults to self-contained. For a full list of available runtime identifiers (RIDs), see the .NET RID Catalog at https://docs.microsoft.com/en-us/dotnet/core/rid-catalog.
Applications can be published directly from Visual Studio or by using the .NET CLI. The command for the CLI is dotnet publish. To see all the options, use dotnet publish -h. Table 16-1 explores the common options used when publishing from the command line.

Table 16-1. Some Options for Application Publishing

Option Meaning in Life
--use-current-runtime Use current runtime as the target runtime.
-o, --output The output directory to place the published artifacts in.
--self-contained Publish the .NET Runtime with your application so the runtime doesn’t need to be installed on the target machine. The default is true if a runtime identifier is specified.
--no-self-contained Publish your application as a framework-dependent application without the .NET Runtime. A supported .NET Runtime must be installed to run your application.
-r --runtime
The target runtime to publish for. This is used when creating a self-contained deployment. The default is to publish a framework-dependent application.
-c debug | release--configuration debug | release The configuration to publish for. The default for most projects is debug.
-v, --verbosity Set the MSBuild verbosity level. Allowed values are q/quiet, m/minimal, n/normal, d/detailed, and diag/diagnostic.

Publishing Framework-Dependent Applications
When a runtime identifier isn’t specified, framework-dependent deployment is the default for the dotnet publish command. To package your application and the required files, all you need to execute with the CLI is the following command:

dotnet publish

■ Note the publish command uses the default configuration for your project, which is typically debug.

This places your application and its supporting files (six files in total) into the bin\Debug\net6.0\ publish directory. Examining the files that were added to that directory, you see two *.dll files (CarLibrary.dll and CSharpCarClient.dll) that contain all the application code. As a reminder,
the CSharpCarClient.exe file is a packaged version of dotnet.exe that is configured to launch CSharpCarClient.dll. The CSharpCarClient.deps.json file lists all the dependencies for the application, and the CSharpCarClient.runtimeconfig.json file specifies the target framework (net6.0) and the framework version. The last file is the debug file for the CSharpCarClient.
To create a release version (which will be placed in the bin\release\net6.0\publish directory), enter the following command:

dotnet publish -c release

Publishing Self-Contained Applications
Like framework-dependent deploys, self-contained deployments include all your application code and referenced assemblies, but also include the .NET Runtime files required by your application. To publish

your application as a self-contained deployment, you must include a runtime identifier in the command (or in your project file). Enter the following CLI command that places the output into a folder named selfcontained. If you are on a Mac or Linux machine, select osx-x64 or linux-x64 instead of win-x64.

dotnet publish -r win-x64 -c release -o selfcontained

■ Note new in .net 6, the --self-contained true option is no longer necessary. if a runtime identifier is specified, the publish command will use the self-contained process.

This places your application and its supporting files (226 files in total) into the selfcontained directory. If you copied these files to another computer that matches the runtime identifier, you can run the application even if the .NET 6 Runtime isn’t installed.

Publishing Self-Contained Applications as a Single File
In most situations, deploying 226 files (for an application that prints a few lines of text) is probably not the most effective way to get your application out to users. Fortunately, .NET 5 greatly improved the ability to publish your application and the cross-platform runtime files into a single file. This process is improved again with .NET 6, eliminating the need to have the native libraries exist outside of the single EXE, which was necessary in .NET 5 for Windows publishing.
The following command creates a single-file, self-contained deployment package for 64-bit Windows operating systems and places the resulting files in a folder named singlefile.

dotnet publish -r win-x64 -c release -o singlefile --self-contained true -p:PublishSingleFile=true

When you examine files that were created, you will find a single executable (CSharpCarClient.exe) and a debug file (CSharpCarClient.pdb). While the previous publish process produced a large number of smaller files, the single file version of CSharpCarClient.exe clocks in at 60 MB! Creating the single file
publication packed all of the 226 files into the new single file. What was made up for in file count reduction was traded for in file size.
To include the debug symbols file in the single file (to truly make it a single file), update your command to the following:

dotnet publish -r win-x64 -c release -o singlefile -p:PublishSingleFile= true -p:DebugType=embedded

Now you have one file that contains everything, but it’s still quite large. One option that might help with file size is compression. The output can be compressed to save space, but this will most likely affect the startup time of your application even more. To enable compression, use the following command (all on one line):

dotnet publish -c release -r win-x64 -o singlefilecompressed -p:PublishSingleFile= true -p:DebugType=embedded -p:EnableCompressionInSingleFile=true

The .NET team has been working on file trimming during the publication process for the past few releases, and with .NET 6, it’s out of preview and ready to be used. The file trimming process determines what can be removed from the runtime based on what your application uses. Part of the reason this process is now live is because the team has been annotating the runtime itself to remove the false warnings that

were prevalent in .NET 5. New in .NET 6, the trimming process doesn’t just look for assemblies that can be removed; it also looks for unused members. Use the following command to trim the single file output (all on one line):

dotnet publish -c release -r win-x64 -o singlefilecompressedandtrimmed -p: PublishSingleFile=true -p:DebugType=embedded -p:EnableCompressionInSingleFile= true -p:PublishTrimmed=true

The final step in this journey is to publish your app in a ready-to-run state. This can improve startup time since some of the JIT compilation is done ahead of time (AOT), during the publish process.

dotnet publish -c release -r win-x64 -o singlefilefinal -p:PublishSingleFile=
true -p:DebugType=embedded -p:EnableCompressionInSingleFile=true -p:PublishTrimmed= true -p:PublishReadyToRun=true

The final size of our application is 11 MB, far less than the 60 MB that we started with.
As a final note, all of these settings can be configured in the project file for your application, like this:


Exe
net6.0
enable
disable
true
true
win-x64
true
embedded
true
true

With those values set in the project file, the command line becomes much shorter:

dotnet publish -c release -o singlefilefinal2

How .NET Locates Assemblies
So far in this book, all the assemblies that you have built were directly related (except for the NuGet example you just completed). You added either a project reference or a direct reference between projects. In these cases (as well as the NuGet example), the dependent assembly was copied directly into the target directory of the client application. Locating the dependent assembly isn’t an issue, since they reside on the disk right next to the application that needs them.
But what about the .NET Framework? How are those located? Previous versions of .NET installed the framework files into the Global Assembly Cache (GAC), and all .NET Framework applications knew how to locate the framework files.
However, the GAC prevents the side-by-side capabilities in .NET Core, so there isn’t a single repository of runtime and framework files. Instead, the files that make up the framework are installed together in C:\ Program Files\dotnet (on Windows), separated by version. Based on the version of the application (as specified in the .csproj file), the necessary runtime and framework files are loaded for an application from the specified version’s directory.

Specifically, when a version of the runtime is started, the runtime host provides a set of probing paths that it will use to find an application’s dependencies. There are five probing properties (each of them optional), as listed in Table 16-2.

Table 16-2. Application Probing Properties

Option Meaning in Life
TRUSTED_PLATFORM_ASSEMBLIES List of platform and application assembly file paths
PLATFORM_RESOURCE_ROOTS List of directory paths to search for satellite resource assemblies
NATIVE_DLL_SEARCH_DIRECTORIES List of directory paths to search for unmanaged (native) libraries
APP_PATHS List of directory paths to search for managed assemblies
APP_NI_PATHS List of directory paths to search for native images of managed assemblies

To see the default paths for these, create a new .NET Console application named FunWithProbingPaths.
Update the Program.cs file to the following top-level statements:

Console.WriteLine("*** Fun with Probing Paths ***"); Console.WriteLine($"TRUSTED_PLATFORM_ASSEMBLIES: ");
//Use ':' on non-Windows platforms
var list = AppContext.GetData("TRUSTED_PLATFORM_ASSEMBLIES")
.ToString().Split(';'); foreach (var dir in list)
{
Console.WriteLine(dir);
}
Console.WriteLine();
Console.WriteLine($"PLATFORM_RESOURCE_ROOTS: {AppContext.GetData ("PLATFORM_RESOURCE_ROOTS")}"); Console.WriteLine();
Console.WriteLine($"NATIVE_DLL_SEARCH_DIRECTORIES: {AppContext.GetData ("NATIVE_DLL_SEARCH_ DIRECTORIES")}");
Console.WriteLine();
Console.WriteLine($"APP_PATHS: {AppContext.GetData("APP_PATHS")}"); Console.WriteLine();
Console.WriteLine($"APP_NI_PATHS: {AppContext.GetData("APP_NI_PATHS")}"); Console.WriteLine();
Console.ReadLine();

When you run this app, you will see most of the values come from the TRUSTED_PLATFORM_ASSEMBLIES variable. In addition to the assembly that is created for this project in the target directory, you will see a list of base class libraries from the current runtime directory, C:\Program Files\dotnet\shared\Microsoft. NETCore.App\6.0.0 (your version number might be different).
Each of the files directly referenced by your application is added to the list as well as any runtime files that are required for your application. The list of runtime libraries is populated by one or more *.deps.json files that are loaded with the .NET Runtime. There are several in the installation directory for the SDK (used for building the software) and the runtime (used for running the software). With our simple example, the only file used is Microsoft.NETCore.App.deps.json.
As your application grows in complexity, so will the list of files in TRUSTED_PLATFORM_ASSEMBLIES. For example, if you add a reference to the Microsoft.EntityFrameworkCore package, the list of required

assemblies grows. To demonstrate this, enter the following command in Package Manager Console (in the same directory as the *.csproj file):

dotnet add package Microsoft.EntityFrameworkCore

Once you have added the package, rerun the application, and notice how many more files are listed. Even though you added only one new reference, the Microsoft.EntityFrameworkCore package has its dependencies, which get added into the trusted file list.

Summary
This chapter examined the role of .NET class libraries (aka .NET *.dlls). As you have seen, class libraries are
.NET binaries that contain logic intended to be reused across a variety of projects.
You learned the details of partitioning types into .NET namespaces and the difference between a .NET and .NET Standard, got started with application configuration, and dove deep into the composition of class libraries. Next you learned how to publish .NET console applications. Finally, you learned how to package your applications using NuGet.

Pro C#10 CHAPTER 15 Multithreaded, Parallel, and Async Programming

CHAPTER 15

Multithreaded, Parallel, and Async Programming

Nobody enjoys working with an application that is sluggish during its execution. Moreover, nobody enjoys starting a task in an application (perhaps initiated by clicking a toolbar item) that prevents other parts of the program from being as responsive as possible. Before the release of .NET (and .NET Core), building applications that had the ability to perform multiple tasks typically required authoring complex C++ code that used the Windows threading APIs. Thankfully, the .NET/.NET Core platform provides several ways for you to build software that can perform complex operations on unique paths of execution, with far fewer pain points.
This chapter begins by defining the overall nature of a “multithreaded application.” Next, you will be introduced to the original threading namespace that has shipped since .NET 1.0, specifically
System.Threading. Here, you will examine numerous types (Thread, ThreadStart, etc.) that allow you to explicitly create additional threads of execution and synchronize your shared resources, which helps ensure that multiple threads can share data in a nonvolatile manner.
The remaining parts of this chapter will examine three more recent techniques .NET Core developers can use to build multithreaded software, specifically the Task Parallel Library (TPL), Parallel LINQ (PLINQ), and the relatively new (as of C# 6) intrinsic asynchronous keywords of C# (async and await). As you will see, these features can dramatically simplify how you can build responsive multithreaded software applications.

The Process/AppDomain/Context/Thread Relationship
In Chapter 14, a thread was defined as a path of execution within an executable application. While many
.NET Core applications can live happy and productive single-threaded lives, an assembly’s primary thread (spawned by the runtime when the application’s entry point executes) may create secondary threads of execution at any time to perform additional units of work. By creating additional threads, you can build more responsive (but not necessarily faster executing on single-core machines) applications.
The System.Threading namespace was released with .NET 1.0 and offers one approach to build multithreaded applications. The Thread class is perhaps the core type, as it represents a given thread. If you want to programmatically obtain a reference to the thread currently executing a given member, simply call the static Thread.CurrentThread property, like so:

static void ExtractExecutingThread()
{
// Get the thread currently
// executing this method.
Thread currThread = Thread.CurrentThread;
}

© Andrew Troelsen, Phil Japikse 2022
A. Troelsen and P. Japikse, Pro C# 10 with .NET 6, https://doi.org/10.1007/978-1-4842-7869-7_15

571

Recall that with .NET Core, there is only a single AppDomain. Even though extra AppDomain’s cannot be created, an application’s AppDomain can have numerous threads executing within it at any given time. To get a reference to the AppDomain that is hosting the application, call the static Thread.GetDomain() method, like so:

static void ExtractAppDomainHostingThread()
{
// Obtain the AppDomain hosting the current thread. AppDomain ad = Thread.GetDomain();
}

A single thread may also be moved into an execution context at any given time, and it may be relocated within a new execution context at the whim of the .NET Core Runtime. When you want to obtain the current execution context a thread happens to be executing in, use the static Thread.CurrentThread.
ExecutionContext property, like so:

static void ExtractCurrentThreadExecutionContext()
{
// Obtain the execution context under which the
// current thread is operating. ExecutionContext ctx =
Thread.CurrentThread.ExecutionContext;
}

Again, the .NET Core Runtime oversees moving threads into (and out of) execution contexts. As a .NET Core developer, you can usually remain blissfully unaware where a given thread ends up. Nevertheless, you should be aware of the various ways of obtaining the underlying primitives.

The Problem of Concurrency
One of the many “joys” (read: painful aspects) of multithreaded programming is that you have little control over how the underlying operating system or the runtime uses its threads. For example, if you craft a block of code that creates a new thread of execution, you cannot guarantee that the thread executes immediately.
Rather, such code only instructs the OS/Runtime to execute the thread as soon as possible (which is typically when the thread scheduler gets around to it).
Furthermore, given that threads can be moved between application and contextual boundaries as required by the runtime, you must be mindful of which aspects of your application are thread-volatile (e.g., subject to multithreaded access) and which operations are atomic (thread-volatile operations are the dangerous ones!).
To illustrate the problem, assume a thread is invoking a method of a specific object. Now assume that this thread is instructed by the thread scheduler to suspend its activity to allow another thread to access the same method of the same object.
If the original thread was not finished with its operation, the second incoming thread may be viewing an object in a partially modified state. At this point, the second thread is basically reading bogus data, which is sure to give way to extremely odd (and hard to find) bugs, which are even harder to replicate and debug.
Atomic operations, on the other hand, are always safe in a multithreaded environment. Sadly, there are few operations in the .NET Core base class libraries that are guaranteed to be atomic. Even the act of assigning a value to a member variable is not atomic! Unless the .NET Core documentation specifically says an operation is atomic, you must assume it is thread-volatile and take precautions.

The Role of Thread Synchronization
At this point, it should be clear that multithreaded programs are in themselves quite volatile, as numerous threads can operate on the shared resources at (more or less) the same time. To protect an application’s resources from possible corruption, .NET Core developers must use any number of threading primitives (such as locks, monitors, and the [Synchronization] attribute or language keyword support) to control access among the executing threads.
Although the .NET Core platform cannot make the difficulties of building robust multithreaded applications completely disappear, the process has been simplified considerably. Using types defined within the System.Threading namespace, the Task Parallel Library, and the C# async and await language keywords, you can work with multiple threads with minimal fuss and bother.
Before diving into the System.Threading namespace, the TPL, and the C# async and await keywords, you will begin by examining how the .NET Core delegate type can be used to invoke a method in an asynchronous manner. While it is most certainly true that since .NET 4.6 the new C# async and await keywords offer a simpler alternative to asynchronous delegates, it is still important that you know how to interact with code using this approach (trust me, there is a ton of code in production that uses asynchronous delegates).

The System.Threading Namespace
Under the .NET and .NET Core platforms, the System.Threading namespace provides types that enable the direct construction of multithreaded applications. In addition to providing types that allow you to interact with a .NET Core Runtime thread, this namespace defines types that allow access to the .NET Core Runtime– maintained thread pool, a simple (non-GUI-based) Timer class, and numerous types used to provide synchronized access to shared resources. Table 15-1 lists some of the important members of this namespace. (Be sure to consult the .NET Core SDK documentation for full details.)

Table 15-1. Core Types of the System.Threading Namespace

Type Meaning in Life
Interlocked This type provides atomic operations for variables that are shared by multiple threads.
Monitor This type provides the synchronization of threading objects using locks and wait/signals. The C# lock keyword uses a Monitor object under the hood.
Mutex This synchronization primitive can be used for synchronization between application domain boundaries.
ParameterizedThreadStart This delegate allows a thread to call methods that take any number of arguments.
Semaphore This type allows you to limit the number of threads that can access a resource concurrently.
Thread This type represents a thread that executes within the .NET Core Runtime. Using this type, you can spawn additional threads in the originating AppDomain.
ThreadPool This type allows you to interact with the .NET Core Runtime–maintained thread pool within a given process.
(continued)

Table 15-1. (continued)

Type Meaning in Life
ThreadPriority This enum represents a thread’s priority level (Highest, Normal, etc.).
ThreadStart This delegate is used to specify the method to call for a given thread. Unlike the ParameterizedThreadStart delegate, targets of ThreadStart must always have the same prototype.
ThreadState This enum specifies the valid states a thread may take (Running, Aborted, etc.).
Timer This type provides a mechanism for executing a method at specified intervals.
TimerCallback This delegate type is used in conjunction with Timer types.

The System.Threading.Thread Class
The most primitive of all types in the System.Threading namespace is Thread. This class represents an object-oriented wrapper around a given path of execution within an AppDomain. This type also defines several methods (both static and instance level) that allow you to create new threads within the current AppDomain, as well as to suspend, stop, and destroy a thread. Consider the list of key static members in Table 15-2.

Table 15-2. Key Static Members of the Thread Type

Static Member Meaning in Life
ExecutionContext This read-only property returns information relevant to the logical thread of execution, including security, call, synchronization, localization, and transaction contexts.
CurrentThread This read-only property returns a reference to the currently running thread.
Sleep() This method suspends the current thread for a specified time.

The Thread class also supports several instance-level members, some of which are shown in Table 15-3.

Table 15-3. Select Instance-Level Members of the Thread Type

Instance-Level Member Meaning in Life
IsAlive Returns a Boolean that indicates whether this thread has been started (and has not yet terminated or aborted).
IsBackground Gets or sets a value indicating whether this thread is a “background thread” (more details in just a moment).
Name Allows you to establish a friendly text name of the thread.
Priority Gets or sets the priority of a thread, which may be assigned a value from the
ThreadPriority enumeration.
(continued)

Table 15-3. (continued)

Instance-Level Member Meaning in Life
ThreadState Gets the state of this thread, which may be assigned a value from the
ThreadState enumeration.
Abort() Instructs the .NET Core Runtime to terminate the thread as soon as possible.
Interrupt() Interrupts (e.g., wakes) the current thread from a suitable wait period.
Join() Blocks the calling thread until the specified thread (the one on which Join() is called) exits.
Resume() Resumes a thread that has been previously suspended.
Start() Instructs the .NET Core Runtime to execute the thread ASAP.
Suspend() Suspends the thread. If the thread is already suspended, a call to Suspend() has no effect.

■Note aborting or suspending an active thread is generally considered a bad idea. When you do so, there is a chance (however small) that a thread could “leak” its workload when disturbed or terminated.

Obtaining Statistics About the Current Thread of Execution
Recall that the entry point of an executable assembly (i.e., the top-level statements or the Main() method) runs on the primary thread of execution. To illustrate the basic use of the Thread type, assume you have
a new Console Application project named ThreadStats. As you know, the static Thread.CurrentThread property retrieves a Thread object that represents the currently executing thread. Once you have obtained the current thread, you are able to print out various statistics, like so:

Console.WriteLine(" Primary Thread stats \n");

// Obtain and name the current thread. Thread primaryThread = Thread.CurrentThread; primaryThread.Name = "ThePrimaryThread";

// Print out some stats about this thread. Console.WriteLine("ID of current thread: {0}",
primaryThread.ManagedThreadId); Console.WriteLine("Thread Name: {0}",
primaryThread.Name);
Console.WriteLine("Has thread started?: {0}", primaryThread.IsAlive);
Console.WriteLine("Priority Level: {0}", primaryThread.Priority);
Console.WriteLine("Thread State: {0}", primaryThread.ThreadState);
Console.ReadLine();

Here is the current output:

Primary Thread stats ID of current thread: 1
Thread Name: ThePrimaryThread Has thread started?: True Priority Level: Normal
Thread State: Running

The Name Property
Notice that the Thread class supports a property called Name. If you do not set this value, Name will return an empty string. However, once you assign a friendly string moniker to a given Thread object, you can greatly simplify your debugging endeavors. If you are using Visual Studio, you may access the Threads window during a debugging session (select Debug ➤ Windows ➤ Threads when the program is running). As you can see from Figure 15-1, you can quickly identify the thread you want to diagnose.

Figure 15-1. Debugging a thread with Visual Studio

The Priority Property
Next, notice that the Thread type defines a property named Priority. By default, all threads have a priority level of Normal. However, you can change this at any point in the thread’s lifetime using the Priority property and the related System.Threading.ThreadPriority enumeration, like so:

public enum ThreadPriority
{
Lowest, BelowNormal,
Normal, // Default value. AboveNormal,
Highest
}

If you were to assign a thread’s priority level to a value other than the default (ThreadPriority.Normal), understand that you would have no direct control over when the thread scheduler switches between threads. A thread’s priority level offers a hint to the .NET Core Runtime regarding the importance of the thread’s activity. Thus, a thread with the value ThreadPriority.Highest is not necessarily guaranteed to be given the highest precedence.
Again, if the thread scheduler is preoccupied with a given task (e.g., synchronizing an object, switching threads, or moving threads), the priority level will most likely be altered accordingly. However, all things being equal, the .NET Core Runtime will read these values and instruct the thread scheduler how to best allocate time slices. Threads with an identical thread priority should each receive the same amount of time to perform their work.

In most cases, you will seldom (if ever) need to directly alter a thread’s priority level. In theory, it is possible to jack up the priority level on a set of threads, thereby preventing lower-priority threads from executing at their required levels (so use caution).

Manually Creating Secondary Threads
When you want to programmatically create additional threads to carry on some unit of work, follow this predictable process when using the types of the System.Threading namespace:
1.Create a method to be the entry point for the new thread.
2.Create a new ParameterizedThreadStart (or ThreadStart) delegate, passing the address of the method defined in step 1 to the constructor.
3.Create a Thread object, passing the ParameterizedThreadStart/ThreadStart
delegate as a constructor argument.
4.Establish any initial thread characteristics (name, priority, etc.).
5.Call the Thread.Start() method. This starts the thread at the method referenced by the delegate created in step 2 as soon as possible.
As stated in step 2, you may use two distinct delegate types to “point to” the method that the secondary thread will execute. The ThreadStart delegate can point to any method that takes no arguments and returns nothing. This delegate can be helpful when the method is designed to simply run in the background without further interaction.
The limitation of ThreadStart is that you are unable to pass in parameters for processing. However, the ParameterizedThreadStart delegate type allows a single parameter of type System.Object. Given that anything can be represented as a System.Object, you can pass in any number of parameters via a custom class or structure. Do note, however, that the ThreadStart and ParameterizedThreadStart delegates can only point to methods that return void.

Working with the ThreadStart Delegate
To illustrate the process of building a multithreaded application (as well as to demonstrate the usefulness of doing so), assume you have a Console Application project named SimpleMultiThreadApp that allows the end user to choose whether the application will perform its duties using the single primary thread or whether it will split its workload using two separate threads of execution.
Assuming you have imported the System.Threading namespace, your first step is to define a method to perform the work of the (possible) secondary thread. To keep focused on the mechanics of building multithreaded programs, this method will simply print out a sequence of numbers to the console window, pausing for approximately two seconds with each pass. Here is the full definition of the Printer class:

namespace SimpleMultiThreadApp; public class Printer
{
public void PrintNumbers()
{
// Display Thread info.
Console.WriteLine("-> {0} is executing PrintNumbers()", Thread.CurrentThread.Name);

// Print out numbers. Console.Write("Your numbers: "); for(int i = 0; i < 10; i++)
{
Console.Write("{0}, ", i); Thread.Sleep(2000);
}
Console.WriteLine();
}
}

Now, within Program.cs, you will add top-level statements to first prompt the user to determine whether one or two threads will be used to perform the application’s work. If the user requests a single thread, you will simply invoke the PrintNumbers() method within the primary thread. However, if the user specifies two threads, you will create a ThreadStart delegate that points to PrintNumbers(), pass this delegate object into the constructor of a new Thread object, and call Start() to inform the .NET Core Runtime this thread is ready for processing. Here is the complete implementation:

using SimpleMultiThreadApp;

Console.WriteLine(" The Amazing Thread App \n"); Console.Write("Do you want [1] or [2] threads? ");
string threadCount = Console.ReadLine();

// Name the current thread.
Thread primaryThread = Thread.CurrentThread; primaryThread.Name = "Primary";

// Display Thread info.
Console.WriteLine("-> {0} is executing Main()", Thread.CurrentThread.Name);

// Make worker class. Printer p = new Printer();

switch(threadCount)
{
case "2":
// Now make the thread. Thread backgroundThread =
new Thread(new ThreadStart(p.PrintNumbers)); backgroundThread.Name = "Secondary"; backgroundThread.Start();
break; case "1":
p.PrintNumbers(); break;
default:
Console.WriteLine("I don't know what you want...you get 1 thread."); goto case "1";
}

// Do some additional work.
Console.WriteLine("This is on the main thread, and we are finished."); Console.ReadLine();

Now, if you run this program with a single thread, you will find that the final console output will not display the message until the entire sequence of numbers has printed to the console. As you are explicitly pausing for approximately two seconds after each number is printed, this will result in a less-than-stellar end-user experience. However, if you select two threads, the message box displays instantly, given that a unique Thread object is responsible for printing the numbers to the console.

Working with the ParameterizedThreadStart Delegate
Recall that the ThreadStart delegate can point only to methods that return void and take no arguments. While this might fit the bill in some cases, if you want to pass data to the method executing on the secondary thread, you will need to use the ParameterizedThreadStart delegate type. To illustrate, create a new Console Application project named AddWithThreads. Now, given that ParameterizedThreadStart can point to any method taking a System.Object parameter, you will create a custom type containing the numbers to be added, like so:

namespace AddWithThreads; class AddParams
{
public int a, b;

public AddParams(int numb1, int numb2)
{
a = numb1; b = numb2;
}
}

Next, create a method in the Program.cs file that will take an AddParams parameter and print the sum of the two numbers involved, as follows:

void Add(object data)
{
if (data is AddParams ap)
{
Console.WriteLine("ID of thread in Add(): {0}", Environment.CurrentManagedThreadId);

Console.WriteLine("{0} + {1} is {2}",
ap.a, ap.b, ap.a + ap.b);
}
}

The code within Program.cs is straightforward. Simply use ParameterizedThreadStart rather than
ThreadStart, like so:

using AddWithThreads;

Console.WriteLine(" Adding with Thread objects "); Console.WriteLine("ID of thread in Main(): {0}",
Environment.CurrentManagedThreadId);

// Make an AddParams object to pass to the secondary thread. AddParams ap = new AddParams(10, 10);
Thread t = new Thread(new ParameterizedThreadStart(Add)); t.Start(ap);

// Force a wait to let other thread finish. Thread.Sleep(5);
Console.ReadLine();

The AutoResetEvent Class
In these first few examples, there is not a clean way to know when the secondary thread has completed its work. In the last example, Sleep was called with an arbitrary time to let the other thread finish. One simple and thread-safe way to force a thread to wait until another is completed is to use the AutoResetEvent class. In the thread that needs to wait, create an instance of this class and pass in false to the constructor to signify you have not yet been notified. Then, at the point at which you are willing to wait, call the WaitOne() method. Here is the update to the Program.cs class, which will do this very thing using a static-level AutoResetEvent member variable:

AutoResetEvent _waitHandle = new AutoResetEvent(false);

Console.WriteLine(" Adding with Thread objects "); Console.WriteLine("ID of thread in Main(): {0}",
Environment.CurrentManagedThreadId); AddParams ap = new AddParams(10, 10);
Thread t = new Thread(new ParameterizedThreadStart(Add)); t.Start(ap);

// Wait here until you are notified!
_waitHandle.WaitOne(); Console.WriteLine("Other thread is done!");

Console.ReadLine();
...

When the other thread is completed with its workload, it will call the Set() method on the same instance of the AutoResetEvent type.

void Add(object data)
{
if (data is AddParams ap)
{

Console.WriteLine("ID of thread in Add(): {0}", Environment.CurrentManagedThreadId);

Console.WriteLine("{0} + {1} is {2}",
ap.a, ap.b, ap.a + ap.b);

// Tell other thread we are done.
_waitHandle.Set();
}
}

Foreground Threads and Background Threads
Now that you have seen how to programmatically create new threads of execution using the System. Threading namespace, let’s formalize the distinction between foreground threads and background threads:
•Foreground threads can prevent the current application from terminating. The .NET Core Runtime will not shut down an application (which is to say, unload the hosting AppDomain) until all foreground threads have ended.
•Background threads (sometimes called daemon threads) are viewed by the .NET Core Runtime as expendable paths of execution that can be ignored at any point in time (even if they are currently laboring over some unit of work). Thus, if all
foreground threads have terminated, all background threads are automatically killed when the application domain unloads.
It is important to note that foreground and background threads are not synonymous with primary and worker threads. By default, every thread you create via the Thread.Start() method is automatically a
foreground thread. Again, this means that the AppDomain will not unload until all threads of execution have completed their units of work. In most cases, this is exactly the behavior you require.
For the sake of argument, however, assume that you want to invoke Printer.PrintNumbers() on a secondary thread that should behave as a background thread. Again, this means the method pointed to by the Thread type (via the ThreadStart or ParameterizedThreadStart delegate) should be able to halt safely as soon as all foreground threads are done with their work. Configuring such a thread is as simple as setting the IsBackground property to true.
Create a new Console Application named BackgroundThreads and copy the Printer.cs file into the new project. Update the namespace of the Printer class to BackgroundThreads, like this:

namespace BackgroundThreads; public class Printer
{
...
}

Update the Program.cs file to match the following:

using BackgroundThreads;

Console.WriteLine(" Background Threads \n"); Printer p = new Printer();
Thread bgroundThread =
new Thread(new ThreadStart(p.PrintNumbers));

// This is now a background thread. bgroundThread.IsBackground = true; bgroundThread.Start();

Notice that this code is not making a call to Console.ReadLine() to force the console to remain visible until you press the Enter key. Thus, when you run the application, it will shut down immediately because the Thread object has been configured as a background thread. Given that the entry point into an application (either top-level statements as shown here or a Main() method) triggers the creation of the primary foreground thread, as soon as the logic in the entry point completes, the AppDomain unloads before the secondary thread can complete its work.
However, if you comment out the line that sets the IsBackground property, you will find that each number prints to the console, as all foreground threads must finish their work before the AppDomain is unloaded from the hosting process.
For the most part, configuring a thread to run as a background type can be helpful when the worker thread in question is performing a noncritical task that is no longer needed when the main task of the program is finished. For example, you could build an application that pings an email server every few minutes for new emails, updates current weather conditions, or performs some other noncritical task.

The Issue of Concurrency
When you build multithreaded applications, your program needs to ensure that any piece of shared data is protected against the possibility of numerous threads changing its value. Given that all threads in an AppDomain have concurrent access to the shared data of the application, imagine what might happen
if multiple threads were accessing the same point of data. As the thread scheduler will force threads to suspend their work at random, what if thread A is kicked out of the way before it has fully completed its work? Thread B is now reading unstable data.
To illustrate the problem of concurrency, let’s build another Console Application project named MultiThreadedPrinting. This application will once again use the Printer class created previously, but this time the PrintNumbers() method will force the current thread to pause for a randomly generated amount of time.

namespace MultiThreadedPrinting; public class Printer
{
public void PrintNumbers()
{
// Display Thread info.
Console.WriteLine("-> {0} is executing PrintNumbers()", Thread.CurrentThread.Name);

// Print out numbers.
for (int i = 0; i < 10; i++)
{
// Put thread to sleep for a random amount of time. Random r = new Random();
Thread.Sleep(1000 * r.Next(5)); Console.Write("{0}, ", i);
}
Console.WriteLine();
}
}

The calling code is responsible for creating an array of ten (uniquely named) Thread objects, each of which is making calls on the same instance of the Printer object as follows:
using MultiThreadedPrinting; Console.WriteLine("Synchronizing Threads \n"); Printer p = new Printer();
// Make 10 threads that are all pointing to the same
// method on the same object. Thread[] threads = new Thread[10]; for (int i = 0; i < 10; i++)
{
threads[i] = new Thread(new ThreadStart(p.PrintNumbers))
{
Name = $"Worker thread #{i}"
};
}
// Now start each one. foreach (Thread t in threads)
{
t.Start();
}
Console.ReadLine();
Before looking at some test runs, let’s recap the problem. The primary thread within this AppDomain begins life by spawning ten secondary worker threads. Each worker thread is told to make calls on the PrintNumbers() method on the same Printer instance. Given that you have taken no precautions to lock down this object’s shared resources (the console), there is a good chance that the current thread will be kicked out of the way before the PrintNumbers() method is able to print the complete results. Because you do not know exactly when (or if ) this might happen, you are bound to get unpredictable results. For example, you might find the output shown here:

Synchronizing Threads
-> Worker thread #3 is executing PrintNumbers()
-> Worker thread #0 is executing PrintNumbers()
-> Worker thread #1 is executing PrintNumbers()
-> Worker thread #2 is executing PrintNumbers()
-> Worker thread #4 is executing PrintNumbers()
-> Worker thread #5 is executing PrintNumbers()
-> Worker thread #6 is executing PrintNumbers()
-> Worker thread #7 is executing PrintNumbers()
-> Worker thread #8 is executing PrintNumbers()
-> Worker thread #9 is executing PrintNumbers()
0, 0, 0, 0, 1, 0, 1, 0, 1, 0, 0, 0, 1, 0, 1, 1, 2, 3, 1, 2, 2, 2, 1, 2, 1, 1, 2, 2, 3, 3, 4,
3, 3, 2, 2, 3, 4, 3, 4, 5, 4, 5, 4, 4, 3, 6, 7, 2, 3, 4, 4, 4, 5, 6, 5, 3, 5, 8, 9,
6, 7, 4, 5, 6, 6, 5, 5, 5, 8, 5, 6, 7, 8, 7, 7, 6, 6, 6, 8, 9,
8, 7, 7, 7, 7, 9,
6, 8, 9,
8, 9,

9, 9,

8, 8, 7, 8, 9,
9,
9,

Now run the application a few more times and examine the output. It will most likely be different each time.

■ Note if you are unable to generate unpredictable outputs, increase the number of threads from 10 to 100 (for example) or introduce another call to Thread.Sleep() within your program. eventually, you will encounter the concurrency issue.

There are clearly some problems here. As each thread is telling the Printer to print the numerical data, the thread scheduler is happily swapping threads in the background. The result is inconsistent output. What you need is a way to programmatically enforce synchronized access to the shared resources. As you would guess, the System.Threading namespace provides several synchronization-centric types. The C# programming language also provides a keyword for the very task of synchronizing shared data in multithreaded applications.

Synchronization Using the C# lock Keyword
The first technique you can use to synchronize access to shared resources is the C# lock keyword. This keyword allows you to define a scope of statements that must be synchronized between threads. By doing so, incoming threads cannot interrupt the current thread, thus preventing it from finishing its work. The lock keyword requires you to specify a token (an object reference) that must be acquired by a thread to enter within the lock scope. When you are attempting to lock down a private instance-level method, you can simply pass in a reference to the current type, as follows:

private void SomePrivateMethod()
{
// Use the current object as the thread token. lock(this)
{
// All code within this scope is thread safe.
}
}

However, if you are locking down a region of code within a public member, it is safer (and a best practice) to declare a private object member variable to serve as the lock token, like so:

public class Printer
{
// Lock token.
private object threadLock = new object();

public void PrintNumbers()
{

// Use the lock token. lock (threadLock)
{
...
}
}
}

In any case, if you examine the PrintNumbers() method, you can see that the shared resource the threads are competing to gain access to is the console window. Scope all interactions with the Console type within a lock scope, as follows:

public void PrintNumbers()
{
// Use the private object lock token. lock (threadLock)
{
// Display Thread info.
Console.WriteLine("-> {0} is executing PrintNumbers()", Thread.CurrentThread.Name);
// Print out numbers. Console.Write("Your numbers: "); for (int i = 0; i < 10; i++)
{
Random r = new Random(); Thread.Sleep(1000 * r.Next(5)); Console.Write("{0}, ", i);
}
Console.WriteLine();
}
}

When doing this, you have effectively designed a method that will allow the current thread to complete its task. Once a thread enters into a lock scope, the lock token (in this case, a reference to the current object) is inaccessible by other threads until the lock is released after the lock scope has exited. Thus, if thread A has obtained the lock token, other threads are unable to enter any scope that uses the same lock token until thread A relinquishes the lock token.

■ Note if you are attempting to lock down code in a static method, simply declare a private static object member variable to serve as the lock token.

If you now run the application, you can see that each thread has ample opportunity to finish its business.

Synchronizing Threads
-> Worker thread #0 is executing PrintNumbers() Your numbers: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
-> Worker thread #1 is executing PrintNumbers() Your numbers: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,

-> Worker thread #3 is executing PrintNumbers()
Your numbers: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
-> Worker thread #2 is executing PrintNumbers()
Your numbers: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
-> Worker thread #4 is executing PrintNumbers()
Your numbers: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
-> Worker thread #5 is executing PrintNumbers()
Your numbers: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
-> Worker thread #7 is executing PrintNumbers()
Your numbers: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
-> Worker thread #6 is executing PrintNumbers()
Your numbers: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
-> Worker thread #8 is executing PrintNumbers()
Your numbers: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,
-> Worker thread #9 is executing PrintNumbers()
Your numbers: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9,

Synchronization Using the System.Threading.Monitor Type
The C# lock statement is a shorthand notation for working with the System.Threading.Monitor class. Once processed by the C# compiler, a lock scope resolves to the following (which you can verify using ildasm.exe):

public void PrintNumbers()
{
Monitor.Enter(threadLock); try
{
// Display Thread info.
Console.WriteLine("-> {0} is executing PrintNumbers()", Thread.CurrentThread.Name);

// Print out numbers. Console.Write("Your numbers: "); for (int i = 0; i < 10; i++)
{
Random r = new Random(); Thread.Sleep(1000 * r.Next(5)); Console.Write("{0}, ", i);
}
Console.WriteLine();
}
finally
{
Monitor.Exit(threadLock);
}
}

First, notice that the Monitor.Enter() method is the ultimate recipient of the thread token you specified as the argument to the lock keyword. Next, all code within a lock scope is wrapped within a try

block. The corresponding finally block ensures that the thread token is released (via the Monitor.Exit() method), regardless of any possible runtime exception. If you were to modify the MultiThreadPrinting program to make direct use of the Monitor type (as just shown), you would find the output is identical.
Now, given that the lock keyword seems to require less code than making explicit use of the System. Threading.Monitor type, you might wonder about the benefits of using the Monitor type directly. The short answer is control. If you use the Monitor type, you can instruct the active thread to wait for some duration of time (via the static Monitor.Wait() method), inform waiting threads when the current thread is completed (via the static Monitor.Pulse() and Monitor.PulseAll() methods), and so on.
As you would expect, in a great number of cases, the C# lock keyword will fit the bill. However, if you are interested in checking out additional members of the Monitor class, consult the .NET Core documentation.

Synchronization Using the System.Threading.Interlocked Type
Although it is always hard to believe, until you look at the underlying CIL code, assignments and simple arithmetic operations are not atomic. For this reason, the System.Threading namespace provides a type that allows you to operate on a single point of data atomically with less overhead than with the Monitor type. The Interlocked class defines the key static members shown in Table 15-4.

Table 15-4. Select Static Members of the System.Threading.Interlocked Type

Member Meaning in Life

CompareExchange() Safely tests two values for equality and, if equal, exchanges one of the values with a third
Decrement() Safely decrements a value by 1 Exchange() Safely swaps two values Increment() Safely increments a value by 1

Although it might not seem like it from the onset, the process of atomically altering a single value is quite common in a multithreaded environment. Assume you have code that increments an integer member variable named intVal. Rather than writing synchronization code such as the following:

int intVal = 5;
object myLockToken = new(); lock(myLockToken)
{
intVal++;
}

you can simplify your code via the static Interlocked.Increment() method. Simply pass in the variable to increment by reference. Do note that the Increment() method not only adjusts the value of the incoming parameter but also returns the new value.

intVal = Interlocked.Increment(ref intVal);

In addition to Increment() and Decrement(), the Interlocked type allows you to atomically assign numerical and object data. For example, if you want to assign the value of a member variable to the

value 83, you can avoid the need to use an explicit lock statement (or explicit Monitor logic) and use the
Interlocked.Exchange() method, like so:

var myInt = 27; Interlocked.Exchange(ref myInt, 83); Console.WriteLine(myInt);

Finally, if you want to test two values for equality and change the point of comparison in a thread-safe manner, you can leverage the Interlocked.CompareExchange() method as follows:

// If the value of i is currently 83, change myInt to 99. Interlocked.CompareExchange(ref myInt, 99, 83);

Programming with Timer Callbacks
Many applications have the need to call a specific method during regular intervals of time. For example, you might have an application that needs to display the current time on a status bar via a given helper function. As another example, you might want to have your application call a helper function every so often to perform noncritical background tasks such as checking for new email messages. For situations such
as these, you can use the System.Threading.Timer type in conjunction with a related delegate named
TimerCallback.
To illustrate, assume you have a Console Application project (TimerApp) that will print the current time every second until the user presses a key to terminate the application. The first obvious step is to write the method that will be called by the Timer type.

Console.WriteLine(" Working with Timer type \n"); Console.ReadLine();

static void PrintTime(object state)
{
Console.WriteLine("Time is: {0}", DateTime.Now.ToLongTimeString());
}

Notice the PrintTime() method has a single parameter of type System.Object and returns void. This is not optional, given that the TimerCallback delegate can only call methods that match this signature. The value passed into the target of your TimerCallback delegate can be any type of object (in the case of the email example, this parameter might represent the name of the Microsoft Exchange Server to interact with during the process). Also note that given that this parameter is indeed a System.Object, you can pass in multiple arguments using a System.Array or custom class/structure.
The next step is to configure an instance of the TimerCallback delegate and pass it into the Timer object. In addition to configuring a TimerCallback delegate, the Timer constructor allows you to specify the optional parameter information to pass into the delegate target (defined as a System.Object), the interval to poll the method, and the amount of time to wait (in milliseconds) before making the first call. Here is an example:

Console.WriteLine(" Working with Timer type \n");

// Create the delegate for the Timer type. TimerCallback timeCB = new TimerCallback(PrintTime);

// Establish timer settings.

Timer t = new Timer(
timeCB, // The TimerCallback delegate object.
null, // Any info to pass into the called method (null for no info). 0, // Amount of time to wait before starting (in milliseconds). 1000); // Interval of time between calls (in milliseconds).

Console.WriteLine("Hit Enter key to terminate..."); Console.ReadLine();

In this case, the PrintTime() method will be called roughly every second and will pass in no additional information to said method. Here is the output:

Working with Timer type Hit key to terminate...
Time is: 6:51:48 PM Time is: 6:51:49 PM Time is: 6:51:50 PM Time is: 6:51:51 PM Time is: 6:51:52 PM
Press any key to continue . . .

If you did want to send in some information for use by the delegate target, simply substitute the null
value of the second constructor parameter with the appropriate information, like so:

// Establish timer settings.
Timer t = new Timer(timeCB, "Hello From C# 10.0", 0, 1000);

You can then obtain the incoming data as follows:

static void PrintTime(object state)
{
Console.WriteLine("Time is: {0}, Param is: {1}", DateTime.Now.ToLongTimeString(), state.ToString());
}

Using a Stand-Alone Discard (New 7.0)
In the previous example, the Timer variable is not used in any execution path, so it can be replaced with a discard, as follows:

_ = new Timer(
timeCB, // The TimerCallback delegate object.
"Hello from C# 10.0", // Any info to pass into the called method
// (null for no info).
0, // Amount of time to wait before starting
//(in milliseconds).
1000); // Interval of time between calls
//(in milliseconds).

Understanding the ThreadPool
The next thread-centric topic you will examine in this chapter is the role of the runtime thread pool. There is a cost with starting a new thread, so for purposes of efficiency, the thread pool holds onto created (but inactive) threads until needed. To allow you to interact with this pool of waiting threads, the System.
Threading namespace provides the ThreadPool class type.
If you want to queue a method call for processing by a worker thread in the pool, you can use the ThreadPool.QueueUserWorkItem() method. This method has been overloaded to allow you to specify an optional System.Object for custom state data in addition to an instance of the WaitCallback delegate.

public static class ThreadPool
{
...
public static bool QueueUserWorkItem(WaitCallback callBack); public static bool QueueUserWorkItem(WaitCallback callBack,
object state);
}

The WaitCallback delegate can point to any method that takes a System.Object as its sole parameter (which represents the optional state data) and returns nothing. Note that if you do not provide a System. Object when calling QueueUserWorkItem(), the .NET Core Runtime automatically passes a null value. To illustrate queuing methods for use by the .NET Core Runtime thread pool, ponder the following program (in a console application called ThreadPoolApp), which uses the Printer type once again (make sure to update the namespace to ThreadPoolApp). In this case, however, you are not manually creating an array of Thread objects; rather, you are assigning members of the pool to the PrintNumbers() method.

using ThreadPoolApp;
Console.WriteLine(" Fun with the .NET Core Runtime Thread Pool \n"); Console.WriteLine("Main thread started. ThreadID = {0}",
Environment.CurrentManagedThreadId); Printer p = new Printer();
WaitCallback workItem = new WaitCallback(PrintTheNumbers);

// Queue the method ten times. for (int i = 0; i < 10; i++)
{
ThreadPool.QueueUserWorkItem(workItem, p);
}
Console.WriteLine("All tasks queued"); Console.ReadLine();

static void PrintTheNumbers(object state)
{
Printer task = (Printer)state; task.PrintNumbers();
}

At this point, you might be wondering if it would be advantageous to use the .NET Core Runtime– maintained thread pool rather than explicitly creating Thread objects. Consider these benefits of leveraging the thread pool:
• The thread pool manages threads efficiently by minimizing the number of threads that must be created, started, and stopped.
• By using the thread pool, you can focus on your business problem rather than the application’s threading infrastructure.
However, using manual thread management is preferred in some cases. Here is an example:
• If you require foreground threads or must set the thread priority. Pooled threads are
always background threads with default priority (ThreadPriority.Normal).
• If you require a thread with a fixed identity to abort it, suspend it, or discover it by name.
That wraps up your investigation of the System.Threading namespace. To be sure, understanding the topics presented thus far in the chapter (especially during your examination of concurrency issues) will be extremely valuable when creating a multithreaded application. Given this foundation, you will now turn your attention to several new thread-centric topics that were introduced in .NET 4.0 and carried forward to
.NET Core. To begin, you will examine the role of an alternative threading model, the Task Parallel Library.

Parallel Programming Using the Task Parallel Library
At this point in the chapter, you have examined the System.Threading namespace objects that allow you to build multithreaded software. Beginning with the release of .NET 4.0, Microsoft introduced a new approach to multithreaded application development using a parallel programming library termed the Task Parallel Library (TPL). Using the types of System.Threading.Tasks, you can build fine-grained, scalable parallel code without having to work directly with threads or the thread pool.
This is not to say, however, that you will not use the types of System.Threading when you use the TPL. Both threading toolkits can work together quite naturally. This is especially true in that the System. Threading namespace still provides most of the synchronization primitives you examined previously (Monitor, Interlocked, etc.). However, you will quite likely find that you will favor working with the TPL
rather than the original System.Threading namespace, given that the same set of tasks can be performed in a more straightforward manner.

The System.Threading.Tasks Namespace
Collectively speaking, the types of System.Threading.Tasks are referred to as the Task Parallel Library. The TPL will automatically distribute your application’s workload across available CPUs dynamically, using the runtime thread pool. The TPL handles the partitioning of the work, thread scheduling, state management, and other low-level details. The result is that you can maximize the performance of your .NET Core applications while being shielded from many of the complexities of directly working with threads.

The Role of the Parallel Class
A key class of the TPL is System.Threading.Tasks.Parallel. This class contains methods that allow you to iterate over a collection of data (specifically, an object implementing IEnumerable) in a parallel fashion, mainly through two primary static methods, Parallel.For() and Parallel.ForEach(), each of which defines numerous overloaded versions.

These methods allow you to author a body of code statements that will be processed in a parallel manner. In concept, these statements are the same sort of logic you would write in a normal looping construct (via the for or foreach C# keyword). The benefit is that the Parallel class will pluck threads from the thread pool (and manage concurrency) on your behalf.
Both methods require you to specify an IEnumerable- or IEnumerable-compatible container that holds the data you need to process in a parallel manner. The container could be a simple array, a nongeneric collection (such as ArrayList), a generic collection (such as List), or the results of a LINQ query.
In addition, you will need to use the System.Func and System.Action delegates to specify the target method that will be called to process the data. You have already encountered the Func delegate in Chapter 13, during your investigation of LINQ to Objects. Recall that Func represents a method that
can have a given return value and a varied number of arguments. The Action delegate is like Func, in that it allows you to point to a method taking some number of parameters. However, Action specifies a method that can only return void.
While you could call the Parallel.For() and Parallel.ForEach() methods and pass a strongly typed Func or Action delegate object, you can simplify your programming by using a fitting C# anonymous method or lambda expression.

Data Parallelism with the Parallel Class
The first way to use the TPL is to perform data parallelism. Simply put, this term refers to the task of iterating over an array or collection in a parallel manner using the Parallel.For() or Parallel.ForEach() method. Assume you need to perform some labor-intensive file I/O operations. Specifically, you need to load a large number of *.jpg files into memory, flip them upside down, and save the modified image data to a new location.
In this example, you will see how to perform the same overall task using a graphical user interface, so you can examine the use of “anonymous delegates” to allow secondary threads to update the primary user interface thread (aka the UI thread).

■ Note When you are building a multithreaded graphical user interface (gui) application, secondary threads can never directly access user interface controls. the reason is that controls (buttons, text boxes, labels, progress bars, etc.) have thread affinity with the thread that created them. in the following example, i will illustrate one way to allow secondary threads to access ui items in a thread-safe manner. you will see a more simplified approach when you examine the C# async and await keywords.

To illustrate, create a new Windows Presentation Foundation application (the template is abbreviated to WPF App (.NET Core)) named DataParallelismWithForEach. To create the project using the CLI and add it to the chapter’s solution, enter the following command:

dotnet new wpf -lang c# -n DataParallelismWithForEach -o .\DataParallelismWithForEach -f net6.0
dotnet sln .\Chapter15_AllProjects.sln add .\DataParallelismWithForEach

■ Note Windows presentation Foundation (WpF) is Windows only in this version of .net Core and will be covered in detail in Chapters 24–28. if you have not worked with WpF or you don’t have access to a Windows machine, everything you need for this example is listed here. if you would rather just follow along with a completed solution, you can find dataparallelismWithForeach in the Chapter 15 folder of the github repo. WpF development works with Visual studio Code, although there is no designer support. For a richer development experience, i suggest you use Visual studio 2022 for the WpF examples in this chapter.

At the time of this writing, the WPF project templates do not add support for global implicit using statements. Update the main PropertyGroup in the DataParallelismWithForEach.csproj file to the following, which also disables nullable reference types (updates in bold):


WinExe
net6.0-windows
enable
disable
true

Double-click the MainWindow.xaml file in Solution Explorer, and replace the XAML with the following:

<Window x:Class="DataParallelismWithForEach.MainWindow" xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation" xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:local="clr-namespace:DataParallelismWithForEach" mc:Ignorable="d"
Title="Fun with TPL" Height="400" Width="800">















Again, do not worry about what the markup means or how it works; you will spend plenty of time with WPF later in this book. The GUI of the application consists of a multiline TextBox and a single Button (named cmdProcess). The purpose of the text area is to allow you to enter data while the work is being performed in the background, thus illustrating the nonblocking nature of the parallel task.
For this example, an additional NuGet package (System.Drawing.Common) is required. To add it to your project, enter the following line (all on one line) in the command line (in the same directory as your solution file) or Package Manager Console in Visual Studio:

dotnet add DataParallelismWithForEach package System.Drawing.Common

Open the MainWindow.xaml.cs file (double-click it in Visual Studio—you might have to expand the arrow by MainWindow.xaml), and add the following using statements to the top of the file:

using System.Drawing; using System.Windows; using System.IO;

■ Note you should update the string passed into the following Directory.GetFiles() method call to point to a path on your computer that has some image files (such as a personal folder of family pictures). i have included some sample images (that ship with the Windows operating system) in the chapter’s sample code for your convenience.

public partial class MainWindow : Window
{
public MainWindow()
{
InitializeComponent();
}

private void cmdCancel_Click(object sender, EventArgs e)
{
// This will be updated shortly
}

private void cmdProcess_Click(object sender, EventArgs e)
{
this.Title = $"Starting..."; ProcessFiles();
this.Title = "Processing Complete";
}

private void ProcessFiles()
{
// Load up all .jpg files, and make a new folder for the
// modified data.
//Get the directory path where the file is executing
//For VS 2022 debugging, the current directory will be \bin\debug\ net6.0-windows
//For VS Code or “dotnet run”, the current directory will be var basePath = Directory.GetCurrentDirectory();
var pictureDirectory = Path.Combine(basePath, "TestPictures");
var outputDirectory =
Path.Combine(basePath, "ModifiedPictures");
//Clear out any existing files
if (Directory.Exists(outputDirectory))
{
Directory.Delete(outputDirectory, true);
}
Directory.CreateDirectory(outputDirectory);
string[] files = Directory.GetFiles(pictureDirectory, "
.jpg", SearchOption.AllDirectories);

// Process the image data in a blocking manner. foreach (string currentFile in files)
{
string filename = Path.GetFileName(currentFile);
// Print out the ID of the thread processing the current image.
this.Title = $"Processing {filename} on thread {Environment.CurrentManagedThreadId}"; using (Bitmap bitmap = new Bitmap(currentFile))
{
bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone); bitmap.Save(Path.Combine(outputDirectory, filename));
}
}
}
}

■ Note if you receive an error that Path is an ambiguous reference between System.IO.Path and
System.Windows.Shapes.Path, either remove the using for System.Windows.Shapes or add System.IO to Path: System.IO.Path.Combine(...).

Notice that the ProcessFiles() method will rotate each *.jpg file under the specified directory.
Currently, all the work is happening on the primary thread of the executable. Therefore, when the button to process the files is clicked, the program will appear to hang. Furthermore, the caption of the window will also report that the same primary thread is processing the file, as we have only a single thread of execution.
To process the files on as many CPUs as possible, you can rewrite the current foreach loop to use Parallel.ForEach(). Recall that this method has been overloaded numerous times; however, in the simplest form, you must specify the IEnumerable-compatible object that contains the items to process (that would be the files string array) and an Action delegate that points to the method that will perform the work.

Here is the relevant update, using the C# lambda operator in place of a literal Action delegate object. Notice that you are currently commenting out the line of code that displayed the ID of the thread executing the current image file. See the next section to find out the reason why.

// Process the image data in a parallel manner!
Parallel.ForEach(files, currentFile =>
{
string filename = Path.GetFileName(currentFile);

// This code statement is now a problem! See next section.
// this.Title = $"Processing {filename} on thread
// {Environment.CurrentManagedThreadId}";

using (Bitmap bitmap = new Bitmap(currentFile))
{
bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone); bitmap.Save(Path.Combine(outputDirectory, filename));
}
}
);

Accessing UI Elements on Secondary Threads
You will notice that I have commented out the previous line of code that updated the caption of the main window with the ID of the currently executing thread. As noted previously, GUI controls have “thread affinity” with the thread that created them. If secondary threads attempt to access a control they did not directly create, it will cause a runtime exception. However, this isn’t guaranteed to happen, and exceptions might not be raised. Threading issues depend on a lot of different variables and are typically intermittent and very hard to reproduce.

■ Note let me reiterate the previous point: when you debug a multithreaded application, you can sometimes catch errors that arise when a secondary thread is “touching” a control created on the primary thread. however, oftentimes when you run the application, the application could appear to run correctly (or it might error straightaway). until you take precautions (examined next), your application has the potential of raising a runtime error under such circumstances.

One approach that you can use to allow these secondary threads to access the controls in a thread- safe manner is yet another delegate-centric technique, specifically an anonymous delegate. The Control parent class in WPF defines a Dispatcher object, which manages the work items for a thread. This object has a method named Invoke(), which takes a System.Delegate as input. You can call this method when you are in a coding context involving secondary threads to provide a thread-safe manner to update the UI of the given control. Now, while you could write all the required delegate code directly, most developers use expression syntax as a simple alternative. Here is the relevant update to the content with the previously commented-out code statement:

// Eek! This will not work anymore!
//this.Title = $"Processing {filename} on thread {Environment.CurrentManagedThreadId}";

// Invoke on the Form object, to allow secondary threads to access controls
// in a thread-safe manner.
Dispatcher?.Invoke(() =>
{
this.Title = $"Processing {filename}";
});
using (Bitmap bitmap = new Bitmap(currentFile))
{
bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone); bitmap.Save(Path.Combine(outputDirectory, filename));
}

Now, if you run the program, the TPL will indeed distribute the workload to multiple threads from the thread pool, using as many CPUs as possible. However, since the Title is always updated from the main thread, the Title update code no longer displays the current thread, and you will not see anything if you type in the text box until all the images have been processed! The reason is that the primary UI thread is still blocked, waiting for all the other threads to finish up their business.

The Task Class
The Task class allows you to easily invoke a method on a secondary thread and can be used as a simple alternative to using asynchronous delegates. Update the Click handler of your Button control as so:

private void cmdProcess_Click(object sender, EventArgs e)
{
this.Title = $"Starting...";
// Start a new "task" to process the files. Task.Factory.StartNew(() => ProcessFiles());
//Can also be written this way
//Task.Factory.StartNew(ProcessFiles);

}

The Factory property of Task returns a TaskFactory object. When you call its StartNew() method, you pass in an Action delegate (here, hidden away with a fitting lambda expression) that points to the method to invoke in an asynchronous manner. With this small update, you will now find that the window’s
title will show which thread from the thread pool is processing a given file, and better yet, the text area is able to receive input, as the UI thread is no longer blocked.

Handling Cancellation Request
One improvement you can make to the current example is to provide a way for the user to stop the processing of the image data, via a second (aptly named) Cancel button. Thankfully, the Parallel. For() and Parallel.ForEach() methods both support cancellation with cancellation tokens. When you invoke methods on Parallel, you can pass in a ParallelOptions object, which in turn contains a CancellationTokenSource object.

First, define the following new private member variable in your Form-derived class of type
CancellationTokenSource named _cancelToken:

public partial class MainWindow :Window
{
// New Window-level variable.
private CancellationTokenSource _cancelToken = new CancellationTokenSource();
...
}

Update the Cancel button Click event to the following code:

private void cmdCancel_Click(object sender, EventArgs e)
{
// This will be used to tell all the worker threads to stop!
_cancelToken.Cancel();
}

Now, the real modifications need to occur within the ProcessFiles() method. Consider the final implementation:

private void ProcessFiles()
{
// Use ParallelOptions instance to store the CancellationToken. ParallelOptions parOpts = new ParallelOptions(); parOpts.CancellationToken = _cancelToken.Token; parOpts.MaxDegreeOfParallelism = System.Environment.ProcessorCount;

// Load up all .jpg files, and make a new folder for the modified data. string[] files = Directory.GetFiles(@".\TestPictures", ".jpg", SearchOption. AllDirectories);
string outputDirectory = @".\ModifiedPictures"; Directory.CreateDirectory(outputDirectory);

try
{
// Process the image data in a parallel manner! Parallel.ForEach(files, parOpts, currentFile =>
{
parOpts
.CancellationToken.ThrowIfCancellationRequested();

string filename = Path.GetFileName(currentFile); Dispatcher?.Invoke(() =>
{
this.Title =
$"Processing {filename} on thread {Environment.CurrentManagedThreadId}";
});
using (Bitmap bitmap = new Bitmap(currentFile))
{
bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone);

bitmap.Save(Path.Combine(outputDirectory, filename));
}
});
Dispatcher?.Invoke(()=>this.Title = "Done!");
}
catch (OperationCanceledException ex)
{
Dispatcher?.Invoke(()=>this.Title = ex.Message);
}
}

Notice that you begin the method by configuring a ParallelOptions object, setting the CancellationToken property to use the CancellationTokenSource token. Also note that when you call the Parallel.ForEach() method, you pass in the ParallelOptions object as the second parameter.
Within the scope of the looping logic, you make a call to ThrowIfCancellationRequested() on the token, which will ensure if the user clicks the Cancel button, all threads will stop, and you will be notified via a runtime exception. When you catch the OperationCanceledException error, you will set the text of the main window to the error message.

■ Note new in C# 10, there is a new async method on the Parallel class named ForEachAsync(). this is covered later in this chapter.

Task Parallelism Using the Parallel Class
In addition to data parallelism, the TPL can also be used to easily fire off any number of asynchronous tasks using the Parallel.Invoke() method. This approach is a bit more straightforward than using members from System.Threading; however, if you require more control over the way tasks are executed, you could forgo use of Parallel.Invoke() and use the Task class directly, as you did in the previous example.
To illustrate task parallelism, create a new console application called MyEBookReader and be sure the System.Text and System.Net namespaces are imported at the top of Program.cs (this example is a
modification of a useful example in the .NET Core documentation). Here, you will fetch a publicly available ebook from Project Gutenberg (www.gutenberg.org) and then perform a set of lengthy tasks in parallel.
The book is downloaded in the GetBook() method, shown here:

using System.Net; using System.Text;

string _theEBook = ""; GetBook();
Console.WriteLine("Downloading book..."); Console.ReadLine();

void GetBook()
{
//NOTE: The WebClient is obsolete.
//We will revisit this example using HttpClient when we discuss async/await using WebClient wc = new WebClient();
wc.DownloadStringCompleted += (s, eArgs) =>

{
_theEBook = eArgs.Result; Console.WriteLine("Download complete."); GetStats();
};

// The Project Gutenberg EBook of A Tale of Two Cities, by Charles Dickens
// You might have to run it twice if you’ve never visited the site before, since the first
// time you visit there is a message box that pops up, and breaks this code.
wc.DownloadStringAsync(new Uri("http://www.gutenberg.org/files/98/98-0.txt"));
}

The WebClient class is a member of System.Net. This class provides methods for sending data to and receiving data from a resource identified by a URI. As it turns out, many of these methods have an asynchronous version, such as DownloadStringAsync(). This method will spin up a new thread from the
.NET Core Runtime thread pool automatically. When the WebClient is done obtaining the data, it will fire the
DownloadStringCompleted event, which you are handling here using a C# lambda expression. If you were to call the synchronous version of this method (DownloadString()), the “Downloading” message would not show until the download was complete.

■ Note the WebClient is obsolete and has been replaced by the HttpClient. We will revisit this example using the HttpClient class in the “async Calls with async/await” section of this chapter.

Next, the GetStats() method is implemented to extract the individual words contained in the theEBook
variable and then pass the string array to a few helper functions for processing as follows:

void GetStats()
{
// Get the words from the ebook.
string[] words = _theEBook.Split(new char[]
{ ' ', '\u000A', ',', '.', ';', ':', '-', '?', '/' },
StringSplitOptions.RemoveEmptyEntries);

// Now, find the ten most common words.
string[] tenMostCommon = FindTenMostCommon(words);

// Get the longest word.
string longestWord = FindLongestWord(words);

// Now that all tasks are complete, build a string to show all stats. StringBuilder bookStats = new StringBuilder("Ten Most Common Words are:\n"); foreach (string s in tenMostCommon)
{
bookStats.AppendLine(s);
}

bookStats.AppendFormat("Longest word is: {0}", longestWord); bookStats.AppendLine(); Console.WriteLine(bookStats.ToString(), "Book info");
}

The FindTenMostCommon() method uses a LINQ query to obtain a list of string objects that occur most often in the string array, while FindLongestWord() locates, well, the longest word.

string[] FindTenMostCommon(string[] words)
{
var frequencyOrder = from word in words
where word.Length > 6 group word by word into g
orderby g.Count() descending select g.Key;
string[] commonWords = (frequencyOrder.Take(10)).ToArray(); return commonWords;
}
string FindLongestWord(string[] words)
{
return (from w in words orderby w.Length descending select w).FirstOrDefault();
}

If you were to run this project, performing all the tasks could take a significant amount of time, based on the CPU count of your machine and overall processor speed. Eventually, you should see the output shown here:

Downloading book...
Download complete.
Ten Most Common Words are:
Defarge himself Manette through nothing business another looking prisoner Cruncher
Longest word is: undistinguishable

You can help ensure that your application uses all available CPUs on the host machine by invoking the FindTenMostCommon() and FindLongestWord() methods in parallel. To do so, modify your GetStats() method as so:

void GetStats()
{
// Get the words from the ebook. string[] words = _theEBook.Split(
new char[] { ' ', '\u000A', ',', '.', ';', ':', '-', '?', '/' },
StringSplitOptions.RemoveEmptyEntries); string[] tenMostCommon = null;
string longestWord = string.Empty;

Parallel.Invoke( () =>
{
// Now, find the ten most common words. tenMostCommon = FindTenMostCommon(words);
},
() =>
{
// Get the longest word.
longestWord = FindLongestWord(words);
});

// Now that all tasks are complete, build a string to show all stats.
...
}

The Parallel.Invoke() method expects a parameter array of Action<> delegates, which you have supplied indirectly using lambda expressions. Again, while the output is identical, the benefit is that the TPL will now use all possible processors on the machine to invoke each method in parallel if possible.

Parallel LINQ Queries (PLINQ)
To wrap up your look at the TPL, be aware that there is another way you can incorporate parallel tasks into your .NET Core applications. If you choose, you can use a set of extension methods that allow you to construct a LINQ query that will perform its workload in parallel (if possible). Fittingly, LINQ queries that are designed to run in parallel are termed PLINQ queries.
Like parallel code authored using the Parallel class, PLINQ has the option of ignoring your request to process the collection in parallel if need be. The PLINQ framework has been optimized in numerous ways, which includes determining whether a query would, in fact, perform faster in a synchronous manner.
At runtime, PLINQ analyzes the overall structure of the query, and if the query is likely to benefit from parallelization, it will run concurrently. However, if parallelizing a query would hurt performance, PLINQ just runs the query sequentially. If PLINQ has a choice between a potentially expensive parallel algorithm or an inexpensive sequential algorithm, it chooses the sequential algorithm by default.
The necessary extension methods are found within the ParallelEnumerable class of the System.Linq
namespace. Table 15-5 documents some useful PLINQ extensions.

Table 15-5. Select Members of the ParallelEnumerable Class

Member Meaning in Life
AsParallel() Specifies that the rest of the query should be parallelized, if possible
WithCancellation() Specifies that PLINQ should periodically monitor the state of the provided cancellation token and cancel execution if it is requested
WithDegreeOfParallelism() Specifies the maximum number of processors that PLINQ should use to parallelize the query
ForAll() Enables results to be processed in parallel without first merging back to the consumer thread, as would be the case when enumerating a LINQ result using the foreach keyword

To see PLINQ in action, create a console application named PLINQDataProcessingWithCancellation. When processing starts, the program will fire off a new Task, which executes a LINQ query that investigates a large array of integers, looking for only the items where x % 3 == 0 is true. Here is a nonparallel version of the query:

Console.WriteLine("Start any key to start processing"); Console.ReadKey();

Console.WriteLine("Processing"); Task.Factory.StartNew(ProcessIntData); Console.ReadLine();

void ProcessIntData()
{
// Get a very large array of integers.
int[] source = Enumerable.Range(1, 10_000_000).ToArray();
// Find the numbers where num % 3 == 0 is true, returned
// in descending order. int[] modThreeIsZero = (
from num in source where num % 3 == 0 orderby num descending select num).ToArray();
Console.WriteLine($"Found { modThreeIsZero.Count()} numbers that match query!");
}

Opting in to a PLINQ Query
If you want to inform the TPL to execute this query in parallel (if possible), you will want to use the
AsParallel() extension method as so:

int[] modThreeIsZero = (
from num in source.AsParallel() where num % 3 == 0
orderby num descending select num).ToArray();

Notice how the overall format of the LINQ query is identical to what you saw in previous chapters.
However, by including a call to AsParallel(), the TPL will attempt to pass the workload off to any available CPU.

Cancelling a PLINQ Query
It is also possible to use a CancellationTokenSource object to inform a PLINQ query to stop processing under the correct conditions (typically because of user intervention). Declare a class-level
CancellationTokenSource object named _cancelToken and update the top-level statements method to take input from the user. Here is the relevant code update:

CancellationTokenSource _cancelToken = new CancellationTokenSource();

do
{
Console.WriteLine("Start any key to start processing"); Console.ReadKey();
Console.WriteLine("Processing"); Task.Factory.StartNew(ProcessIntData); Console.Write("Enter Q to quit: "); string answer = Console.ReadLine();
// Does user want to quit? if (answer.Equals("Q",
StringComparison.OrdinalIgnoreCase))
{
_cancelToken.Cancel(); break;
}
}
while (true); Console.ReadLine();
Now, inform the PLINQ query that it should be on the lookout for an incoming cancellation request by chaining on the WithCancellation() extension method and passing in the token. In addition, you will want to wrap this PLINQ query in a proper try/catch scope and deal with the possible exception. Here is the final version of the ProcessIntData() method:

void ProcessIntData()
{
// Get a very large array of integers.
int[] source = Enumerable.Range(1, 10_000_000).ToArray();
// Find the numbers where num % 3 == 0 is true, returned
// in descending order. int[] modThreeIsZero = null; try
{
modThreeIsZero = (from num in source.AsParallel().WithCancellation(_cancelToken.Token) where num % 3 == 0
orderby num descending select num).ToArray();
Console.WriteLine();
Console.WriteLine($"Found {modThreeIsZero.Count()} numbers that match query!");
}
catch (OperationCanceledException ex)
{
Console.WriteLine(ex.Message);
}
}

When running this, you will want to hit Q and enter quickly to see the message from the cancellation token. On my development machine, I had less than one second to quit before it finished on its own. You can add a call to Thread.Sleep() to make it easier to cancel the process:

try
{
Thread.Sleep(3000);
modThreeIsZero = (from num in source.AsParallel().WithCancellation(_cancelToken.Token) where num % 3 == 0
orderby num descending select num).ToArray();
Console.WriteLine();
Console.WriteLine($"Found {modThreeIsZero.Count()} numbers that match query!");
}

Async Calls Using the async/await Pattern
I have covered a lot of material in this (rather lengthy) chapter. To be sure, building, debugging, and understanding complex multithreaded applications are challenging in any framework. While the TPL, PLINQ, and the delegate type can simplify matters to some extent (especially when compared to other platforms and languages), developers are still required to know the ins and outs of various advanced techniques.
Since the release of .NET 4.5, the C# programming language has been updated with two new keywords that further simplify the process of authoring asynchronous code. In contrast to all the examples in this chapter, when you use the new async and await keywords, the compiler will generate a good deal of threading code on your behalf, using numerous members of the System.Threading and System.Threading. Tasks namespaces.

A First Look at the C# async and await Keywords (Updated 7.1, 9.0)
The async keyword of C# is used to qualify that a method, lambda expression, or anonymous method should be called in an asynchronous manner automatically. Yes, it is true. Simply by marking a method with the async modifier, the .NET Core Runtime will create a new thread of execution to handle the task at hand. Furthermore, when you are calling an async method, the await keyword will automatically pause the
current thread from any further activity until the task is complete, leaving the calling thread free to continue.
To illustrate, create a console application named FunWithCSharpAsync and add a method named
DoWork(), which forces the calling thread to wait for five seconds. Here is the story thus far:

Console.WriteLine(" Fun With Async ===>"); Console.WriteLine(DoWork()); Console.WriteLine("Completed"); Console.ReadLine();

static string DoWork()
{
Thread.Sleep(5_000); return "Done with work!";
}

Now, given your work in this chapter, you know that if you were to run the program, you would need to wait five seconds before anything else can happen. If this were a graphical application, the entire screen would be locked until the work was completed.
If you were to use any of the previous techniques shown in this chapter to make your program more responsive, you would have a good deal of work ahead of you. However, since .NET 4.5, you can author the following C# code base:

...
string message = await DoWorkAsync(); Console.WriteLine(message);
...

static async Task DoWorkAsync()
{
return await Task.Run(() =>
{
Thread.Sleep(5_000); return "Done with work!";
});
}

If you are using a Main() method as your entry point (instead of top-level statements), you need to mark the method as async, introduced in C# 7.1.

static async Task Main(string[] args)
{
...
string message = await DoWorkAsync(); Console.WriteLine(message);
...
}

■ Note the ability to decorate the Main() method with async is new as of C# 7.1. top-level statements, added in C# 9.0, are implicitly async.

Notice the await keyword before naming the method that will be called in an asynchronous manner. This is important: if you decorate a method with the async keyword but do not have at least one internal await-centric method call, you have essentially built a synchronous method call (in fact, you will be given a compiler warning to this effect).
Now, notice that you are required to use the Task class from the System.Threading.Tasks
namespace to refactor your Main() (if you are using Main()) and DoWork() methods (the latter is added as DoWorkAsync()). Basically, rather than returning a specific return value straightaway (a string object in the current example), you return a Task object, where the generic type parameter T is the underlying, actual return value. If the method does not have a return value (as in the Main() method), then instead of Task you just use Task.
The implementation of DoWorkAsync() now directly returns a Task object, which is the return value of Task.Run(). The Run() method takes a Func<> or Action<> delegate, and as you know by this point in the text, you can simplify your life by using a lambda expression. Basically, your new version of DoWorkAsync() is essentially saying the following:

When you call me, I will run a new task. This task will cause the calling thread to sleep for five seconds, and when it is done, it gives me a string return value. I will put this string in a new Task object and return it to the caller.
Having translated this new implementation of DoWorkAsync() into more natural (poetic) language, you gain some insight into the real role of the await token. This keyword will always modify a method that
returns a Task object. When the flow of logic reaches the await token, the calling thread is suspended in this method until the call completes. If this were a graphical application, the user could continue to use the UI while the DoWorkAsync() method executes.

SynchronizationContext and async/await
The official definition of the SynchronizationContext is a base class that provides free-threaded context with no synchronization. While that initial definition is not very descriptive, the official documentation goes on to say:

The purpose of the synchronization model implemented by this class is to allow the internal asynchronous/synchronous operations of the common language runtime to behave properly with different synchronization models.

That statement, along with what you know about multithreading, sheds light on the matter. Recall that GUI applications (WinForms, WPF) do not allow secondary threads to access controls directly but must delegate that access. We have already seen the Dispatcher object in the WPF example. For the console applications that did not use WPF, this restriction is not in place. These are the different synchronization models that are referred to. With that in mind, let’s dig deeper into the SynchronizationContext.
The SynchonizationContext is a type that provides a virtual post method, which takes a delegate to
be executed asynchronously. This provides a pattern for frameworks to appropriately handle asynchronous requests (dispatching for WPF/WinForms, directly executing for non-GUI applications, etc.). It provides a way to queue a unit of work to a context and keeps count of outstanding async operations.
As we discussed earlier, when a delegate is queued to run asynchronously, it is scheduled to run on a separate thread. This detail is handled by the .NET Core Runtime. This is typically done using the .NET Core Runtime managed thread pool but can be overridden with a custom implementation.
While this plumbing work can be manually managed through code, the async/await pattern does most of the heavy lifting. When an async method is awaited, it leverages the SynchronizationContext and TaskScheduler implementations of the target framework. For example, if you are using async/await in a WPF application, the WPF framework manages the dispatching of the delegate and calling back into the state machine when the awaited task completes in order to safely update controls.

The Role of ConfigureAwait
Now that you understand the SynchronizationContext a little better, it is time to cover the role of the ConfigureAwait() method. By default, awaiting a Task will result in a synchronization context being utilized. When developing GUI applications (WinForms, WPF), this is the behavior that you want. However, if you are writing non-GUI application code, the overhead of queuing the original context when not needed can potentially cause performance issues in your application.
To see this in action, update your top-level statements to the following:

Console.WriteLine(" Fun With Async ===>");
//Console.WriteLine(DoWork());
string message = await DoWorkAsync();

Console.WriteLine($"0 - {message}");

string message1 = await DoWorkAsync().ConfigureAwait(false); Console.WriteLine($"1 - {message1}");

The original code block is using the SynchronizationContext supplied by the framework (in this case, the .NET Core Runtime). It is equivalent to calling ConfigureAwait(true). The second example ignores the current context and scheduler.
Guidance from the .NET Core team suggests that when developing application code (WinForms, WPF, etc.), leave the default behavior in place. If you are writing nonapplication code (e.g., library code), then use ConfigureAwait(false). The one exception for this is ASP.NET Core (covered starting with Chapter 29).
ASP.NET Core does not create a custom SynchronizationContext; therefore, ConfigureAwait(false) does
not provide the benefit when using other frameworks.

Naming Conventions for Asynchronous Methods
Of course, you noticed the name change from DoWork() to DoWorkAsync(), but why the change? Let’s say that the new version of the method was still named DoWork(); however, the calling code has been implemented as so:

//Oops! No await keyword here! string message = DoWork();

Notice you did indeed mark the method with the async keyword, but you neglected to use the await keyword as a decorator before calling the DoWork() method call. At this point, you will have compiler errors, as the return value of DoWork() is a Task object, which you are attempting to assign directly to a string variable. Remember, the await token extracts the internal return value contained in the Task object. Since you have not used this token, you have a type mismatch.

■ Note an “awaitable” method is simply a method that returns a Task or Task.

Given that methods that return Task objects can now be called in a nonblocking manner via the async and await tokens, the recommendation is to name any method returning a Task with an Async suffix. In this way, developers who know the naming convention receive a visual reminder that the await keyword is required, if they intend to invoke the method within an asynchronous context.

■ Note event handlers for gui controls (such as a button Click handler) as well as action methods in MVC- style apps that use the async/await keywords do not follow this naming convention (by convention—pardon the redundancy!).

Async Methods That Don’t Return Data
Currently, your DoWorkAsync() method is returning a Task, which contains “real data” for the caller that will be obtained transparently via the await keyword. However, what if you want to build an asynchronous method that doesn’t return anything? While there are two ways to do this, there is really only one right way to do it. First, let’s look at the problems with defining an async void method.

Async Void Methods
The following is an example of an async method that uses void as the return type instead of Task:

static async void MethodReturningVoidAsync()
{
await Task.Run(() =>
{
/ Do some work here... / Thread.Sleep(4_000);
});
Console.WriteLine("Fire and forget void method completed");
}

If you were to call this method, it would run on its own without blocking the main thread. The following code will show the “Completed” message before the message from the MethodReturningVoidAsync() method:

MethodReturningVoidAsync(); Console.WriteLine("Completed"); Console.ReadLine();

While this might seem like a viable option for “fire and forget” scenarios, there is a larger problem. If the method throws an exception, it has nowhere to go except to the synchronization context of the calling method. Update the method to the following:

static async void MethodReturningVoidAsync()
{
await Task.Run(() =>
{
/ Do some work here... / Thread.Sleep(4_000);
throw new Exception("Something bad happened");
});
Console.WriteLine("Fire and forget void method completed");
}

To be safe, wrap the call to this method in a try-catch block, and run the program again:

try
{
MethodReturningVoidAsync();
}
catch (Exception e)
{
Console.WriteLine(e);
}

Not only does the catch block not catch the exception, but the exception is placed on the threading execution context. So, while this might seem like a good idea for “fire and forget” scenarios, you better hope there isn’t an exception thrown in the async void method or your whole application might come crashing down.

Async Void Methods Using Task
The better way is to have your method use Task instead of void. Update the method to the following:

static async Task MethodReturningVoidTaskAsync()
{
await Task.Run(() =>
{
/ Do some work here... / Thread.Sleep(4_000);
});
Console.WriteLine("Void method completed");
}

If you call with method without the await keyword, the same result will happen as in the previous example:

MethodReturningVoidTaskAsync(); Console.WriteLine("Void method complete");

Update the MethodReturningVoidTaskAsync() to throw an exception:

static async Task MethodReturningVoidTaskAsync()
{
await Task.Run(() =>
{

});

/ Do some work here... / Thread.Sleep(4_000);
throw new Exception("Something bad happened");

Console.WriteLine("Void method completed");
}

Now wrap the call to this method in a try-catch block:

try
{

}

MethodReturningVoidTaskAsync();

catch (Exception ex)
{
Console.WriteLine(ex);
}

When you run the program and the exception is thrown, two interesting things happen. The first is that the whole program doesn’t crash, and the second is that the catch block doesn’t catch the exception. When an exception is thrown out of a Task/Task method, the exception is captured and placed on the Task object. When using await, the Exception (or AggregateException) is available to handle.

Update the calling code to await the method, and the catch block will now work as expected:

try
{

}

await MethodReturningVoidTaskAsync();

catch (Exception ex)
{
Console.WriteLine(ex);
}

To sum it up, you should avoid creating async void methods and use async Task methods. Whether or not you await them becomes a business decision, but either way, at least you won’t crash your app!

Async Methods with Multiple Awaits
It is completely permissible for a single async method to have multiple await contexts within its implementation. The following example shows this in action:

static async Task MultipleAwaits()
{
await Task.Run(() => { Thread.Sleep(2_000); }); Console.WriteLine("Done with first task!");

await Task.Run(() => { Thread.Sleep(2_000); }); Console.WriteLine("Done with second task!");

await Task.Run(() => { Thread.Sleep(2_000); }); Console.WriteLine("Done with third task!");
}

Again, here each task is not doing much more than suspending the current thread for a spell; however, any unit of work could be represented by these tasks (calling a web service, reading a database, etc.).
Another option is to not await each task but await them all together and return when all of the tasks are completed. This is a more likely scenario, where there are three things (check email, update server, download files) that must be completed in a batch, but can be done in parallel. Here is the code updated using the Task.WhenAll() method:

static async Task MultipleAwaitsAsync()
{
await Task.WhenAll(Task.Run(() =>
{
Thread.Sleep(2_000); Console.WriteLine("Done with first task!");
}), Task.Run(() =>
{
Thread.Sleep(1_000);
Console.WriteLine("Done with second task!");
}), Task.Run(() =>

{
Thread.Sleep(1_000); Console.WriteLine("Done with third task!");
}));
}

When you run the program now, you see that the three tasks fire in order of the lowest Sleep time.

Fun With Async ===> Done with second task! Done with third task! Done with first task! Completed

There is also a WhenAny(), which signals that one of the tasks have completed. The method returns the first task that completed. To demonstrate WhenAny(), change the last line of the MultipleAwaitsAsync to this:

await Task.WhenAny(Task.Run(() =>
{
Thread.Sleep(2_000); Console.WriteLine("Done with first task!");
}), Task.Run(() =>
{
Thread.Sleep(1_000);
Console.WriteLine("Done with second task!");
}), Task.Run(() =>
{
Thread.Sleep(1_000); Console.WriteLine("Done with third task!");
}));

When you do this, the output changes to this:

Fun With Async ===> Done with second task! Completed
Done with third task!
Done with first task!

Each of these methods also works with an array of tasks. To demonstrate this, create a new method named MultipleAwaitsTake2Async(). In this method, create a List, add the three tasks to it, and then call Task.WhenAll() or Task.WhenAny():

static async Task MultipleAwaitsTake2()
{
var tasks = new List(); tasks.Add(Task.Run(() =>
{

Thread.Sleep(2_000); Console.WriteLine("Done with first task!");
}));
tasks.Add(Task.Run(() =>
{
Thread.Sleep(1_000);
Console.WriteLine("Done with second task!");
}));
tasks.Add(Task.Run(() =>
{
Thread.Sleep(1_000); Console.WriteLine("Done with third task!");
}));
//await Task.WhenAny(tasks); await Task.WhenAll(tasks);
}

Calling Async Methods from Synchronous Methods
Each of the previous examples used the async keyword to return the thread to calling code while the async method executes. In review, you can use the await keyword only in a method marked async. What if you cannot (or do not want to) mark a method async?
Fortunately, there are ways to call async methods when in a synchronous context. Unfortunately, most of them are bad. The first option is to simply forego the await keyword, allowing the original thread to continue execution while the async method runs on a separate thread, never returning to the caller. This behaves like the previous example of calling async Task methods. Any values returned from the method are lost, and exceptions are swallowed.
This might fit your needs, but if it doesn’t, you have three choices. The first is to simply use the Result
property on the Task or the Wait() method on Task methods. If the method fails, any exceptions are wrapped in an AggregateException, potentially complicating error handling. You can also call
GetAwaiter().GetResult(). This behaves in the same way as the Wait() and Result calls, with the small difference of not wrapping exceptions into an AggregateException. While the GetAwaiter().GetResult() methods work on both methods with a return value and methods without a return value, they are marked in the documentation as “not for external use,” which means they might change or go away at some point in the future.
While those two options seem harmless replacements for using await in an async method, there is a more serious issue with using them. Calling either Wait(), Result, or GetAwaiter().GetResult() blocks the calling thread, processes the async method on another thread, then returns back to the calling thread, tying up two threads to get the work performed. Worse yet, each of these can cause deadlocks, especially if the calling thread is from the UI of the application.
To help discover and correct improper async/await code (and naming conventions), add the Microsoft.VisualStudio.Threading.Analyzers package to the project. This package adds analyzers that will provide compiler warnings when improper threading code is discovered, including bad naming conventions. To see this in action, add the following code to the top level statements:

= DoWorkAsyncO.Result;
_ = DoWorkAsyncO.GetAwaiter().GetResult();

This causes the following compiler warning:

VSTHRD002 Synchronously waiting on tasks or awaiters may cause deadlocks. Use await or JoinableTaskFactory.Run instead.

Not only is there a compiler warning, but the recommended solution is also provided! To use the JoinableTaskFactory class, you need to add the Microsoft.VisualStudio.Threading NuGet package and the following using statement to the top of the Program.cs file:

using Microsoft.VisualStudio.Threading;

The JoinableTaskFactory needs a JoinableTaskContext in the constructor:

JoinableTaskFactory joinableTaskFactory = new JoinableTaskFactory(new JoinableTaskContext());

With this in place, you can use the Run() method to safely execute an async method, such as the
DoWork() method, from a synchronous context:

string message2 = joinableTaskFactory.Run(async 0 => await DoWorkAsync());

As you know, the DoWork() method returns Task, and that value is indeed returned from the
Run() method. You can also call methods that just return Task, as follows:

joinableTaskFactory.Run(async () =>
{
await MethodReturningVoidTaskAsync(); await SomeOtherAsyncMethod();
});

■ Note even though the packages have Visualstudio in the name, the packages do not depend on Visual studio. they are .net packages that can be used with or without Visual studio installed.

Await in catch and finally Blocks
C# 6 introduced the ability to place await calls in catch and finally blocks. The method itself must be async
to do this. The following code example demonstrates the capability:

static async Task MethodWithTryCatch()
{
try
{
//Do some work return "Hello";
}
catch (Exception ex)
{
await LogTheErrors(); throw;
}

finally
{
await DoMagicCleanUp();
}
}

Generalized Async Return Types (New 7.0)
Prior to C# 7, the only return options for async methods were Task, Task, and void. C# 7 enables additional return types, if they follow the async pattern. One concrete example is ValueTask. To see this in action, create code like this:

static async ValueTask ReturnAnInt()
{
await Task.Delay(1_000); return 5;
}

The same rules apply for ValueTask as for Task, since ValueTask is just a Task for value types instead of forcing allocation of an object on the heap.

Local Functions with async/await (New 7.0)
Local functions were introduced in Chapter 4 and used throughout this book. They can also be beneficial for async methods. To demonstrate the benefit, you need to first see the problem. Add a new method named MethodWithProblems() and add the following code:

static async Task MethodWithProblems(int firstParam, int secondParam)
{
Console.WriteLine("Enter"); await Task.Run(() =>
{
//Call long running method Thread.Sleep(4_000); Console.WriteLine("First Complete");
//Call another long running method that fails because
//the second parameter is out of range Console.WriteLine("Something bad happened");
});
}

The scenario is that the second long-running task fails because of invalid input data. You can (and should) add checks to the beginning of the method, but since the entire method is asynchronous, there is no guarantee when the checks will be executed. It would be better for the checks to happen right away before the calling code moves on. In the following update, the checks are done in a synchronous manner, and then the private function is executed asynchronously:

static async Task MethodWithProblemsFixed(int firstParam, int secondParam)
{
Console.WriteLine(“Enter”);

if (secondParam < 0)
{
Console.WriteLine("Bad data"); return;
}

await actualImplementation();

async Task actualImplementation()
{
await Task.Run(() =>
{
//Call long running method Thread.Sleep(4_000); Console.WriteLine("First Complete");
//Call another long running method that fails because
//the second parameter is out of range Console.WriteLine("Something bad happened");
});
}
}

Cancelling async/await Operations
Cancellation is also possible with the async/await pattern and much simpler than with Parallel.ForEach. To demonstrate, we will use the same WPF project from earlier in the chapter. You can either reuse that project or add a new WPF app (.NET Core) to the solution and add the System.Drawing.Common package to the project by executing the following CLI commands:

dotnet new wpf -lang c# -n PictureHandlerWithAsyncAwait -o .\PictureHandlerWithAsyncAwait -f net6.0 dotnet sln .\Chapter15_AllProjects.sln add .\PictureHandlerWithAsyncAwait
dotnet add PictureHandlerWithAsyncAwait package System.Drawing.Common

If you are using Visual Studio, do this by right-clicking the solution name in Solution Explorer, selecting Add ➤ Project, and naming it PictureHandlerWithAsyncAwait. Make sure to set the new project as the startup project by right-clicking the new project name and selecting Set as StartUp Project from the context menu. Add the System.Drawing.Common NuGet package.

dotnet add PictureHandlerWithAsyncAwait package System.Drawing.Common

Replace the XAML to match the previous WPF project (DataParallelismWithForEach), except change the title to Picture Handler with Async/Await. Also make sure to update the main PropertyGroup in the project file to disable nullable reference types.


WinExe
net6.0-windows
enable
disable
true

In the MainWindow.xaml.cs file, make sure that the following using statements are in place:

using System.IO; using System.Windows; using System.Drawing;

Next, add a class-level variable for the CancellationToken and add the Cancel button event handler:

private CancellationTokenSource _cancelToken = null; private void cmdCancel_Click(object sender, EventArgs e)
{
_cancelToken.Cancel();
}

The process is the same as the previous example: get the picture directory, create the output directory, and get the picture files, rotate them, and save them to the new directory. Instead of using Parallel.
ForEach(), this new version will use async methods to do the work, and the method signatures accept a
CancellationToken as a parameter. Enter the following code:

private async void cmdProcess_Click(object sender, EventArgs e)
{
_cancelToken = new CancellationTokenSource(); var basePath = Directory.GetCurrentDirectory(); var pictureDirectory =
Path.Combine(basePath, "TestPictures"); var outputDirectory =
Path.Combine(basePath, "ModifiedPictures");
//Clear out any existing files
if (Directory.Exists(outputDirectory))
{
Directory.Delete(outputDirectory, true);
}
Directory.CreateDirectory(outputDirectory); string[] files = Directory.GetFiles(
pictureDirectory, "*.jpg", SearchOption.AllDirectories); try
{
foreach(string file in files)
{
try
{
await ProcessFileAsync(file, outputDirectory,_cancelToken.Token);
}
catch (OperationCanceledException ex)
{
Console.WriteLine(ex); throw;
}
}
}
catch (OperationCanceledException ex)

{
Console.WriteLine(ex); throw;
}
catch (Exception ex)
{
Console.WriteLine(ex); throw;
}
_cancelToken = null;
this.Title = "Processing complete";
}

After the initial setup, the code loops through the files and calls ProcessFileAsync() asynchronously for each file. The call to ProcessFileAsync() is wrapped in a try/catch block, and the CancellationToken is passed into the ProcessFile() method. If Cancel() is executed on the CancellationTokenSource (such as when the user clicks the Cancel button), an OperationCanceledException is thrown.

■ Note the try/catch code can be anywhere in the calling chain (as you shall soon see). Whether to place it at the first call or in the asynchronous method itself is a matter of pure preference and application needs.

The final method to add is the ProcessFileAsync() method.

private async Task ProcessFileAsync(string currentFile, string outputDirectory, CancellationToken token)
{
string filename = Path.GetFileName(currentFile); using (Bitmap bitmap = new Bitmap(currentFile))
{
try
{
await Task.Run(() =>
{
Dispatcher?.Invoke(() =>
{
this.Title = $"Processing {filename}";
});
bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone); bitmap.Save(Path.Combine(outputDirectory, filename));
}
,token);
}
catch (OperationCanceledException ex)
{
Console.WriteLine(ex); throw;
}
}
}

This method uses another overload of the Task.Run command, taking in the CancellationToken as a parameter. The Task.Run command is wrapped in a try/catch block (just like the calling code) in case the user clicks the Cancel button.

Cancelling async/await operations with WaitAsync() (New 10.0)
New in C# 10.0, async calls can be cancelled with a cancellation token and/or after a time limit has been reached by using the WaitAsync() method. The following examples (located in the FunWithCSharpAsync project in this chapter’s code) show all three variations of this new feature:

CancellationTokenSource tokenSource = new CancellationTokenSource();
= await DoWorkAsync().WaitAsync(TimeSpan.FromSeconds(10));
= await DoWorkAsync().WaitAsync(tokenSource.Token);
_ = await DoWorkAsync().WaitAsync(TimeSpan.FromSeconds(10), tokenSource.Token);

Cancelling async/await operations in Synchronous Calls
The Wait() method can also take a cancellation token when calling async methods from a non-async method. This can be used with or without a timeout specified. When used with a timeout, the timeout must be in milliseconds:

CancellationTokenSource tokenSource = new CancellationTokenSource(); MethodReturningTaskOfVoidAsync().Wait(tokenSource.Token); MethodReturningTaskOfVoidAsync().Wait(10000,tokenSource.Token);

You can also use the JoinableTaskFactory and the new WaitAsync() method when calling from synchronous code:

JoinableTaskFactory joinableTaskFactory2 = new JoinableTaskFactory(new JoinableTaskContext());
CancellationTokenSource tokenSource2 = new CancellationTokenSource(); joinableTaskFactory2.Run(async () =>
{
await MethodReturningVoidTaskAsync().WaitAsync(tokenSource.Token); await MethodReturningVoidTaskAsync().WaitAsync(TimeSpan.
FromSeconds(10),tokenSource.Token);
});

Asynchronous Streams (New 8.0)
New in C# 8.0, streams (covered in greater detail in Chapter 19) can be created and consumed asynchronously. A method that returns an asynchronous stream
•Is declared with the async modifier
•Returns an IAsyncEnumerable
•Contains yield return statements (covered in Chapter 8) to return successive elements in the asynchronous stream

Take the following example:

public static async IAsyncEnumerable GenerateSequence()
{
for (int i = 0; i < 20; i++)
{
await Task.Delay(100); yield return i;
}
}

The method is declared as async, returns an IAsyncEnumerable, and uses the yield return to return integers in from a sequence. To call this method, add the following to your calling code:

await foreach (var number in GenerateSequence())
{
Console.WriteLine(number);
}

The Parallel.ForEachAsync() Method (New 10.0)
New in C# 10, the Parallel class has a new ForEachAsync() method, which is (as the name suggests) async and also provides an async method for the body. Back in the DataParallelismWithForEach project, create a new async method named ProcessFilesAsync() and copy the code from the ProcessFiles() method.
Once the code is copied, replace the previous call to Parallel.ForEach() with await Parallel.
ForEachAsync(). The first two parameters (files, parOpts) are the same in both calls. The change is in the body, which takes two parameters, the currentFile and the cancellation token:

//Old code:
//Parallel.ForEach(files, parOpts, currentFile =>
//New code:
await Parallel.ForEachAsync(files, parOpts, async (currentFile, token) =>

The cancellation token that is passed into the body is the same cancellation token as the one in the ParallelOptions instance. This means we can check the token parameter for requested cancellation requests instead of the one on the ParallelOptions instance:

//Old code:
//parOpts.CancellationToken.ThrowIfCancellationRequested();
//New code: token.ThrowIfCancellationRequested();

The complete method is shown here:

private async Task ProcessFilesAsync()
{
// Load up all *.jpg files, and make a new folder for the modified data. var basePath = Directory.GetCurrentDirectory();
var pictureDirectory = Path.Combine(basePath, "TestPictures"); var outputDirectory = Path.Combine(basePath, "ModifiedPictures");
//Clear out any existing files

if (Directory.Exists(outputDirectory))
{
Directory.Delete(outputDirectory, true);
}
Directory.CreateDirectory(outputDirectory);
string[] files = Directory.GetFiles(pictureDirectory, "*.jpg", SearchOption. AllDirectories);
// Use ParallelOptions instance to store the CancellationToken. ParallelOptions parOpts = new ParallelOptions(); parOpts.CancellationToken = _cancelToken.Token; parOpts.MaxDegreeOfParallelism = System.Environment.ProcessorCount; try
{
// Process the image data in a parallel manner!
await Parallel.ForEachAsync(files, parOpts, async (currentFile,token) =>
{
token.ThrowIfCancellationRequested();
string filename = Path.GetFileName(currentFile); Dispatcher?.Invoke(() =>
{
this.Title = $"Processing {filename} on thread {Environment. CurrentManagedThreadId}";
}
);
using (Bitmap bitmap = new Bitmap(currentFile))
{
bitmap.RotateFlip(RotateFlipType.Rotate180FlipNone); bitmap.Save(Path.Combine(outputDirectory, filename));
}
Thread.Sleep(2000);
});
Dispatcher?.Invoke(() => this.Title = "Done!");
}
catch (OperationCanceledException ex)
{
Dispatcher?.Invoke(() => this.Title = ex.Message);
}
}

The final change is to call this new method when the button is clicked:

private void cmdProcess_Click(object sender, EventArgs e)
{
this.Title = $"Starting..."; ProcessFilesAsync().Wait();
}

Update the Book Reader App with async/await
Now that you understand the async/await pattern, let’s update the Book Reader app to use the HttpClient
instead of the WebClient. Create a new method named GetBookAsync().

async Task GetBookAsync()
{
HttpClient client = new HttpClient();
_theEBook = await client.GetStringAsync("http://www.gutenberg.org/files/98/98-0.txt"); Console.WriteLine("Download complete.");
GetStats();
}

After creating a new instance of the HttpClient class, the code calls the GetStringAsync() method to make an HTTP Get call to retrieve the text from the website. As you can see, the HttpClient provides a much more concise way to make HTTP calls. The HttpClient class will be explored in much more detail in the chapters that cover ASP.NET Core.

Wrapping Up async and await
This section contained a lot of examples; here are the key points of this section:
•Methods (as well as lambda expressions or anonymous methods) can be marked with the async keyword to enable the method to do work in a nonblocking manner.
•Methods (as well as lambda expressions or anonymous methods) marked with the
async keyword will run synchronously until the await keyword is encountered.
•A single async method can have multiple await contexts.
•When the await expression is encountered, the calling thread is suspended until the awaited task is complete. In the meantime, control is returned to the caller of the method.
•The await keyword will hide the returned Task object from view, appearing to directly return the underlying return value. Methods with no return value simply return void.
•Parameter checking and other error handling should be done in the main section of the method, with the actual async portion moved to a private function.
•For stack variables, the ValueTask is more efficient than the Task object, which might cause boxing and unboxing.
•As a naming convention, methods that are to be called asynchronously should be marked with the Async suffix.

Summary
This chapter began by examining the role of the System.Threading namespace. As you learned, when an application creates additional threads of execution, the result is that the program in question can carry out numerous tasks at (what appears to be) the same time. You also examined several manners in which you can protect thread-sensitive blocks of code to ensure that shared resources do not become unusable units of bogus data.
This chapter then examined some new models for working with multithreaded development introduced with .NET 4.0, specifically the Task Parallel Library and PLINQ. I wrapped things up by covering the role of the async and await keywords. As you have seen, these keywords are using many types of the TPL framework in the background; however, the compiler does most of the work to create the complex threading and synchronization code on your behalf.

Pro C#10 CHAPTER 14 Processes, AppDomains, and Load Contexts

PART V

Programming with .NET Core Assemblies

CHAPTER 14

Processes, AppDomains, and Load Contexts

In this chapter, you’ll drill deep into the details of how an assembly is hosted by the runtime and come to understand the relationship between processes, application domains, and object contexts.
In a nutshell, application domains (or simply AppDomains) are logical subdivisions within a given process that host a set of related .NET Core assemblies. As you will see, an AppDomain is further subdivided into contextual boundaries, which are used to group like-minded .NET Core objects. Using the notion of context, the runtime can ensure that objects with special requirements are handled appropriately.
While it is true that many of your day-to-day programming tasks might not involve directly working with processes, AppDomains, or object contexts, understanding these topics is important when working with numerous .NET Core APIs, including multithreading, parallel processing, and object serialization.

The Role of a Windows Process
The concept of a “process” existed within Windows-based operating systems well before the release of the
.NET/.NET Core platforms. In simple terms, a process is a running program. However, formally speaking, a process is an operating system–level concept used to describe a set of resources (such as external code libraries and the primary thread) and the necessary memory allocations used by a running application.
For each .NET Core application loaded into memory, the OS creates a separate and isolated process for use during its lifetime.
Using this approach to application isolation, the result is a much more robust and stable runtime environment, given that the failure of one process does not affect the functioning of another. Furthermore, data in one process cannot be directly accessed by another process, unless you use specific tools such as System.IO.Pipes or the MemoryMappedFile class. Given these points, you can regard the process as a fixed, safe boundary for a running application.
Every Windows process is assigned a unique process identifier (PID) and may be independently loaded and unloaded by the OS as necessary (as well as programmatically). As you might be aware, the Processes tab of the Windows Task Manager utility (activated via the Ctrl+Shift+Esc keystroke combination on Windows) allows you to view various statistics regarding the processes running on a given machine. The Details tab allows you to view the assigned PID and image name (see Figure 14-1).

© Andrew Troelsen, Phil Japikse 2022
A. Troelsen and P. Japikse, Pro C# 10 with .NET 6, https://doi.org/10.1007/978-1-4842-7869-7_14

551

Figure 14-1. The Windows Task Manager

The Role of Threads
Every Windows process contains an initial “thread” that functions as the entry point for the application. Chapter 15 examines the details of building multithreaded applications under the .NET Core platform; however, to facilitate the topics presented here, you need a few working definitions. First, a thread is a path of execution within a process. Formally speaking, the first thread created by a process’s entry point is termed the primary thread. Any .NET Core program (console application, Windows service, WPF application, etc.) marks its entry point with the Main() method or a file containing top-level statements (which gets converted to a Program class and Main() method, as demonstrated earlier in this book). When this code is invoked, the primary thread is created automatically.
Processes that contain a single primary thread of execution are intrinsically thread-safe, given that there is only one thread that can access the data in the application at a given time. However, a single-threaded process (especially one that is GUI based) will often appear a bit unresponsive to the user if this single thread is performing a complex operation (such as printing out a lengthy text file, performing a mathematically intensive calculation, or attempting to connect to a remote server located thousands of miles away).
Given this potential drawback of single-threaded applications, the operating systems that are supported by .NET Core (as well as the .NET Core platform) make it possible for the primary thread to spawn additional

secondary threads (also termed worker threads) using a handful of API functions such as CreateThread(). Each thread (primary or secondary) becomes a unique path of execution in the process and has concurrent access to all shared points of data within the process.
As you might have guessed, developers typically create additional threads to help improve the program’s overall responsiveness. Multithreaded processes provide the illusion that numerous activities are happening at the same time. For example, an application may spawn a worker thread to perform a labor- intensive unit of work (again, such as printing a large text file). As this secondary thread is churning away, the main thread is still responsive to user input, which gives the entire process the potential of delivering greater performance. However, this may not actually be the case: using too many threads in a single process can actually degrade performance, as the CPU must switch between the active threads in the process (which takes time).
On some machines, multithreading is most commonly an illusion provided by the OS. Machines that host a single (nonhyperthreaded) CPU do not have the ability to literally handle multiple threads at the same time. Rather, a single CPU will execute one thread for a unit of time (called a time slice) based in part on the thread’s priority level. When a thread’s time slice is up, the existing thread is suspended to allow another thread to perform its business. For a thread to remember what was happening before it was kicked out of the way, each thread is given the ability to write to Thread Local Storage (TLS) and is provided with a separate call stack, as illustrated in Figure 14-2.

Figure 14-2. The Windows process/thread relationship

If the subject of threads is new to you, don’t sweat the details. At this point, just remember that a thread is a unique path of execution within a Windows process. Every process has a primary thread (created via the executable’s entry point) and may contain additional threads that have been programmatically created.

Interacting with Processes Using .NET Core
Although processes and threads are nothing new, the way you interact with these primitives under the
.NET Core platform has changed quite a bit (for the better). To pave the way to understanding the world of building multithreaded assemblies (see Chapter 15), let’s begin by checking out how to interact with processes using the .NET Core base class libraries.
The System.Diagnostics namespace defines several types that allow you to programmatically interact with processes and various diagnostic-related types such as the system event log and performance counters. In this chapter, you are concerned with only the process-centric types defined in Table 14-1.

Table 14-1. Select Members of the System.Diagnostics Namespace

Process-Centric Types of the
System.Diagnostics Namespace Meaning in Life
Process The Process class provides access to local and remote processes and allows you to programmatically start and stop processes.
ProcessModule This type represents a module (.dll or .exe) that is loaded into a process. Understand that the ProcessModule type can represent any module—COM-based, .NET-based, or traditional C-based binaries.
ProcessModuleCollection This provides a strongly typed collection of ProcessModule objects.
ProcessStartInfo This specifies a set of values used when starting a process via the
Process.Start() method.
ProcessThread This type represents a thread within a given process. Be aware that ProcessThread is a type used to diagnose a process’s thread set and is not used to spawn new threads of execution within a process.
ProcessThreadCollection This provides a strongly typed collection of ProcessThread objects.

The System.Diagnostics.Process class allows you to analyze the processes running on a given machine (local or remote). The Process class also provides members that allow you to programmatically start and terminate processes, view (or modify) a process’s priority level, and obtain a list of active threads and/or loaded modules within a given process. Table 14-2 lists some of the key properties of System.
Diagnostics.Process.
Table 14-2. Select Properties of the Process Type

Property Meaning in Life
ExitTime This property gets the timestamp associated with the process that has terminated (represented with a DateTime type).
Handle This property returns the handle (represented by an IntPtr) associated to the process by the OS. This can be useful when building .NET applications that need to communicate with unmanaged code.
Id This property gets the PID for the associated process.
MachineName This property gets the name of the computer the associated process is running on.
MainWindowTitle MainWindowTitle gets the caption of the main window of the process (if the process does not have a main window, you receive an empty string).
Modules This property provides access to the strongly typed ProcessModuleCollection type, which represents the set of modules (.dll or .exe) loaded within the current process.
ProcessName This property gets the name of the process (which, as you would assume, is the name of the application itself).
Responding This property gets a value indicating whether the user interface of the process is responding to user input (or is currently “hung”).
StartTime This property gets the time that the associated process was started (via a DateTime type).
Threads This property gets the set of threads that are running in the associated process (represented via a collection of ProcessThread objects).

In addition to the properties just examined, System.Diagnostics.Process also defines a few useful methods (see Table 14-3).

Table 14-3. Select Methods of the Process Type

Method Meaning in Life
CloseMainWindow() This method closes a process that has a user interface by sending a close message to its main window.
GetCurrentProcess() This static method returns a new Process object that represents the currently active process.
GetProcesses() This static method returns an array of new Process objects running on a given machine.
Kill() This method immediately stops the associated process.
Start() This method starts a process.

Enumerating Running Processes
To illustrate the process of manipulating Process objects (pardon the redundancy), create a C# Console Application project named ProcessManipulator. Next, define the following static helper method within the Program.cs file:

static void ListAllRunningProcesses()
{
// Get all the processes on the local machine, ordered by
// PID.
var runningProcs = from proc
in Process.GetProcesses(".") orderby proc.Id
select proc;

// Print out PID and name of each process. foreach(var p in runningProcs)
{
string info = $"-> PID: {p.Id}\tName: {p.ProcessName}"; Console.WriteLine(info);
}
Console.WriteLine("****\n");
}

The static Process.GetProcesses() method returns an array of Process objects that represent the running processes on the target machine (the dot notation shown here represents the local computer). After you have obtained the array of Process objects, you are able to invoke any of the members listed in Tables 14-2 and 14-3. Here, you are simply displaying the PID and the name of each process, ordered by PID. Update the top-level statements as follows:

using System.Diagnostics;
Console.WriteLine(" Fun with Processes \n"); ListAllRunningProcesses();
Console.ReadLine();

When you run the application, you will see the names and PIDs for all processes on your local computer. Here is some partial output from my current machine (your output will most likely be different):

Fun with Processes
-> PID: 0 Name: Idle
-> PID: 4 Name: System
-> PID: 104 Name: Secure System
-> PID: 176 Name: Registry
-> PID: 908 Name: svchost
-> PID: 920 Name: smss
-> PID: 1016 Name: csrss
-> PID: 1020 Name: NVDisplay.Container
-> PID: 1104 Name: wininit
-> PID: 1112 Name: csrss


Investigating a Specific Process
In addition to obtaining a complete list of all running processes on a given machine, the static Process. GetProcessById() method allows you to obtain a single Process object via the associated PID. If you request access to a nonexistent PID, an ArgumentException exception is thrown. For example, if you were interested in obtaining a Process object representing a process with the PID of 30592, you could write the following code:

// If there is no process with the PID of 30592, a runtime exception will be thrown. static void GetSpecificProcess()
{
Process theProc = null; try
{
theProc = Process.GetProcessById(30592); Console.WriteLine(theProc?.ProcessName);
}
catch(ArgumentException ex)
{
Console.WriteLine(ex.Message);
}
}

At this point, you have learned how to get a list of all processes, as well as a specific process on a machine via a PID lookup. While it is somewhat useful to discover PIDs and process names, the Process class also allows you to discover the set of current threads and libraries used within a given process. Let’s see how to do so.

Investigating a Process’s Thread Set
The set of threads is represented by the strongly typed ProcessThreadCollection collection, which contains some number of individual ProcessThread objects. To illustrate, add the following additional static helper function to your current application:

static void EnumThreadsForPid(int pID)
{
Process theProc = null; try
{
theProc = Process.GetProcessById(pID);
}
catch(ArgumentException ex)
{
Console.WriteLine(ex.Message); return;
}

// List out stats for each thread in the specified process. Console.WriteLine(
"Here are the threads used by: {0}", theProc.ProcessName); ProcessThreadCollection theThreads = theProc.Threads;

foreach(ProcessThread pt in theThreads)
{
string info =
$"-> Thread ID: {pt.Id}\tStart Time: {pt.StartTime.ToShortTimeString()}\tPriority:
{pt.PriorityLevel}"; Console.WriteLine(info);
}
Console.WriteLine("****\n");
}

As you can see, the Threads property of the System.Diagnostics.Process type provides access to the ProcessThreadCollection class. Here, you are printing the assigned thread ID, start time, and priority level of each thread in the process specified by the client. Now, update your program’s top-level statements to prompt the user for a PID to investigate, as follows:

...
// Prompt user for a PID and print out the set of active threads. Console.WriteLine(" Enter PID of process to investigate "); Console.Write("PID: ");
string pID = Console.ReadLine(); int theProcID = int.Parse(pID);

EnumThreadsForPid(theProcID); Console.ReadLine();

When you run your program, you can now enter the PID of any process on your machine and see the threads used in the process. The following output shows a partial list of the threads used by PID 3804 on my machine, which happens to be hosting Edge:

Enter PID of process to investigate PID: 3804
Here are the threads used by: msedge
-> Thread ID: 3464 Start Time: 01:20 PM Priority: Normal
-> Thread ID: 19420 Start Time: 01:20 PM Priority: Normal
-> Thread ID: 17780 Start Time: 01:20 PM Priority: Normal
-> Thread ID: 22380 Start Time: 01:20 PM Priority: Normal
-> Thread ID: 27580 Start Time: 01:20 PM Priority: -4
...


The ProcessThread type has additional members of interest beyond Id, StartTime, and PriorityLevel.
Table 14-4 documents some members of interest.

Table 14-4. Select Members of the ProcessThread Type

Member Meaning in Life
CurrentPriority Gets the current priority of the thread
Id Gets the unique identifier of the thread
IdealProcessor Sets the preferred processor for this thread to run on
PriorityLevel Gets or sets the priority level of the thread
ProcessorAffinity Sets the processors on which the associated thread can run
StartAddress Gets the memory address of the function that the operating system called that started this thread
StartTime Gets the time that the operating system started the thread
ThreadState Gets the current state of this thread
TotalProcessorTime Gets the total amount of time that this thread has spent using the processor
WaitReason Gets the reason that the thread is waiting

Before you read any further, be aware that the ProcessThread type is not the entity used to create, suspend, or kill threads under the .NET Core platform. Rather, ProcessThread is a vehicle used to obtain diagnostic information for the active Windows threads within a running process. Again, you will investigate how to build multithreaded applications using the System.Threading namespace in Chapter 15.

Investigating a Process’s Module Set
Next up, let’s check out how to iterate over the number of loaded modules that are hosted within a given process. When talking about processes, a module is a general term used to describe a given .dll (or the
.exe itself) that is hosted by a specific process. When you access the ProcessModuleCollection via the
Process.Modules property, you can enumerate over all modules hosted within a process: .NET Core-based,

COM-based, or traditional C-based libraries. Ponder the following additional helper function that will enumerate the modules in a specific process based on the PID:

static void EnumModsForPid(int pID)
{
Process theProc = null; try
{
theProc = Process.GetProcessById(pID);
}
catch(ArgumentException ex)
{
Console.WriteLine(ex.Message); return;
}

Console.WriteLine("Here are the loaded modules for: {0}", theProc.ProcessName);
ProcessModuleCollection theMods = theProc.Modules; foreach(ProcessModule pm in theMods)
{
string info = $"-> Mod Name: {pm.ModuleName}"; Console.WriteLine(info);
}
Console.WriteLine("****\n");
}

To see some possible output, let’s check out the loaded modules for the process hosting the current example program (ProcessManipulator). To do so, run the application, identify the PID assigned to ProcessManipulator.exe (via the Task Manager), and pass this value to the EnumModsForPid() method. Once you do, you might be surprised to see the list of *.dlls used for a simple Console Application project (GDI32.dll, USER32.dll, ole32.dll, etc.). The following output is a partial listing of modules loaded (edited for brevity):

Here are (some of) the loaded modules for: ProcessManipulator Here are the loaded modules for: ProcessManipulator
-> Mod Name: ProcessManipulator.exe
-> Mod Name: ntdll.dll
-> Mod Name: KERNEL32.DLL
-> Mod Name: KERNELBASE.dll
-> Mod Name: USER32.dll
-> Mod Name: win32u.dll
-> Mod Name: GDI32.dll
-> Mod Name: gdi32full.dll
-> Mod Name: msvcp_win.dll
-> Mod Name: ucrtbase.dll
-> Mod Name: SHELL32.dll
-> Mod Name: ADVAPI32.dll
-> Mod Name: msvcrt.dll
-> Mod Name: sechost.dll
-> Mod Name: RPCRT4.dll

-> Mod Name: IMM32.DLL
-> Mod Name: hostfxr.dll
-> Mod Name: hostpolicy.dll
-> Mod Name: coreclr.dll
-> Mod Name: ole32.dll
-> Mod Name: combase.dll
-> Mod Name: OLEAUT32.dll
-> Mod Name: bcryptPrimitives.dll
-> Mod Name: System.Private.CoreLib.dll
...


Starting and Stopping Processes Programmatically
The final aspects of the System.Diagnostics.Process class examined here are the Start() and Kill() methods. As you can gather by their names, these members provide a way to programmatically launch and terminate a process, respectively. For example, consider the following static StartAndKillProcess() helper method.

■ Note Depending on your operating system’s security settings, you might need to be running with administrator rights to start new processes.

static void StartAndKillProcess()
{
Process proc = null;

// Launch Edge, and go to Facebook! try
{
proc = Process.Start(@"C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe", "www.facebook.com");
}
catch (InvalidOperationException ex)
{
Console.WriteLine(ex.Message);
}

Console.Write("--> Hit enter to kill {0}...", proc.ProcessName);
Console.ReadLine();

// Kill all of the msedge.exe processes. try
{
foreach (var p in Process.GetProcessesByName("MsEdge"))
{
p.Kill(true);
}

}
catch (InvalidOperationException ex)
{
Console.WriteLine(ex.Message);
}
}

The static Process.Start() method has been overloaded a few times. At a minimum, you will need to specify the path and filename of the process you want to launch. This example uses a variation of the
Start() method that allows you to specify any additional arguments to pass into the program’s entry point, in this case, the web page to load.
After you call the Start() method, you are returned a reference to the newly activated process.
When you want to terminate the process, simply call the instance-level Kill() method. In this example, since Microsoft Edge launches a lot of processes, you are looping through to kill all the launched processes. You are also wrapping the calls to Start() and Kill() within a try/catch block to handle any InvalidOperationException errors. This is especially important when calling the Kill() method, as this error will be raised if the process has already been terminated prior to calling Kill().

■ Note When using the .net Framework (prior to .net Core), the Process.Start() method allowed for either the full path and filename or the operating system shortcut (e.g., msedge) of the process to start. With
.net Core and the cross-platform support, you must specify the full path and filename. operating system associations can be leveraged using the ProcessStartInfo, covered in the next two sections.

Controlling Process Startup Using the ProcessStartInfo Class
The Process.Start() method also allows you to pass in a System.Diagnostics.ProcessStartInfo type to specify additional bits of information regarding how a given process should come to life. Here is a partial definition of ProcessStartInfo (see the documentation for full details):

public sealed class ProcessStartInfo : object
{
public ProcessStartInfo();
public ProcessStartInfo(string fileName);
public ProcessStartInfo(string fileName, string arguments); public string Arguments { get; set; }
public bool CreateNoWindow { get; set; }
public StringDictionary EnvironmentVariables { get; } public bool ErrorDialog { get; set; }
public IntPtr ErrorDialogParentHandle { get; set; } public string FileName { get; set; }
public bool LoadUserProfile { get; set; } public SecureString Password { get; set; } public bool RedirectStandardError { get; set; } public bool RedirectStandardInput { get; set; }
public bool RedirectStandardOutput { get; set; } public Encoding StandardErrorEncoding { get; set; } public Encoding StandardOutputEncoding { get; set; } public bool UseShellExecute { get; set; }

public string Verb { get; set; } public string[] Verbs { get; }
public ProcessWindowStyle WindowStyle { get; set; } public string WorkingDirectory { get; set; }
}

To illustrate how to fine-tune your process startup, here is a modified version of StartAndKillProcess(), which will load Microsoft Edge and navigate to www.facebook.com, using the Windows association MsEdge:

static void StartAndKillProcess()
{
Process proc = null;

// Launch Microsoft Edge, and go to Facebook, with maximized window. try
{
ProcessStartInfo startInfo = new ProcessStartInfo("MsEdge", "www.facebook.com");
startInfo.UseShellExecute = true; proc = Process.Start(startInfo);
}
catch (InvalidOperationException ex)
{
Console.WriteLine(ex.Message);
}
...
}

In .NET Core, the UseShellExecute property defaults to false, while in prior versions of .NET, the UseShellExecute property defaults to true. This is the reason that the previous version of Process.Start(), shown here, no longer works without using ProcessStartInfo and setting the UseShellExecute property to true:

Process.Start("msedge")

Leveraging OS Verbs with ProcessStartInfo
In addition to using the OS shortcuts to launch applications, you can also take advantage of file associations with ProcessStartInfo. On Windows, if you right-click a Word document, there are options to edit or print the document. Let’s use the ProcessStartInfo to determine the verbs available and then use them to manipulate the process.
Create a new method with the following code:

static void UseApplicationVerbs()
{
int i = 0;
//adjust this path and name to a document on your machine ProcessStartInfo si =
new ProcessStartInfo(@"..\TestPage.docx"); foreach (var verb in si.Verbs)

{
Console.WriteLine($" {i++}. {verb}");
}
si.WindowStyle = ProcessWindowStyle.Maximized; si.Verb = "Edit";
si.UseShellExecute = true; Process.Start(si);
}

When you run this code, the first part prints out all the available verbs for a Word document, as the following shows:

Fun with Processes
0.Edit
1.OnenotePrintto
2.Open
3.OpenAsReadOnly
4.Print
5.Printto
6.ViewProtected

After setting WindowStyle to maximized, the verb is set to Edit, which opens the document in edit mode. If you set the verb to Print, the document will be sent straight to the printer.
Now that you understand the role of Windows processes and how to interact with them from C# code, you are ready to investigate the concept of a .NET application domain.

■ Note the directory in which the application runs is dependent on how you run the sample application. if you use the CLi command dotnet run, the current directory is the same as where the project file is located. if you are using Visual studio, the current directory will be the directory of the compiled assembly, which is .\ bin\debug\net6.0. You will need to adjust the path to the Word document accordingly.

Understanding .NET Application Domains
Under the .NET and .NET Core platforms, executables are not hosted directly within a Windows process, as is the case in traditional unmanaged applications. Rather, .NET and .NET Core executables are hosted by
a logical partition within a process called an application domain. This partition of a traditional Windows process offers several benefits, some of which are as follows:
• AppDomains are a key aspect of the OS-neutral nature of the .NET Core platform, given that this logical division abstracts away the differences in how an underlying OS represents a loaded executable.
• AppDomains are far less expensive in terms of processing power and memory than a full-blown process. Thus, the CoreCLR can load and unload application domains much quicker than a formal process and can drastically improve scalability of server applications.

AppDomains are fully and completely isolated from other AppDomains within a process. Given this fact, be aware that an application running in one AppDomain is unable to obtain data of any kind (global variables or static fields) within another AppDomain, unless they use a distributed programming protocol.

■ Note support for appDomains is changed in .net Core. in .net Core, there is exactly one appDomain. Creation of new appDomains is no longer supported because they require runtime support and are generally expensive to create. the ApplicationLoadContext (covered later in this chapter) provides assembly isolation in .net Core.

The System.AppDomain Class
The AppDomain class is largely deprecated with .NET Core. While most of the remaining support is designed to make migrating from .NET 4.x to .NET Core easier, the remaining features can still provide value, as covered in the next two sections.

Interacting with the Default Application Domain
Your application has access to the default application domain using the static AppDomain.CurrentDomain property. After you have this access point, you can use the methods and properties of AppDomain to perform some runtime diagnostics.
To learn how to interact with the default application domain, begin by creating a new Console Application project named DefaultAppDomainApp and disable nullability in the project file. Now, update your Program.cs file with the following logic, which will simply display some details about the default application domain, using a number of members of the AppDomain class:

using System.Reflection; using System.Runtime.Loader;

Console.WriteLine(" Fun with the default AppDomain \n"); DisplayDADStats();
Console.ReadLine();

static void DisplayDADStats()
{
// Get access to the AppDomain for the current thread. AppDomain defaultAD = AppDomain.CurrentDomain;
// Print out various stats about this domain. Console.WriteLine("Name of this domain: {0}",
defaultAD.FriendlyName);
Console.WriteLine("ID of domain in this process: {0}", defaultAD.Id);
Console.WriteLine("Is this the default domain?: {0}", defaultAD.IsDefaultAppDomain());
Console.WriteLine("Base directory of this domain: {0}", defaultAD.BaseDirectory);
Console.WriteLine("Setup Information for this domain:"); Console.WriteLine("\t Application Base: {0}",

defaultAD.SetupInformation.ApplicationBase); Console.WriteLine("\t Target Framework: {0}",
defaultAD.SetupInformation.TargetFrameworkName);
}

The output of this example is shown here:

Fun with the default AppDomain Name of this domain: DefaultAppDomainApp ID of domain in this process: 1
Is this the default domain?: True
Base directory of this domain: C:\GitHub\Books\csharp8-wf\Code\Chapter_14\ DefaultAppDomainApp\DefaultAppDomainApp\bin\Debug\net6.0\
Setup Information for this domain:
Application Base: C:\GitHub\Books\csharp8-wf\Code\Chapter_14\DefaultAppDomainApp\ DefaultAppDomainApp\bin\Debug\net6.0\
Target Framework: .NETCoreApp,Version=v5.0

Notice that the name of the default application domain will be identical to the name of the executable that is contained within it (DefaultAppDomainApp.exe, in this example). Also notice that the base directory value, which will be used to probe for externally required private assemblies, maps to the current location of the deployed executable.

Enumerating Loaded Assemblies
It is also possible to discover all the loaded .NET Core assemblies within a given application domain using the instance-level GetAssemblies() method. This method will return to you an array of Assembly objects (covered in Chapter 17). To do this, you must have added the System.Reflection namespace to your code file (as you did earlier in this section).
To illustrate, define a new method named ListAllAssembliesInAppDomain() within the Program.cs
file. This helper method will obtain all loaded assemblies and print the friendly name and version of each.

static void ListAllAssembliesInAppDomain()
{
// Get access to the AppDomain for the current thread. AppDomain defaultAD = AppDomain.CurrentDomain;

// Now get all loaded assemblies in the default AppDomain. Assembly[] loadedAssemblies = defaultAD.GetAssemblies();
Console.WriteLine(" Here are the assemblies loaded in {0} \n", defaultAD.FriendlyName);
foreach(Assembly a in loadedAssemblies)
{
Console.WriteLine($"-> Name, Version: {a.GetName().Name}:{a.GetName().Version}" );
}
}

Assuming you have updated your top-level statements to call this new member, you will see that the application domain hosting your executable is currently using the following .NET Core libraries:

Here are the assemblies loaded in DefaultAppDomainApp
-> Name, Version: System.Private.CoreLib:5.0.0.0
-> Name, Version: DefaultAppDomainApp:1.0.0.0
-> Name, Version: System.Runtime:5.0.0.0
-> Name, Version: System.Console:5.0.0.0
-> Name, Version: System.Threading:5.0.0.0
-> Name, Version: System.Text.Encoding.Extensions:5.0

Now understand that the list of loaded assemblies can change at any time as you author new C# code. For example, assume you have updated your ListAllAssembliesInAppDomain() method to make use of a LINQ query, which will order the loaded assemblies by name, as follows:
static void ListAllAssembliesInAppDomain()
{
// Get access to the AppDomain for the current thread. AppDomain defaultAD = AppDomain.CurrentDomain;

// Now get all loaded assemblies in the default AppDomain. var loadedAssemblies =
defaultAD.GetAssemblies().OrderBy(x=>x.GetName().Name); Console.WriteLine(" Here are the assemblies loaded in {0} \n", defaultAD.FriendlyName);
foreach(Assembly a in loadedAssemblies)
{
Console.WriteLine($"-> Name, Version: {a.GetName().Name}:{a.GetName().Version}" );
}
}
If you were to run the program once again, you would see that System.Linq.dll has also been loaded
into memory.

Here are the assemblies loaded in DefaultAppDomainApp
-> Name, Version: DefaultAppDomainApp:1.0.0.0
-> Name, Version: System.Console:5.0.0.0
-> Name, Version: System.Linq:5.0.0.0
-> Name, Version: System.Private.CoreLib:5.0.0.0
-> Name, Version: System.Runtime:5.0.0.0
-> Name, Version: System.Text.Encoding.Extensions:5.0.0.0
-> Name, Version: System.Threading:5.0.0

Assembly Isolation with Application Load Contexts
As you have just seen, AppDomains are logical partitions used to host .NET Core assemblies. Additionally, an application domain may be further subdivided into numerous load context boundaries. Conceptually, a load context creates a scope for loading, resolving, and potentially unloading a set of assemblies. In a
nutshell, a .NET Core load context provides a way for a single AppDomain to establish a “specific home” for a given object.

■ Note While understanding processes and application domains is quite important, most .net Core applications will never demand that you work with object contexts. i’ve included this overview material just to paint a more complete picture.

The AssemblyLoadContext class provides the capability to load additional assemblies into their own contexts. To demonstrate, first add a class library project named ClassLibary1 and add it to your current solution. Using the .NET Core CLI, execute the following commands in the directory containing your current solution:

dotnet new classlib -lang c# -n ClassLibrary1 -o .\ClassLibrary1 -f net6.0 dotnet sln .\Chapter14_AllProjects.sln add .\ClassLibrary1

Next, add a reference from the DefaultAppDomainApp to the ClassLibrary1 project by executing the following CLI command:

dotnet add DefaultAppDomainApp reference ClassLibrary1

If you are using Visual Studio, right-click the solution node in Solution Explorer, select Add ➤ New Project, and add a .NET Core class library named ClassLibrary1. This creates the project and adds it to your solution. Next, add a reference to this new project by right-clicking the DefaultAppDomainApp project
and selecting Add ➤ Project Reference. Select the Projects ➤ Solution option in the left rail, and select the ClassLibrary1 check box, as shown in Figure 14-3.

Figure 14-3. Adding the project reference in Visual Studio

In this new class library, add a Car class, as follows:

namespace ClassLibrary1; public class Car
{
public string PetName { get; set; } public string Make { get; set; } public int Speed { get; set; }
}

With this new assembly in place, make sure the following using statements are at the top of the
Program.cs file in the DefaultAppDomainApp project:

using System.Reflection; using System.Runtime.Loader;

The next method in the DefaultAppDomainApp top level statements is the
LoadAdditionalAssembliesDifferentContexts() method, shown here:

static void LoadAdditionalAssembliesDifferentContexts()
{
var path = Path.Combine(AppDomain.CurrentDomain.BaseDirectory,
"ClassLibrary1.dll"); AssemblyLoadContext lc1 =
new AssemblyLoadContext("NewContext1",false); var cl1 = lc1.LoadFromAssemblyPath(path);
var c1 = cl1.CreateInstance("ClassLibrary1.Car");

AssemblyLoadContext lc2 =
new AssemblyLoadContext("NewContext2",false); var cl2 = lc2.LoadFromAssemblyPath(path);
var c2 = cl2.CreateInstance("ClassLibrary1.Car");
Console.WriteLine(" Loading Additional Assemblies in Different Contexts "); Console.WriteLine($"Assembly1 Equals(Assembly2) {cl1.Equals(cl2)}"); Console.WriteLine($"Assembly1 == Assembly2 {cl1 == cl2}"); Console.WriteLine($"Class1.Equals(Class2) {c1.Equals(c2)}"); Console.WriteLine($"Class1 == Class2 {c1 == c2}");
}

The first line uses the static Path.Combine method to build up the directory for the ClassLibrary1
assembly.

■ Note You might be wondering why you created a reference for an assembly that will be loaded dynamically. this is to make sure that when the project builds, the ClassLibrary1 assembly builds as well and is in the same directory as the DefaultAppDomainApp. this is merely a convenience for this example. there is no need to reference an assembly that you will load dynamically.

Next, the code creates a new AssemblyLoadContext with the name NewContext1 (the first parameter of the method) and does not support unloading (the second parameter). This LoadContext is used to load
the ClassLibrary1 assembly and then create an instance of a Car class. If some of this code is new to you, it will be explained more fully in Chapter 18. The process is repeated with a new AssemblyLoadContext, and then the assemblies and classes are compared for equality. When you run this new method, you will see the following output:

Loading Additional Assemblies in Different Contexts Assembly1 Equals(Assembly2) False

Assembly1 == Assembly2 False Class1.Equals(Class2) False Class1 ==8 Class2 False

This demonstrates that the same assembly has been loaded twice into the app domain. The classes are also different, as should be expected.
Next, add a new method that will load the assembly from the same AssemblyLoadContext.

static void LoadAdditionalAssembliesSameContext()
{
var path = Path.Combine(AppDomain.CurrentDomain.BaseDirectory,
"ClassLibrary1.dll"); AssemblyLoadContext lc1 =
new AssemblyLoadContext(null,false); var cl1 = lc1.LoadFromAssemblyPath(path);
var c1 = cl1.CreateInstance("ClassLibrary1.Car"); var cl2 = lc1.LoadFromAssemblyPath(path);
var c2 = cl2.CreateInstance("ClassLibrary1.Car");
Console.WriteLine(" Loading Additional Assemblies in Same Context "); Console.WriteLine($"Assembly1.Equals(Assembly2) {cl1.Equals(cl2)}"); Console.WriteLine($"Assembly1 == Assembly2 {cl1 == cl2}"); Console.WriteLine($"Class1.Equals(Class2) {c1.Equals(c2)}"); Console.WriteLine($"Class1 == Class2 {c1 == c2}");
}

The main difference in this code is that only one AssemblyLoadContext is created. Now, when the ClassLibrary1 assembly is loaded twice, the second assembly is simply a pointer to the first instance of the assembly. Running the code produces the following output:

Loading Additional Assemblies in Same Context Assembly1.Equals(Assembly2) True
Assembly1 == Assembly2 True Class1.Equals(Class2) False Class1 == Class2 False

Summarizing Processes, AppDomains, and Load Contexts
At this point, you should have a much better idea about how a .NET Core assembly is hosted by the runtime. If the previous pages have seemed to be a bit too low level for your liking, fear not. For the most part, .NET Core automatically deals with the details of processes, application domains, and load contexts on your behalf. The good news, however, is that this information provides a solid foundation for understanding multithreaded programming under the .NET Core platform.

Summary
The point of this chapter was to examine exactly how a .NET Core application is hosted by the .NET Core platform. As you have seen, the long-standing notion of a Windows process has been altered under the hood to accommodate the needs of the CoreCLR. A single process (which can be programmatically manipulated via the System.Diagnostics.Process type) is now composed of an application domain, which represents isolated and independent boundaries within a process.
An application domain is capable of hosting and executing any number of related assemblies.
Furthermore, a single application domain can contain any number of load contexts for further assembly isolation. Using this additional level of type isolation, the CoreCLR can ensure that special-need objects are handled correctly.