插入关系很慢

Question

我必须插入多个关系并遇到 Context.SaveChanges 操作问题，这需要永远完成。我已经尝试了多种方法将这些实体添加到数据库中，但似乎没有任何帮助。

我的模型是按以下方式构建的：

public class Agreement : GdSoftDeleteEntity
{
    public DateTime Date { get; set; }
    public AgreementType AgreementType { get; set; }

    public virtual ICollection<PersonAgreementRelation> PersonAgreementRelations { get; set; }
    public virtual ICollection<ImageSearchAppointment> ImageSearchAppointments { get; set; }
}

public class Person : GdSoftDeleteEntity
{
    public string Name { get; set; }
    public string FirstName { get; set; }

    // E-mail is in identityuser
    //public string EmailAddress { get; set; }

    public virtual PersonType PersonType { get; set; }

    public virtual ICollection<PersonAgreementRelation> PersonAgreementRelations { get; set; }
    public virtual ICollection<PersonPersonRelation> PersonMasters { get; set; }
    public virtual ICollection<PersonPersonRelation> PersonSlaves { get; set; }
}

public class PersonAgreementRelation : GdSoftDeleteEntity
{
    public int PersonId { get; set; }
    public virtual Person Person { get; set; }

    public int AgreementId { get; set; }
    public virtual Agreement Agreement { get; set; }

    public virtual PersonAgreementRole PersonAgreementRole { get; set; }
}

public class ImageSearchAppointment : GdSoftDeleteEntity
{
    public string Name { get; set; }
    public bool ShowResultsToCustomer { get; set; }
    public bool HasImageFeed { get; set; }
    public int AgreementId { get; set; }
    public virtual Agreement Agreement { get; set; }

    public Periodicity Periodicity { get; set; }
    public PeriodicityCategory PeriodicityCategory { get; set; }

    public virtual ICollection<ImageSearchCommand> ImageSearchCommands { get; set; }
    public virtual ICollection<ImageSearchAppointmentWebDomainWhitelist> ImageSearchAppointmentWebDomainWhitelists { get; set; }
    public virtual ICollection<ImageSearchAppointmentWebDomainExtension> ImageSearchAppointmentWebDomainExtensions { get; set; }
}

public class ImageSearchCommand : GdSoftDeleteEntity
{
    public int ImageSearchAppointmentId { get; set; }
    public virtual ImageSearchAppointment ImageSearchAppointment { get; set; }

    public int? ImageSearchAppointmentCredentialsId { get; set; }
    public virtual ImageSearchAppointmentCredentials ImageSearchAppointmentCredentials { get; set; }

    public DateTime Date { get; set; }

    //public bool Invoiced { get; set; }
    public int NumberOfImages { get; set; }
    public DateTime ImageCollectionProcessedDate { get; set; }

    public virtual ICollection<ImageSearchExecution> ImageSearchExecutions { get; set; }
}

在我的服务中，我编写了以下代码：

public int AddAgreement(int personId, AgreementDto agreementDto)
        {
            Context.Configuration.LazyLoadingEnabled = false;
            //var person = Context.Persons.SingleOrDefault(el => el.Id == personId);
            var person = Context.Persons
                .SingleOrDefault(x => x.Id == personId);
            if (person == null)
            {
                throw new GraphicsDetectiveInvalidDataTypeException($"No person found for Id: {personId}");
            }

            if (agreementDto == null)
            {
                throw new GraphicsDetectiveInvalidDataTypeException("Invalid agreementDto");
            }

            //TODO: Check if OKAY!!!

            if (agreementDto.ImageSearchAppointmentDto.Count == 0)
            {
                throw new GraphicsDetectiveInvalidDataTypeException("Count of imagesearchappointments can't be lower than 0");
            }

            //set agreement properties
            var agreement = new Agreement
            {
                Date = agreementDto.DateTime,
                AgreementType = AgreementType.WwwImageSearch,
                //ImageSearchAppointments = new List<ImageSearchAppointment>(),
                //IsDeleted = false
            };
            Context.Agreements.Add(agreement);
            Context.SaveChanges();

            //var personAdminId = Context.Users.Single(x => x.Email == ConfigurationManager.AppSettings["DefaultGdAdminEmail"]).PersonId;
            // Dit werkt niet. Moet in 2 stappen
            //set personagreementrelations for new agreement
            var adminEmail = ConfigurationManager.AppSettings["DefaultGdAdminEmail"];    
            var personAdminId = Context.Users
                .SingleOrDefault(x => x.Email == adminEmail)
                .PersonId;

            var personPmId = Context.Persons.Single(x => x.Name == "My name").Id;
            var personAgreementRelations = new List<PersonAgreementRelation>()
                {
                    new PersonAgreementRelation
                    {
                        AgreementId = agreement.Id,
                        PersonId = personId,
                        PersonAgreementRole = PersonAgreementRole.Client,
                    },
                    new PersonAgreementRelation
                    {
                        AgreementId = agreement.Id,
                        PersonAgreementRole = PersonAgreementRole.Supplier,
                        PersonId = personPmId,
                    },
                     new PersonAgreementRelation
                    {
                        AgreementId = agreement.Id,
                        PersonAgreementRole = PersonAgreementRole.Admin,
                        PersonId = personAdminId,
                    }
                };
            foreach (var personAgreementRelation in personAgreementRelations)
            {
                Context.PersonAgreementRelations.Add(personAgreementRelation);
            }

            Context.Configuration.ValidateOnSaveEnabled = false;
            Context.Configuration.AutoDetectChangesEnabled = false;

            Context.SaveChanges();

            Context.Configuration.ValidateOnSaveEnabled = true;
            Context.Configuration.AutoDetectChangesEnabled = true;
            Context.Configuration.LazyLoadingEnabled = true;

            return agreement.Id;
        }

        public void AddFirstImageSearchAppointmentToAgreement(int agreementId, ImageSearchAppointmentDto imageSearchAppointmentDto)
        {
            Context.Configuration.LazyLoadingEnabled = false;
            var agreement = Context.Agreements.SingleOrDefault(x => x.Id == agreementId);
            if (agreement == null)
            {
                throw new GraphicsDetectiveInvalidDataTypeException($"No agreement found for id {agreementId}");
            }
            var appointmentType = imageSearchAppointmentDto;
            if (appointmentType == null)
            {
                throw new GraphicsDetectiveInvalidDataTypeException($"No valid imageSearchAppointment");
            }
            if (appointmentType.ImageSearchCommandDto.Count == 0)
            {
                throw new GraphicsDetectiveInvalidDataTypeException("No imageSearchCommand");
            }

            var imageSearchAppointment = new ImageSearchAppointment
            {
                AgreementId = agreement.Id,
                Agreement = agreement,
                Name = appointmentType.Name,
                Periodicity = appointmentType.Periodicity,
                PeriodicityCategory = appointmentType.PeriodicityCategory,
                ShowResultsToCustomer = appointmentType.ShowResultsToCustomer,
                ImageSearchAppointmentWebDomainExtensions = new List<ImageSearchAppointmentWebDomainExtension>(),
                ImageSearchCommands = new List<ImageSearchCommand>(),
                ImageSearchAppointmentWebDomainWhitelists = new List<ImageSearchAppointmentWebDomainWhitelist>(),
                IsDeleted = false
            };

            var imageSearchCommandDto = appointmentType.ImageSearchCommandDto.Single();
            var imageSearchCommand = new ImageSearchCommand()
            {
                ImageSearchAppointment = imageSearchAppointment,
                Date = imageSearchCommandDto.Date,
                NumberOfImages = imageSearchCommandDto.NumberOfImages,
                ImageCollectionProcessedDate = imageSearchCommandDto.ImageCollectionProcessedDate,
                IsDeleted = false
            };

            if (imageSearchCommandDto.ImageSearchAppointmentCredentialsDto != null)
            {
                imageSearchCommand.ImageSearchAppointmentCredentials = new ImageSearchAppointmentCredentials
                {
                    FtpProfileType = imageSearchCommandDto.ImageSearchAppointmentCredentialsDto.FtpProfileType,
                    Location = imageSearchCommandDto.ImageSearchAppointmentCredentialsDto.Location,
                    Username = imageSearchCommandDto.ImageSearchAppointmentCredentialsDto.Username,
                    Password = imageSearchCommandDto.ImageSearchAppointmentCredentialsDto.Password,
                    UsePassive = imageSearchCommandDto.ImageSearchAppointmentCredentialsDto.UsePassive,
                    IsDeleted = false
                };
            }
            imageSearchAppointment.ImageSearchCommands.Add(imageSearchCommand);

            if (!imageSearchAppointment.ShowResultsToCustomer)
            {
                var webDomainExtensions = appointmentType.WebDomainExtensionDtos
                    .Select(x => new ImageSearchAppointmentWebDomainExtension()
                    {
                        ImageSearchAppointment = imageSearchAppointment,
                        WebDomainExtensionId = x.Id
                    })
                    .ToList();

                imageSearchAppointment.ImageSearchAppointmentWebDomainExtensions = webDomainExtensions;
            }

            Context.ImageSearchAppointments.Add(imageSearchAppointment);
            Context.SaveChanges();

            Context.Configuration.LazyLoadingEnabled = true;
        }

我使用 dotTrace 分析这些函数，将新实体添加到我的数据库大约需要 9 分钟。

数据库是 Azure SQL 数据库，层 S3

我尝试了建议的解决方案并修改了我的代码如下：

public int AddAgreement(int personId, AgreementDto agreementDto)
        {
            var agreementId = 0;
            using (var context = new GdDbContext())
            {
                GdDbConfiguration.SuspendExecutionStrategy = true;
                context.Configuration.LazyLoadingEnabled = true;
                //var person = Context.Persons.SingleOrDefault(el => el.Id == personId);
                var person = context.Persons
                    .SingleOrDefault(x => x.Id == personId);
                if (person == null)
                {
                    throw new GraphicsDetectiveInvalidDataTypeException($"No person found for Id: {personId}");
                }

                //var personAdminId = Context.Users.Single(x => x.Email == ConfigurationManager.AppSettings["DefaultGdAdminEmail"]).PersonId;
                // Dit werkt niet. Moet in 2 stappen
                //set personagreementrelations for new agreement
                var adminEmail = ConfigurationManager.AppSettings["DefaultGdAdminEmail"];
                var personAdminId = context.Users
                    .Where(x => x.Email == adminEmail)
                    .Include(x => x.Person)
                    .First()
                    .Person.Id;

                var personPmId = context.Persons.First(x => x.Name == "My name").Id;

                using (var dbContextTransaction = context.Database.BeginTransaction())
                {
                    try
                    {

                        if (agreementDto == null)
                        {
                            throw new GraphicsDetectiveInvalidDataTypeException("Invalid agreementDto");
                        }

                        //TODO: Check if OKAY!!!

                        if (agreementDto.ImageSearchAppointmentDto.Count == 0)
                        {
                            throw new GraphicsDetectiveInvalidDataTypeException("Count of imagesearchappointments can't be lower than 0");
                        }

                        //set agreement properties
                        var agreement = new Agreement
                        {
                            Date = agreementDto.DateTime,
                            AgreementType = AgreementType.WwwImageSearch,
                            //ImageSearchAppointments = new List<ImageSearchAppointment>(),
                            //IsDeleted = false
                        };
                        context.Agreements.Add(agreement);
                        //Context.SaveChanges();

                        var personAgreementRelations = new List<PersonAgreementRelation>()
                        {
                            new PersonAgreementRelation
                            {
                                //Agreement = agreement,
                                AgreementId = agreement.Id,
                                PersonId = personId,
                                //Person = person,
                                PersonAgreementRole = PersonAgreementRole.Client,
                                //IsDeleted = false
                            },
                            new PersonAgreementRelation
                            {
                                //Agreement = agreement,
                                AgreementId = agreement.Id,
                                PersonAgreementRole = PersonAgreementRole.Supplier,
                                PersonId = personPmId,
                                //Person = personPm,
                                //IsDeleted = false
                            },
                             new PersonAgreementRelation
                            {
                                //Agreement = agreement,
                                AgreementId = agreement.Id,
                                PersonAgreementRole = PersonAgreementRole.Admin,
                                PersonId = personAdminId,
                                //Person = personAdmin,
                            }
                        };

                        foreach (var personAgreementRelation in personAgreementRelations)
                        {
                            context.PersonAgreementRelations.Add(personAgreementRelation);
                        }
                        //agreement.PersonAgreementRelations = personAgreementRelations;

                        //Context.Agreements.Add(agreement);

                        context.Configuration.ValidateOnSaveEnabled = false;
                        context.Configuration.AutoDetectChangesEnabled = false;

                        //await Context.SaveChangesAsync();

                        context.SaveChanges();
                        dbContextTransaction.Commit();
                        //await Task.Run(async () => await Context.SaveChangesAsync());

                        context.Configuration.ValidateOnSaveEnabled = true;
                        context.Configuration.AutoDetectChangesEnabled = true;
                        context.Configuration.LazyLoadingEnabled = false;

                        agreementId = agreement.Id;
                    }
                    catch (Exception ex)
                    {
                        dbContextTransaction.Rollback();
                        throw ex;
                    }
                }
                GdDbConfiguration.SuspendExecutionStrategy = false;
            }

            return agreementId;
        }

但它需要和以前一样多的时间

Answer 1

您可以按照下面提到的建议来提高上述方法的性能。

1. 使用FirstOrDefault() 代替SingleOrDefault().FirstOrDefault() 是最快的方法。

2. 我可以看到你在同一个方法上使用了Context.SaveChanges()方法的次数。这会降低方法的性能。所以你必须避免这种情况。而不是使用事务。

像这样：EF Transactions

using (var context = new YourContext()) 
            { 
                using (var dbContextTransaction = context.Database.BeginTransaction()) 
                { 
                    try 
                    { 
                        // your operations here

                        context.SaveChanges(); //this called only once
                        dbContextTransaction.Commit(); 
                    } 
                    catch (Exception) 
                    { 
                        dbContextTransaction.Rollback(); 
                    } 
                } 
            } 

1. 如果上面的改进不够，你可以考虑存储过程的实现。

Answer 2

您的代码存在一些性能问题

增加性能

foreach (var personAgreementRelation in personAgreementRelations)
{
    Context.PersonAgreementRelations.Add(personAgreementRelation);
}


Context.Configuration.ValidateOnSaveEnabled = false;
Context.Configuration.AutoDetectChangesEnabled = false;

您添加多个实体，然后禁用 AutoDetectChanges。你通常做相反的事情

根据您的上下文中的实体数量，它可能会严重影响您的性能

在“AddFirstImageSearchAppointmentToAgreement”方法中，您似乎使用了外部上下文，如果它已经包含数千个实体，这可能会非常糟糕。

见：Improve Entity Framework Add Performance

使用不当，使用 Add 方法将实体添加到上下文中比将其保存在数据库中需要更多时间！

SaveChanges vs. Bulk Insert vs. BulkSaveChanges

SaveChanges 非常慢。对于要保存的每条记录，都需要一次数据库往返。由于额外的延迟，SQL Azure 用户尤其如此。

某些库允许您执行批量插入

见：

• Entity Framework Bulk Insert Library
• Entity Framework Bulk SaveChanges Library

免责声明：我是项目的所有者Entity Framework Extensions

此库具有 BulkSaveChanges 功能。它的工作原理与 SaveChanges 完全相同，但 速度更快！

// Context.SaveChanges();
Context.BulkSaveChanges();

编辑：添加附加信息 #1

我在 Pastebin 中粘贴了我的新代码：link

交易

为什么在选择数据并将实体添加到上下文时开始事务？这只是对交易的一种非常糟糕的使用。

事务必须尽可能晚地开始。由于 BulkSaveChanges 已在事务中执行，因此没有必要创建它。

Async.Result

 var personAdminId = context.Users.FirstOrDefaultAsync(x => x.Email == adminEmail).Result.PersonId;

我不明白你为什么在这里使用异步方法...

• 在最佳情况下，您可以获得与使用非异步方法相似的性能
• 在更糟糕的情况下，您会遇到一些使用异步方法的 performance issue

缓存项

var adminEmail = ConfigurationManager.AppSettings["DefaultGdAdminEmail"];
var personAdminId = context.Users.FirstOrDefaultAsync(x => x.Email == adminEmail).Result.PersonId;

我不知道你调用 AddAgreement 方法多少次，但我怀疑管理员会改变。

因此，如果您调用它 10,000 次，您需要进行 10,000 次数据库往返，以便每次都获得相同的准确值。

改为创建一个静态变量，并且只获取一次值！你肯定会在这里节省很多时间

这是我通常处理这种静态变量的方式：

var personAdminId = My.UserAdmin.Id;

public static class My
{
    private static User _userAdmin;
    public static User UserAdmin
    {
        get
        {
            if (_userAdmin == null)
            {
                using (var context = new GdDbContext())
                {
                    var adminEmail = ConfigurationManager.AppSettings["DefaultGdAdminEmail"];
                    _userAdmin = context.Users.FirstOrDefault(x => x.Email == adminEmail);
                }
            }

            return _userAdmin;
        }
    }
}

LazyLoadingEnabled

在第一个代码中，您将 LazyLoadingEnabled 设置为 false，但在您的 Pastebin 代码中没有，

禁用 LazyLoading 会有所帮助，因为它不会创建代理实例。

取 10m 而不是 9m

如果性能好一点，请在删除事务并再次禁用 LazyLoading 后告诉我。

下一步是了解一些统计数据：

• 大约调用 AddAgreement 方法的时间
• 您的数据库中大约有多少人
• AddAgreement 方法平均保存了大约多少个实体

编辑：添加附加信息 #2

目前，真正提高性能的唯一方法是减少数据库往返次数。

我看到您每次仍在搜索 personAdminId。通过将此值缓存在某个静态变量之类的地方，您可以节省 30 到 1 分钟。

这三个问题你还没回答：

• 大约调用 AddAgreement 方法的时间
• 您的数据库中大约有多少人
• AddAgreement 方法平均保存了大约多少个实体

这些问题的目的是了解什么是慢！

例如，如果您调用 AddAgreement 方法 10,000 次，而您的数据库中只有 2000 人，您可能最好将这些 2000 人缓存在两个字典中，以节省 20,000 次数据库往返（节省一到两分钟？ )。