如何批量插入SQL Server数据库？

Question

我们执行 SQL Server 存储过程mySP 插入数据库（假设它会调用存储过程大约 300,000 次）。

ALTER PROCEDURE [dbo].[mySP]
    @ThePath VARCHAR(255),
    @TheID INT
AS
    INSERT INTO myTbl (TheTime, ThePath, TheID)
    VALUES (GETDATE(), @ThePath, @TheID)

这是执行存储过程mySP的代码：

using (SqlConnection con = new SqlConnection(connectionString))
{
:
     foreach (.....)  //about 300,000 times
     {
         SqlCommand MyCmd = new SqlCommand("mySP", con);
         MyCmd.CommandType = CommandType.StoredProcedure;
         MyCmd.Parameters.Add(new SqlParameter("ThePath", sMyPath));
         MyCmd.Parameters.Add(new SqlParameter("TheID", sMyID));

         iRow = MyCmd.ExecuteNonQuery();

         MyCmd.Dispose();
         MyCmd = null;
     }
}

如何在一批说 5000 的情况下执行此操作，然后提交事务？

谢谢

Answer 1

当一次插入数千行时，最好使用 .Net 自己的 SqlBulkCopy 之类的东西，这基本上意味着您必须先填充 DataTable，然后使用 SqlBulkCopy 将其写入数据库。请参阅https://sqlbulkcopy-tutorial.net/ 以获得很好的示例。一个小建议是玩弄 BatchSize 属性；通常保持原样就足够了。不要忘记也使用事务。

例子：

using (SqlConnection connection = new SqlConnection("server=local);database=MyDatabase;integrated security=SSPI"))
{
    connection.Open();

    DirectoryInfo directory = new DirectoryInfo(@"C:\Temp\");

    var files = directory.GetFiles("*.dat");

    foreach (var file in files)
    {
        if (file.Length > 0)
        {
            string fileName = Path.GetFileNameWithoutExtension(file.FullName);

            SqlCommand cmd = new SqlCommand("dbo.uspStagingGetTableStructure", connection);
            cmd.CommandType = CommandType.StoredProcedure;
            cmd.Parameters.AddWithValue("@tableName", fileName);

            using (SqlDataReader reader = cmd.ExecuteReader())
            {
                while (reader.Read())
                {
                }
            }

            //
            // Prepare Bulkcopy target (datatable)
            //

            string target = "MyBulkCopyTarget";

            DataTable dataTable = new DataTable(target);
            SqlBulkCopy bulkcopy = new SqlBulkCopy(connection);
            bulkcopy.DestinationTableName = target;
            bulkcopy.BulkCopyTimeout = 600;

            //
            // Map columns source -> target
            //

            //foreach (var dataField in dataFields)
            //{
            //    if (dataField.sqlDbType == SqlDbType.Int)
            //        dataTable.Columns.Add(dataField.fieldName, typeof(int));
            //    else
            //        dataTable.Columns.Add(dataField.fieldName, typeof(string));
            //    bulkcopy.ColumnMappings.Add(dataField.fieldName, dataField.fieldName);
            //}

            //
            // Populate Bulkcopy target (datatable)
            //

            string line = string.Empty;
            using (StreamReader reader = file.OpenText())
            {
                while ((line = reader.ReadLine()) != null)
                {
                    DataRow row = dataTable.NewRow();

                    //
                    // Use proper data types
                    //

                    //foreach (var dataField in dataFields)
                    //{
                    //    if (dataField.sqlDbType == SqlDbType.Int)
                    //        row[dataField.fieldName] = Convert.ToInt32(line.Substring(dataField.fieldStartPos, dataField.fieldLength));
                    //    else
                    //        row[dataField.fieldName] = line.Substring(dataField.fieldStartPos, dataField.fieldLength);
                    //}

                    dataTable.Rows.Add(row);
                }
            }

            //
            // Push datatable to server
            //

            bulkcopy.WriteToServer(dataTable);
        }
    }
}
}
}

Answer 2

在 SQL Server 中创建类型并将表作为参数传递。请往下看

How to pass table value parameters to stored procedure from .net code

https://www.c-sharpcorner.com/UploadFile/ff2f08/table-value-parameter-use-with-C-Sharp/

一旦将表变量传递给存储过程，就可以在一个插入选择语句中插入数据。