【问题标题】:Sequelize - How to insert csv data into multiple tables in mysql using sequelizeSequelize - 如何使用 sequelize 将 csv 数据插入 mysql 中的多个表中
【发布时间】:2020-07-31 10:28:51
【问题描述】:

我在用什么

node.js, express, sequelize 6.3.3, fast-csv 4.3.1 and mysql database.

我有什么

我有一个包含以下标题和数据的 CSV 文件。

no, name, email, township, father_name, father_phone, mother_name, mother_phone, parent_township

在这个 CSV 中,我有两种类型的数据:studentsparents

在我的MySQL 数据库中,我有三个表:

students, parents and townships 

学生桌:

id, name, email, township_id

父母表:

id, student_id, father_name, father_phone, mother_name, mother_phone, parent_township_id

乡镇表:

id, name

我做了什么

我用fast-csv npm 包读取了CSV,代码如下。

let csvData = [];

fs.createReadStream(req.file.path)
    .pipe(
      csv.parse({
        ignoreEmpty: true,
        trim: true,
        skipLines: 6,
      })
    )
    .on("error", (error) => console.error(error))
    .on("data", (row) => {
      csvData.push(getHeaders(_.compact(row), csvStudentDataEntryFields));
    })
    .on("end", () => {
      fs.unlinkSync(req.file.path);

      const data = csvData.map((data) => {
        const student = _.pick(data, studentFields);
        const parent = _.pick(data, parentFields);
        return [student, parent];
      });

      return res.status(201).send({
        status: "success",
        data,
      });
    });

我得到了什么

使用上面的代码,我得到了data 和以下值。

[
    [ // row 1 from CSV
        {
            "name": "John",
            "email": "john@gmail.com",
            "township": "NYC",
        },
        {
            "fatherName": "Smith",
            "fatherPhone": "9111111111",
            "motherName": "Mary",
            "motherPhone": "9111111111",
            "parentTownship": "NYC"
         }
    ],
    [ // row 2 from CSV
        {
            "name": "Cutter",
            "email": "cutter@gmail.com",
            "township": "NYC",
        },
        {
            "fatherName": "Laseli",
            "fatherPhone": "9111111111",
            "motherName": "Mary",
            "motherPhone": "9111111111",
            "parentTownship": "NYC"
        }
    ]
]

我想要什么

我想将data 中的row 1row 2 存储到数据库中的相应表中。

问题

我认为我需要将那些township 文本数据替换为真实ID,因为我有如上所述的外键。

我怎样才能实现它?我想在数据库级别做到这一点。我不想在单独的 js 模块中查找该乡镇名称的 id

更新

学生模型

class Student extends Model {
    static associate(models) {
      Student.belongsTo(models.Township, {
        foreignKey: "townshipId",
        as: "township",
        targetKey: "townshipId",
      });

     Student.hasOne(models.Parent, {
        foreignKey: "studentId",
        as: "parent",
        sourceKey: "studentId",
      });
    }
  }

父模型

class Parent extends Model {
    static associate(models) {
      Parent.belongsTo(models.Student, {
        foreignKey: "studentId",
        as: "student",
        targetKey: "studentId",
      });

      Parent.belongsTo(models.Township, {
        foreignKey: "parentTownshipId",
        as: "township",
        targetKey: "townshipId",
      });
    }
  }

乡镇模式

class Township extends Model {
    static associate(models) {
      Township.hasMany(models.Student, {
        foreignKey: "townshipId",
        as: "township",
        sourceKey: "townshipId",
      });

      Township.hasMany(models.Parent, {
        foreignKey: "townshipId",
        as: "parentTownship",
        sourceKey: "townshipId",
      });
    }
  }

更新:

在我的控制器中,

let std = await models.Student.create(
    {
        nameEn: "John",
        email: "john@gmail.com",
        townshipId: 1,
        parent: [{ ...d[1], parentTownshipId: 1 }],
    },
    { include: ["parent"] }
);

【问题讨论】:

    标签: mysql node.js csv sequelize.js fast-csv


    【解决方案1】:

    我认为您可以将所有关系插入在一起。

    这是来自sequelize文档

    const amidala = await User.create({
      username: 'p4dm3',
      points: 1000,
      profiles: [{
        name: 'Queen',
        User_Profile: {
          selfGranted: true
        }
      }]
    }, {
      include: Profile
    });
    
    const result = await User.findOne({
      where: { username: 'p4dm3' },
      include: Profile
    });
    
    console.log(result);
    

    设置源密钥

     const Foo = sequelize.define('foo', {
      name: { type: DataTypes.TEXT, unique: true }
    }, { timestamps: false });
    const Bar = sequelize.define('bar', {
      title: { type: DataTypes.TEXT, unique: true }
    }, { timestamps: false });
    const Baz = sequelize.define('baz', { summary: DataTypes.TEXT }, { timestamps: false });
    Foo.hasOne(Bar, { sourceKey: 'name', foreignKey: 'fooName' });
    Bar.hasMany(Baz, { sourceKey: 'title', foreignKey: 'barTitle' });
    // [...]
    await Bar.setFoo("Foo's Name Here");
    await Baz.addBar("Bar's Title Here");
    

    参考https://sequelize.org/master/manual/advanced-many-to-many.html#using-one-to-many-relationships-instead

    【讨论】:

    • 非常感谢您的回答。但我收到以下错误。 SequelizeDatabaseError: Incorrect integer value: 'Falam' for column 'township_id'
    • 这可能是关联问题。
    • 我已经更新并在我的问题中添加了associations。请看这个。
    • 尝试添加目标键 source.belongsTo(target, {foreignKey: 'fkname', targetKey: 'id'});
    • 谢谢。我清楚地看到您首先检查传入NYC 是否存在于townships 表中的第一种方法,如果存在,请抓住id 并将其替换为NYC。我认为upset 不行,因为如果NYC 不在townships 中,它将插入。它可能允许插入虚假数据。对不起,我不明白你在最后评论中推荐的方式。 (会有单独的路由到 CRUD townships 资源。)
    猜你喜欢
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2020-08-20
    • 2020-04-30
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 2018-11-14
    相关资源
    最近更新 更多