LOAD DATA INFILE - MySQL - 查找和替换（删除）一些关键字 - 在导入之前

Question

我使用 LOAD DATA LOCAL INFILE 并导入了一个大的 TXT 文件，效果很好。它使用以下方法导入mydata Mysql 表中的数据

LOAD DATA LOCAL INFILE 'Base.txt' INTO TABLE mydata CHARACTER 
SET
   latin1 FIELDS TERMINATED BY '\t' ENCLOSED BY '"' LINES TERMINATED BY '\n' IGNORE 1 LINES (keywords , fixedprice, quantity, description, condition1, image1url, merchantcategory, leadtime, Manufacturer, Model_Number, AZ_Code, Volts, Watts, ColorTemp, Shape, Life, Base 

)

在导入数据之前是否可以将Keywords和description列中的stripoutwords替换并删除。所以检查select name from stripoutwords EXISTS in the Keywordsand thedescription column, If EXISTs then remove those keywords from the import

两个表的架构都在这里：

CREATE TABLE `stripoutwords` (
  `id` int(11) NOT NULL,
  `name` text NOT NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;

CREATE TABLE `mydata` (
  `keywords` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
  `fixedprice` float(6,2) DEFAULT NULL,
  `quantity` varchar(255) DEFAULT NULL,
  `description` text CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci,
  `condition1` varchar(400) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL,
  `image1url` text CHARACTER SET latin1 COLLATE latin1_general_ci,
  `merchantcategory` text CHARACTER SET latin1 COLLATE latin1_general_ci,
  `leadtime` varchar(400) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL,
  `Manufacturer` varchar(400) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL,
  `Model_Number` varchar(400) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL,
  `AZ_Code` varchar(400) CHARACTER SET latin1 COLLATE latin1_general_ci NOT NULL,
  `Volts` varchar(400) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL,
  `Watts` varchar(400) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL,
  `ColorTemp` varchar(400) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL,
  `Shape` varchar(400) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL,
  `Life` varchar(400) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL,
  `Base` varchar(255) CHARACTER SET latin1 COLLATE latin1_general_ci DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

Answer 1

您必须使用函数对数据进行预处理

你会写一个这样的 sql 查询

LOAD DATA LOCAL INFILE 'Base.txt' INTO TABLE mydata CHARACTER SET   latin1 
FIELDS TERMINATED BY '\t' 
ENCLOSED BY '"' 
LINES TERMINATED BY '\n' 
IGNORE 1 LINES 
(@keywords , fixedprice, quantity, @description, condition1, image1url, merchantcategory, leadtime, Manufacturer, Model_Number, AZ_Code, Volts, Watts, ColorTemp, Shape, Life, Base 
)
SET keywords = Removetext(@keywords),description = Removetext(@description)

具有类似的功能

CREATE function Removetext (
     texttofilter text
) RETURNS TEXT DETERMINISTIC
BEGIN
    DECLARE finished INTEGER DEFAULT 0;
    DECLARE stopwords TEXT DEFAULT "";

    -- declare cursor for employee email
    DEClARE curlist 
        CURSOR FOR 
            SELECT `name` FROM `stripoutwords`;

    -- declare NOT FOUND handler
    DECLARE CONTINUE HANDLER 
        FOR NOT FOUND SET finished = 1;

    OPEN curlist;

    getwordlist: LOOP
        FETCH curlist INTO stopwords;
        IF finished = 1 THEN 
            LEAVE getwordlist;
        END IF;
        -- build email list
        SET texttofilter = REPLACE (texttofilter,stopwords,"");
    END LOOP getwordlist;
    CLOSE curlist;
RETURN texttofilter;
END

这是一个函数如何工作的例子https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=28cf0a3ba53afcffcb12740e8d07b912