雪花环境下sql查询数据校验

Question

我有两个数据库说雪花的发展和质量。目前我分别在两个数据库表上运行 sql 查询。并单独存储输出。 然后使用 excel 对这两个文件进行手动数据验证。 是通过任何脚本或 sql 查询自动化之类的任何方式。用于进行验证。

谢谢

Answer 1

根据您的要求，有多种验证方法。假设您要验证存在的表employee 的列数据是两个不同的数据库，您可以创建一个SQL 脚本并运行它来验证是否存在任何差异。这是一个示例，您可以进一步扩展它或根据您的用例进行构建。

// Comparing data of tables in different databases
use role accountadmin;

create or replace database exampledb1;

use database exampledb1;
create or replace table employee

(   id int,
    FirstName  varchar(20),
    LastName   varchar(20),
    email      varchar(50) 
);

insert into employee values 
(1,'Sarah', 'Janes','sarah.janes@hotmail.com'),
(2,'Richard', 'Hays','richard.hays@gmail.com'),
(3,'Maria', 'Khan','maria.khan@wow.com');

create or replace database exampledb2;
use database exampledb2;
create or replace table employee

(   id  int,
    FirstName  varchar(20),
    LastName   varchar(20),
    email      varchar(50) 
);

insert into employee values 
(1,'Sarah', 'Janes','sarah.janes@hotmail.com'),
(2,'Richard1', 'Hays','richard.hays@gmail.com'),
(3,'Maria', 'Khan','maria.khan@wow.com');
 
 // Check the variance by comparing hash of each row
 // You should have correct access to both databases, scheam and underlying tables
 
 with qadata as
 (
 select id,hash(FirstName||LastName||email) as r1, FirstName, LastName,Email
 from exampledb2.public.employee as r1
 ),
 proddata as
 (
 select id,hash(t.FirstName||t.LastName||t.email) as r1, FirstName, LastName,Email
 from exampledb1.public.employee as t
  )
  select prod.id,
         case when prod.r1 = qa.r1 then 'Matched' else 'Variance' end as CheckStatus,
         prod.FirstName, 
         prod.LastName,
         prod.Email, 
         qa.FirstName, 
         qa.LastName,
         qa.Email
  from qadata qa 
  inner join proddata prod on qa.id=prod.id;

这是输出列 CheckStatus 将识别行数据是否匹配或是否存在差异，如第 2 行所示。 Comparison results