【问题标题】:Why does my binary response add spaces between characters?为什么我的二进制响应在字符之间添加空格?
【发布时间】:2021-08-12 01:33:05
【问题描述】:

我正在使用 Node、Express、KNEX 和繁琐的驱动程序与 SQL Server 交互。我执行一个返回varbinary(max) 响应的存储过程。我无法更改此存储过程。

当我捕获响应并尝试将其转换为将发送回浏览器的字符串时,预期的响应在每个字符之间有大量空白字符。

我做错了什么?

        // const { StringDecoder } = require('string_decoder');
        // const decoder = new StringDecoder('utf8');

        const response = await Webservice.spWebservice(params)  //exec stored procedure
        let { Response_Binary } = response;

        console.log(Response_Binary)

        const buff = Buffer.from(Response_Binary)
        console.log(buff);
        console.log(buff.toString());


        // console.log(decoder.write(buff));
        // console.log(decoder.end(buff));

实际 SQL Server 响应(通过 KNEX):

<Buffer 3c 00 3f 00 78 00 6d 00 6c 00 20 00 76 00 65 00 72 00 73 00 69 00 6f 00 6e 00 3d 00 22 00 31 00 2e 00 30 00 22 00 20 00 65 00 6e 00 63 00 6f 00 64 00 ... 764 more bytes>

Buffer.from()

<Buffer 3c 00 3f 00 78 00 6d 00 6c 00 20 00 76 00 65 00 72 00 73 00 69 00 6f 00 6e 00 3d 00 22 00 31 00 2e 00 30 00 22 00 20 00 65 00 6e 00 63 00 6f 00 64 00 ... 764 more bytes>

toString()

< ? x m l v e r s i o n = " 1 . 0 " e n c o d i n g = " U T F - 1 6 " ? > < x m l > < s t a t u s > < r e s u l t > S u c c e s s < / r e s u l t > < m e s s a g e > < / m e s s a g e > < / s t a t u s > < d a t a > < p p A c t i o n > i n i t < / A p p A c t i o n > < U s e r I n f o > < U s e r > < / U s e r > < A d d r e s s > : : 1 < / A d d r e s s > < U s e r A g e n t > M o z i l l a / 5 . 0 ( W i n d o w s N T 0 . 0 ; W i n 6 4 ; x 6 4 ) A p p l e W e b K i t / 5 3 7 . 3 6 ( K H T M L , l i k e G e c k o ) C h r o m e / 9 2 . 0 . 4 5 1 5 . 1 3 1 S a f a r i / 5 3 7 . 3 6 < / U s e r A g e n t > < C a l l e r > < C a l l e r > < R e f e r r e r > < / R e f e r r e r > < O p t i o n s > < / O p t i o n s > < / U s e r I n f o > < / d a t a > < / x m l > `

我尝试过的事情[更新]:

        let convertedBuffer = Buffer.from(Response_Binary, 'binary').toString();
        console.log(convertedBuffer) // extra spaces between characters

        convertedBuffer = Buffer.from(Response_Binary, 'ascii').toString();
        console.log(convertedBuffer) // extra spaces between characters
        
        convertedBuffer = Buffer.from(Response_Binary, 'utf-8').toString();
        console.log(convertedBuffer) // extra spaces between characters
        
        convertedBuffer = Buffer.from(Response_Binary, 'utf8').toString();
        console.log(convertedBuffer) // extra spaces between characters
        
        convertedBuffer = Buffer.from(Response_Binary, 'base64').toString();
        console.log(convertedBuffer) // extra spaces between characters
        
        convertedBuffer = Buffer.from(Response_Binary, 'utf16le').toString();
        console.log(convertedBuffer) // extra spaces between characters


        console.log(`Is Response_Binary from KNEX/MSSQL actually a buffer? ===>>> `)
        console.log(Buffer.isBuffer(Response_Binary)); // returns true;


        console.log(Response_Binary.toString('ascii'));   // extra spaces between characters
        console.log(Response_Binary.toString('utf-8'));   // extra spaces between characters
        console.log(Response_Binary.toString('utf8'));    // extra spaces between characters
        console.log(Response_Binary.toString('base64'));  // not for humans
        console.log(Response_Binary.toString('utf16le')); // CORRECT ANSWER!!!!!


Response_Binary.toString('utf16le') 返回:

<?xml version="1.0" encoding="UTF-16"?><xml><status><result>Success</result><message></message></status><data><AppAction>init</AppAction><UserInfo><User>redacted</User><Address>::1</Address><UserAgent>Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/92.0.4515.131 Safari/537.36</UserAgent><Caller></Caller><Referrer></Referrer><Options></Options></UserInfo></data></xml>

【问题讨论】:

  • 有人可能会问为什么要将字符串数据存储为varbinary
  • 我正在尝试从 Java Servlet (Tomcat) 过渡到 Node.js。存储过程的设计者,我的老板,将二进制从 SQL 发送到 Tomcat,所以我想学习如何正确解码二进制响应。他报告说,来自 SQL 的文本非常慢,而二进制则更快。 20 年世界发生了很大的变化,我也许可以和他谈判,但我想尽量保持主存储过程不被修改。
  • 似乎您的二进制列中存储了 unicode 字符。您写道您“玩弄了编码-究竟是怎么回事?默认是utf8,这似乎不正确。您尝试了哪些编码值?
  • 对我来说这看起来像 UTF-16 little-endian。
  • 伙计们:我以您的评论为灵感。我已更新“我尝试过的内容”部分。事实证明,我个人可以让它工作的唯一方法是使用这个:Response_Binary.toString('utf16le')。我在微软的网站上查看了varbinary(max) 的编码方式,但找不到答案。所以,我想我真正的问题一定是:我应该如何知道varbinary(max) 的编码方式?

标签: node.js sql-server express binary knex.js


【解决方案1】:

回答我自己的问题,但我会接受任何更好的解释:

我推测 TSQL 对 vabinary(max) 的默认字符编码是 utf16le

KNEX 将 TSQL 的二进制响应作为准备就绪的缓冲区返回。 Buffer.isBuffer(Response_Binary) //it's a buffer!

这意味着我可以直接在Response_Binary上调用Buffer的.toString()

Response_Binary.toString('utf16le')

我想指出,我正在向浏览器发送响应,上面的方法很慢。翻阅我的笔记,我发现res.end() 不仅速度更快,而且还能“转换”它:

慢*:

     strXML = Response_Binary.toString('utf16le');
     return res.send(strXML);

快速 *:

     return res.end(Response_Binary, 'binary');

来源灵感:https://spin.atomicobject.com/2019/06/17/knex-postgresql-blobs/

===== [更新] 慢与快:=====

经过进一步测试,我无法区分这两种方法之间的明显区别。

【讨论】:

    猜你喜欢
    • 2012-06-24
    • 2017-03-11
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    • 1970-01-01
    相关资源
    最近更新 更多