【发布时间】：2019-04-26 23:17:08
【问题描述】：
我正在测试 Apache Ignite 2.7 Linux ODBC 驱动程序，但由于某种原因它无法正常工作。
当我运行 SQL 语句（如 SELECT Id FROM MyTable）时，该值是“乱码”，为 Id 列返回一个类似于 8\00\05\07\09\0d\09\08\0-\0 的值。不过，它与我期望的值相似，即80579d98-9010-4610-b12e-ed33ed7d3c62。我不知道这些是 ASCII 空字符 (\0) 还是什么，但有些不对劲，我想不通。
当我使用 SQLLine 工具时，这些值不会“乱码”。我还安装了 Windows ODBC 驱动程序，效果很好，但我需要 Linux ODBC 驱动程序。
我使用以下库构建了驱动程序：
gcc 6.3.0 20170516 (Debian 6.3.0-18+deb9u1)
GNU Make 4.1
libtoolize 2.4.6
aclocal 1.15
autoheader 2.69
automake 1.15
autoreconf 2.69
unixodbc 2.3.4-1
libssl 1.0.2r-1~deb9u1
这是 ignite-odbc-install.ini 的内容：
# cat /src/apache-ignite-2.7.0-bin/platforms/cpp/odbc/install/ignite-odbc-install.ini
[Apache Ignite]
Description=Apache Ignite
Driver=/usr/local/lib/libignite-odbc.so
Setup=/usr/local/lib/libignite-odbc.so
DriverODBCVer=03.00
FileUsage=0
这是运行 odbcinst 的结果：
# odbcinst -i -d -f /src/apache-ignite-2.7.0-bin/platforms/cpp/odbc/install/ignite-odbc-install.ini
odbcinst: Driver installed. Usage count increased to 1.
Target directory is /etc
这是 /usr/local/lib 文件夹的内容：
# cd /usr/local/lib
# ls
libignite-binary-2.7.0.33575.so.0 libignite-common-2.7.0.33575.so.0 libignite-odbc-2.7.0.33575.so.0 libignite-thin-client-2.7.0.33575.so.0
libignite-binary-2.7.0.33575.so.0.0.0 libignite-common-2.7.0.33575.so.0.0.0 libignite-odbc-2.7.0.33575.so.0.0.0 libignite-thin-client-2.7.0.33575.so.0.0.0
libignite-binary.a libignite-common.a libignite-odbc.a libignite-thin-client.a
libignite-binary.la libignite-common.la libignite-odbc.la libignite-thin-client.la
libignite-binary.so libignite-common.so libignite-odbc.so libignite-thin-client.so
这是 ldd 命令的结果：
# ldd /usr/local/lib/libignite-odbc.so
linux-vdso.so.1 (0x00007fffa46d0000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f18d554c000)
libignite-binary-2.7.0.33575.so.0 => /usr/local/lib/libignite-binary-2.7.0.33575.so.0 (0x00007f18d531d000)
libodbcinst.so.2 => /usr/lib/x86_64-linux-gnu/libodbcinst.so.2 (0x00007f18d5108000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f18d4d86000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f18d4a82000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f18d46e3000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f18d44cc000)
/lib64/ld-linux-x86-64.so.2 (0x00007f18d59f8000)
libignite-common-2.7.0.33575.so.0 => /usr/local/lib/libignite-common-2.7.0.33575.so.0 (0x00007f18d42b5000)
libltdl.so.7 => /usr/lib/x86_64-linux-gnu/libltdl.so.7 (0x00007f18d40ab000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f18d3e8e000)
更新 1
我从 .Net Core 2.2 C# 控制台程序运行它，在 Docker Linux 环境中只有 System.Data.Odbc 4.5.0 NuGet 参考。
IgniteGarledDataReproducer.csproj
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<OutputType>Exe</OutputType>
<TargetFramework>netcoreapp2.2</TargetFramework>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="System.Data.Odbc" Version="4.5.0" />
</ItemGroup>
</Project>
Program.cs
using System;
using System.Data.Odbc;
namespace IgniteGarbledDataReproducer {
internal class Program {
private static void Main(string[] args) {
Console.WriteLine("START");
var connectionString = Environment.GetEnvironmentVariable("IGNITE_CONNECTION_STRING");
Console.WriteLine(connectionString);
using (var conn = new OdbcConnection(connectionString)) {
conn.Open();
Console.WriteLine("Connection Opened");
using (var cmd = conn.CreateCommand()) {
cmd.CommandText = "DROP TABLE IF EXISTS MyTable;";
cmd.ExecuteNonQuery();
}
Console.WriteLine("Table Dropped (if it existed)");
using (var cmd = conn.CreateCommand()) {
cmd.CommandText = "CREATE TABLE MyTable (Id varchar(36) not null, FirstName varchar(255), LastName varchar(255), PRIMARY KEY (Id));";
cmd.ExecuteNonQuery();
}
Console.WriteLine("Table Created");
using (var cmd = conn.CreateCommand()) {
cmd.CommandText = "INSERT INTO MyTable (Id) VALUES ('80579d98-9010-4610-b12e-ed33ed7d3c62');";
cmd.ExecuteNonQuery();
}
Console.WriteLine("Data Inserted");
using (var cmd = conn.CreateCommand()) {
cmd.CommandText = "SELECT Id FROM MyTable;";
using (var rdr = cmd.ExecuteReader()) {
while (rdr.Read())
Console.WriteLine($"Id = '{rdr["Id"]}'");
}
}
Console.WriteLine("Data Returned");
}
Console.WriteLine("DONE");
}
}
}
Dockerfile
FROM microsoft/dotnet:2.2-runtime AS base
WORKDIR /app
WORKDIR /src
# Install pre-requisites for building and installing Ignite ODBC Driver
RUN apt-get update \
&& apt-get install -y gcc g++ make automake autotools-dev libtool m4 libssl1.0-dev unixodbc-dev unixodbc-bin unzip \
&& apt-get clean -y
# Download and unzip Ignite binary release
#ADD http://apache.mirrors.hoobly.com/ignite/2.7.0/apache-ignite-2.7.0-bin.zip /src/
COPY apache-ignite-2.7.0-bin.zip .
RUN unzip -q /src/apache-ignite-2.7.0-bin.zip \
&& rm /src/apache-ignite-2.7.0-bin.zip
# Build Ignite ODBC Driver
ARG IGNITE_HOME=/src/apache-ignite-2.7.0-bin
WORKDIR /src/apache-ignite-2.7.0-bin/platforms/cpp
RUN libtoolize \
&& aclocal \
&& autoheader \
&& automake --add-missing \
&& autoreconf \
&& ./configure --enable-odbc --disable-core --disable-node \
&& make \
&& make install
# Install Ignite ODBC Driver
RUN apt-get update \
&& apt-get install -y unixodbc \
&& apt-get clean -y \
&& odbcinst -i -d -f /src/apache-ignite-2.7.0-bin/platforms/cpp/odbc/install/ignite-odbc-install.ini
FROM microsoft/dotnet:2.2-sdk AS build
WORKDIR /src
COPY IgniteGarbledDataReproducer/IgniteGarbledDataReproducer.csproj IgniteGarbledDataReproducer/
RUN dotnet restore IgniteGarbledDataReproducer/IgniteGarbledDataReproducer.csproj
COPY . .
WORKDIR /src/IgniteGarbledDataReproducer
RUN dotnet build IgniteGarbledDataReproducer.csproj -c Release -o /app
FROM build AS publish
RUN dotnet publish IgniteGarbledDataReproducer.csproj -c Release -o /app
FROM base AS final
WORKDIR /app
COPY --from=publish /app .
ENTRYPOINT ["dotnet", "IgniteGarbledDataReproducer.dll"]
docker-compose.yml
version: '3.4'
services:
ignitegarbleddatareproducer:
image: ${DOCKER_REGISTRY-}ignite-garbled-data-reproducer
build:
context: .
dockerfile: IgniteGarbledDataReproducer/Dockerfile
docker-compose-override.yml
version: '3.4'
services:
ignitegarbleddatareproducer:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- IGNITE_CONNECTION_STRING=DRIVER={Apache Ignite};ADDRESS=my-ignite-3e8b8cb9a4e192af.elb.us-east-1.amazonaws.com:10800;SSL_MODE=disable;
构建
docker build . -t ignitegarbleddatareproducer -f IgniteGarbledDataReproducer/Dockerfile
执行
PS C:\IgniteGarbledDataReproducer> docker-compose up
Starting ignitegarbleddatareproducer_ignitegarbleddatareproducer_1 ... done
Attaching to ignitegarbleddatareproducer_ignitegarbleddatareproducer_1
ignitegarbleddatareproducer_1 | START
ignitegarbleddatareproducer_1 | DRIVER={Apache Ignite};ADDRESS=my-ignite-3e8b8cb9a4e192af.elb.us-east-1.amazonaws.com:10800;SSL_MODE=disable;
ignitegarbleddatareproducer_1 | Connection Opened
ignitegarbleddatareproducer_1 | DROP TABLE IF EXISTS MyTable;
ignitegarbleddatareproducer_1 | Table Dropped (if it existed)
ignitegarbleddatareproducer_1 | CREATE TABLE MyTable (Id varchar(36) not null, FirstName varchar(255), LastName varchar(255), PRIMARY KEY (Id));
ignitegarbleddatareproducer_1 | Table Created
ignitegarbleddatareproducer_1 | INSERT INTO MyTable (Id) VALUES ('80579d98-9010-4610-b12e-ed33ed7d3c62');
ignitegarbleddatareproducer_1 | Data Inserted
ignitegarbleddatareproducer_1 | SELECT Id FROM MyTable;
ignitegarbleddatareproducer_1 | Id = '8 0 5 7 9 d 9 8 - '
ignitegarbleddatareproducer_1 | Data Returned
ignitegarbleddatareproducer_1 | DONE
ignitegarbleddatareproducer_ignitegarbleddatareproducer_1 exited with code 0
更新 2
我通过将以下内容添加到/src/apache-ignite-2.7.0-bin/platforms/cpp/odbc/install/ignite-odbc-install.ini 变成了on ODBC logging，您可以得到the logs here。
[ODBC]
Trace=yes
TraceFile=/logs/odbc.log
为方便起见，SELECT Id From MyTable; 语句后的日志结尾如下。
[ODBC][1][1556564832.827181][SQLExecDirectW.c][177]
Entry:
Statement = 0xf27a80
SQL = [SELECT Id FROM MyTable;][length = 23 (SQL_NTS)]
[ODBC][1][1556564832.847882][SQLExecDirectW.c][445]
Exit:[SQL_SUCCESS]
[ODBC][1][1556564832.852889][SQLRowCount.c][173]
Entry:
Statement = 0xf27a80
Row Count = 0x7ffd7da98f00
[ODBC][1][1556564832.856831][SQLRowCount.c][247]
Exit:[SQL_SUCCESS]
Row Count = 0x7ffd7da98f00 -> 0
[ODBC][1][1556564832.860335][SQLNumResultCols.c][156]
Entry:
Statement = 0xf27a80
Column Count = 0x7ffd7da98f50
[ODBC][1][1556564832.864944][SQLNumResultCols.c][251]
Exit:[SQL_SUCCESS]
Count = 0x7ffd7da98f50 -> 1
[ODBC][1][1556564832.869909][SQLFetch.c][162]
Entry:
Statement = 0xf27a80
[ODBC][1][1556564832.884876][SQLFetch.c][351]
Exit:[SQL_SUCCESS]
[ODBC][1][1556564832.894173][SQLColAttributeW.c][156]
Entry:
Statement = 0xf27a80
Column Number = 1
Field Identifier = SQL_DESC_NAME
Character Attr = 0xf2e190
Buffer Length = 4096
String Length = 0x7ffd7da98f20
Numeric Attribute = 0x7ffd7da98ee0
[ODBC][1][1556564832.898409][SQLColAttributeW.c][523]
Exit:[SQL_SUCCESS]
[ODBC][1][1556564832.910800][SQLColAttributeW.c][156]
Entry:
Statement = 0xf27a80
Column Number = 1
Field Identifier = SQL_DESC_CONCISE_TYPE
Character Attr = 0xf2e190
Buffer Length = 4096
String Length = 0x7ffd7da98f40
Numeric Attribute = 0x7ffd7da98f00
[ODBC][1][1556564832.914551][SQLColAttributeW.c][523]
Exit:[SQL_SUCCESS]
[ODBC][1][1556564832.925913][SQLGetData.c][237]
Entry:
Statement = 0xf27a80
Column Number = 1
Target Type = -8 SQL_WCHAR
Buffer Length = 4094
Target Value = 0xf2e190
StrLen Or Ind = 0x7ffd7da98ee0
[ODBC][1][1556564832.929959][SQLGetData.c][534]
Exit:[SQL_SUCCESS]
Buffer = [8](unicode)
Strlen Or Ind = 0x7ffd7da98ee0 -> 36
[ODBC][1][1556564832.934835][SQLFetch.c][162]
Entry:
Statement = 0xf27a80
[ODBC][1][1556564832.938930][SQLFetch.c][351]
Exit:[SQL_NO_DATA]
[ODBC][1][1556564832.943970][SQLMoreResults.c][162]
Entry:
Statement = 0xf27a80
[ODBC][1][1556564832.960311][SQLMoreResults.c][345]
Exit:[SQL_NO_DATA]
[ODBC][1][1556564832.965960][SQLFreeStmt.c][144]
Entry:
Statement = 0xf27a80
Option = 0
[ODBC][1][1556564832.970025][SQLFreeStmt.c][266]
Exit:[SQL_SUCCESS]
[ODBC][1][1556564832.975135][SQLFreeHandle.c][381]
Entry:
Handle Type = 3
Input Handle = 0xf27a80
[ODBC][1][1556564832.981240][SQLFreeHandle.c][494]
Exit:[SQL_SUCCESS]
[ODBC][1][1556564832.991337][SQLDisconnect.c][208]
Entry:
Connection = 0xf0a6f0
[ODBC][1][1556564832.996778][SQLDisconnect.c][379]
Exit:[SQL_SUCCESS]
[ODBC][1][1556564833.001505][SQLFreeHandle.c][284]
Entry:
Handle Type = 2
Input Handle = 0xf0a6f0
[ODBC][1][1556564833.006038][SQLFreeHandle.c][333]
Exit:[SQL_SUCCESS]
更新 3
我有一种预感，它可能与 UTF-16 有关，因为 similar question 中的字符用 \0 分隔。
也许与 ODBC 日志中这一行中的 UNICODE 设置有关？
[ODBC][1][1556564832.214691][SQLDriverConnectW.c][290]
Entry:
Connection = 0xf0a6f0
Window Hdl = (nil)
Str In = [DRIVER={Apache Ignite};ADDRESS=my-ignite-3e8b8cb9a4e192af.elb.us-east-1.amazonaws.com:10800;SSL_MODE=disable;][length = 115 (SQL_NTS)]
Str Out = (nil)
Str Out Max = 0
Str Out Ptr = 0x7ffd7da98dd0
Completion = 0
UNICODE Using encoding ASCII 'ANSI_X3.4-1968' and UNICODE 'UCS-2LE'
更新 4
我找到了这个document from Microsoft，上面写着
如果ASCII字符编码不是UTF-8，例如：
UNICODE Using encoding ASCII 'ISO8859-1' and UNICODE 'UCS-2LE'那里 安装了多个驱动程序管理器并且您的应用程序是 使用了错误的，或者驱动程序管理器没有正确构建。
我假设我的驱动程序没有正确构建。
而this document 说要使用--enable-iconv --with-iconv-char-enc=UTF8 --with-iconv-ucode-enc=UTF16LE。
更新 5
我还发现这个git issue 与 .Net Core、unixODBC 和 UTF16 相关。
更新 6
我尝试在调用 unixODBC 的 configure 时添加 --enable-iconv --with-iconv-char-enc=UTF8 --with-iconv-ucode-enc=UTF16LE，这将 ODBC 连接更改为 UNICODE Using encoding ASCII 'UTF8' and UNICODE 'UTF16LE'，但无论如何我都得到了相同的错误数据。我也试过--with-iconv-ucode-enc=UNICODE，但没有变化。
这就是我更改 Dockerfile 的方式。我从 apt-get 中删除了 unixodbc 并将以下内容添加到我的 Dockerfile 中
ADD ftp://ftp.unixodbc.org/pub/unixODBC/unixODBC-2.3.7.tar.gz .
RUN gunzip unixODBC*.tar.gz && tar xvf unixODBC*.tar
WORKDIR /src/unixODBC-2.3.7
RUN ./configure --enable-gui=no --enable-drivers=no --enable-iconv --with-iconv-char-enc=UTF8 --with-iconv-ucode-enc=UTF16LE
RUN make && make install
ENV LD_LIBRARY_PATH="/usr/local/lib"
这就是 odbc 日志现在的样子：
[ODBC][7][1556723781.839804][SQLDriverConnectW.c][290]
Entry:
Connection = 0x1ddff70
Window Hdl = (nil)
Str In = [DRIVER={Apache Ignite};ADDRESS=my-ignite-3e8b8cb9a4e192af.elb.us-east-1.amazonaws.com:10800;SSL_MODE=disable;][length = 115 (SQL_NTS)]
Str Out = (nil)
Str Out Max = 0
Str Out Ptr = 0x7ffeca3331a0
Completion = 0
UNICODE Using encoding ASCII 'UTF8' and UNICODE 'UTF16LE'
更新 7
我通过isql 连接并得到了正确的响应，但是当我通过 iusql 连接时，我从我的复制器那里得到了“坏数据”。
更新 8
添加一些 odbc 安装信息：
root@4efdb5ed98b5:/app# odbc_config --cflags
-DHAVE_UNISTD_H -DHAVE_PWD_H -DHAVE_SYS_TYPES_H -DHAVE_LONG_LONG -DSIZEOF_LONG_INT=8 -I/usr/local/include
root@4efdb5ed98b5:/app# odbc_config --ulen
-DSIZEOF_SQLULEN=8
root@4efdb5ed98b5:/app# odbcinst -j
unixODBC 2.3.7
DRIVERS............: /usr/local/etc/odbcinst.ini
SYSTEM DATA SOURCES: /usr/local/etc/odbc.ini
FILE DATA SOURCES..: /usr/local/etc/ODBCDataSources
USER DATA SOURCES..: /root/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8
【问题讨论】：
-
Id 列的类型是什么？以及如何检索数据？请出示代码。
-
@NellieDanielyan 抱歉耽搁了。我不得不创建一个复制器，因为我的代码库太复杂而无法发布。使用重现问题的代码更新了问题。我在该 AWS 地址后面的 ignite 是不可公开访问的，因此您将无法访问它。您需要提供自己的 Ignite 2.7.0 实例。
-
尝试
rdr.GetString(GetOrdinal("Id"))而不是rdr["Id"] -
@NellieDanielyan 我已经尝试过了，但它产生了相同的结果。我认为 Apache Ignite 人员必须修复 ODBC 驱动程序中的问题。我还尝试为每种数据类型插入数据，并尝试取回相同的数据，我注意到 char、varchar、uuid 和 date 数据类型无法正常工作。例如，当我将数据插入 UUID 列时，我会返回一个数据类型实际上是 byte[] 的对象，我必须自己将其转换为字符串，例如 (new System.IO.StreamReader(new System .IO.MemoryStream((byte[])value))).ReadToEnd().