【问题标题】:Storing binary data in QR codes在二维码中存储二进制数据
【发布时间】:2016-06-23 15:38:03
【问题描述】:

我正在尝试将二进制数据存储在二维码中。显然 QR 码确实支持存储原始二进制数据(或 ISO-8859-1 / Latin1)。这是我要编码的内容(十六进制):

d1 50 01 00 00 00 f6 5f 05 2d 8f 0b 40 e2 01

我尝试了以下编码器:

  1. qr.js

  1. Google Charts

  1. qrcode.js

使用zxing.org 解码会产生各种不正确的结果。两个 javascript 产生这个(这是错误的;第一个文本字符应该是 Ñ。

而 Google Charts 会生成这个...

发生了什么事?这些都是正确的吗?真正奇怪的是,如果我对这个序列(至少使用 JS 序列)进行编码,那么它可以正常工作 - 我原以为问题是非 ASCII 字符,但 Ñ (0xd1) 是非 ASCII。

d1 50 01 00 00 00 01 02 03 04 05 06 40 e2 01

有人知道怎么回事吗?

更新

我突然想到尝试使用我找到的基于 ZBar 的扫描仪应用程序来扫描它们。它可以扫描两个 JS 版本(至少它们以 ÑP 开头)。谷歌图表是错误的。所以看起来问题出在 ZXing 上(这太糟糕了——我不会向任何人推荐它)。

更新 2

ZBar 不能处理空字节。 :-(

【问题讨论】:

  • 各种编码都是可能的,只是解释的问题。我猜你会使用binary mode (0100) 所以你的输入显然应该有效,但输出需要你自己编码。像 Binary2Hex..
  • 好吧 qrcode.js 至少声称仅支持 8 位模式。我很确定这只是解码器中的错误(ZXing 完全搞砸了,ZBar 使用以空值结尾的字符串;是的 C)。
  • 我认为,上面的二维码是不可能的。它们是通过那些 javascript 库人工创建的,但不可能从真正的 QR 码库(非 javascript)生成。 00 字节的存在将立即切断,不会生成那些二维码。许多人对它无法读取感到难过,但我说是的,这些二维码一开始就无法创建

标签: qr-code zxing


【解决方案1】:

“发生了什么?这些都是正确的吗?”

除了谷歌图表(只是空的),你的二维码都是正确的。

您可以看到来自 zxing 的二进制数据是您所期望的:

4: Byte mode indicator  
0f: length of 15 byte  
d15001...: your 15 bytes of data  
ec11 is just padding  

问题来自解码。因为大多数解码器会尝试将其解释为文本。但由于它是二进制数据,因此您不应尝试将其作为文本处理。即使您认为可以将其从文本转换为二进制,但正如您所见,这可能会导致无效文本值出现问题。

所以解决方案是使用解码器,它会输出二进制数据,而不是文本数据。

现在关于将二维码二进制数据解释为文本,您说第一个字符应该是“Ñ”,如果将其解释为“ISO-8859-1”,则为真, 根据二维码标准,没有定义ECI模式时应该怎么做。

但实际上,在这种情况下,大多数智能手机 QR 码阅读器会将其解释为 UTF-8(或至少尝试自动检测编码)。

尽管这不是标准,但这已成为普遍做法: 无 ECI 的二进制模式,UTF-8 编码文本。

也许背后的原因是没有人愿意浪费这些宝贵的字节添加指定 UTF-8 的 ECI 模式。实际上,并非所有解码器都支持 ECI。

【讨论】:

    【解决方案2】:

    在二维码中存储二进制数据需要克服两个问题。

    1. ISO-8859-1 不允许 00-1F 和 7F-9F 范围内的字节。如果你 无论如何都需要对这些字节进行编码,引用或编码它们,即使用 引用可打印或 Base-64 编码来避免这些范围。

    2. 由于您尝试将二进制数据存储在 QR 码中,因此您必须 仅依靠您自己的扫描仪来处理此二进制数据。你 不必通过其他软件显示您的二维码中的文本, 就像 zxing.org 上的 web 应用程序,因为大多数 QR 解码器, 包括 zxing.org 使用启发式检测字符 设置使用。这些启发式可能会检测到一个字符集,而不是 ISO-8859-1,因此无法正确显示您的二进制数据。一些 扫描仪使用启发式方法来检测字符集,即使 字符集由 ECI 明确给出。这就是为什么提供 ECI 可能没有多大帮助 - 即使使用 ECI,扫描仪仍然使用启发式算法。

    因此,仅使用 US-ASCII 可打印字符(例如,在将其传递给 QR 码生成器之前以 Base64 编码的二进制数据)是针对启发式的 QR 码最安全的选择。这也将克服另一个复杂性:ISO-8859-1 不是 2000 年发布的早期 QR 码标准 (ISO/IEC 18004:2000) 中的默认编码。该标准确实根据 JIS X 0201(JIS8 也称为 ISO-2022-JP)指定了 8 位拉丁/假名字符集作为 8 位模式的默认编码,而 2005 年发布的更新标准确实将默认设置更改为ISO-8859-1。

    作为 Base-64 的替代方案,您可以使用两个十六进制字符(0-9、AF)对每个字节进行编码,因此,在 QR 码中,您的数据将以字母数字模式编码,而不是 8 位模式.这肯定会禁用所有启发式算法,并且不会产生比 Base-64 更大的 QR 码,因为字母数字模式下的每个字符在 QR 码流中只占用 6 位。

    【讨论】:

    • 您在哪里找到00-1F 和7F-9F 的范围?被拒绝并被 3 个字节替换的字节不在您指定的范围内。例如,这些字节保持不变:42, 03, 24, 6A, 6E, 77。但是这些字节不见了:EE, DB, C1, ...
    • 如果您查看代码页布局en.wikipedia.org/wiki/ISO/IEC_8859-1#Code_page_layout 中的字符映射,您将看到值 00-1F 和 7F-9F 是“未定义”,即不编码任何已定义的字符。 ISO 8859-1 对它所指的“拉丁字母 1 号”进行编码,由 191 个拉丁字母字符组成。
    • 谢谢!赞成的答案。
    【解决方案3】:

    乍一看,qr 格式是不同的。我会比较一下 qr 格式,看看是不是纠错或编码问题或其他问题。

    【讨论】:

    • 这应该是一条评论,因为它没有回答问题。
    • 我很确定 1. 和 3. 都是 8 位模式。 Enc 位似乎是错误的,因此您无法直接读取它们。
    【解决方案4】:

    事实证明 ZXing 只是垃圾,ZBar 对数据做了一些奇怪的事情(例如将其转换为 UTF-8)。我设法让它输出包括空字节在内的原始数据。 Here is a patch 是我找到的最好的 Android ZBar 库,现已合并。

    【讨论】:

      【解决方案5】:

      我使用System.Convert.ToBase64String将提供的样本字节数组转换为Base64编码的字符串,然后使用ZXing创建一个二维码图像。

      接下来我调用 ZXing 从生成的二维码中读回字符串,然后调用System.Convert.FromBase64String 将字符串转换回字节数组。

      我确认数据已成功完成往返。

      【讨论】:

        【解决方案6】:

        更新: 我最近回去并将引用的代码作为一个项目发布在 GitHub 上,供任何想要使用它的人使用。 https://github.com/yurelle/Base45Encoder


        这有点死机,但我刚刚遇到了这个问题,并想出了一个解决方案。

        使用 ZXING 读取 QR 码的问题在于它假定所有 QR Payload 都是字符串。如果你愿意用 ZXING 在 java 中生成二维码,我开发了一个解决方案,可以将二进制有效负载存储在 ZXING 二维码中,存储效率损失仅为 -8%;比 Base64 的 4 倍通胀要好得多。

        它利用基于纯字母字符串的 ZXING 库的内部压缩优化。如果您想要完整的解释,包括数学和单元测试,请查看my other answer。

        但简短的回答是这样的:

        解决方案

        我将它实现为一个自包含的静态实用程序类,所以你所要做的就是调用:

        //Encode
        final byte[] myBinaryData = ...;
        final String encodedStr = BinaryToBase45Encoder.encodeToBase45QrPayload(myBinaryData);
        
        //Decode
        final byte[] decodedBytes = BinaryToBase45Encoder.decodeBase45QrPayload(encodedStr);
        

        或者,您也可以通过 InputStreams 来完成:

        //Encode
        final InputStream in_1 = ... ;
        final String encodedStr = BinaryToBase45Encoder.encodeToBase45QrPayload(in_1);
        
        //Decode
        final InputStream in_2 = ... ;
        final byte[] decodedBytes = BinaryToBase45Encoder.decodeBase45QrPayload(in_2);
        

        这是实现

        import java.io.ByteArrayInputStream;
        import java.io.ByteArrayOutputStream;
        import java.io.IOException;
        import java.io.InputStream;
        import java.lang.reflect.Field;
        import java.util.HashMap;
        import java.util.LinkedList;
        import java.util.Map;
        
        /**
         * For some reason none of the Java QR Code libraries support binary payloads. At least, none that
         * I could find anyway. The commonly suggested workaround for this is to use Base64 encoding.
         * However, this results in a 4x payload size inflation. If your payload is already near the size
         * limit of QR codes, this is not possible.
         *
         * This class implements an encoder which takes advantage of a built-in compression optimization
         * of the ZXING QR Code library, to enable the storage of Binary data into a QR Code, with a
         * storage efficiency loss of only -8%.
         *
         * The built-in optimization is this: ZXING will automatically detect if your String payload is
         * purely AlphaNumeric (by their own definition), and if so, it will automatically compress 2
         * AlphaNumeric characters into 11 bits.
         *
         *
         * ----------------------
         *
         *
         * The included ALPHANUMERIC_TABLE is the conversion table used by the ZXING library as a reverse
         * index for determining if a given input data should be classified as alphanumeric.
         *
         * See:
         *
         *      com.google.zxing.qrcode.encoder.Encoder.chooseMode(String content, String encoding)
         *
         * which scans through the input string one character at a time and passes them to:
         *
         *      getAlphanumericCode(int code)
         *
         * in the same class, which uses that character as a numeric index into the the
         * ALPHANUMERIC_TABLE.
         *
         * If you examine the values, you'll notice that it ignores / disqualifies certain values, and
         * effectively converts the input into base 45 (0 -> 44; -1 is interpreted by the calling code
         * to mean a failure). This is confirmed in the function:
         *
         *      appendAlphanumericBytes(CharSequence content, BitArray bits)
         *
         * where they pack 2 of these base 45 digits into 11 bits. This presents us with an opportunity.
         * If we can take our data, and convert it into a compatible base 45 alphanumeric representation,
         * then the QR Encoder will automatically pack that data into sub-byte chunks.
         *
         * 2 digits in base 45 is 2,025 possible values. 11 bits has a maximum storage capacity of 2,048
         * possible states. This is only a loss of 1.1% in storage efficiency behind raw binary.
         *
         *      45 ^ 2 = 2,025
         *      2 ^ 11 = 2,048
         *      2,048 - 2,025 = 23
         *      23 / 2,048 = 0.01123046875 = 1.123%
         *
         * However, this is the ideal / theoretical efficiency. This implementation processes data in
         * chunks, using a Long as a computational buffer. However, since Java Long's are singed, we
         * can only use the lower 7 bytes. The conversion code requires continuously positive values;
         * using the highest 8th byte would contaminate the sign bit and randomly produce negative
         * values.
         *
         *
         * Real-World Test:
         *
         * Using a 7 byte Long to encode a 2KB buffer of random bytes, we get the following results.
         *
         *      Raw Binary Size:        2,048
         *      Encoded String Size:    3,218
         *      QR Code Alphanum Size:  2,213 (after the QR Code compresses 2 base45 digits to 11 bits)
         *
         * This is a real-world storage efficiency loss of only 8%.
         *
         *      2,213 - 2,048 = 165
         *      165 / 2,048 = 0.08056640625 = 8.0566%
         */
        public class BinaryToBase45Encoder {
            public final static int[] ALPHANUMERIC_TABLE;
        
            /*
             * You could probably just copy & paste the array literal from the ZXING source code; it's only
             * an array definition. But I was unsure of the licensing issues with posting it on the internet,
             * so I did it this way.
             */
            static {
                final Field SOURCE_ALPHANUMERIC_TABLE;
                int[] tmp;
        
                //Copy lookup table from ZXING Encoder class
                try {
                    SOURCE_ALPHANUMERIC_TABLE = com.google.zxing.qrcode.encoder.Encoder.class.getDeclaredField("ALPHANUMERIC_TABLE");
                    SOURCE_ALPHANUMERIC_TABLE.setAccessible(true);
                    tmp = (int[]) SOURCE_ALPHANUMERIC_TABLE.get(null);
                } catch (NoSuchFieldException e) {
                    e.printStackTrace();//Shouldn't happen
                    tmp = null;
                } catch (IllegalAccessException e) {
                    e.printStackTrace();//Shouldn't happen
                    tmp = null;
                }
        
                //Store
                ALPHANUMERIC_TABLE = tmp;
            }
        
            public static final int NUM_DISTINCT_ALPHANUM_VALUES = 45;
            public static final char[] alphaNumReverseIndex = new char[NUM_DISTINCT_ALPHANUM_VALUES];
        
            static {
                //Build AlphaNum Index
                final int len = ALPHANUMERIC_TABLE.length;
                for (int x = 0; x < len; x++) {
                    // The base45 result which the alphanum lookup table produces.
                    // i.e. the base45 digit value which String characters are
                    // converted into.
                    //
                    // We use this value to build a reverse lookup table to find
                    // the String character we have to send to the encoder, to
                    // make it produce the given base45 digit value.
                    final int base45DigitValue = ALPHANUMERIC_TABLE[x];
        
                    //Ignore the -1 records
                    if (base45DigitValue > -1) {
                        //The index into the lookup table which produces the given base45 digit value.
                        //
                        //i.e. to produce a base45 digit with the numeric value in base45DigitValue, we need
                        //to send the Encoder a String character with the numeric value in x.
                        alphaNumReverseIndex[base45DigitValue] = (char) x;
                    }
                }
            }
        
            /*
             * The storage capacity of one digit in the number system; i.e. the maximum
             * possible number of distinct values which can be stored in 1 logical digit
             */
            public static final int QR_PAYLOAD_NUMERIC_BASE = NUM_DISTINCT_ALPHANUM_VALUES;
        
            /*
             * We can't use all 8 bytes, because the Long is signed, and the conversion math
             * requires consistently positive values. If we populated all 8 bytes, then the
             * last byte has the potential to contaminate the sign bit, and break the
             * conversion math. So, we only use the lower 7 bytes, and avoid this problem.
             */
            public static final int LONG_USABLE_BYTES = Long.BYTES - 1;
        
            //The following mapping was determined by brute-forcing -1 Long (all bits 1), and compressing to base45 until it hit zero.
            public static final int[] BINARY_TO_BASE45_DIGIT_COUNT_CONVERSION = new int[] {0,2,3,5,6,8,9,11,12};
            public static final int NUM_BASE45_DIGITS_PER_LONG = BINARY_TO_BASE45_DIGIT_COUNT_CONVERSION[LONG_USABLE_BYTES];
            public static final Map<Integer, Integer> BASE45_TO_BINARY_DIGIT_COUNT_CONVERSION = new HashMap<>();
        
            static {
                //Build Reverse Lookup
                int len = BINARY_TO_BASE45_DIGIT_COUNT_CONVERSION.length;
                for (int x=0; x<len; x++) {
                    int numB45Digits = BINARY_TO_BASE45_DIGIT_COUNT_CONVERSION[x];
                    BASE45_TO_BINARY_DIGIT_COUNT_CONVERSION.put(numB45Digits, x);
                }
            }
        
            public static String encodeToBase45QrPayload(final byte[] inputData) throws IOException {
                return encodeToBase45QrPayload(new ByteArrayInputStream(inputData));
            }
        
            public static String encodeToBase45QrPayload(final InputStream in) throws IOException {
                //Init conversion state vars
                final StringBuilder strOut = new StringBuilder();
                int data;
                long buf = 0;
        
                // Process all input data in chunks of size LONG.BYTES, this allows for economies of scale
                // so we can process more digits of arbitrary size before we hit the wall of the binary
                // chunk size in a power of 2, and have to transmit a sub-optimal chunk of the "crumbs"
                // left over; i.e. the slack space between where the multiples of QR_PAYLOAD_NUMERIC_BASE
                // and the powers of 2 don't quite line up.
                while(in.available() > 0) {
                    //Fill buffer
                    int numBytesStored = 0;
                    while (numBytesStored < LONG_USABLE_BYTES && in.available() > 0) {
                        //Read next byte
                        data = in.read();
        
                        //Push byte into buffer
                        buf = (buf << 8) | data; //8 bits per byte
        
                        //Increment
                        numBytesStored++;
                    }
        
                    //Write out in lower base
                    final StringBuilder outputChunkBuffer = new StringBuilder();
                    final int numBase45Digits = BINARY_TO_BASE45_DIGIT_COUNT_CONVERSION[numBytesStored];
                    int numB45DigitsProcessed = 0;
                    while(numB45DigitsProcessed < numBase45Digits) {
                        //Chunk out a digit
                        final byte digit = (byte) (buf % QR_PAYLOAD_NUMERIC_BASE);
        
                        //Drop digit data from buffer
                        buf = buf / QR_PAYLOAD_NUMERIC_BASE;
        
                        //Write Digit
                        outputChunkBuffer.append(alphaNumReverseIndex[(int) digit]);
        
                        //Track output digits
                        numB45DigitsProcessed++;
                    }
        
                    /*
                     * The way this code works, the processing output results in a First-In-Last-Out digit
                     * reversal. So, we need to buffer the chunk output, and feed it to the OutputStream
                     * backwards to correct this.
                     *
                     * We could probably get away with writing the bytes out in inverted order, and then
                     * flipping them back on the decode side, but just to be safe, I'm always keeping
                     * them in the proper order.
                     */
                    strOut.append(outputChunkBuffer.reverse().toString());
                }
        
                //Return
                return strOut.toString();
            }
        
            public static byte[] decodeBase45QrPayload(final String inputStr) throws IOException {
                //Prep for InputStream
                final byte[] buf = inputStr.getBytes();//Use the default encoding (the same encoding that the 'char' primitive uses)
        
                return decodeBase45QrPayload(new ByteArrayInputStream(buf));
            }
        
            public static byte[] decodeBase45QrPayload(final InputStream in) throws IOException {
                //Init conversion state vars
                final ByteArrayOutputStream out = new ByteArrayOutputStream();
                int data;
                long buf = 0;
                int x=0;
        
                // Process all input data in chunks of size LONG.BYTES, this allows for economies of scale
                // so we can process more digits of arbitrary size before we hit the wall of the binary
                // chunk size in a power of 2, and have to transmit a sub-optimal chunk of the "crumbs"
                // left over; i.e. the slack space between where the multiples of QR_PAYLOAD_NUMERIC_BASE
                // and the powers of 2 don't quite line up.
                while(in.available() > 0) {
                    //Convert & Fill Buffer
                    int numB45Digits = 0;
                    while (numB45Digits < NUM_BASE45_DIGITS_PER_LONG && in.available() > 0) {
                        //Read in next char
                        char c = (char) in.read();
        
                        //Translate back through lookup table
                        int digit = ALPHANUMERIC_TABLE[(int) c];
        
                        //Shift buffer up one digit to make room
                        buf *= QR_PAYLOAD_NUMERIC_BASE;
        
                        //Append next digit
                        buf += digit;
        
                        //Increment
                        numB45Digits++;
                    }
        
                    //Write out in higher base
                    final LinkedList<Byte> outputChunkBuffer = new LinkedList<>();
                    final int numBytes = BASE45_TO_BINARY_DIGIT_COUNT_CONVERSION.get(numB45Digits);
                    int numBytesProcessed = 0;
                    while(numBytesProcessed < numBytes) {
                        //Chunk out 1 byte
                        final byte chunk = (byte) buf;
        
                        //Shift buffer to next byte
                        buf = buf >> 8; //8 bits per byte
        
                        //Write byte to output
                        //
                        //Again, we need to invert the order of the bytes, so as we chunk them off, push
                        //them onto a FILO stack; inverting their order.
                        outputChunkBuffer.push(chunk);
        
                        //Increment
                        numBytesProcessed++;
                    }
        
                    //Write chunk buffer to output stream (in reverse order)
                    while (outputChunkBuffer.size() > 0) {
                        out.write(outputChunkBuffer.pop());
                    }
                }
        
                //Return
                out.flush();
                out.close();
                return out.toByteArray();
            }
        }
        

        【讨论】:

          猜你喜欢
          • 1970-01-01
          • 2010-11-09
          • 1970-01-01
          • 1970-01-01
          • 2013-12-20
          • 1970-01-01
          • 2018-02-12
          • 2021-06-13
          • 1970-01-01
          相关资源
          最近更新 更多