Python - 删除扩展 ascii

Question

好的，所以我是整个 python 世界的新手，所以请耐心等待。

背景：我们正在尝试将日志卸载到 mongo 以便能够更快地查询和搜索它们。设备已经以合适的格式打印它们，除了在每个 }{ 开始和结束数据对象之间，如下所示：

¾ïúÀï{"id":"xxx","timestamp":xxx,"payloadType":"xxx","payload":{"protocol":"xxx","zoneID":xxx,"zoneName":"xxx","eventType":"xxx"}}’ÂCº¾ïúÀï{"id":"xxx","timestamp":xxx,"payloadType":"xxx","payload":{"protocol":"xxx","zoneID":xxx,"zoneName":"xxx","eventType":"xx}}

使用以下内容，我已经能够将其转换为字节，然后转换回输出为的字符串：

f = open('logfile', 'r')
file_data = f.read()
f.close()

data = file_data.encode('utf-8')

print(str(data))

>>>b'\xc2\xbe\xc3\xaf\xc3\xba\xc3\x80\x01\xc3\xaf{"id":"xxx","timestamp":xxx,"payloadType":"xxx","payload":{"protocol":"xxx","zoneID":xxx,"zoneName":"xxx","eventType":"xxx"}}

在我看来，这比上面看到的丑陋角色更容易处理，但我不知道。

这只是一个示例..此日志中返回了成千上万行。在我看来，理想情况下，解决此问题的最佳方法是删除 { 之前字符串开头的所有字符以及 }}{

之间的所有字符

Answer 1

将字符串编码为字节，然后解码回ASCII：

data.encode().decode('ascii',errors='ignore')
# {"id":"xxx","timestamp":xxx,...}}

您还可以使用正则表达式删除最外层花括号之外的所有字符：

re.sub(r'^[^{]*(?={)|(?<=})[^}{]*(?={)|(?<=})[^}]*$', '', data)

后一种机制顺便也删除了你不想要的 ASCII 'C' 字符。

Answer 2

import re

str='¾ïúÀï{"id":"xxx","timestamp":xxx,"payloadType":"xxx","payload":{"protocol":"xxx","zoneID":xxx,"zoneName":"xxx","eventType":"xxx"}}’ÂCº¾ïúÀï{"id":"xxx","timestamp":xxx,"payloadType":"xxx","payload":{"protocol":"xxx","zoneID":xxx,"zoneName":"xxx","eventType":"xx}}'

str=re.sub('[^\x00-\x7F]','',str)
print(str)

应该产生输出为...

'{"id":"xxx","timestamp":xxx,"payloadType":"xxx","payload":{"protocol":"xxx","zoneID":xxx,"zoneName":"xxx","eventType":"xxx"}}C{"id":"xxx","timestamp":xxx,"payloadType":"xxx","payload":{"protocol":"xxx","zoneID":xxx,"zoneName":"xxx","eventType":"xx}}'

Answer 3

类似的东西呢：

import string

cleaned_string = ''
for char in ugly_string:
    if char in string.printable:
        cleaned_string += char

This question 也处理类似的问题。

Answer 4

如果垃圾字节不包含左花括号，您可以执行以下操作：

def decode_all(data):
    decoder = JSONDecoder()
    end_index = 0

    while data:
        try:
            data = data[data.index('{', end_index):]
        except ValueError:
            break

        obj, end_index = decoder.raw_decode(data)
        yield obj

否则，在不知道这些垃圾字节可以包含什么以及您的 JSON 是否是纯 ASCII 的情况下，我认为最好的解决方案是尝试一遍又一遍地从数据中解析 JSON 编码的对象并跳过垃圾字节隐含地：

from json import JSONDecoder

data = '''¾ïúÀï{"id":"123","timestamp":123,"payloadType":"123","payload":{"protocol":"123","zoneID":123,"zoneName":"123","eventType":"123"}}’ÂCº¾ïúÀï{"id":"123","timestamp":123,"payloadType":"123","payload":{"protocol":"123","zoneID":123,"zoneName":"123","eventType":"xx"}}'''

def decode_all(data):
    decoder = JSONDecoder()

    while data:
        try:
            obj, end_index = decoder.raw_decode(data)
            data = data[end_index:]

            yield obj
        except ValueError:
            end_index = None

        start = data.find('{')

        if start == -1:
            break
        elif end_index is None and start == 0:
            start = 1

        data = data[start:]

for o in decode_all(data):
    print(o)

Answer 5

我使用了 exifread。简而言之，使用 .printable 来获取值的字符串。 下面是获取一些选择标签的值并将它们打包到一个名为 context 的字典中的代码：

photo_file = '/absolute/path/photo.jpg'
f = open(latest_f, 'rb')
tags = exifread.process_file(f)
select_tags = ['Image Make', 'Image Model','EXIF DateTimeOriginal', 'EXIF ExposureTime']
context = dict()
for i, tag in enumerate(select_tags):
   context[tag] = tags[select_tags[i]]*.printable*
print(context)