标记包含数字的文本

Question

我想创建一个文本分类器，模型的输入包含数字以及包含重要信息的文本（不要以为我可以扔掉数字）。有没有办法标记这种输入？

输入看起来像这样：

     input:
     -------
     Please have a look at case#345
     injector 1 and injector 3 is not responding for model 8
     Car has been running for 2345 km, try to do this procedure 
     .....
     .....

Answer 1

这有帮助：

from keras.preprocessing.text import text_to_word_sequence

text = 'Please have a look at case#345. injector1 and injector3 is not responding for model8. Car has been running for 2345 km, try to do this procedure .'
# tokenize the document
result = text_to_word_sequence(text)
print(result)

输出：

['please', 'have', 'a', 'look', 'at', 'case', '345', 'injector1', 'and', 'injector3', 'is', 'not', 'responding', 'for', 'model8', 'car', 'has', 'been', 'running', 'for', '2345', 'km', 'try', 'to', 'do', 'this', 'procedure']