如何使用 NLP 从 PDF 文件中提取关键字及其页码？

Question

在上面的 PDF 文件中，我的代码必须从给定 PDF 的所有页面中提取关键字和表名称，例如 Table 1、Table 2、带有粗体字母的标题，例如 INTRODUCTION、CASE PRESENTATION。

写了一个小程序从PDF文件中提取文本

punctuations = ['(',')',';',':','[',']',',','^','=','-','!','.','{','}','/','#','^','&']

stop_words = stopwords.words('English')

keywords = [word for word in tokens if not word in stop_words and not word in punctuations]

print(keywords)

我得到的输出如下

从上面的输出中，如何提取 INTRODUCTION、CASE PRESENTATION、Table 1 等关键字以及页码并将它们保存在输出文件中。

输出格式

INTRODUCTION in Page 1

CASE PRESENTATION in Page 3

Table 1 (Descriptive Statistics) in Page 5

在获取此格式的输出时需要帮助。

代码

def main():

        file_name = open("Test1.pdf","rb")
        readpdf = PyPDF2.PdfFileReader(file_name)
    

    #Parse thru each page to extract the texts
        pdfPages = readpdf.numPages
        count=0
        text=""
        print()
        #The while loop will read each page.
        while count < pdfPages:
            pageObj = readpdf.getPage(count)
            count +=1
            text += pageObj.extractText()

        #This if statement exists to check if the above library returned words. It's done because PyPDF2 cannot read scanned files.
        if text != "":
            text = text
        #If the above returns as False, we run the OCR library textract to #convert scanned/image based PDF files into text.
        else:
            text = textract.process(fileurl, method='tesseract', language='eng')

        #PRINT THE TEXT EXTRACTED FROM GIVEN PDF
        #print(text)

        #The function will break text into individual words
    
        tokens = word_tokenize(text)
        #print('TOKENS')
        #print(tokens)

        #Clean the punctuations not required.
        punctuations = ['(',')',';',':','[',']',',','^','=','-','!','.','{','}','/','#','^','&']
        
        stop_words = stopwords.words('English')
        
        keywords = [word for word in tokens if not word in stop_words and not word in punctuations]
   
        print(keywords)

Answer 1

如果您想知道哪个页面是一些文本，那么您不应该将所有内容添加到一个字符串中，而是应该单独处理每个页面（在 for-loop` 中）

可能与此类似。它是没有 tesseract 的代码，需要将 PDF 拆分为单独的页面并单独处理每个页面的方法

pdfPages = readpdf.numPages

# create it before loop
punctuations = ['(',')',';',':','[',']',',','^','=','-','!','.','{','}','/','#','^','&']
stop_words = stopwords.words('English')

#all_pages = []

# work with every page separatelly
for count in range(pdfPages):

    pageObj = readpdf.getPage(count)

    page_text = pageObj.extractText()
    
    page_tokens = word_tokenize(page_text)

    page_keywords = [word for word in page_tokens if not word in stop_words and not word in punctuations]

    page_uppercase_words = [word for word in page_keywords if word.isupper()]

    #all_pages.append( (count, page_keywords, page_uppercase_words) )

    print('page:', count)
    print('keywords:', page_keywords) 
    print('uppercase:', page_uppercase_words)

    # TODO: append/save page to file 

Answer 2

问题在这里部分解决：https://github.com/konfuzio-ai/document-ai-python-sdk/issues/6#issue-876036328

检查：https://github.com/konfuzio-ai/document-ai-python-sdk

# pip install konfuzio_sdk
# in working directory
# konfuzio_sdk init

from konfuzio_sdk.api import get_document_annotations
    
document_first_annotation = get_document_annotations(document_id=1111)[0]
page_index = document_first_annotation['bbox']['page_index']
keyword = document_first_annotation['offset_string']

Konfuzio SDK 中的对象注释允许直接访问关键字字符串，但目前不能直接访问页面索引。此属性将很快添加。 访问项目第一个培训文档中第一个注释的示例如下：

# pip install konfuzio_sdk
# in working directory
# konfuzio_sdk init

from konfuzio_sdk.data import Project

my_project = Project()

annotations_first_doc = my_project.documents[0].annotations()
first_annotation = annotations_first_doc[0]
keyword = first_annotation.offset_string

Answer 3

 import PyPDF2
 import pandas 
 import numpy
 import re
 import os,sys
 import nltk
 import fitz
 
 def main():
     file_name = open("File1.pdf","rb")
     readPDF = PyPDF2.PdfFileReader(file_name)
     call_function(file_name,readPDF)   

 def call_function(fname,readpdf)
     pdfPages = readpdf.numPages
          
     for pageno in range(pdfPages):
        doc_name = fitz.open(fname.name)
        page = word_tokenize(doc_name[pageno].get_text())
        page_texts = [word for word in page if not word in stop_words and not word in punctuations]
        print('Page Number:',pageno)
        print('Page Texts :',page_texts)