更新 Apache Lucene 索引文件

Question

我正在使用 Apache Lucene 库为我的网站创建搜索功能。该网站从 Sharepoint RSSFeeds 获取所有内容，因此每次我都必须浏览所有 RSSFeed 网址并阅读内容。为了使搜索功能更快，我创建了一个计划任务来每隔一小时进行一次索引：

    <bean id="rssIndexerService" class="com.lloydsbanking.webmi.service.RSSIndexerService" />
<task:scheduled-tasks> <task scheduled ref="rssIndexerService" method="indexUrls" cron="0 0 * * * MON-FRI" /></task:scheduled-tasks>

问题是，如果我创建一个新内容，那么在服务器运行时搜索不显示新内容，并且在调用调度任务之后，如果我删除一个条目，它仍然不显示删除的条目从索引文件。这是索引代码：

@Service
public class RSSIndexerService extends RSSReader {

    @Autowired
    private RSSFeedUrl rssFeedUrl;

    private IndexWriter indexWriter = null;

    private String indexPath = "C:\\MI\\index";

    Logger log = Logger.getLogger(RSSIndexerService.class.getName());

    public void indexUrls() throws IOException {
        Date start = new Date();
        IndexWriter writer = getIndexWriter();
        log.info("Reading all the Urls in the Sharepoint");     
        Iterator<Entry<String, String>> entries = rssFeedUrl.getUrlMap().entrySet().iterator();
        try {
            while (entries.hasNext()) {
                Entry<String, String> mapEntry = entries.next();
                String url = mapEntry.getValue();
                SyndFeed feed = rssReader(url);
                for (Object entry : feed.getEntries()) {
                    SyndEntry syndEntry = (SyndEntry) entry;
                    SyndContent desc = syndEntry.getDescription();
                    if (desc != null) {
                        String text = desc.getValue();
                        if ("text/html".equals(desc.getType())) {
                            Document doc = new Document();
                            text = extractText(text);
                            Field fieldTitle = new StringField("title", syndEntry.getTitle(), Field.Store.YES);
                            doc.add(fieldTitle);
                            Field pathField = new StringField("path", url, Field.Store.YES);
                            doc.add(pathField);
                            doc.add(new TextField("contents", text, Field.Store.YES));

                            // New index, so we just add the document (no old document can be there):
                            writer.addDocument(doc);
                        }
                    }
                }

            }

        } finally {

            // closeIndexWriter();
        }
        Date end = new Date();
        log.info(end.getTime() - start.getTime() + " total milliseconds");
    }

    public IndexWriter getIndexWriter() throws IOException {

        if (indexWriter == null) {
            Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_47);

            log.info("Indexing to directory '" + indexPath + "'...");
            Directory dir = FSDirectory.open(new File(indexPath));
            IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_47, analyzer);

            config.setOpenMode(OpenMode.CREATE_OR_APPEND);
            indexWriter = new IndexWriter(dir, config);
        }
        return indexWriter;
    }

    @PreDestroy
    public void closeIndexWriter() throws IOException {
        if (indexWriter != null) {
            System.out.println("Done with indexing ...");
            indexWriter.close();
        }
    }

}

我知道问题可能是由 config.setOpenMode(OpenMode.CREATE_OR_APPEND); 引起的，但我不知道如何解决。

Answer 1

好吧，我想出了检查目录之前是否为空的想法，如果不是，则删除以前的索引，然后每次都在 OpenMode.Create 中进行索引：

File path = new File(System.getProperty("java.io.tmpdir")+"\\index");
        Directory dir = FSDirectory.open(path);

        Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_47);
        IndexWriterConfig config = new IndexWriterConfig(Version.LUCENE_47, analyzer);

        if (path.list() != null) {
            log.info("Delete previous indexes ...");
            FileUtils.cleanDirectory(path);
        }
        config.setOpenMode(OpenMode.CREATE);

然后我简单地使用 addDocument():

if ("text/html".equals(desc.getType())) {
                        ...
                        // New index, so we just add the document (no old document can be there):
                        indexWriter.addDocument(doc);
                    }