为 DS-2542（XOAI 日期粒度）修补 Dspace 不会导致行为改变

Question

我正在尝试将补丁应用于运行 Dspace 5.1（带有一些附加模块）的测试环境，以修复由于日期粒度不正确而导致的 OAI 收集问题。问题是DS-2542 XOAI does not support non granular YYYY-MM-DD harvesting properly

环境

• Dspace 5.1（继承，随后从 4.2 升级）
• Tomcat 版本 6
• Oracle Java 1.7 更新 79
• 红帽企业 Linux 6.6 版

我如何应用补丁

首先我需要从https://github.com/DSpace/DSpace/pull/912 获取 Dspace (dspace-oai) 的补丁。这给了我以下补丁 (as a diff)，我将其放入 /tmp/

diff --git a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
index 1995fc0..cdb17d9 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
@@ -50,6 +50,11 @@ public boolean isShown(DSpaceItem item)
     public SolrFilterResult buildSolrQuery()
     {
         String format = dateProvider.format(date).replace("Z", ".999Z"); // Tweak to set the millisecon
+        if (format.substring(11, 19).equals("00:00:00"))
+        {
+            format = format.substring(0, 11) + "23:59:59" + format.substring(19);
+        }
+
         return new SolrFilterResult("item.lastmodified:[* TO "
                 + ClientUtils.escapeQueryChars(format) + "]");
     }
diff --git a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
index e968414..b73955d 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
@@ -31,8 +31,7 @@ public static String format(Date date)
     }
     public static String format(Date date, boolean init)
     {
-       SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'000Z'");
-       if (!init) sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'999Z'");
+        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
         // We indicate that the returned date is in Zulu time (UTC) so we have
         // to set the time zone of sdf correct.
         sdf.setTimeZone(TimeZone.getTimeZone("ZULU"));

我已使用以下命令序列应用了补丁：

sudo su - builder
  cd /usr/local/src/
    git clone 'https://github.com/lyncode/xoai.git' xoai
    cd xoai/
      git branch -a
      git checkout 3.x
      less pom.xml    <-- verify that com.lyncode version is 3.2.10-SNAPSHOT
      mvn package install
      cd ..
    cd dspace-src/
      patch -p1 < /tmp/473f2faaba99671b55372bcca1604aea2acf9601.diff    # (needed tweaking)
      vim dspace-oai/pom.xml   # change <xoai.version> from 3.2.9 to 3.2.10-SNAPSHOT
      mvn package -U -P \!dspace-jspui,\!dspace-lni,\!dspace-rest,\!dspace-swordv2,\!dspace-rdf,\!dspace-xmlui-mirage2
      exit
cd /usr/local/src/dspace-src/dspace/target/dspace-installer/
  ant update
  chown -Rc tomcat:tomcat /usr/local/dspace/
  service tomcat6 restart

测试方法

补丁后的结果与我的补丁前相同。使用 2015-04-07 仍然失败，但应该在修补后通过。 2015-04-07T00:00:00Z 工作相同并通过。

$ curl 'https://test-dspace.example.com/oai/request?verb=ListRecords&metadataPrefix=oai_dc&until=2015-04-07'
...<error code="badArgument">Invalid date given in until parameter</error></OAI-PMH>

$ curl 'https://test-dspace.example.com/oai/request?verb=ListRecords&metadataPrefix=oai_dc&until=2015-04-07T00:00:00Z'
... completeListSize="3731" cursor="0">oai_dc//2015-04-07T00:00:00.000Z//100</resumptionToken></ListRecords></OAI-PMH>

我已经多次看到并阅读了DSpace - Tech email entitled 'Building DSpace after bug fix (DS-2542)'，尽管我已经按照该电子邮件跟踪做了我应该做的所有事情（包括“正确的”——编辑你的 POM——和“快速和脏”——替换jar——其中列出的方法），问题仍然存在。

我尝试过的事情（都无济于事）

• 确保 tomcat 正确停止，然后启动。
• 重复完整的构建过程（mvn -U clean package）
• 确保在/usr/local/dspace/lib/ 和/usr/local/dspace/webapps/oai/WEB-INF/lib/ 中除了xoai-3.2.10-SNAPSHOT.jar 之外没有xoai-*.jar
• 为 com.lyncode.xoai 和 org.dspace.xoai 启用 DEBUG 日志记录
• 替换了 jar - 认为可能是 dspace 的构建过程（mvn 或 ant）正在用其他方式替换它。
• 删除 ~/.m2/repository 并从头开始重新构建（包括所有模块）。
• 在 Tomcat 启动参数中添加了 -verbose:class 以查看加载的位置（请参阅附加附录）

我目前的想法/疑问

我确信 jar 正在按预期创建，并且安装良好，并且没有旧的 jar 在起作用。

我不确信我认为应该运行的代码实际上正在运行，或者是否有其他东西掩盖了代码......我不知道如何在雄猫。

是否有一个命令（大概是dspace dsrun ...）我可以用来在 Tomcat 之外复制这个测试？

我应该指出我是系统工程师，而不是 Java 开发人员。

附录

日志设置

在 dspace.log 的 appender 部分的末尾添加

log4j.logger.com.lyncode.xoai=DEBUG, A1
log4j.logger.org.dspace.xoai=DEBUG, A1

这让我可以看到如下日志：

2015-07-28 02:34:07,222 DEBUG org.dspace.xoai.services.impl.solr.DSpaceSolrServerResolver @ Solr Server Initialized

2015-07-28 02:34:07,233 DEBUG com.lyncode.xoai.dataprovider.OAIRequestParameters @ RootParameterMap 'until' = '2015-04-07'

如果我编辑 /usr/local/src/xoai/src/main/java/com/lyncode/xoai/dataprovider/OAIDataProvider.java，我可以把它变成

2015-07-28 02:50:15,374 DEBUG com.lyncode.xoai.dataprovider.OAIRequestParameters @ CAMERON-RootParameterMap 'until' = '2015-04-07'

但是将 CAMERON 添加到看似唯一生成错误文本的位置的开头不会显示测试输出的变化（这在同一目录中的 OAIDataProvider.java 中）：

throw new BadArgumentException("CAMERON Invalid date given in until parameter");

dspace-src 目录的git diff（相对于预补丁）

# git diff
diff --git a/build.properties b/build.properties
index 844cb93..09cb5c3 100644
--- a/build.properties
+++ b/build.properties
@@ -166,6 +166,7 @@ http.proxy.port = 3128
 loglevel.other = WARN
 # loglevel.other: Log level for other third-party tools/APIs used by DSpace
 # Possible values (from most to least info): DEBUG, INFO, WARN, ERROR, FATAL
-loglevel.dspace = INFO
+#loglevel.dspace = INFO
+loglevel.dspace = DEBUG
 # loglevel.dspace: Log level for all DSpace-specific code (org.dspace.*)
 # Possible values (from most to least info): DEBUG, INFO, WARN, ERROR, FATAL
diff --git a/dspace-oai/pom.xml b/dspace-oai/pom.xml
index 6e56a52..136a10f 100644
--- a/dspace-oai/pom.xml
+++ b/dspace-oai/pom.xml
@@ -16,7 +16,7 @@
         <!-- This is the path to the root [dspace-src] directory. -->
         <root.basedir>${basedir}/..</root.basedir>
         <spring.version>3.2.5.RELEASE</spring.version>
-        <xoai.version>3.2.9</xoai.version>
+        <xoai.version>3.2.10-SNAPSHOT</xoai.version>
         <jtwig.version>2.0.1</jtwig.version>
     </properties>

diff --git a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
index 1995fc0..cdb17d9 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/filter/DateUntilFilter.java
@@ -50,6 +50,11 @@ public class DateUntilFilter extends DSpaceFilter
     public SolrFilterResult buildSolrQuery()
     {
         String format = dateProvider.format(date).replace("Z", ".999Z"); // Tweak to set the millisecon
+        if (format.substring(11, 19).equals("00:00:00"))
+        {
+            format = format.substring(0, 11) + "23:59:59" + format.substring(19);
+        }
+
         return new SolrFilterResult("item.lastmodified:[* TO "
                 + ClientUtils.escapeQueryChars(format) + "]");
     }
diff --git a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
index e968414..b73955d 100644
--- a/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
+++ b/dspace-oai/src/main/java/org/dspace/xoai/util/DateUtils.java
@@ -31,8 +31,7 @@ public class DateUtils
     }
     public static String format(Date date, boolean init)
     {
-       SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'000Z'");
-       if (!init) sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.'999Z'");
+        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss'Z'");
         // We indicate that the returned date is in Zulu time (UTC) so we have
         // to set the time zone of sdf correct.
         sdf.setTimeZone(TimeZone.getTimeZone("ZULU"));

来自 /usr/local/src/xoai 的 Git 日志（github.com/lyncode/xoai checkout of branch 3.x）

commit f6721be7ef5bf75d790e220ee81821e8eb70986e
Merge: dd9ef83 71ae14e
Author: João Melo <jmelo@lyncode.com>
Date:   Thu Oct 9 18:33:57 2014 +0100

    Merge pull request #34 from kosarko/dates_fix

    from and until granularity

commit dd9ef830a16209e624bd03c3e91e7c5d7bdbd449
Merge: 8e778c2 c04b0bc
Author: João Melo <jmelo@lyncode.com>
Date:   Thu Oct 9 18:33:29 2014 +0100

    Merge pull request #33 from kosarko/filter_fix

    fixed typo

commit 71ae14ed5ee9e6a5a6e3d6253424c212ac98081a
Author: Ondřej Košarko <kosarko@ufal.mff.cuni.cz>
Date:   Thu Oct 9 19:05:03 2014 +0200

    from and until granularity

    Regardles of the repository setting the from and until parameters must
    handle day granularity. Error message fixed

commit c04b0bcd177007f618f18008e53a61a81aabdb06
Author: Ondřej Košarko <kosarko@ufal.mff.cuni.cz>
Date:   Thu Oct 9 19:01:08 2014 +0200

    fixed typo

xoai-*.jar 的实例

$ sudo updatedb
$ locate xoai- | grep '\.jar$' | grep -v bak | xargs ls -l
-rwxr-xr-x 1 tomcat  tomcat   321775 Jul 28 03:04 /usr/local/dspace/lib/xoai-3.2.10-SNAPSHOT.jar
-rwxr-xr-x 1 tomcat  tomcat   321775 Jul 28 03:04 /usr/local/dspace/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace/modules/oai/target/oai-5.1/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace-oai/target/dspace-oai-5.1/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace/target/dspace-installer/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321832 Jul 28 01:34 /usr/local/src/dspace-src/dspace/target/dspace-installer/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder  321775 Jul 28 03:04 /usr/local/src/xoai/target/xoai-3.2.10-SNAPSHOT.jar
-rw-rw-r-- 1 builder builder 1571092 Jul 28 03:04 /usr/local/src/xoai/target/xoai-3.2.10-SNAPSHOT-javadoc.jar
-rw-rw-r-- 1 builder builder  222305 Jul 28 03:04 /usr/local/src/xoai/target/xoai-3.2.10-SNAPSHOT-sources.jar

验证 tomcat 应该看到相同的内容（如果链接断开，似乎没有链接）

$ find -L /usr/share/tomcat6 -type f -name 'xoai-*.jar' -print | xargs ls -l
-rwxr-xr-x 1 tomcat tomcat 321775 Jul 28 03:04 /usr/share/tomcat6/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar

Tomcat -verbose:class 结果

所有 com.lyncode.xoai。类来自文件：/usr/local/dspace/webapps/oai/WEB-INF/lib/xoai-3.2.10-SNAPSHOT.jar

非常感谢您的阅读，

卡梅隆

Answer 1

您是否尝试过清除 oai 响应缓存？我不知道这种类型的东西是否会被缓存，但这是我唯一能想到的你还没有提到的东西。

[dspace]/bin/dspace oai clean-cache

https://wiki.duraspace.org/display/DSDOC5x/OAI+2.0+Server#OAI2.0Server-OAIManager%28SolrDataSource%29