SonarQube 5.1.1 - 使用 Oracle DB 分析非常慢

Question

我们想使用 SonarQube 5.1.1 来分析一个包含大约 25.000 个文件的 Java 项目。但是“应用项目排除”步骤非常慢，需要 2.5 小时。同时，AIX-DB-Server 上的 CPU 活动非常高 - 60% CPU。

在 TRACE-Mode 下启动 SonarQube-Analysis (ANT-Task) 时，我们可以看到，以下 SQL 的执行速度非常慢。 每个 SQL 大约需要 0.5 秒。看来，该 SQL 是针对 25.000 个文件中的每一个执行的。

我不是 Oracle 专家，但问题似乎是对 25.000 个文件中的每一个文件的“project_measures”表进行“全表扫描”。

关于变通办法或解决方案的任何想法？

我不确定这是否是同一个问题：sonarqube-apply-project-exclusions-very-slow

来自 ANT-Analyzer 的日志（TRACE 级别）：

[sonar:sonar] 10:59:29.342 DEBUG - Opening JDBC Connection
[sonar:sonar] 10:59:29.358 DEBUG - Resetting autocommit to true on JDBC Connection [jdbc:oracle:thin:@XXX:XXX:XXX, UserName=SONAR, Oracle JDBC driver]
[sonar:sonar] 10:59:29.358 DEBUG - Closing JDBC Connection [jdbc:oracle:thin:@XXX:XXX:XXX, UserName=SONAR, Oracle JDBC driver]
[sonar:sonar] 10:59:29.358 DEBUG - Load technical debt model (done) | time=16ms
[sonar:sonar] 10:59:29.374 INFO  - Apply project exclusions
[sonar:sonar] 10:59:29.374 DEBUG - Opening JDBC Connection
[sonar:sonar] 10:59:29.389 DEBUG - Resetting autocommit to true on JDBC Connection [jdbc:oracle:thin:@XXX:XXX:XXX, UserName=SONAR, Oracle JDBC driver]
[sonar:sonar] 10:59:29.389 DEBUG - Closing JDBC Connection [jdbc:oracle:thin:@XXX:XXX:XXX, UserName=SONAR, Oracle JDBC driver]
[sonar:sonar] 10:59:29.592 DEBUG - select metric1_.name as col_0_0_, measuremod0_.id as col_1_0_, measuremod0_.id as id5_, measuremod0_.alert_status as alert2_5_, measuremod0_.alert_text as alert3_5_, measuremod0_.characteristic_id as characte4_5_, measuremod0_.measure_data as measure5_5_, measuremod0_.description as descript6_5_, measuremod0_.metric_id as metric7_5_, measuremod0_.person_id as person8_5_, measuremod0_.project_id as project9_5_, measuremod0_.rule_id as rule10_5_, measuremod0_.rule_priority as rule11_5_, measuremod0_.snapshot_id as snapshot12_5_, measuremod0_.tendency as tendency5_, measuremod0_.text_value as text14_5_, measuremod0_.url as url5_, measuremod0_.value as value5_, measuremod0_.variation_value_1 as variation17_5_, measuremod0_.variation_value_2 as variation18_5_, measuremod0_.variation_value_3 as variation19_5_, measuremod0_.variation_value_4 as variation20_5_, measuremod0_.variation_value_5 as variation21_5_ from project_measures measuremod0_, metrics metric1_, projects resourcemo2_, snapshots snapshot3_ where metric1_.id=measuremod0_.metric_id and measuremod0_.snapshot_id=snapshot3_.id and snapshot3_.project_id=resourcemo2_.id and resourcemo2_.kee=? and snapshot3_.status=? and snapshot3_.qualifier<>? and (measuremod0_.characteristic_id is null) and (measuremod0_.person_id is null) and (measuremod0_.rule_id is null) and (measuremod0_.rule_priority is null) and (metric1_.name in (? , ? , ?)) and snapshot3_.islast=1 order by snapshot3_.created_at
[sonar:sonar] 10:59:32.077 DEBUG - select metric1_.name as col_0_0_, measuremod0_.id as col_1_0_, measuremod0_.id as id5_, measuremod0_.alert_status as alert2_5_, measuremod0_.alert_text as alert3_5_, measuremod0_.characteristic_id as characte4_5_, measuremod0_.measure_data as measure5_5_, measuremod0_.description as descript6_5_, measuremod0_.metric_id as metric7_5_, measuremod0_.person_id as person8_5_, measuremod0_.project_id as project9_5_, measuremod0_.rule_id as rule10_5_, measuremod0_.rule_priority as rule11_5_, measuremod0_.snapshot_id as snapshot12_5_, measuremod0_.tendency as tendency5_, measuremod0_.text_value as text14_5_, measuremod0_.url as url5_, measuremod0_.value as value5_, measuremod0_.variation_value_1 as variation17_5_, measuremod0_.variation_value_2 as variation18_5_, measuremod0_.variation_value_3 as variation19_5_, measuremod0_.variation_value_4 as variation20_5_, measuremod0_.variation_value_5 as variation21_5_ from project_measures measuremod0_, metrics metric1_, projects resourcemo2_, snapshots snapshot3_ where metric1_.id=measuremod0_.metric_id and measuremod0_.snapshot_id=snapshot3_.id and snapshot3_.project_id=resourcemo2_.id and resourcemo2_.kee=? and snapshot3_.status=? and snapshot3_.qualifier<>? and (measuremod0_.characteristic_id is null) and (measuremod0_.person_id is null) and (measuremod0_.rule_id is null) and (measuremod0_.rule_priority is null) and (metric1_.name in (? , ? , ?)) and snapshot3_.islast=1 order by snapshot3_.created_at
[sonar:sonar] 10:59:33.295 DEBUG - select metric1_.name as col_0_0_, measuremod0_.id as col_1_0_, measuremod0_.id as id5_, measuremod0_.alert_status as alert2_5_, measuremod0_.alert_text as alert3_5_, measuremod0_.characteristic_id as characte4_5_, measuremod0_.measure_data as measure5_5_, measuremod0_.description as descript6_5_, measuremod0_.metric_id as metric7_5_, measuremod0_.person_id as person8_5_, measuremod0_.project_id as project9_5_, measuremod0_.rule_id as rule10_5_, measuremod0_.rule_priority as rule11_5_, measuremod0_.snapshot_id as snapshot12_5_, measuremod0_.tendency as tendency5_, measuremod0_.text_value as text14_5_, measuremod0_.url as url5_, measuremod0_.value as value5_, measuremod0_.variation_value_1 as variation17_5_, measuremod0_.variation_value_2 as variation18_5_, measuremod0_.variation_value_3 as variation19_5_, measuremod0_.variation_value_4 as variation20_5_, measuremod0_.variation_value_5 as variation21_5_ from project_measures measuremod0_, metrics metric1_, projects resourcemo2_, snapshots snapshot3_ where metric1_.id=measuremod0_.metric_id and measuremod0_.snapshot_id=snapshot3_.id and snapshot3_.project_id=resourcemo2_.id and resourcemo2_.kee=? and snapshot3_.status=? and snapshot3_.qualifier<>? and (measuremod0_.characteristic_id is null) and (measuremod0_.person_id is null) and (measuremod0_.rule_id is null) and (measuremod0_.rule_priority is null) and (metric1_.name in (? , ? , ?)) and snapshot3_.islast=1 order by snapshot3_.created_at
[sonar:sonar] 10:59:34.405 DEBUG - select metric1_.name as col_0_0_, measuremod0_.id as col_1_0_, measuremod0_.id as id5_, measuremod0_.alert_status as alert2_5_, measuremod0_.alert_text as alert3_5_, measuremod0_.characteristic_id as characte4_5_, measuremod0_.measure_data as measure5_5_, measuremod0_.description as descript6_5_, measuremod0_.metric_id as metric7_5_, measuremod0_.person_id as person8_5_, measuremod0_.project_id as project9_5_, measuremod0_.rule_id as rule10_5_, measuremod0_.rule_priority as rule11_5_, measuremod0_.snapshot_id as snapshot12_5_, measuremod0_.tendency as tendency5_, measuremod0_.text_value as text14_5_, measuremod0_.url as url5_, measuremod0_.value as value5_, measuremod0_.variation_value_1 as variation17_5_, measuremod0_.variation_value_2 as variation18_5_, measuremod0_.variation_value_3 as variation19_5_, measuremod0_.variation_value_4 as variation20_5_, measuremod0_.variation_value_5 as variation21_5_ from project_measures measuremod0_, metrics metric1_, projects resourcemo2_, snapshots snapshot3_ where metric1_.id=measuremod0_.metric_id and measuremod0_.snapshot_id=snapshot3_.id and snapshot3_.project_id=resourcemo2_.id and resourcemo2_.kee=? and snapshot3_.status=? and snapshot3_.qualifier<>? and (measuremod0_.characteristic_id is null) and (measuremod0_.person_id is null) and (measuremod0_.rule_id is null) and (measuremod0_.rule_priority is null) and (metric1_.name in (? , ? , ?)) and snapshot3_.islast=1 order by snapshot3_.created_at
[sonar:sonar] 10:59:35.108 DEBUG - select metric1_.name as col_0_0_, measuremod0_.id as col_1_0_, measuremod0_.id as id5_, measuremod0_.alert_status as alert2_5_, measuremod0_.alert_text as alert3_5_, measuremod0_.characteristic_id as characte4_5_, measuremod0_.measure_data as measure5_5_, measuremod0_.description as descript6_5_, measuremod0_.metric_id as metric7_5_, measuremod0_.person_id as person8_5_, measuremod0_.project_id as project9_5_, measuremod0_.rule_id as rule10_5_, measuremod0_.rule_priority as rule11_5_, measuremod0_.snapshot_id as snapshot12_5_, measuremod0_.tendency as tendency5_, measuremod0_.text_value as text14_5_, measuremod0_.url as url5_, measuremod0_.value as value5_, measuremod0_.variation_value_1 as variation17_5_, measuremod0_.variation_value_2 as variation18_5_, measuremod0_.variation_value_3 as variation19_5_, measuremod0_.variation_value_4 as variation20_5_, measuremod0_.variation_value_5 as variation21_5_ from project_measures measuremod0_, metrics metric1_, projects resourcemo2_, snapshots snapshot3_ where metric1_.id=measuremod0_.metric_id and measuremod0_.snapshot_id=snapshot3_.id and snapshot3_.project_id=resourcemo2_.id and resourcemo2_.kee=? and snapshot3_.status=? and snapshot3_.qualifier<>? and (measuremod0_.characteristic_id is null) and (measuremod0_.person_id is null) and (measuremod0_.rule_id is null) and (measuremod0_.rule_priority is null) and (metric1_.name in (? , ? , ?)) and snapshot3_.islast=1 order by snapshot3_.created_at
[sonar:sonar] 10:59:35.733 DEBUG - select metric1_.name as col_0_0_, measuremod0_.id as col_1_0_, measuremod0_.id as id5_, measuremod0_.alert_status as alert2_5_, measuremod0_.alert_text as alert3_5_, measuremod0_.characteristic_id as characte4_5_, measuremod0_.measure_data as measure5_5_, measuremod0_.description as descript6_5_, measuremod0_.metric_id as metric7_5_, measuremod0_.person_id as person8_5_, measuremod0_.project_id as project9_5_, measuremod0_.rule_id as rule10_5_, measuremod0_.rule_priority as rule11_5_, measuremod0_.snapshot_id as snapshot12_5_, measuremod0_.tendency as tendency5_, measuremod0_.text_value as text14_5_, measuremod0_.url as url5_, measuremod0_.value as value5_, measuremod0_.variation_value_1 as variation17_5_, measuremod0_.variation_value_2 as variation18_5_, measuremod0_.variation_value_3 as variation19_5_, measuremod0_.variation_value_4 as variation20_5_, measuremod0_.variation_value_5 as variation21_5_ from project_measures measuremod0_, metrics metric1_, projects resourcemo2_, snapshots snapshot3_ where metric1_.id=measuremod0_.metric_id and measuremod0_.snapshot_id=snapshot3_.id and snapshot3_.project_id=resourcemo2_.id and resourcemo2_.kee=? and snapshot3_.status=? and snapshot3_.qualifier<>? and (measuremod0_.characteristic_id is null) and (measuremod0_.person_id is null) and (measuremod0_.rule_id is null) and (measuremod0_.rule_priority is null) and (metric1_.name in (? , ? , ?)) and snapshot3_.islast=1 order by snapshot3_.created_at

可能导致问题的SQL：

SELECT metric1_.name AS col_0_0_,
         measuremod0_.id AS col_1_0_,
         measuremod0_.id AS id5_,
         measuremod0_.alert_status AS alert2_5_,
         measuremod0_.alert_text AS alert3_5_,
         measuremod0_.characteristic_id AS characte4_5_,
         measuremod0_.measure_data AS measure5_5_,
         measuremod0_.description AS descript6_5_,
         measuremod0_.metric_id AS metric7_5_,
         measuremod0_.person_id AS person8_5_,
         measuremod0_.project_id AS project9_5_,
         measuremod0_.rule_id AS rule10_5_,
         measuremod0_.rule_priority AS rule11_5_,
         measuremod0_.snapshot_id AS snapshot12_5_,
         measuremod0_.tendency AS tendency5_,
         measuremod0_.text_value AS text14_5_,
         measuremod0_.url AS url5_,
         measuremod0_.VALUE AS value5_,
         measuremod0_.variation_value_1 AS variation17_5_,
         measuremod0_.variation_value_2 AS variation18_5_,
         measuremod0_.variation_value_3 AS variation19_5_,
         measuremod0_.variation_value_4 AS variation20_5_,
         measuremod0_.variation_value_5 AS variation21_5_
    FROM project_measures measuremod0_,
         metrics metric1_,
         projects resourcemo2_,
         snapshots snapshot3_
   WHERE     metric1_.id = measuremod0_.metric_id
         AND measuremod0_.snapshot_id = snapshot3_.id
         AND snapshot3_.project_id = resourcemo2_.id
         AND resourcemo2_.kee = :1
         AND snapshot3_.status = :2
         AND snapshot3_.qualifier <> :3
         AND (measuremod0_.characteristic_id IS NULL)
         AND (measuremod0_.person_id IS NULL)
         AND (measuremod0_.rule_id IS NULL)
         AND (measuremod0_.rule_priority IS NULL)
         AND (metric1_.name IN ( :4, :5, :6))
         AND snapshot3_.islast = 1
ORDER BY snapshot3_.created_at

Answer 1

这不是答案，只是可能对您有所帮助的几点。 AFAIK 没有人可以帮助你，因为你没有提供足够的信息

• 联系您的 DBA 寻求合作
• 从另一方（DB）开始问题调查。生成 AWR/Statspack 报告，您将看到。一般来说，与 Java 相比，Oracle 提供了强大的性能分析工具。
• 获取查询的执行计划
• 默认情况下，Oracle 索引不索引 NULL 值。

Answer 2

我不是来自 SonarSource，但它看起来像你指出的票，所以我相信 5.2 将是解决方案。

同时，Oracle DB方面的一些观点：

1. 您是否为 SonarQube 用户使用专用表空间？请参阅我的博客上的http://qualilogy.com/en/install-sonar-oracle-user/。如果您使用与其他 Oracle 用户/数据库共享的表空间，尤其是在虚拟服务器上，那么您更容易出现性能问题。
2. 从一开始就检查表空间是否足够大，并且“自动扩展”参数也足够高。如果表空间需要增长，并且以小增量进行，那么 Oracle 将更多地致力于不断增加一些小空间。
3. 还要检查其他系统表空间，尤其是专门用于索引的表空间。它越大越老，你的性能就越差。

同样，我不认为这会解决您的问题，但它可能会有所帮助，或者至少，如果您使用 Oracle 操作 SonarQube，可能会有所帮助。

最好的问候。

Answer 3

有关此性能问题的任何更新，请观看、投票或评论以下 Jira 票：https://jira.sonarsource.com/browse/SONAR-6839