(自 2015 年 9 月 4 日起更正以包括可重复的基准和结论)
当然values.clear() 比创建新对象更快(只需将最后一项索引设置为零)。
几乎可以肯定,values.clear() 会比创建新对象更快。对于您最初使用的 ArrayList,它只会将插入索引设置为零。
正如我在 PD#1 中评论的那样,对于这种元素为整数的情况,BitSet 可能是一种最快的方法(假设值的范围不太广泛。但是,这可能对任何其他类型的元素都没有用。
-
另外正如 正如我所遇到的 Assylias answer,HashSet 是比 ArrayList 更好的选择(假设 @987654328 @ 提供了一个不错的分布,但不会导致我们达到 O(N) 性能)。
在这种HashSet 的情况下,直觉还表明clear()(基本上将“鸽子洞”的HashSet#table 数组设置为null)比构建一个全新的集合(无论如何都需要同一个表被初始化/重置为零)。但在这种特殊情况下,事情会反过来发生。 Assylias 发表了他的研究结果。不幸的是,我不得不自己编写我的基准测试代码,以了解这怎么会发生。我在 P.D.#3 中讨论了这个问题
无论如何,主要的事情是,因为为每次迭代创建一个全新的 HashSet 并没有实质性的损失,所以这样做是有意义的(因为它更简单),除非我们必须更加关注性能和资源。
另一个关于性能的问题是I/O。示例代码中的System.out.println() 可能对每一行都执行flush(),这会自动将瓶颈转移到控制台/标准输出。解决方法可能是添加到StringBuffer。除非有读取器进程热切地等待该输出,否则将写入延迟到循环结束可能是有意义的。
这是我的尝试:
Set<Integer> values = new HashSet<Integer>();
// (PD 1) Or new BitSet(max_x - min_x + 1);
// (PD 2) Or new HashSet((int)Math.ceil(n/0.75));
StringBuffer sb = new StringBuffer(); // appends row values for printing together.
for (int j = 0, x; j < m; j++) {
values.clear();
sb.setLength(0);
for (int i = 0; i < n; i++) {
x = X[i][j];
if (! values.contains(x)){
sb.append(x+"\n");
values.add(x);
}
}
System.out.print(sb);
}
P.D.1。另外,如果您可以考虑使用 BitSet。它具有 O(1) 的访问性能(即使在最坏的情况下,也没有 没有冲突)。它最适合范围从 0 开始的整数(否则可能需要转换)以及在可能的分布中足够密集的实际值群体。
- 例如,如果您检查 Unicode 代码点的出现,您将需要一个 139,264 字节长的数组 (17 (planes) * 216 (codepoints/plane) / 8),您可能正在使用100 个字符长的短信中只有 40 个不同的字符,这可能有点过头了。但是,如果您仅限于 ISO-Latin-1 中的 256 个可能值。 (8 字节位集),这实际上是一个完美的选择。
P.D.2.此外,正如 Assylias 所说,为 HashSet 设置初始大小可能会有所帮助。作为 threshold = (int)(capacity * loadFactor) ,您可能需要 initialCapacity=(int)Math.ceil(n/0.75) 以确保没有调整大小。 这个问题属于 Assylias 帖子(我没有为自己使用)并且不适合在此讨论方式
PD3(2015 年 9 月:3 年后) 我碰巧重新审视了这个问题,我对 Assylas 的结果非常感兴趣,我编写了自己的微基准(我包括在内,因此任何人都可以复制)。以下是我的结论:
- 我提出的
BitSet(注意:不适合非整数和非常稀疏的分布)明显优于 HashSet 的所有风格(在密集的情况下大约快 4 倍分布)
- 对大小为 1000 的 高度填充集 的测试显示出轻微的优势,有利于创建 新 集合(7.7" vs 9.8" )。但是,
HashSet#clear() 与 new HashSet() 的“试运行”将产生相反的结果(9.5“与 7.5”)。我的猜测是,这是因为重置 HashSet.table(设置 null 而不是 null)时缓存失效的惩罚。
- 另外,事先知道最佳尺寸也是一大优势(这可能并不总是可行)。
HashSet.clear() 方法更具适应性,并且可以更好地低估大小。高估不会有太大的不同,但如果内存是个问题,这可能不是一个好策略。
- 结果清楚地表明,如今创建对象和分配内存并不是什么大问题(参见Programmers.SE)。 但是,重用对象仍然应该是一种选择来考虑。参见drmirror 中的示例,即使在 JDK 1.6 演进之后,重用实例 (CharBuffer) 也会使性能翻倍。
loadFactor==0.75f 需要 1.33 倍的表空间,以换取避免 25% 的冲突。我的测试没有显示这种情况下默认设置的任何优势。
这是我用于测试的类。抱歉,如果它可能在某些方面过冲而在其他方面有所欠缺(无需预热,只需执行足够长的时间,以便实现有机会因自己的垃圾而窒息)。
/**
* Messing around this StackOverflow question: https://stackoverflow.com/questions/11740013/fastest-way-to-recreate-the-arraylist-in-a-for-loop/ .
* Quite surprisingly new HashSet() (which should imply actual memory initialization) is faster than HashSet.clear() in the given scenario.
* Primary goal is to test this phenomenon (new vs clear) under different scenarios.
* Secondarily a bit about the BitSet and the HashSet loadFactor is tested.
* @author Javier
*/
public class TestSetClear2 {
public static interface MicroBenchmark {
public String getName();
/**
*
* @param dataSet Data set to insert in the collection
* @param initialSize Initial size for the collection. Can try to be optimal or try to fool.
* @param iterations Number of times to go through the dataSet over and over
*/
public void run(int[] dataSet, int initialSize, int iterations);
}
/** Bad initial case. Based in question code */
public static class MBList implements MicroBenchmark {
@Override public String getName() { return "ArrayList.clear()"; }
@Override public void run(int[] data, int initialSize, int n) {
// Not taking initial size into account may result in a resizing penalty in the first iteration
// But will have an adequate size in following iterations, and wont be fooled by bad estimations.
List<Integer> values = new ArrayList<Integer>();
for (int iter = 0; iter < n; iter++) {
values.clear();
for (int i = 0; i < data.length; i++) {
int x = data[i];
if (values.contains(x)) continue;
values.add(x);
}
}
}
}
/** new HashSet(N,1) for every iteration. Reported as best by assylias. */
public static class MBNewHashSetN1 implements MicroBenchmark {
@Override public String getName() { return "new HashSet(N,1)"; }
@Override public void run(int[] data, int initialSize, int n) {
for (int iter = 0; iter < n; iter++) {
Set<Integer> values = new HashSet<>(initialSize, 1.0f); // 1.0 loadfactor optimal if no collisions.
for (int i = 0; i < data.length; i++) {
int x = data[i];
if (values.contains(x)) continue;
values.add(x);
}
}
}
}
// No need to implement raw new HashSet() (reported as worse). Will be enough fooling to initialize to 16 so it succumbs to resizing.
/** HashsetClear for every iteration. Attempted by Assylias and Javier. Clear() does not perform as well as expected under basic tests. */
public static class MBHashSetClear implements MicroBenchmark {
private float loadFactor; // Allow loadFactor to check how much 1.0 factor affects if there are collisions.
private String name;
public MBHashSetClear(float loadFactor) {
this.loadFactor = loadFactor;
name = String.format(Locale.ENGLISH, "HashSet(N,%f).clear()", loadFactor);
}
@Override public String getName() { return name; }
@Override public void run(int[] data, int initialSize, int n) {
HashSet<Integer> values = new HashSet<>((int)Math.ceil(initialSize/loadFactor), loadFactor);// Just the size for loadfactor so it wont resize.
for (int iter = 0; iter < n; iter++) {
values.clear();
for (int i = 0; i < data.length; i++) {
int x = data[i];
if (values.contains(x)) continue;
values.add(x);
}
}
}
}
/** Javier BitSet. Might clearly outperform HashSet, but only on the very specific constraints of the test (non negative integers, not hugely big). */
public static class MBBitSet implements MicroBenchmark {
@Override public String getName() { return "BitSet.clear()"; }
@Override public void run(int[] data, int distributionSize, int n) {
BitSet values = new BitSet(distributionSize);
for (int iter = 0; iter < n; iter++) {
values.clear();
for (int i = 0; i < data.length; i++) {
int x = data[i];
if (values.get(x)) continue;
values.set(x);
}
}
}
}
public static void main(String[] args) {
final MicroBenchmark mbNew = new MBNewHashSetN1();
// Create with same loadFactor as MBNewHashSetN1. So we compare apples with apples (same size of handled table, same collisions).
final MicroBenchmark mbClear = new MBHashSetClear(1.0f);
final MicroBenchmark mbClear075 = new MBHashSetClear(0.75f);
final MicroBenchmark mbBitset = new MBBitSet();
final MicroBenchmark mbList = new MBList(); // Will have a taste of O(N) with a not too bit dataset.
// warmup. trigger the cpu high performance mode? Fill the heap with garbage?
//mbNew.run(dataSetE3xE3, 1000, (int)1e5); // Using new HS might give a bit advantage?
int timePerTest = 10000;
int distributionSize, initialCapacity, datasetLength;
// 1000 long and values 0..999 (1e3 x 1e3). Optimal initial capacity
distributionSize = 1000; datasetLength = 1000; initialCapacity = 1000;
final int[] dataSetE3xE3 = generateRandomSet(1000,1000);
runBenchmark("E3xE3", dataSetE3xE3, distributionSize, timePerTest, initialCapacity,
mbNew, mbClear, mbClear075, mbBitset);
// repeat with underestimated initial size. Will incur in resizing penalty
initialCapacity = 16; // Default initial
runBenchmark("E3xE3+underSize", dataSetE3xE3, distributionSize, timePerTest, initialCapacity,
mbNew, mbClear, mbBitset);
// repeat with overestimated initial size. larger garbage and clearing.
initialCapacity = 100000; // oversized will force to handle large tables filled with 0 / null.
runBenchmark("E3xE3+overSize", dataSetE3xE3, distributionSize, timePerTest, initialCapacity,
mbNew, mbClear, mbBitset);
// Dry run (not rum). what if we focus on the new and clear operations. Just 1 item so clear() is forced to traverse the table.
datasetLength = 1; distributionSize = 1000; initialCapacity = 1000;
runBenchmark("E3xE3-DryRun", generateRandomSet(datasetLength, distributionSize),
distributionSize, timePerTest, initialCapacity,
mbNew, mbClear);
// check for * 100 and / 100 sizes.
distributionSize = datasetLength = initialCapacity = 10;
runBenchmark("E1xE1",
generateRandomSet(datasetLength, distributionSize),
distributionSize, timePerTest, initialCapacity,
mbNew, mbClear, mbList);
distributionSize = datasetLength = initialCapacity = 100000;
runBenchmark("E5xE5", generateRandomSet(datasetLength, distributionSize),
distributionSize, timePerTest, initialCapacity,
mbNew, mbClear);
// Concentrated distributions might behave as with oversized?
datasetLength=10000; distributionSize=10; initialCapacity=Math.min(datasetLength, distributionSize);
runBenchmark("E4xE1",
generateRandomSet(datasetLength, distributionSize),
distributionSize, timePerTest, initialCapacity,
mbNew, mbClear);
// Sparse distributions might allow mild collision. Also adverse for BitSet.
// TODO Generate a greater/known amount of collisions
datasetLength=10000; distributionSize=(int)1e6; initialCapacity=Math.min(datasetLength, distributionSize);
runBenchmark("E4xE6",
generateRandomSet(datasetLength, distributionSize),
distributionSize, timePerTest, initialCapacity,
mbNew, mbClear, mbClear075);
}
private static void runBenchmark(String testName, int[] dataSet, int distributionSize, int timePerTest
, int initialCapacity, MicroBenchmark ... testees /* not testes */) {
// How many iterations? Will use first testee to callibrate.
MicroBenchmark curTest = testees[0];
long t0 = System.nanoTime();
long ellapsed = 0L;
final long minToCallibrate = (long)0.5e9; // half second
int iterations = 1;
while (ellapsed < minToCallibrate) {
curTest.run(dataSet, initialCapacity, iterations);
iterations *= 2; // same as <<= 1
ellapsed = System.nanoTime() - t0;
}
// calculation is not laser-sharp precise (actually executed iterations -1, and some extra initializations).
final int nIterations = (int) ((double)iterations * timePerTest * 1e6 /* nanos/millis */ / ellapsed);
// Do actual benchmark
System.out.printf(Locale.ENGLISH, "dataset:{name=%s,length:%d,distrib:%d,capacity0:%d,iterations:%d}\n",
testName, dataSet.length, distributionSize, initialCapacity, nIterations);
for (MicroBenchmark testee : testees) {
t0 = System.nanoTime();
testee.run(dataSet, initialCapacity, nIterations);
ellapsed = System.nanoTime() - t0;
System.out.printf(Locale.ENGLISH, "%s : %5.3f\n", testee.getName(), ellapsed/1e9 );
}
}
private static int[] generateRandomSet(int lengthOfSet, int distributionSize) {
Random r = new Random();
int[] result = new int[lengthOfSet];
for (int i = 0; i < lengthOfSet; i++) {
result[i] = r.nextInt(distributionSize);
}
return result;
}
}
这是我的结果(使用 JDK 1.8.0_31 - 64 位 - Windows 7)
dataset:{name=E3xE3,length:1000,distrib:1000,capacity0:1000,iterations:514241}
new HashSet(N,1) : 7.688
HashSet(N,1.000000).clear() : 9.796
HashSet(N,0.750000).clear() : 9.923
BitSet.clear() : 1.990
dataset:{name=E3xE3+underSize,length:1000,distrib:1000,capacity0:16,iterations:420572}
new HashSet(N,1) : 9.735
HashSet(N,1.000000).clear() : 6.637
BitSet.clear() : 1.611
dataset:{name=E3xE3+overSize,length:1000,distrib:1000,capacity0:100000,iterations:143032}
new HashSet(N,1) : 9.948
HashSet(N,1.000000).clear() : 10.377
BitSet.clear() : 0.447
dataset:{name=E3xE3-DryRun,length:1,distrib:1000,capacity0:1000,iterations:18511486}
new HashSet(N,1) : 9.583
HashSet(N,1.000000).clear() : 7.523
dataset:{name=E1xE1,length:10,distrib:10,capacity0:10,iterations:76177852}
new HashSet(N,1) : 9.988
HashSet(N,1.000000).clear() : 10.521
ArrayList.clear() : 7.915
dataset:{name=E5xE5,length:100000,distrib:100000,capacity0:100000,iterations:2892}
new HashSet(N,1) : 9.764
HashSet(N,1.000000).clear() : 9.615
dataset:{name=E4xE1,length:10000,distrib:10,capacity0:10,iterations:170386}
new HashSet(N,1) : 9.843
HashSet(N,1.000000).clear() : 9.708
dataset:{name=E4xE6,length:10000,distrib:1000000,capacity0:10000,iterations:36497}
new HashSet(N,1) : 9.686
HashSet(N,1.000000).clear() : 10.079
HashSet(N,0.750000).clear() : 10.008