【问题标题】:Data structure with unique elements and fast add and remove具有独特元素和快速添加和删除的数据结构
【发布时间】:2011-08-31 18:36:13
【问题描述】:

我需要一个具有以下属性的数据结构:

  • 结构的每个元素都必须是唯一的。
  • 添加:将一个元素添加到数据结构中,除非该元素已经 存在。
  • Pop:从数据结构中删除一个元素并返回该元素 删除。删除哪个元素并不重要。

此结构不需要其他操作。一个简单的列表实现将需要几乎 O(1) 时间来弹出和 O(N) 时间来添加(因为必须检查整个列表以确保 唯一性)。我目前正在使用红黑树来满足这种数据结构的需求,但我想知道是否可以使用不太复杂的东西来实现几乎相同的性能。

我更喜欢 C# 的答案,但 Java、Javascript 和 C++ 也可以接受。

我的问题类似于this question,但是我不需要查找或删除最大值或最小值(或者实际上是任何特定类型的值),所以我希望在这方面会有改进。但是,如果该问题中的任何结构在这里合适,请告诉我。

那么,什么数据结构只允许唯一元素,支持快速添加和删除,并且比红黑树更简单?

【问题讨论】:

    标签: c# algorithm data-structures


    【解决方案1】:

    内置的HashSet<T>呢?

    它只包含独特的元素。 Remove (pop) 是 O(1),Add 是 O(1),除非必须调整内部数组的大小。

    【讨论】:

    • 感谢您的指点。不过,我对如何实现 HashSet 很感兴趣,以防我需要为其他语言做同样的事情。据我了解,HashSet 仅在 .NET 3.5 中引入。不过,我担心 HashSet 因其名称而仅与哈希码进行比较,而不是相互比较元素。如果是这样的话,HashSet 就不是真的合适了。
    • 顾名思义,它是作为哈希表实现的,因此可以快速查找元素。
    • 看看我的编辑。恐怕这里不适合散列集。
    • @Peter,哈希码是第一轮检查,相等是第二轮。换句话说,当哈希码匹配时 -> 检查是否完全相等。这就是它快速的原因,它可以通过哈希码进行内部平衡,并且只检查少数元素是否相等。
    • 啊,那会让哈希集更合适。不过,如果有人能解释有关如何自己实现哈希集的详细信息,我将不胜感激。这对其他人也将有用。与此同时,这个答案似乎是最好的。
    【解决方案2】:

    正如 Meta-Knight 所说,HashSet 是最快的数据结构。查找和删除需要恒定的 O(1) 时间(除非在极少数情况下,您的哈希很糟糕,然后您需要多次重新哈希或使用存储桶哈希集)。哈希集上的所有操作都需要 O(1) 时间,唯一的缺点是它需要更多内存,因为哈希被用作数组(或其他分配的内存块)的索引。因此,除非您对内存非常严格,否则请使用 HashSet。我只是在解释为什么你应该采用这种方法并且你应该首先接受 Meta-Knights 的回答。

    使用哈希是可以的,因为通常你会覆盖 HashCode() 和 Equals() 函数。 HashSet 在内部做的是生成哈希,然后如果相等则检查是否相等(以防哈希冲突)。如果不是,则必须调用一种方法来执行称为 rehashing 的操作,该方法会生成一个新哈希,该哈希通常与原始哈希的偏移量为奇数(不确定 .NET 是否这样做,但其他语言是否这样做)并根据需要重复该过程.

    【讨论】:

    【解决方案3】:

    从哈希集或字典中删除随机元素非常容易。 一切平均为 O(1),在现实世界中意味着 O(1)。 示例:

    public class MyNode
    {
        ...
    }
    
    public class MyDataStructure
    {
        private HashSet<MyNode> nodes = new HashSet<MyNode>();
    
        /// <summary>
        /// Inserts an element to this data structure. 
        /// If the element already exists, returns false.
        /// Complexity is averaged O(1).
        /// </summary>
        public bool Add(MyNode node)
        {
            return node != null && this.nodes.Add(node);
        }
    
        /// <summary>
        /// Removes a random element from the data structure.
        /// Returns the element if an element was found.
        /// Returns null if the data structure is empty.
        /// Complexity is averaged O(1).
        /// </summary>
        public MyNode Pop()
        {
            // This loop can execute 1 or 0 times.
            foreach (MyNode node in nodes)
            {
                this.nodes.Remove(node);
                return node;
            }
            return null;
        }
    }
    

    根据我的经验,几乎所有可以比较的东西也可以被散列:)。 我想知道是否有人知道无法散列的东西。

    根据我的经验,这也适用于一些使用特殊技术进行容差的浮点比较。

    散列表的散列函数不需要完美,只要足够好。 此外,如果您的数据非常复杂,通常哈希函数不如红黑树或 avl 树复杂。 它们很有用,因为它们可以让事情井井有条,但您不需要这个。

    为了展示如何做一个简单的哈希集,我将考虑一个带有整数键的简单字典。 这个实现非常快,非常适合用于示例的稀疏数组。 我没有编写增加存储桶表的代码,因为它很烦人并且通常是大错误的来源,但由于这是一个概念证明,所以应该足够了。 我也没有写迭代器。

    我写的,可能有bug。

    public class FixedIntDictionary<T>
    {
        // Our internal node structure.
        // We use structs instead of objects to not add pressure to the garbage collector.
        // We mantains our own way to manage garbage through the use of a free list.
        private struct Entry
        {
            // The key of the node
            internal int Key;
    
            // Next index in pEntries array.
            // This field is both used in the free list, if node was removed
            // or in the table, if node was inserted.
            // -1 means null.
            internal int Next;
    
            // The value of the node.
            internal T Value;
        }
    
        // The actual hash table. Contains indices to pEntries array.
        // The hash table can be seen as an array of singlt linked list.
        // We store indices to pEntries array instead of objects for performance
        // and to avoid pressure to the garbage collector.
        // An index -1 means null.
        private int[] pBuckets;
    
        // This array contains the memory for the nodes of the dictionary.
        private Entry[] pEntries;
    
        // This is the first node of a singly linked list of free nodes.
        // This data structure is called the FreeList and we use it to
        // reuse removed nodes instead of allocating new ones.
        private int pFirstFreeEntry;
    
        // Contains simply the number of items in this dictionary.
        private int pCount;
    
        // Contains the number of used entries (both in the dictionary or in the free list) in pEntries array.
        // This field is going only to grow with insertions.
        private int pEntriesCount;
    
        ///<summary>
        /// Creates a new FixedIntDictionary. 
        /// tableBucketsCount should be a prime number
        /// greater than the number of items that this
        /// dictionary should store.
        /// The performance of this hash table will be very bad
        /// if you don't follow this rule!
        /// </summary>
        public FixedIntDictionary<T>(int tableBucketsCount)
        {
            // Our free list is initially empty.
            this.pFirstFreeEntry = -1;
    
            // Initializes the entries array with a minimal amount of items.
            this.pEntries = new Entry[8];
    
            // Allocate buckets and initialize every linked list as empty.
            int[] buckets = new int[capacity];
            for (int i = 0; i < buckets.Length; ++i)
                buckets[i] = -1;
    
            this.pBuckets = buckets;
        }
    
        ///<summary>Gets the number of items in this dictionary. Complexity is O(1).</summary>
        public int Count
        {
            get { return this.pCount; }
        }
    
        ///<summary>
        /// Adds a key value pair to the dictionary.
        /// Complexity is averaged O(1).
        /// Returns false if the key already exists.
        /// </summary>
        public bool Add(int key, T value)
        {
            // The hash table can be seen as an array of linked list.
            // We find the right linked list using hash codes.
            // Since the hash code of an integer is the integer itself, we have a perfect hash.
    
            // After we get the hash code we need to remove the sign of it.
            // To do that in a fast way we and it with 0x7FFFFFFF, that means, we remove the sign bit.
            // Then we have to do the modulus of the found hash code with the size of our buckets array.
    
            // For this reason the size of our bucket array should be a prime number,
            // this because the more big is the prime number, the less is the chance to find an
            // hash code that is divisible for that number. This reduces collisions.
    
            // This implementation will not grow the buckets table when needed, this is the major
            // problem with this implementation.
            // Growing requires a little more code that i don't want to write now
            // (we need a function that finds prime numbers, and it should be fast and we
            // need to rehash everything using the new buckets array).
    
            int bucketIndex = (key & 0x7FFFFFFF) % this.pBuckets.Length;
            int bucket = this.pBuckets[bucketIndex];
    
            // Now we iterate in the linked list of nodes.
            // Since this is an hash table we hope these lists are very small.
            // If the number of buckets is good and the hash function is good this will translate usually 
            // in a O(1) operation.
    
            Entry[] entries = this.pEntries;
            for (int current = entries[bucket]; current != -1; current = entries[current].Next)
            {
                if (entries[current].Key == key)
                {
                    // Entry already exists.
                    return false;
                }
            }
    
            // Ok, key not found, we can add the new key and value pair.
    
            int entry = this.pFirstFreeEntry;
            if (entry != -1)
            {
                // We found a deleted node in the free list.
                // We can use that node without "allocating" another one.
                this.pFirstFreeEntry = entries[entry].Next;
            }
            else
            {
                // Mhhh ok, the free list is empty, we need to allocate a new node.
                // First we try to use an unused node from the array.
                entry = this.pEntriesCount++;
                if (entry >= this.pEntries)
                {
                    // Mhhh ok, the entries array is full, we need to make it bigger.
                    // Here should go also the code for growing the bucket table, but i'm not writing it here.
                    Array.Resize(ref this.pEntries, this.pEntriesCount * 2);
                    entries = this.pEntries;
                }
            }
    
            // Ok now we can add our item.
            // We just overwrite key and value in the struct stored in entries array.
    
            entries[entry].Key = key;
            entries[entry].Value = value;
    
            // Now we add the entry in the right linked list of the table.
    
            entries[entry].Next = this.pBuckets[bucketIndex];
            this.pBuckets[bucketIndex] = entry;
    
            // Increments total number of items.
            ++this.pCount;
    
            return true;
        }
    
        /// <summary>
        /// Gets a value that indicates wether the specified key exists or not in this table.
        /// Complexity is averaged O(1).
        /// </summary>
        public bool Contains(int key)
        {
            // This translate in a simple linear search in the linked list for the right bucket.
            // The operation, if array size is well balanced and hash function is good, will be almost O(1).
    
            int bucket = this.pBuckets[(key & 0x7FFFFFFF) % this.pBuckets.Length];
            Entry[] entries = this.pEntries;
            for (int current = entries[bucket]; current != -1; current = entries[current].Next)
            {
                if (entries[current].Key == key)
                {
                    return true;
                }
            }
            return false;
        }
    
        /// <summary>
        /// Removes the specified item from the dictionary.
        /// Returns true if item was found and removed, false if item doesn't exists.
        /// Complexity is averaged O(1).
        /// </summary>
        public bool Remove(int key)
        {
            // Removal translate in a simple contains and removal from a singly linked list.
            // Quite simple.
    
            int bucketIndex = (key & 0x7FFFFFFF) % this.pBuckets.Length;
            int bucket = this.pBuckets[bucketIndex];
            Entry[] entries = this.pEntries;
            int next;
            int prev = -1;
            int current = entries[bucket];
    
            while (current != -1)
            {
                next = entries[current].Next;
    
                if (entries[current].Key == key)
                {
                    // Found! Remove from linked list.
                    if (prev != -1)
                        entries[prev].Next = next;
                    else
                        this.pBuckets[bucketIndex] = next;
    
                    // We now add the removed node to the free list,
                    // so we can use it later if we add new elements.
                    entries[current].Next = this.pFirstFreeEntry;
                    this.pFirstFreeEntry = current;
    
                    // Decrements total number of items.
                    --this.pCount;
    
                    return true;
                }
    
                prev = current;
                current = next;
            }
            return false;
        }
    
    }
    

    如果你徘徊这个实现是否好,它是一个非常类似于 .NET 框架为 Dictionary 类所做的实现:)

    要使其成为哈希集,只需删除 T 即可获得整数哈希集。 如果您需要获取通用对象的哈希码,只需使用 x.GetHashCode 或提供您的哈希码函数。

    要编写迭代器,您需要修改几处,但不想在这篇文章中添加太多其他内容:)

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 2012-08-17
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2021-07-22
      • 1970-01-01
      • 2011-05-17
      相关资源
      最近更新 更多