【问题标题】:manipulating an atom containing a ref collection in clojure在 clojure 中操作包含 ref 集合的原子
【发布时间】:2018-05-12 01:24:34
【问题描述】:

我有一个应用程序,它应该在指定预算范围内为客户预订航班。因此,我有客户数据和可用航班数据。然后我在 Clojure 中开发解决方案如下。

首先,我创建了一个航班原子:

(def flights
   (atom []))

然后我创建了一个函数来将航班初始化为一个包含 ref 集合的 atom。在这里,我传递了包含在这篇文章后面的航班数据。

(defn initialize-flights [initial-flights]
   (reset! flights (map ref initial-flights)))

然后我通过 process-customers 函数来处理客户,如下所示。这就是真正令人困惑的地方。

(defn process-customers [customers]

(doseq [customer1 (partitionCustomerInput N-THREADS customers)]


  (doseq [customer2  customer1]

    (swap! flights
      (fn [flights_collection]
        (if-let [updated-flight (process-customer flights_collection customer2)]

          (assoc flights (:id updated-flight) updated-flight)
          flights_collection)))))


  (reset! finished-processing? true))

在流程客户内部,我将航班集合传递给流程客户(注意流程客户是流程客户的辅助函数,它们不是同一个函数)。此时的Flights-collection 是一个flight refs 过程的集合——客户应该在列表中进行搜索,如果客户有资格获得其中的航班,它会使用book 功能来编辑航班。我应该如何将航班集合传递给流程客户?实际上,流程客户不会搜索航班参考,也不会更改航班参考?

下面是流程客户函数及其辅助函数。

(defn- process-customer [flights customer]
  "Try to book a flight from `flights` for `customer`, returning the updated
  flight if found, or nil if no suitable flight was found."
  (if-let [{:keys [flight price]} (find-flight flights customer)]
    (let [updated-flight (book flight price (:seats customer))]
      (log "Customer" (:id customer) "booked" (:seats customer)
        "seats on flight" (:id updated-flight) "at $" price " (< budget of $"
        (:budget customer) ").")
      updated-flight)
    (do
      (log "Customer" (:id customer) "did not find a flight.")
      nil)))


(defn filter-pricing-with-n-seats [pricing seats]
  "Get `pricing` for which there are at least `seats` empty seats available."
  (filter #(>= (second %) seats) pricing))

(defn lowest-available-price [flight seats]
  "Returns the lowest price in `flight` for which at least `seats` empty seats
  are available, or nil if none found."
  (-> (:pricing flight)                 ; [[price available taken]]
    (filter-pricing-with-n-seats seats)
    (sort-pricing)
    (first)                             ; [price available taken]
    (first)))                           ; price

(defn- find-flight [flights customer]
  "Find a flight in `flights` that is on the route and within the budget of
  `customer`. If a flight was found, returns {:flight flight :price price},
  else returns nil."
  (let [{:keys [_id from to seats budget]}
          customer
        flights-and-prices
          ; flights that are on the route and within budget, and their price
          (for [f flights
                :when (and (= (:from f) from) (= (:to f) to))
                :let [lowest-price (lowest-available-price f seats)]
                :when (and (some? lowest-price) (<= lowest-price budget))]
            {:flight f :price lowest-price})
        cheapest-flight-and-price
          (first (sort-by :price flights-and-prices))]
    cheapest-flight-and-price))

(defn- book [flight price seats]
  "Updates `flight` to book `seats` at `price`."
  (update flight :pricing
    (fn [pricing]
      (for [[p a t] pricing]
        (if (= p price)
          [p (- a seats) (+ t seats)]
          [p a t])))))



(def finished-processing?
  "Set to true once all customers have been processed, so that sales process
  can end."
  (atom false))

(defn partitionCustomerInput 
  [threads customers]
  (let [partitions (partition-all 
     (Math/ceil (/ (count customers) threads))  customers)]
        partitions))

下面是主要功能。它初始化航班并启动客户处理

 (defn main []
      (initialize-flights input/flights)
       (let [f1 (future (time (process-customers input/customers)))


        @f1

        )

      (println "Flights:")
      (print-flights (map deref @flights)))
    (main)
    (shutdown-agents)

以下是客户和航班集合。

(def flights
      [{:id 0
        :from "BRU" :to "ATL"
        :carrier "Delta"
        :pricing [[600 150 0] ; price; # seats available at that price; # seats taken at that price
                  [650  50 0]
                  [700  50 0]
                  [800  50 0]]}
       {:id 1
        :from "BRU" :to "LON"
        :carrier "Brussels Airlines"
        :pricing [[300 150 0]
                  [350  50 0]
                  [370  20 0]
                  [380  30 0]]}
       {:id 2
        :from "BRU" :to "LON"
        :carrier "Brussels Airlines"
        :pricing [[250 100 0]
                  [300  50 0]]}
       {:id 3
        :from "BRU" :to "MAD"
        :carrier "Brussels Airlines"
        :pricing [[200 150 0]
                  [250  50 0]
                  [300 100 0]]}
       {:id 4
        :from "BRU" :to "MAD"
        :carrier "Iberia"
        :pricing [[250 150 0]
                  [300  50 0]]}])

(def customers
  [{:id  0 :from "BRU" :to "ATL" :seats 5 :budget 700}
   {:id  1 :from "BRU" :to "ATL" :seats 5 :budget 550}
   {:id  2 :from "BRU" :to "LON" :seats 6 :budget 270}
   {:id  3 :from "BRU" :to "ATL" :seats 4 :budget 600}
   {:id  4 :from "BRU" :to "LON" :seats 3 :budget 270}
   {:id  5 :from "BRU" :to "LON" :seats 9 :budget 250}
   {:id  6 :from "BRU" :to "MAD" :seats 5 :budget 200}
   {:id  7 :from "BRU" :to "MAD" :seats 9 :budget 150}
   {:id  8 :from "BRU" :to "LON" :seats 5 :budget 250}
   {:id  9 :from "BRU" :to "ATL" :seats 4 :budget 500}
   {:id 10 :from "BRU" :to "MAD" :seats 1 :budget 180}
   {:id 11 :from "BRU" :to "LON" :seats 2 :budget 320}
   {:id 12 :from "BRU" :to "ATL" :seats 3 :budget 850}
   {:id 13 :from "BRU" :to "ATL" :seats 4 :budget 200}])

另外,请注意,我想在此实现中使用 refs 来更改航班,因为 ref 支持协调读写以原子地更改航班。我的目标是为这个应用程序制定一个高度并行化的解决方案,不能容忍冲突。

【问题讨论】:

  • 当我看到一个包裹 refs 的 atom 时,我有点怀疑。您是否已经在不使用 atom 和 refs 的情况下实现了该算法?如果没有,我建议你先这样做。然后我们可以看看如何使用多个线程让它运行得更快。也许我们不需要这么多原子/引用?
  • 请看这个链接:stackoverflow.com/questions/50253397/…。我在那里有两个实施,但他们没有加快速度。即它们还不是平行的。
  • 这就是为什么我选择一个作为飞行参考集合的原子:首先,应该可以改变整个飞行集合和单个飞行。我还希望我的 refs 具有精细的粒度,以确保在更新航班时减少冲突的数量。未能对整个集合使用原子将意味着我无法改变集合,并且对整个集合仅使用一个 ref 并且取消原子将意味着大量的读写冲突。
  • 与您的previous post 一样,我们需要消化大量代码。另外,您的问题到底是什么?此外,当你想“同时”改变两件事时,参考是用的,我在你的代码中看不到这一点。使用原子代替,更容易。也许您想让事情变得更加并行,但是您从哪里开始新线程?我只能看到一个电话给future,这是针对所有客户的,而不是针对每个客户的。如果我是你,我会先用更简单的算法来玩转 futures、atoms 和 refs。

标签: clojure refs


【解决方案1】:

我认为您需要一个 ref 而不是顶层的 atom。看来您需要协调对单个航班的更改并更改为航班列表。如果一个线程正在修改航班,而另一个线程将其从列表中删除怎么办?您的process-customer 副作用必须全部在(dosync) 内完成。

性能方面,应该没问题,因为如果您不在事务中修改您的航班列表参考,它不会导致其他更改它的事务重试。

另一个原因是因为您违反了swap! 的一条非常重要的规则。传递给swap! 的函数必须没有副作用,因为它可以被 STM 重试。更改 ref 是副作用,可能会导致难以理解的错误。

所以我会做类似的事情

(def flights 
  (ref [(ref {:id "flight-1"}) 
        (ref {:id "flight-2"})]))

;; Run a bunch of these in their own threads...
(doseq [customer partitioned-customers]
  (dosync (process-customer customer flights)))

然后您可以使用 altercommuteensure 微调 process-customer,以最大限度地提高并发性并最大限度地减少重试次数。

希望这会有所帮助,祝你好运!

【讨论】:

    猜你喜欢
    • 2012-12-17
    • 1970-01-01
    • 2018-05-22
    • 1970-01-01
    • 1970-01-01
    • 2020-07-09
    • 2011-12-21
    • 1970-01-01
    • 2019-12-25
    相关资源
    最近更新 更多