功能上按空格分割字符串，按引号分组！

Question

在 Clojure[1] 中编写惯用的函数式代码，如何编写一个用空格分割字符串但保持引用短语完整的函数？一个快速的解决方案当然是使用正则表达式，但如果没有它们，这应该是可能的。乍一看，似乎很难！我用命令式语言写过类似的东西，但我想看看函数式递归方法是如何工作的。

快速检查一下我们的函数应该做什么：

"Hello there!"  -> ["Hello", "there!"]
"'A quoted phrase'" -> ["A quoted phrase"]
"'a' 'b' c d" -> ["a", "b", "c", "d"]
"'a b' 'c d'" -> ["a b", "c d"]
"Mid'dle 'quotes do not concern me'" -> ["Mid'dle", "quotes do not concern me"]

我不介意引号之间的间距是否发生变化（因此可以先使用空格进行简单分割）。

"'lots    of   spacing' there" -> ["lots of spacing", "there"] ;is ok to me

[1] 这个问题可以笼统地回答，但我想 Clojure 中的函数式方法可以轻松地转换为 Haskell、ML 等。

Answer 1

这是一个返回惰性单词序列/引用字符串的版本：

(defn splitter [s]
  (lazy-seq
   (when-let [c (first s)]
     (cond
      (Character/isSpace c)
      (splitter (rest s))
      (= \' c)
      (let [[w* r*] (split-with #(not= \' %) (rest s))]
        (if (= \' (first r*))
          (cons (apply str w*) (splitter (rest r*)))
          (cons (apply str w*) nil)))
      :else
      (let [[w r] (split-with #(not (Character/isSpace %)) s)]
        (cons (apply str w) (splitter r)))))))

测试运行：

user> (doseq [x ["Hello there!"
                 "'A quoted phrase'"
                 "'a' 'b' c d"
                 "'a b' 'c d'"
                 "Mid'dle 'quotes do not concern me'"
                 "'lots    of   spacing' there"]]
        (prn (splitter x)))
("Hello" "there!")
("A quoted phrase")
("a" "b" "c" "d")
("a b" "c d")
("Mid'dle" "quotes do not concern me")
("lots    of   spacing" "there")
nil

如果输入中的单引号没有正确匹配，则从最后一个单引号开始的所有内容都将被视为一个“单词”：

user> (splitter "'asdf")
("asdf")

---

更新：回应 edbond 评论的另一个版本，更好地处理单词中的引号字符：

(defn splitter [s]
  ((fn step [xys]
     (lazy-seq
      (when-let [c (ffirst xys)]
        (cond
         (Character/isSpace c)
         (step (rest xys))
         (= \' c)
         (let [[w* r*]
               (split-with (fn [[x y]]
                             (or (not= \' x)
                                 (not (or (nil? y)
                                          (Character/isSpace y)))))
                           (rest xys))]
           (if (= \' (ffirst r*))
             (cons (apply str (map first w*)) (step (rest r*)))
             (cons (apply str (map first w*)) nil)))
         :else
         (let [[w r] (split-with (fn [[x y]] (not (Character/isSpace x))) xys)]
           (cons (apply str (map first w)) (step r)))))))
   (partition 2 1 (lazy-cat s [nil]))))

测试运行：

user> (doseq [x ["Hello there!"
                 "'A quoted phrase'"
                 "'a' 'b' c d"
                 "'a b' 'c d'"
                 "Mid'dle 'quotes do not concern me'"
                 "'lots    of   spacing' there"
                 "Mid'dle 'quotes do no't concern me'"
                 "'asdf"]]
        (prn (splitter x)))
("Hello" "there!")
("A quoted phrase")
("a" "b" "c" "d")
("a b" "c d")
("Mid'dle" "quotes do not concern me")
("lots    of   spacing" "there")
("Mid'dle" "quotes do no't concern me")
("asdf")
nil

Answer 2

这个解决方案在haskell中，但主要思想也应该适用于clojure。
解析器的两种状态（引号内或引号外）由两个相互递归的函数表示。

splitq = outside [] . (' ':)

add c res = if null res then [[c]] else map (++[c]) res

outside res xs = case xs of
    ' '  : ' '  : ys -> outside res $ ' ' : ys
    ' '  : '\'' : ys -> res ++ inside [] ys
    ' '  : ys        -> res ++ outside [] ys
    c    : ys        -> outside (add c res) ys
    _                -> res

inside res xs = case xs of
    ' '  : ' ' : ys -> inside res $ ' ' : ys
    '\'' : ' ' : ys -> res ++ outside [] (' ' : ys)
    '\'' : []       -> res
    c    : ys       -> inside (add c res) ys
    _               -> res

Answer 3

这是 Clojure 版本。对于非常大的输入，这可能会破坏堆栈。正则表达式或真正的解析器生成器会更简洁。

(declare parse*)
(defn slurp-word [words xs terminator]
  (loop [res "" xs xs]
    (condp = (first xs)
      nil  ;; end of string after this word
      (conj words res)

      terminator ;; end of word
      (parse* (conj words res) (rest xs))

      ;; else
      (recur (str res (first xs)) (rest xs)))))

(defn parse* [words xs]
  (condp = (first xs)
    nil ;; end of string
    words

    \space  ;; skip leading spaces
    (parse* words (rest xs))

    \' ;; start quoted part
    (slurp-word words (rest xs) \')

    ;; else slurp until space
    (slurp-word words xs \space)))

(defn parse [s]
  (parse* [] s))

您的意见：

user> (doseq [x ["Hello there!"
                 "'A quoted phrase'"
                 "'a' 'b' c d"
                 "'a b' 'c d'"
                 "Mid'dle 'quotes do not concern me'"
                 "'lots    of   spacing' there"]]
        (prn (parse x)))

["Hello" "there!"]
["A quoted phrase"]
["a" "b" "c" "d"]
["a b" "c d"]
["Mid'dle" "quotes do not concern me"]
["lots    of   spacing" "there"]
nil

Answer 4

能够修改 Brian 以使用 trampoline 以使其不会耗尽堆栈空间。基本上让slurp-word 和parse* 返回函数而不是执行它们，然后将parse 更改为使用trampoline

(defn slurp-word [words xs terminator]
  (loop [res "" xs xs]
    (condp = (first xs)
        nil  ;; end of string after this word
      (conj words res)

      terminator ;; end of word
      #(parse* (conj words res) (rest xs))

      ;; else
      (recur (str res (first xs)) (rest xs)))))

(defn parse* [words xs]
  (condp = (first xs)
      nil ;; end of string
    words

    \space  ;; skip leading spaces
    (parse* words (rest xs))

    \' ;; start quoted part
    #(slurp-word words (rest xs) \')

    ;; else slurp until space
    #(slurp-word words xs \space)))

    (defn parse [s]
      (trampoline #(parse* [] s)))


(defn test-parse []
  (doseq [x ["Hello there!"
             "'A quoted phrase'"
             "'a' 'b' c d"
             "'a b' 'c d'"
             "Mid'dle 'quotes do not concern me'"
             "'lots    of   spacing' there"
             (apply str (repeat 30000 "'lots    of   spacing' there"))]]
    (prn (parse x))))

Answer 5

例如fnparse 允许您以函数式方式编写解析器。

Answer 6

使用正则表达式：

 (defn my-split [string]
  (let [criterion " +(?=([^']*'[^']*')*[^']*$)"]
   (for [s (into [] (.split string criterion))] (.replace s "'" ""))))

正则表达式中的第一个字符是您要分割字符串的字符 - 这里至少是一个空格..

如果您想更改引用字符，只需将每个 ' 更改为 /" 之类的其他内容。

编辑：我刚刚看到您明确提到您不想使用正则表达式。对不起！

Answer 7

哦，天哪，既然我的测试成功了，给出的答案似乎比我的要好。不管怎样，我把它贴在这里是为了乞求一些 cmet 将代码惯用化。

我画了一个haskellish伪：

pl p w:ws = | if w:ws empty
               => p
            | if w begins with a quote
               => pli p w:ws
            | otherwise
               => pl (p ++ w) ws

pli p w:ws = | if w:ws empty
                => p
             | if w begins with a quote
                => pli (p ++ w) ws
             | if w ends with a quote
                => pl (init p ++ (tail p ++ w)) ws
             | otherwise
                => pli (init p ++ (tail p ++ w)) ws

好吧，名字不好。那里

• 函数pl 处理未引用的单词
• 函数pli（i as in inner）处理引用的短语
• 参数（列表）p是已经处理（完成）的信息
• 参数（列表）w:ws是要处理的信息

我是这样翻译伪的：

(def quote-chars '(\" \')) ;'

; rewrite .startsWith and .endsWith to support multiple choices
(defn- starts-with?
  "See if given string begins with selected characters."
  [word choices]
  (some #(.startsWith word (str %)) choices))

(defn- ends-with?
  "See if given string ends with selected characters."
  [word choices]
  (some #(.endsWith word (str %)) choices))

(declare pli)
(defn- pl [p w:ws]
    (let [w (first w:ws)
          ws (rest w:ws)]
     (cond
        (nil? w)
            p
        (starts-with? w quote-chars)
            #(pli p w:ws)
        true
            #(pl (concat p [w]) ws))))

(defn- pli [p w:ws]
    (let [w (first w:ws)
          ws (rest w:ws)]
     (cond
        (nil? w)
            p
        (starts-with? w quote-chars)
            #(pli (concat p [w]) ws)
        (ends-with? w quote-chars)
            #(pl (concat 
                  (drop-last p)
                  [(str (last p) " " w)])
                ws)
        true
            #(pli (concat 
                  (drop-last p)
                  [(str (last p) " " w)])
                ws))))

(defn split-line
    "Split a line by spaces, leave quoted groups intact."
    [input]
    (let [splt (.split input " +")]
        (map strip-input 
            (trampoline pl [] splt))))

不是很 Clojuresque，细节。此外，我在拆分和剥离引号时依赖于正则表达式，因此我应该得到一些反对意见。