ValueError：传递的项目数错误 1，位置暗示 2

Question

我通过多处理计算出了错误 ROC 曲线和运行大型数据集。此过程运行正常，但 30 分钟后，我突然收到错误消息。

多处理代码已更改：

for i in range(0, len(values) - 1):
        for j in range(i + 1, len(values)):
            # print(values[i], " and ", values[j])
            positive = []
            positive.append(values[i])
            positive.append(values[j])
            print(positive)
            proc = multiprocessing.Process(args=(positive,)) 
            print(proc)
            positives.append(proc)
            proc.start()

        for proc in positives:
            proc.join()

positives = pd.DataFrame(positives, columns=["file_x", "file_y"])

追溯

    ['1013_StephenHawking_67_m.jpg', '1023_StephenHawking_45_m.jpg']
<Process(Process-236167, initial)>
Traceback (most recent call last):
  File "/home/khawar/.local/lib/python3.6/site-packages/pandas/core/internals/managers.py", line 1671, in create_block_manager_from_blocks
    make_block(values=blocks[0], placement=slice(0, len(axes[0])))
  File "/home/khawar/.local/lib/python3.6/site-packages/pandas/core/internals/blocks.py", line 2744, in make_block
    return klass(values, ndim=ndim, placement=placement)
  File "/home/khawar/.local/lib/python3.6/site-packages/pandas/core/internals/blocks.py", line 2400, in __init__
    super().__init__(values, ndim=ndim, placement=placement)
  File "/home/khawar/.local/lib/python3.6/site-packages/pandas/core/internals/blocks.py", line 131, in __init__
    f"Wrong number of items passed {len(self.values)}, "
ValueError: Wrong number of items passed 1, placement implies 2

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/khawar/deepface/tests/Ensemble-Face-Recognition.py", line 53, in <module>
    positives = pd.DataFrame(positives, columns=["file_x", "file_y"])
  File "/home/khawar/.local/lib/python3.6/site-packages/pandas/core/frame.py", line 523, in __init__
    mgr = init_ndarray(data, index, columns, dtype=dtype, copy=copy)
  File "/home/khawar/.local/lib/python3.6/site-packages/pandas/core/internals/construction.py", line 234, in init_ndarray
    return create_block_manager_from_blocks(block_values, [columns, index])
  File "/home/khawar/.local/lib/python3.6/site-packages/pandas/core/internals/managers.py", line 1681, in create_block_manager_from_blocks
    raise construction_error(tot_items, blocks[0].shape[1:], axes, e)
ValueError: Shape of passed values is (236167, 1), indices imply (236167, 2)

Answer 1

这可能是索引混乱的情况。

尝试使用.loc[df.index.drop_duplicates()] 删除任何潜在的重复索引：

positives = pd.DataFrame(positives, columns=["file_x", "file_y"]).loc[positives.index.drop_duplicates()]

Answer 2

已编辑

我想再次强调您的问题是关于ValueError: Wrong number of items passed 1, placement implies 2。

ValueError 发生是因为您的变量 positives 存储的是 multiporcessing.Process 的对象而不是 (str, str)，这是有问题的。

由于您只是配对字符串，我会改为执行以下操作：

from itertools import combinations

data = idendities # using your json data 

# list comprehensive, create pairwise combination 
positives = [ list(combinations(v, 2) for key, value in data.items() ]
 
# unset nested lists 
positives = [ pair for l in positives for pair in l ]
# len(positives) = 3300 

positives = pd.DataFrame(positives, columns=["file_x", "file_y"])

输出

# positives before doing pd.DataFrame 

[('1020_StephenHawking_65_m.jpg', '1004_StephenHawking_43_m.jpg'),
 ('1020_StephenHawking_65_m.jpg', '1017_StephenHawking_65_m.jpg'),
 ('1020_StephenHawking_65_m.jpg', '1014_StephenHawking_67_m.jpg'),
 ('1020_StephenHawking_65_m.jpg', '1006_StephenHawking_36_m.jpg'),
 ('1020_StephenHawking_65_m.jpg', '1000_StephenHawking_1_m.jpg'),
 ('1020_StephenHawking_65_m.jpg', '1018_StephenHawking_66_m.jpg'),
   ... ... ]

# positives after pd.DataFrame

                         file_x                        file_y
0  1020_StephenHawking_65_m.jpg  1004_StephenHawking_43_m.jpg
1  1020_StephenHawking_65_m.jpg  1017_StephenHawking_65_m.jpg
2  1020_StephenHawking_65_m.jpg  1014_StephenHawking_67_m.jpg
3  1020_StephenHawking_65_m.jpg  1006_StephenHawking_36_m.jpg
4  1020_StephenHawking_65_m.jpg   1000_StephenHawking_1_m.jpg

而且，内存问题肯定不是来自这部分代码。