Pandas：根据开始/结束分割点的字符串列表（重叠）将字符串列拆分为组件列

Question

在我的 Pandas 字符串数据框中，在一列中我有一个大字符串，我想将其拆分为单独的字符串，每个字符串都有自己的行一个新的数据框。第二列是一个标签，相同的标签应该出现在每个字符串组件上。

开始和结束分割点应该由一组字符串确定。每个组件字符串都将从遇到该集合中的一个字符串开始。每个字符串的起点应该在它自己的行的列中，而不应该在拆分的字符串中。

这是一个例子

我有这个数据框

testdf = pd.DataFrame([
    [ 'BACKGROUND\nDiagnostic uncertainty in ALS has serious management implications and delays recruitment into clinical trials. Emerging evidence of presymptomatic disease-burden provides the rationale to develop diagnostic applications based on the evaluation of in-vivo pathological patterns early in the disease.\n\n\nOBJECTIVES\nTo outline and test a diagnostic classification approach based on an array of complementary imaging metrics in key disease-associated anatomical structures.\n\n\nMETHODS\nData from 75 ALS patients and 75 healthy controls were randomly allocated in a training and validation cohort. Spatial masks were created for anatomical foci which best discriminate patients from controls in the training sample. In a virtual brain biopsy, data was then retrieved from these key disease-associated brain regions. White matter diffusivity indices, grey matter T1-signal intensity values and basal ganglia volumes were evaluated as predictor variables in a canonical discriminant function.\n\n\nRESULTS\nFollowing predictor variable selection, a classification specificity of 85.5% and sensitivity of 89.1% was achieved in the training sample and 90% specificity and 90% sensitivity in the validation sample.\n\n\nDISCUSSION\nThis study evaluates disease-associated imaging measures in a dummy diagnostic application. Although larger samples will be required for robust validation, the study confirms the potential of multimodal quantitative imaging in future clinical applications.' , 'Entry1'], 
                       [ '\nProblem statement: The industrialization of the world from whole to s ite as a result of technological innovation made many industries adopt ing Information and Communication Technology (ICT) for processing of all their activities from i nception to completion, especially in the developed nations. But, the developing nations appear to make sluggish progress towards ICT adoption due to apprehensiveness that their fraudulent activities c an easily be traced. \nApproach: The purpose of this study was to evaluate the contractor’s perception t oward ICT innovation acceptance for construction site management and the effectiveness of the innova tion. A 519 questionnaire survey was employed for the data collection, while SPSS version 17.0 wa s used for the descriptive statistic and factorial analysis of the data. \nResults: The findings show ICT innovation was effective for site management but there were positive and negative factors that affec t the ICT innovation based on the contractors view. \nConclusion: By evaluating the ICT innovation, empirical eviden c has been provided for the ‘wait and see contractors’ to adopt ICT in construction site management and by making adequate provisions against the negative factors. ' , 'Entry2'], 
                       ['BACKGROUND AND PURPOSE\nRotator cuff tears are associated with secondary rotator cuff muscle pathology, which is definitive for the prognosis of rotator cuff repair. There is little information regarding the early histological and immunohistochemical nature of these muscle changes in humans. We analyzed muscle biopsies from patients with supraspinatus tendon tears.\n\n\nMETHODS\nSupraspinatus muscle biopsies were obtained from 24 patients undergoing arthroscopic repair of partial- or full-thickness supraspinatus tendon tears. Tissue was formalin-fixed and processed for histology (for assessment of fatty infiltration and other degenerative changes) or immunohistochemistry (to identify satellite cells (CD56+), proliferating cells (Ki67+), and myofibers containing predominantly type 1 or 2 myosin heavy chain (MHC)). Myofiber diameters and the relative content of MHC1 and MHC2 were determined morphometrically.\n\n\nRESULTS\nDegenerative changes were present in both patient groups (partial and full-thickness tears). Patients with full-thickness tears had a reduced density of satellite cells, fewer proliferating cells, atrophy of MHC1+ and MHC2+ myofibers, and reduced MHC1 content.\n\n\nINTERPRETATION\nFull-thickness tears show significantly reduced muscle proliferative capacity, myofiber atrophy, and loss of MHC1 content compared to partial-thickness supraspinatus tendon tears.' ,  'Entry3']
] )
testdf.columns = ['A', 'B']
testdf.head(10)

这个输出

A   B
0   BACKGROUND\nDiagnostic uncertainty in ALS has serious management implications and delays recruitment into clinical trials. Emerging evidence of presymptomatic disease-burden provides the rationale to develop diagnostic applications based on the evaluation of in-vivo pathological patterns early in the disease.\n\n\nOBJECTIVES\nTo outline and test a diagnostic classification approach based on an array of complementary imaging metrics in key disease-associated anatomical structures.\n\n\nMETHODS\nData from 75 ALS patients and 75 healthy controls were randomly allocated in a training and validation cohort. Spatial masks were created for anatomical foci which best discriminate patients from controls in the training sample. In a virtual brain biopsy, data was then retrieved from these key disease-associated brain regions. White matter diffusivity indices, grey matter T1-signal intensity values and basal ganglia volumes were evaluated as predictor variables in a canonical discriminant function.\n\n\nRESULTS\nFollowing predictor variable selection, a classification specificity of 85.5% and sensitivity of 89.1% was achieved in the training sample and 90% specificity and 90% sensitivity in the validation sample.\n\n\nDISCUSSION\nThis study evaluates disease-associated imaging measures in a dummy diagnostic application. Although larger samples will be required for robust validation, the study confirms the potential of multimodal quantitative imaging in future clinical applications. Entry1
1   \nProblem statement: The industrialization of the world from whole to s ite as a result of technological innovation made many industries adopt ing Information and Communication Technology (ICT) for processing of all their activities from i nception to completion, especially in the developed nations. But, the developing nations appear to make sluggish progress towards ICT adoption due to apprehensiveness that their fraudulent activities c an easily be traced. \nApproach: The purpose of this study was to evaluate the contractor’s perception t oward ICT innovation acceptance for construction site management and the effectiveness of the innova tion. A 519 questionnaire survey was employed for the data collection, while SPSS version 17.0 wa s used for the descriptive statistic and factorial analysis of the data. \nResults: The findings show ICT innovation was effective for site management but there were positive and negative factors that affec t the ICT innovation based on the contractors view. \nConclusion: By evaluating the ICT innovation, empirical eviden c has been provided for the ‘wait and see contractors’ to adopt ICT in construction site management and by making adequate provisions against the negative factors.\t Entry2
2   BACKGROUND AND PURPOSE\nRotator cuff tears are associated with secondary rotator cuff muscle pathology, which is definitive for the prognosis of rotator cuff repair. There is little information regarding the early histological and immunohistochemical nature of these muscle changes in humans. We analyzed muscle biopsies from patients with supraspinatus tendon tears.\n\n\nMETHODS\nSupraspinatus muscle biopsies were obtained from 24 patients undergoing arthroscopic repair of partial- or full-thickness supraspinatus tendon tears. Tissue was formalin-fixed and processed for histology (for assessment of fatty infiltration and other degenerative changes) or immunohistochemistry (to identify satellite cells (CD56+), proliferating cells (Ki67+), and myofibers containing predominantly type 1 or 2 myosin heavy chain (MHC)). Myofiber diameters and the relative content of MHC1 and MHC2 were determined morphometrically.\n\n\nRESULTS\nDegenerative changes were present in both patient groups (partial and full-thickness tears). Patients with full-thickness tears had a reduced density of satellite cells, fewer proliferating cells, atrophy of MHC1+ and MHC2+ myofibers, and reduced MHC1 content.\n\n\nINTERPRETATION\nFull-thickness tears show significantly reduced muscle proliferative capacity, myofiber atrophy, and loss of MHC1 content compared to partial-thickness supraspinatus tendon tears. Entry3

我有一组这样的字符串

listStrings = { 
'\nIntroduction' , '\nCase' , 
'\nLiterature' , '\nBackground',  '\nRelated' , 
'\nMethods' , '\nMethod',
'\nTechniques', '\nMethodology',
'\nResults', '\nResult', '\nExperimental',
'\nExperiments', '\nExperiment',
'\nDiscussion' , '\nLimitations',
'\nConclusion' , '\nConclusions',
'\nConcluding' ,
'Introduction\n' , 'Case\n' , 
'Literature\n' , 'Background\n',  'Related\n' , 
'Methods\n' , 'Method\n',
'Techniques\n', 'Methodology\n',
'Results\n', 'Result\n', 'Experimental\n',
'Experiments\n', 'Experiment\n',
'Discussion\n' , 'Limitations\n',
'Conclusion\n' , 'Conclusions\n',
'Concluding\n' ,
'INTRODUCTION' , 'CASE' , 
'LITERATURE' , 'BACKGROUND',  'RELATED' , 
'METHODS' , 'METHOD',
'TECHNIQUES', 'METHODOLOGY',
'RESULTS', 'RESULT', 'EXPERIMENTAL',
'EXPERIMENTS', 'EXPERIMENT',
'DISCUSSION' , 'LIMITATIONS',
'CONCLUSION' , 'CONCLUSIONS',
'CONCLUDING' ,
'Introduction:' , 'Case:' , 
'Literature:' , 'Background:',  'Related:' , 
'Methods:' , 'Method:',
'Techniques:', 'Methodology:',
'Results:', 'Result:', 'Experimental:',
'Experiments:', 'Experiment:',
'Discussion:' , 'Limitations:',
'Conclusion:' , 'Conclusions:',
'Concluding:' ,
}

直到 A 列中的字符串到达​​ listStrings 中的字符串之一，不要保存任何内容。一旦它到达listStrings 中的一个字符串，将该listStrings 字符串作为它自己的单独列放在新数据帧的一行中。然后将 listStrings 字符串之后的所有内容放在一个新行中，直到该段到达来自 listStrings 的另一个字符串。然后重复该过程：将该字符串放在新列中，并为新段创建新行，依此类推。

这是理想结果的示例

testdf2 = pd.DataFrame([
    [ 'BACKGROUND' , '\nDiagnostic uncertainty in ALS has serious management implications and delays recruitment into clinical trials. Emerging evidence of presymptomatic disease-burden provides the rationale to develop diagnostic applications based on the evaluation of in-vivo pathological patterns early in the disease.\n\nOBJECTIVES\nTo outline and test a diagnostic classification approach based on an array of complementary imaging metrics in key disease-associated anatomical structures.\n\n', 'Entry1'],
    ['METHODS', 'Data from 75 ALS patients and 75 healthy controls were randomly allocated in a training and validation cohort. Spatial masks were created for anatomical foci which best discriminate patients from controls in the training sample. In a virtual brain biopsy, data was then retrieved from these key disease-associated brain regions. White matter diffusivity indices, grey matter T1-signal intensity values and basal ganglia volumes were evaluated as predictor variables in a canonical discriminant function.\n\n', 'Entry1'],
    ['RESULTS', '\nFollowing predictor variable selection, a classification specificity of 85.5% and sensitivity of 89.1% was achieved in the training sample and 90% specificity and 90% sensitivity in the validation sample.\n\n', 'Entry1'],
    ['DISCUSSION', '\nThis study evaluates disease-associated imaging measures in a dummy diagnostic application. Although larger samples will be required for robust validation, the study confirms the potential of multimodal quantitative imaging in future clinical applications.' , 'Entry1'], 
                        ['\nResults:', ' The findings show ICT innovation was effective for site management but there were positive and negative factors that affec t the ICT innovation based on the contractors view. ', 'Entry2'],
                         ['\nConclusion:',' By evaluating the ICT innovation, empirical eviden c has been provided for the wait and see contractors to adopt ICT in construction site management and by making adequate provisions against the negative factors.', 'Entry2'], 
                       ['BACKGROUND',  'AND PURPOSE\nRotator cuff tears are associated with secondary rotator cuff muscle pathology, which is definitive for the prognosis of rotator cuff repair. There is little information regarding the early histological and immunohistochemical nature of these muscle changes in humans. We analyzed muscle biopsies from patients with supraspinatus tendon tears.\n\n', 'Entry3'],
                      [ 'METHODS', '\nSupraspinatus muscle biopsies were obtained from 24 patients undergoing arthroscopic repair of partial- or full-thickness supraspinatus tendon tears. Tissue was formalin-fixed and processed for histology (for assessment of fatty infiltration and other degenerative changes) or immunohistochemistry (to identify satellite cells (CD56+), proliferating cells (Ki67+), and myofibers containing predominantly type 1 or 2 myosin heavy chain (MHC)). Myofiber diameters and the relative content of MHC1 and MHC2 were determined morphometrically.\n\n',  'Entry3'],
                      [ 'RESULTS', '\nDegenerative changes were present in both patient groups (partial and full-thickness tears). Patients with full-thickness tears had a reduced density of satellite cells, fewer proliferating cells, atrophy of MHC1+ and MHC2+ myofibers, and reduced MHC1 content.\n\n\nINTERPRETATION\nFull-thickness tears show significantly reduced muscle proliferative capacity, myofiber atrophy, and loss of MHC1 content compared to partial-thickness supraspinatus tendon tears.', 'Entry3']
])
testdf2.columns = ['C' , 'D', 'E']
testdf2.head(20)

这将导致

C   D   E
0   BACKGROUND  \nDiagnostic uncertainty in ALS has serious management implications and delays recruitment into clinical trials. Emerging evidence of presymptomatic disease-burden provides the rationale to develop diagnostic applications based on the evaluation of in-vivo pathological patterns early in the disease.\n\nOBJECTIVES\nTo outline and test a diagnostic classification approach based on an array of complementary imaging metrics in key disease-associated anatomical structures.\n\n    Entry1
1   METHODS Data from 75 ALS patients and 75 healthy controls were randomly allocated in a training and validation cohort. Spatial masks were created for anatomical foci which best discriminate patients from controls in the training sample. In a virtual brain biopsy, data was then retrieved from these key disease-associated brain regions. White matter diffusivity indices, grey matter T1-signal intensity values and basal ganglia volumes were evaluated as predictor variables in a canonical discriminant function.\n\n Entry1
2   RESULTS \nFollowing predictor variable selection, a classification specificity of 85.5% and sensitivity of 89.1% was achieved in the training sample and 90% specificity and 90% sensitivity in the validation sample.\n\n  Entry1
3   DISCUSSION  \nThis study evaluates disease-associated imaging measures in a dummy diagnostic application. Although larger samples will be required for robust validation, the study confirms the potential of multimodal quantitative imaging in future clinical applications.  Entry1
4   \nResults:  The findings show ICT innovation was effective for site management but there were positive and negative factors that affec t the ICT innovation based on the contractors view.  Entry2
5   \nConclusion:   By evaluating the ICT innovation, empirical eviden c has been provided for the wait and see contractors to adopt ICT in construction site management and by making adequate provisions against the negative factors.    Entry2
6   BACKGROUND  AND PURPOSE\nRotator cuff tears are associated with secondary rotator cuff muscle pathology, which is definitive for the prognosis of rotator cuff repair. There is little information regarding the early histological and immunohistochemical nature of these muscle changes in humans. We analyzed muscle biopsies from patients with supraspinatus tendon tears.\n\n    Entry3
7   METHODS \nSupraspinatus muscle biopsies were obtained from 24 patients undergoing arthroscopic repair of partial- or full-thickness supraspinatus tendon tears. Tissue was formalin-fixed and processed for histology (for assessment of fatty infiltration and other degenerative changes) or immunohistochemistry (to identify satellite cells (CD56+), proliferating cells (Ki67+), and myofibers containing predominantly type 1 or 2 myosin heavy chain (MHC)). Myofiber diameters and the relative content of MHC1 and MHC2 were determined morphometrically.\n\n Entry3
8   RESULTS \nDegenerative changes were present in both patient groups (partial and full-thickness tears). Patients with full-thickness tears had a reduced density of satellite cells, fewer proliferating cells, atrophy of MHC1+ and MHC2+ myofibers, and reduced MHC1 content.\n\n\nINTERPRETATION\nFull-thickness tears show significantly reduced muscle proliferative capacity, myofiber atrophy, and loss of MHC1 content compared to partial-thickness supraspinatus tendon tears. Entry3

对应于第一个数据帧中每个原始字符串的原始标签列 B 被赋予新数据帧的字符串段的每一行。

listStrings 中的字符串被排除在段中，而是被提取到它自己的列中。

最后，listStrings 中的字符串可能存在重叠。重叠首先出现在哪一侧，或者为列选择哪一侧，或者是否使用组合（即'\nResult;'）都无关紧要。如第二个示例数据帧的第 4 行和第 5 行所示。

编辑：

对于一个解决方案，我需要将 B 列转移到上面给出的示例解决方案中的 E 列，B 列中的每个条目都是一个重要的标签，我需要 E 列中的每个组件字符串来包含它的原始标签。

Answer 1

这是一种方法，我不确定大数据集的效率：

# first we build a big regex pattern
pat = '|'.join(listStrings)

# find all keywords in the series
new_df = testdf.A.str.findall(pat)
# 0    [BACKGROUND, METHODS, RESULT, DISCUSSION]
# 1                    [\nResults, \nConclusion]
# 2                [BACKGROUND, METHODS, RESULT]
# Name: A, dtype: object

# find all the chunks by splitting the text with the found keywords
chunks = pd.concat([testdf.A.iloc[[i]].str.split('|'.join(new_df.iloc[i]), expand=True) 
             for i in range(len(testdf))]).stack()

# stack the keywords:
keys = new_df.str.join(' ').str.split(' ', expand=True).stack()

# out return dataframe
# note that we shift the chunks to match the keywords
pd.DataFrame({'D': keys, 'E': chunks.groupby(level=0).shift(-1)})

输出：

                D                                                  E
0 0    BACKGROUND  \nDiagnostic uncertainty in ALS has serious ma...
  1       METHODS  \nData from 75 ALS patients and 75 healthy con...
  2        RESULT  S\nFollowing predictor variable selection, a c...
  3    DISCUSSION  \nThis study evaluates disease-associated imag...
  4           NaN                                                NaN
1 0     \nResults  : The findings show ICT innovation was effecti...
  1  \nConclusion  : By evaluating the ICT innovation, empirical ...
  2           NaN                                                NaN
2 0    BACKGROUND   AND PURPOSE\nRotator cuff tears are associate...
  1       METHODS  \nSupraspinatus muscle biopsies were obtained ...
  2        RESULT  S\nDegenerative changes were present in both p...
  3           NaN                                                NaN

编辑：

这是给出问题中指定的确切输出的解决方案的一个版本

# first we build a big regex pattern
pat = '|'.join(listStrings)

# find all keywords in the series
new_df = testdf.A.str.findall(pat)
# 0    [BACKGROUND, METHODS, RESULT, DISCUSSION]
# 1                    [\nResults, \nConclusion]
# 2                [BACKGROUND, METHODS, RESULT]
# Name: A, dtype: object

# find all the chunks by splitting the text with the found keywords
chunks = pd.concat([testdf.A.iloc[[i]].str.split('|'.join(new_df.iloc[i]), expand=True) 
             for i in range(len(testdf))]).stack()

# stack the keywords:
keys = np.concatenate(new_df.values) # Flatten the keywords array
values = chunks.groupby(level=0).shift(-1).dropna().values
labels = np.concatenate([len(val) * [testdf['B'][ind]] for ind, val in enumerate(new_df.values)]) 
# out return dataframe
# note that we shift the chunks to match the keywords
pd.DataFrame({'C': keys, 'D': values, 'E': labels})

输出：

C   D   E
0   BACKGROUND  \nDiagnostic uncertainty in ALS has serious ma...   Entry1
1   METHODS \nData from 75 ALS patients and 75 healthy con...   Entry1
2   RESULTS \nFollowing predictor variable selection, a cl...   Entry1
3   DISCUSSION  \nThis study evaluates disease-associated imag...   Entry1
4   \nResult    s: The findings show ICT innovation was effect...   Entry2
5   \nConclusion    : By evaluating the ICT innovation, empirical ...   Entry2
6   BACKGROUND  AND PURPOSE\nRotator cuff tears are associate...    Entry3
7   METHODS \nSupraspinatus muscle biopsies were obtained ...   Entry3
8   RESULTS \nDegenerative changes were present in both pa...   Entry3