python数据中的熊猫数据框没有以正确的方式出现？

Question

import requests
from bs4 import BeautifulSoup
import pandas as pd
start_url="https://www.indeed.co.in/jobs?q=teacher&l=India"
page_data=requests.get(start_url)
soup=BeautifulSoup(page_data.content,"html.parser")
Title=[]
Company=[]
Salary=[]
Summary=[]
Location=[]
link_to_apply=[]

for job_tag in soup.find_all("div",class_="jobsearch-SerpJobCard unifiedRow row result"):
    title=job_tag.find("h2",class_="title")
    company=job_tag.find("span",class_="company")
    location=job_tag.find(class_="location accessible-contrast-color-location").text.strip()
    summary=job_tag.find("div",class_="summary")
    link=job_tag.find("a",href=True)
    base_url="https://www.indeed.com"
    final_link=base_url+link["href"]

    Title.append(title.text.strip())
    Company.append(company.text.strip())
    Location.append(location)
    Summary.append(summary.text.strip())
    link_to_apply.append(final_link)


data=list(zip(Title,Company,Location,Summary))
p=pd.DataFrame(data,columns=["Title","Company","Location","Summary"])
p.to_csv("new.csv")
pd.set_option("display.max_colwidth",None)
pd.set_option("display.max_rows",None)
pd.set_option("display.max_rows",None)
pd.set_option("display.width",None)
pd.set_option('display.max_columns',None)

the output of the following code

                                                                 Title  \
0                                              all Subject Teacher\nnew   
1                                           Part time Teacher / Trainer   
2   Online Tutor / Teachers - Women Candidates only - Non Metro...\nnew   
3                                          Science & Maths Teacher\nnew   
4                                                Primary School Teacher   
5                                                Preschool Teacher\nnew   
6       Online Tutor / Teachers - Women Candidates only- Across Indi...   
7                                                         Maths Teacher   
8                               wanted faculty for teaching cbse school   
9                                               Primary English Teacher   
10                                                 Training Facilitator   
11                                                         Head Teacher   
12                                                    Math Teacher\nnew   
13                                                     Teacher for Kids   
14                                            Preschool Teacher - Vizag   

                                                  Company  \
0                                               Home Guru   
1                                               Home Guru   
2                 Whitehat Education Technology Pvt. Ltd.   
3                                               Home Guru   
4                       Newdimension International School   
5                                               Home Guru   
6                 Whitehat Education Technology Pvt. Ltd.   
7                            Dheeraj International School   
8   TILFORD SCHOOL,JANGAREDDIGUDEM,W.G.DIST,ANDHRA PRA...   
9                                       Stones2Milestones   
10                                      Stones2Milestones   
11                                           Wunderschool   
12                             GEMS Public School,Patiala   
13                                            ANAR EdTech   
14                                        Koala Preschool   

                                  Location  \
0                                   Remote   
1                                   Remote   
2                            Kochi, Kerala   
3                                   Remote   
4                  Bhongir, Andhra Pradesh   
5                      Mumbai, Maharashtra   
6                     Bengaluru, Karnataka   
7                        Pune, Maharashtra   
8   Jangareddi Gudem Bazar, Andhra Pradesh   
9                         Gurgaon, Haryana   
10                        Gurgaon, Haryana   
11                  Chandigarh, Chandigarh   
12                Univ P O Patiala, Punjab   
13                           Kochi, Kerala   
14           Visakhapatnam, Andhra Pradesh   

                                                                                                                                                                Summary  
0    We are recognized leaders in one to one coaching of all subjects & Courses, both online and offline.\nStudents and Gurus from all across India are invited to get…  
1    We are recognized leaders in one to one coaching of all subjects & Courses, both online and offline.\nStudents and Gurus from all across India are invited to get…  
2              Teacher, Work from Home, Online Tutor, Teaching, Home Tutor, coding teacher, computer teacher, home teacher, kids teacher.\nYou've found your dream job.  
3    We are recognized leaders in one to one coaching of all subjects & Courses, both online and offline.\nStudents and Gurus from all across India are invited to get…  
4      Proven experience as a teacher.\nCollaborate with other teachers, parents and stakeholders and participate in regular meetings.\nTotal work: 1 year (Preferred).  
5     We Provide Virtual recorded sessions of Preschool curriculum,.\nHave to record Virtual Classes of preschool curriculum.\nMust have preschool teaching experience.  
6              Teacher, Work from Home, Online Tutor, Teaching, Home Tutor, coding teacher, computer teacher, home teacher, kids teacher.\nYou've found your dream job.  
7                       We are looking for qualified and experienced candidates who can join immediately for the following posts at Dheeraj International School, Pune.  
8                                    WANTED FACULTY FOR TEACHING @CBSE SCHOOL.\nSalary not constraint for deserving candidates. *.\nSalary: Up to ₹50,000.00 per month.  
9        We are looking to bring into our fold an incredible teacher/facilitator to conduct online classes for children of ages 6-10.\nWorking with: Consumer App Team.  
10      We are looking to bring into our fold an incredible teacher/facilitator to conduct online classes for children of ages 6-10.\nWorking with: Consumer App Team*.  
11       The Head teacher shall influence the thinking and practice of students, teachers and parents.\nPlanning, Documentation , Evaluation, Mentorship, Guidance and…  
12  Welcome to Gems Public School, Patiala...\nGEMS is an international education company.\nIt is a global advisory and educational management firm, with a network of…  
13     We are* looking for dynamic and vibrant lady tutors with a passion for inspiring our kids to flourish and reach their potentials.\nWork at your convenient time.  
14    Our growing Preschool facility is looking for a Young energetic Female Preschool Teacher who can help to create a fun environment and incorporate educational…  

我希望输出位于适当的表格列中。此外，表格需要保存到 CSV 文件 这就是我选择熊猫的原因。请快速回复，谢谢大家 有人知道为什么我没有得到正确的数据框吗？大熊猫作为输出 还有其他方法吗？ 同样在创建的 csv 文件中，输出正常，但在 python 中却不是

Answer 1

在大多数情况下，当前的代码似乎可以工作。如果您删除换行符，那可能会有所帮助。至于'...'，数据在网站上被截断。您将需要像 Selenium 这样的网络自动化工具来单击每个链接。您可以使用 tabulate 来格式化控制台中的输出表格。

这是删除换行符并修复缩进的代码。

import requests
from bs4 import BeautifulSoup
import pandas as pd

start_url="https://www.indeed.co.in/jobs?q=teacher&l=India"
page_data=requests.get(start_url)
soup=BeautifulSoup(page_data.content,"html.parser")
Title=[]
Company=[]
Salary=[]
Summary=[]
Location=[]
link_to_apply=[]

for job_tag in soup.find_all("div",class_="jobsearch-SerpJobCard unifiedRow row result"):
    title=job_tag.find("h2",class_="title")
    company=job_tag.find("span",class_="company")
    location=job_tag.find(class_="location accessible-contrast-color-location").text.strip()
    summary=job_tag.find("div",class_="summary")
    link=job_tag.find("a",href=True)
    base_url="https://www.indeed.com"
    final_link=base_url+link["href"]

    Title.append(title.text.replace('\n'," ").strip())
    Company.append(company.text.replace('\n'," ").strip())
    Location.append(location.replace('\n'," "))
    Summary.append(summary.text.replace('\n'," ").strip())
    link_to_apply.append(final_link.replace('\n'," "))
    
data=list(zip(Title,Company,Location,Summary))
p=pd.DataFrame(data,columns=["Title","Company","Location","Summary"])
p.to_csv("new.csv", index=False)
pd.set_option("display.max_colwidth",None)
pd.set_option("display.max_rows",None)
pd.set_option("display.max_rows",None)
pd.set_option("display.width",None)
pd.set_option('display.max_columns',None)
pd.options.display.max_colwidth = None
print(p.to_string(index=False))

# from tabulate import tabulate
# print(tabulate(p, headers='keys', tablefmt='psql'))  # terminal
#p   # Jupyter only, direct output
print(formatdf(p, 25))   # Idle

输出（Jupyter，截断）

对于空闲，我必须编写一个函数来格式化数据。这将根据最大列宽将列数据包装在数据框中。

def formatdf(df, mxcolwidth):
    outstr = ''
    for c in df.columns:
        df[c] = df[c].str.wrap(mxcolwidth)  # insert newlines
        
    # get max width for each column
    wdic = {}
    for c in df.columns:
       s = df[c]
       mx = 0
       for r in s:
           for ln in r.split('\n'):
               if len(ln) > mx:
                   mx = len(ln)
       wdic[c] = mx  # dictionary, max line length of each column

    # create row divider string 
    rowstr = ''
    for c in df.columns:
        rowstr += '-' * (wdic[c])
    rowstr += '-' * (len(df.columns)*3) + '-\n'


    outstr += '| '  # start row line
    # column headers
    for c in df.columns:
        outstr += c.ljust(wdic[c]) + ' | '
    outstr += '\n'
    outstr += rowstr
    
    # each row in dataframe
    for ir, r in df.iterrows():
        mxln = 0
        for c in df.columns:  # get mxx lines for this data row
            lncnt = len(r[c].split('\n'))
            if lncnt > mxln: mxln = lncnt
            
        for i in range(mxln): # for each line in data cell
            for ic, c in enumerate(df.columns):  # each column
                if ic == 0: outstr += '| '   # left border of table
                lns = r[c].split('\n')  # split, each line of text in data cell
                if i < len(lns):
                    outstr += lns[i].ljust(wdic[c]) + ' | '  # single line of text
                else:
                    outstr += " ".ljust(wdic[c]) + ' | '   # empty line
            outstr+= "\n"
        outstr += rowstr  # row divider
              
    return outstr

输出（空闲，截断）

| Title                     | Company                   | Location                 | Summary                   | 
----------------------------------------------------------------------------------------------------------------
| Online Tutor / Teachers - | Whitehat Education        | Srinagar, Jammu and      | Teacher, Work from Home,  | 
| Women Candidates only -   | Technology Pvt. Ltd.      | Kashmir                  | Online Tutor, Teaching,   | 
| Non Metro... new          |                           |                          | Home Tutor, coding        | 
|                           |                           |                          | teacher, computer         | 
|                           |                           |                          | teacher, home teacher,    | 
|                           |                           |                          | kids teacher. You've      | 
|                           |                           |                          | found your dream job.     | 
----------------------------------------------------------------------------------------------------------------
| Post Graduate Teacher and | North Eastern Railway     | Gorakhpur, Uttar Pradesh | North Eastern Railway     | 
| Trained Graduate Teacher  |                           |                          | Recruitment 2020 - Post   | 
| new                       |                           |                          | Graduate Teacher and      | 
|                           |                           |                          | Trained Graduate Teacher  | 
|                           |                           |                          | Vacancies - Apply         | 
|                           |                           |                          | NowNorth Eastern Railway  | 
|                           |                           |                          | Recruitment 2020-21:…     | 
----------------------------------------------------------------------------------------------------------------