如何使用 puppeteer 从 iframe 标签中抓取日期

Question

我尝试从 iframe/frame 标记中抓取一些数据，但我被 puppeteer 的代码卡住了。我是新手，所以请多多包涵。这是该网站的链接。当我点击时在那里在第一帧的名称中，我再次在女巫的第二帧中获得了一些数据，我可以单击并在第三帧中获取数据。 在代码中，我尝试循环真正的第一帧以获取第二帧和第三帧的所有数据。

感谢您的任何提示。

我已经运行了这个命令： document.querySelector("正文 > 表单 > 字体 > 选择 > 选项") 在控制台中，但我找不到在 puppeteer 中运行它的方法。

const puppeteer = require("puppeteer");

(async () => {

    const browser = await puppeteer.launch();

    const page = await browser.newPage();
    await page.goto('');

    const iframeParagraph = await page.evaluate(() => {

        const iframe = document.getElementsByName("stanga");

        // grab iframe's document object
        const iframeDoc = iframe.contentDocument || iframe.contentWindow.document;

        const iframeP = iframeDoc.getElementsByName("fmtstatii");

        return iframeP.innerHTML;
    });

    console.log(iframeParagraph); 

    await browser.close();

})();

或

const puppeteer = require('puppeteer');

let scrape = async () => {
    const browser = await puppeteer.launch({headless: false});
    const page = await browser.newPage();

    await page.goto('');
    await page.click('document.querySelector("body > form > font > select")');
    await page.waitFor(1000);

    const result = await page.evaluate(() => {
        let statie = document.querySelector('document.querySelector("body > form > font > select > option")').innerText;

        return {
            statie
        }

    });

    browser.close();
    return result;
};

scrape().then((value) => {
    console.log(value); // Success!
});

这是我得到的错误：

[(node:13308) UnhandledPromiseRejectionWarning: Error: Evaluation failed: DOMException: Failed to execute 'querySelector' on 'Document': 'document.querySelector("body > form > font > select")' is not a
valid selector.
    at __puppeteer_evaluation_script__:1:33
    at ExecutionContext._evaluateInternal (D:\Zero\ratt_scrap\node_modules\puppeteer\lib\ExecutionContext.js:122:13)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  -- ASYNC --
    at ExecutionContext.<anonymous> (D:\Zero\ratt_scrap\node_modules\puppeteer\lib\helper.js:111:15)
    at ElementHandle.$ (D:\Zero\ratt_scrap\node_modules\puppeteer\lib\JSHandle.js:395:50)
    at ElementHandle.<anonymous> (D:\Zero\ratt_scrap\node_modules\puppeteer\lib\helper.js:112:23)
    at DOMWorld.$ (D:\Zero\ratt_scrap\node_modules\puppeteer\lib\DOMWorld.js:121:34)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  -- ASYNC --
    at Frame.<anonymous> (D:\Zero\ratt_scrap\node_modules\puppeteer\lib\helper.js:111:15)
    at Page.click (D:\Zero\ratt_scrap\node_modules\puppeteer\lib\Page.js:986:29)
    at scrape (D:\Zero\ratt_scrap\scrape.js:23:16)
    at process._tickCallback (internal/process/next_tick.js:68:7)
(node:13308) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:13308) \[DEP0018\] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.]

Answer 1

你犯了几个错误：

1. 您应该与Frame 而不是Page 对象进行交互。

   const frame = await page.frames().find(frame => frame.name() === 'stanga'); // Find the right frame.
   

2. click() 方法需要selector <string>，因此您无需在click() 方法中添加document.querySelector。

   await frame.click('body > form > font > select');
   

3. 要获取所有innerText，您必须遍历元素。

4. 别忘了添加await。你错过了close 方法。

   await browser.close();
   

解决方案：

const puppeteer = require('puppeteer');

let scrape = async () => {
const browser = await puppeteer.launch({headless: false});
const page = await browser.newPage();

await page.goto('http://ratt.ro/txt');
const frame = await page.frames().find(frame => frame.name() === 'stanga');
await frame.click('body > form > font > select');
await page.waitFor(1000);


const optionsResult = await frame.$$eval('body > form > font > select > option', (options) => {
    const result = options.map(option => option.innerText);

    return result;
});

await browser.close();

return optionsResult;
};

scrape().then((value) => {
  console.log(value); // Success!
});