Flutter(Dart) , webscraper 插件在不同的 url 上给出错误

Question

 _getData() async {
    webScraper = WebScraper('https://www.yesilyurtgame.com');
    print("İm waiting");

    if (await webScraper.loadWebPage('/steamko')) {
      print("İm got in");
      List<Map<String, dynamic>> results =
      webScraper.getElement('div.center', ['title']);
      setState(() {
        loaded = true;
        popNum = results[0]['title'];
      });
    }
  }

这是我的代码，用于使用网络抓取工具抓取一些网站。 我明白了

Restarted application in 269ms.
İm waiting
Error: Instance of 'WebScraperException'
    at Object.throw_ [as throw] (http://localhost:55475/dart_sdk.js:4328:11)
    at web_scraper.WebScraper.new.loadWebPage (http://localhost:55475/packages/web_scraper/web_scraper.dart.lib.js:68:23)
    at loadWebPage.throw (<anonymous>)
    at http://localhost:55475/dart_sdk.js:37599:38
    at _RootZone.runBinary (http://localhost:55475/dart_sdk.js:37452:58)
    at _FutureListener.thenAwait.handleError (http://localhost:55475/dart_sdk.js:32436:48)
    at handleError (http://localhost:55475/dart_sdk.js:32987:51)
    at Function._propagateToListeners (http://localhost:55475/dart_sdk.js:33013:17)
    at _Future.new.[_completeError] (http://localhost:55475/dart_sdk.js:32860:23)
    at async._AsyncCallbackEntry.new.callback (http://localhost:55475/dart_sdk.js:32898:31)
    at Object._microtaskLoop (http://localhost:55475/dart_sdk.js:37708:13)
    at _startMicrotaskLoop (http://localhost:55475/dart_sdk.js:37714:13)
    at http://localhost:55475/dart_sdk.js:33226:9
Application finished.

但是当我尝试抓取例如：

  _getData() async {
    webScraper = WebScraper('https://worldpopulationreview.com');
    if (await webScraper.loadWebPage('/')) {
      List<Map<String, dynamic>> results =
      webScraper.getElement('div.center', ['title']);
      setState(() {
        loaded = true;
        popNum = results[0]['title'];
      });
    }
  }

这行得通。我也尝试了其他一些 URL，但我只能找到这个链接（我从教程中获得）有效。我需要抓取不同种类的网站，所以我想知道是否有更好的方法来抓取或使这个插件工作？

Answer 1

您的源页面不包含任何具有中心类的 div。所以它将为空。您应该在抓取网页时考虑这些元素。在您的源代码中，下面有一个网络元素：

       <h1 class="gfont">Knight Online (SteamKO) GB</h1>

这是一个 h1 元素，它的类是 gfont。然后您可以使用以下代码获取该元素：

  _getData() async {
      webScraper = WebScraper('https://worldpopulationreview.com');
      if (await webScraper.loadWebPage('/')) {
        List<Map<String, dynamic>> results =
        webScraper.getElement('h1.gfont', []);
        setState(() {
          loaded = true;
          popNum = results[0];
        });
      }
    }