【问题标题】:Error navigation error with nightmare js?噩梦js的错误导航错误?
【发布时间】:2018-06-13 11:49:19
【问题描述】:

我是 Nightmare js 的新手,正在尝试从网站上抓取一些数据。但是后来我收到了这个错误,我无法理解我做错了什么。有人可以解释一下是什么问题吗? 我要抓取的元素如下。我正在尝试提取标签之间的文本,如下<span class="home__matches__match__header__game-id">Game ID 9490 - </span> Kenya - Premier League

<div class="home__sports">
    <div class="home__sports__title">
        <div class="home__sports__title__category">Kenya - Premier League</div>
        <div class="home__sports__title__copy">
            <span title="Copy" class="icon home__sports__title__copy__icon icon__copy has-tooltip">
                <svg
                    xmlns="http://www.w3.org/2000/svg"
                    xmlns:xlink="http://www.w3.org/1999/xlink" version="1.1" id="Capa_1" x="0px" y="0px" viewBox="0 0 488.244 488.244" xml:space="preserve" width="15px">
                    <g>
                        <g>
                            <g>
                                <path d="M284.141,138.477V72.896H43.678v415.348h306.045V138.477H284.141z" fill="#FFFFFF"></path>
                                <polygon points="290.682,72.723 290.682,131.247 349.205,131.247     " fill="#FFFFFF"></polygon>
                            </g>
                            <g>
                                <polygon points="385.525,0 385.525,58.524 444.048,58.524     " fill="#FFFFFF"></polygon>
                                <polygon points="378.985,65.753 378.985,0.172 138.521,0.172 138.521,65.581 291.543,65.581 357.124,131.162 357.124,415.519       444.566,415.519 444.566,65.753     " fill="#FFFFFF"></polygon>
                            </g>
                        </g>
                    </g>
                </svg>
            </span>
            <span>Copy</span>
        </div>
    </div>
    <div class="home__sports__matches scrollable" tabindex="0">
        <!---->
        <!---->
        <!---->
        <!---->
        <!---->
        <div class="home__matches__match">
            <div class="home__matches__match__header clearfix">
                <span class="home__matches__match__header__game-id">Game ID 9490 - </span> Kenya - Premier League 
                <span class="pull-right">13/06, 15:00</span>
            </div>
            <div class="home__matches__match__odds">
                <button class="home__matches__match__odds__odd bold clearfix">
    Wazito

                    <span class="pull-right bold">8.84</span>
                </button>
                <button class="home__matches__match__odds__odd bold clearfix">
    X

                    <span class="pull-right bold">4.67</span>
                </button>
                <button class="home__matches__match__odds__odd bold clearfix">
    GOR Mahia FC

                    <span class="pull-right bold">1.41</span>
                </button>
                <button class="home__matches__match__odds__markets clearfix bold">
            +25
        </button>
            </div>
            <!---->
        </div>
        <div class="home__matches__match">
            <div class="home__matches__match__header clearfix">
                <span class="home__matches__match__header__game-id">Game ID 3924 - </span> Kenya - Premier League 
                <span class="pull-right">13/06, 15:00</span>
            </div>
            <div class="home__matches__match__odds">
                <button class="home__matches__match__odds__odd bold clearfix">
    Kariobangi Sharks

                    <span class="pull-right bold">2.61</span>
                </button>
                <button class="home__matches__match__odds__odd bold clearfix">
    X

                    <span class="pull-right bold">3.05</span>
                </button>
                <button class="home__matches__match__odds__odd bold clearfix">
    Nzoia United

                    <span class="pull-right bold">2.93</span>
                </button>
                <button class="home__matches__match__odds__markets clearfix bold">
            +25
        </button>
            </div>
            <!---->
        </div>
        <div class="home__matches__match">
            <div class="home__matches__match__header clearfix">
                <span class="home__matches__match__header__game-id">Game ID 8769 - </span> Kenya - Premier League 
                <span class="pull-right">13/06, 15:00</span>
            </div>
            <div class="home__matches__match__odds">
                <button class="home__matches__match__odds__odd bold clearfix">
    Ulinzi Stars FC

                    <span class="pull-right bold">2.3</span>
                </button>
                <button class="home__matches__match__odds__odd bold clearfix">
    X

                    <span class="pull-right bold">3.05</span>
                </button>
                <button class="home__matches__match__odds__odd bold clearfix">
    Kakamega Homeboyz

                    <span class="pull-right bold">4.2</span>
                </button>
                <button class="home__matches__match__odds__markets clearfix bold">
            +25
        </button>
            </div>
            <!---->
        </div>
        <div class="home__matches__match">
            <div class="home__matches__match__header clearfix">
                <span class="home__matches__match__header__game-id">Game ID 3021 - </span> Kenya - Premier League 
                <span class="pull-right">13/06, 15:00</span>
            </div>
            <div class="home__matches__match__odds">
                <button class="home__matches__match__odds__odd bold clearfix">
    AFC Leopards SC

                    <span class="pull-right bold">2.66</span>
                </button>
                <button class="home__matches__match__odds__odd bold clearfix">
    X

                    <span class="pull-right bold">3.21</span>
                </button>
                <button class="home__matches__match__odds__odd bold clearfix">
    Sofapaka FC

                    <span class="pull-right bold">2.61</span>
                </button>
                <button class="home__matches__match__odds__markets clearfix bold">
            +25
        </button>
            </div>
            <!---->
        </div>
    </div>
    <!---->
</div>
   var jquery = require('jquery');
var Nightmare = require('nightmare');
var express = require('express');
nightmare = Nightmare()

var app = express();

    nightmare.goto('https://www.betika.com/competition/701')
    .wait(2000)
    .evaluate(function () {
        var gameids=[]
        var res;
        var gameids=[]
        $('.home__matches__match__header__game-id').each(function(){

            console.log('Entered')
            item ={}
            item['gameid'] = $(this).text()
            gameids.push(item);


        })

        return {
               res:gameids


            };
        },function (value) {
          /*for (gameids in value){
              console.log(value[gameids.gameid])

          }*/
          console.log("gotcha",value)

        }
    )
    .run(function (err, nightmare) {
        if (err) return console.log(err);

    });

结果:-

{ Error: navigation error
    at unserializeError (/home/surajit/WebstormProjects/Odds/node_modules/nightmare/lib/ipc.js:162:13)
    at EventEmitter.<anonymous> (/home/surajit/WebstormProjects/Odds/node_modules/nightmare/lib/ipc.js:89:13)
    at Object.onceWrapper (events.js:312:19)
    at emitTwo (events.js:125:13)
    at EventEmitter.emit (events.js:213:7)
    at ChildProcess.<anonymous> (/home/surajit/WebstormProjects/Odds/node_modules/nightmare/lib/ipc.js:49:10)
    at emitTwo (events.js:125:13)
    at ChildProcess.emit (events.js:213:7)
    at process.nextTick (internal/child_process.js:755:12)
    at _combinedTickCallback (internal/process/next_tick.js:95:7)
  code: -7,
  details: 'Navigation timed out after 30000 ms',
  url: 'https://www.betika.com/competition/701' }

【问题讨论】:

    标签: node.js web-scraping nightmare


    【解决方案1】:

    看起来网站或数据即使在我的普通浏览器上也需要很长时间才能加载。已经快 2 分钟了,但仍然没有加载。 nightmare 设置的导航超时时间为 30 秒。

    然后,看起来如果 URL 无效,它将永远不会加载。所以我点击了一个有效的比赛网址,它终于加载了。

    这与您的代码无关,这与您如何处理或提供目标网站有关。

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2016-08-11
      • 1970-01-01
      • 1970-01-01
      • 2020-04-10
      • 2018-02-13
      • 1970-01-01
      相关资源
      最近更新 更多