【问题标题】:Jsoup.parse takes 10x more time on 19+ android devicesJsoup.parse 在 19+ android 设备上花费 10 倍以上的时间
【发布时间】:2014-01-16 18:48:57
【问题描述】:

由于某种原因,在 kitkat 设备上使用 Jsoup.parse 比在旧设备上花费的时间多 10 倍,起初我认为它与 ART 运行时有关,但改回 dalvik 并没有帮助

这是我正在使用的代码:

        downloadedHtml = NetworkHelper.downloadString("https://en.m.wikipedia.org/wiki/Dusseldorf");

        AppLog.i("Downloaded data, Jsoup is parsing the html");

        hDoc = Jsoup.parse(downloadedHtml);

        Element htmlElement = hDoc.select("html").first();

        String langCode = htmlElement.attributes().get("lang");

        ArticleInfo articleInfo = new ArticleInfo(getWikiLanguage(langCode), langCode, href);

        article = new Article(articleInfo, href);

        String title = hDoc.getElementById("section_0").text();

        article.set_title(title);

        Document documentNode = hDoc.ownerDocument(); 

        Elements contents = documentNode.getElementsByClass("content");

        if (contents == null || contents.isEmpty())
            throw new IllegalArgumentException("content");

        Element content = contents.first();

        Elements imgElements = content.select("img");

        Element htmlNode;

        for (int i = 0; i < imgElements.size(); i++)
        {
            htmlNode = imgElements.get(i);

            if (!htmlNode.hasAttr("src"))
                continue;

            String src = htmlNode.attr("src");

            if (src.startsWith("//"))
                htmlNode.attr("src", String.format("http:%s", src));
            //else
            //throw new UnsupportedOperationException();
        }

        //get section headings

        Elements headlines = documentNode.getElementsByClass("mw-headline");

        if (headlines != null)
        {
            Element headline;

            for (int i = 0; i < headlines.size(); i++)
            {
                headline = headlines.get(i);

                String headline_link = headline.id();
                String headline_title = headline.text();

                SectionHeadline sectionHeadline = new SectionHeadline(headline_title, headline_link);
                article.get_sectionHeadlines().add(sectionHeadline);
            }
        }

        article.set_html(content.outerHtml());

        //get languages
        //language list

        Element languageSection = content.getElementById("mw-mf-language-section");

        if (languageSection != null)
        {
            Elements languageLinks = languageSection.select("li");

            Element languageLink;

            for (int i = 0; i < languageLinks.size(); i++)
            {
                languageLink = languageLinks.get(i);

                Element link = null;
                Elements ls = languageLink.select("a");

                if (ls == null || ls.size() == 0)
                    continue;

                link = ls.first();

                if (!link.hasAttr("href"))
                    continue;

                String linkHref = link.attr("href");

                if (linkHref != null && link.text() != null)
                {
                    String languageCode = link.attr("lang");

                    if (linkHref.startsWith("//"))
                        linkHref = String.format("http:%s", linkHref);

                    ArticleInfo languageInfo = new ArticleInfo(getWikiLanguage(languageCode), languageCode, linkHref);

                    if (languageInfo.get_language() == "Unknown")
                        continue;

                    article.get_languages().add(languageInfo);
                }
            }

        }

任何想法可能是什么问题?

【问题讨论】:

    标签: android web-scraping jsoup android-4.4-kitkat


    【解决方案1】:

    问题中的代码选择文档的一部分,将其保存到变量中,选择该变量的一部分,将其保存到新变量中,等等。另一种可能的实现是更多地使用selector syntax 来仅选择需要的元素,而不是将这些中间步骤保存在新对象中。

    下面的代码在我的机器上执行了 2 秒。上面的类似摘录在大约 4 秒内执行。随后的时间更接近,相差大约 50 毫秒,因此请谨慎对待。

    我不知道 kitkat 是否存在性能问题。您可能会发现向您的 kitkat 和 dalvik 版本添加计时器以隔离是否存在性能瓶颈以及在何处存在性能瓶颈会很有帮助。

    这是我的代码:

    long start = System.currentTimeMillis();
    Document hDoc = Jsoup.
        connect("https://en.m.wikipedia.org/wiki/Dusseldorf").
        userAgent("Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.17 (KHTML, like Gecko) Chrome/24.0.1312.57 Safari/537.17").
        get();
    
    //select the first html element, then take the value of the lang attribute
    String langCode = hDoc.select("html:eq(0)").attr("lang");
    String title = hDoc.getElementById("section_0").text();
    
    Document documentNode = hDoc.ownerDocument();
    
    //select all the image elements having the attribute src which are
    //descended from the first element with the content class
    Elements imgElementsHavingSrcAttr = documentNode.select("*.content:eq(0) img[src]");
    Element htmlNode;
    
    //for each img element
    for (Element img : imgElementsHavingSrcAttr)
    {
        htmlNode = img;
        String src = img.attr("src");
    
        if (src.startsWith("//"))
        {
            htmlNode.attr("src", String.format("http:%s", src));
        }
    }
    System.out.println("Function took " + (System.currentTimeMillis()-start) + "ms");
    

    【讨论】:

      猜你喜欢
      • 1970-01-01
      • 1970-01-01
      • 2020-12-06
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2014-08-30
      • 1970-01-01
      相关资源
      最近更新 更多