如何在 python 中集成 Wikidata 查询答案

【问题标题】：How to integrate Wikidata query in python如何在 python 中集成 Wikidata 查询
【发布时间】：2019-05-02 23:52:39
【问题描述】：

我目前正在使用Wikidata Query Service 运行我的维基数据查询。

例如，我的一个 Wikidata 查询如下所示。

SELECT ?sLabel {
    SERVICE wikibase:mwapi {
        bd:serviceParam wikibase:api "EntitySearch".
        bd:serviceParam wikibase:endpoint "www.wikidata.org".
        bd:serviceParam mwapi:search "natural language processing".
        bd:serviceParam mwapi:language "en".
        ?item wikibase:apiOutputItem mwapi:item.
        ?num wikibase:apiOrdinal true.
    }
    ?s wdt:P279|wdt:P31 ?item .
    SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en" }
}
ORDER BY ?num
LIMIT 10

我想知道我们是否可以在 python 程序中使用这些查询？如果是这样，我们如何在 python 中集成查询？

如果需要，我很乐意提供更多详细信息。

【问题讨论】：

不允许在 Stackoverflow 上索取工具、库等，或者说它被认为是题外话。并且使用“Python + SPARQL”的搜索引擎应该将sparqlwrapper 作为RDFlib 的一部分显示为第一个命中。
@AKSW 以前不知道。谢谢并指出:)
欢迎通过softwarerecs.stackexchange.com 提出工具请求：-)

标签： python sparql wikidata-query-service

【解决方案1】：

如果您想在没有 SPARQL 特定库的情况下执行此操作：

import requests

url = 'https://query.wikidata.org/sparql'
query = '''
SELECT ?item ?itemLabel ?linkcount WHERE {
    ?item wdt:P31/wdt:P279* wd:Q35666 .
    ?item wikibase:sitelinks ?linkcount .
FILTER (?linkcount >= 1) .
SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en" . }
}
GROUP BY ?item ?itemLabel ?linkcount
ORDER BY DESC(?linkcount)
'''
r = requests.get(url, params = {'format': 'json', 'query': query})
data = r.json()

【讨论】：

能否请您添加代码以转换为data.frame
下面看我的回答——dicts列表可以直接导入pandas

【解决方案2】：

sparqlwrapper 可以处理。您可以找到更多信息here

【讨论】：

【解决方案3】：

作为http://wiki.bitplan.com/index.php/PyLoDStorage 的提交者，我推荐这个库，因为它以一种让您立即获得正确的python 类型的方式包装了sparqlwrapper。因此，您将能够将数据导入其他库，例如 pandas 或使用 csv、json、xml、sql 或任何您的后续需求。

我已经稍微修改和扩展了您的查询

# WF 2021-10-30
# see https://stackoverflow.com/questions/55961615/how-to-integrate-wikidata-query-in-python/69771615#69771615
SELECT ?s ?sLabel ?item ?itemLabel ?sourceCode ?webSite ?stackexchangeTag  {
    SERVICE wikibase:mwapi {
        bd:serviceParam wikibase:api "EntitySearch".
        bd:serviceParam wikibase:endpoint "www.wikidata.org".
        bd:serviceParam mwapi:search "natural language processing".
        bd:serviceParam mwapi:language "en".
        ?item wikibase:apiOutputItem mwapi:item.
        ?num wikibase:apiOrdinal true.
    }
    ?s wdt:P279|wdt:P31 ?item .
    OPTIONAL { 
      ?s wdt:P1324 ?sourceCode.
    }
    OPTIONAL {    
      ?s wdt:P856 ?webSite.
    }
    OPTIONAL {    
      ?s wdt:P1482 ?stackexchangeTag.
    }
    SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en" }
}
ORDER BY ?itemLabel ?sLabel

try it!

然后将其用作 pyLoDStorage 库的展示：

def testStackoverflow55961615Query(self):
        '''
        see 
        https://stackoverflow.com/questions/55961615/how-to-integrate-wikidata-query-in-python
        https://stackoverflow.com/a/69771615/1497139
        '''
        endpoint="https://query.wikidata.org/sparql"
        wd=SPARQL(endpoint)
        queryString="""SELECT ?s ?sLabel ?item ?itemLabel ?sourceCode ?webSite ?stackexchangeTag  {
    SERVICE wikibase:mwapi {
        bd:serviceParam wikibase:api "EntitySearch".
        bd:serviceParam wikibase:endpoint "www.wikidata.org".
        bd:serviceParam mwapi:search "natural language processing".
        bd:serviceParam mwapi:language "en".
        ?item wikibase:apiOutputItem mwapi:item.
        ?num wikibase:apiOrdinal true.
    }
    ?s wdt:P279|wdt:P31 ?item .
    OPTIONAL { 
      ?s wdt:P1324 ?sourceCode.
    }
    OPTIONAL {    
      ?s wdt:P856 ?webSite.
    }
    OPTIONAL {    
      ?s wdt:P1482 ?stackexchangeTag.
    }
    SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en" }
}
ORDER BY ?itemLabel ?sLabel"""
        qlod=wd.queryAsListOfDicts(queryString,fixNone=True)
        query=Query(name="EntitySearch",query=queryString,lang='sparql')
        debug=self.debug
        for tablefmt in ["github","mediawiki","latex"]:
            lod=copy.deepcopy(qlod)
            qdoc=query.documentQueryResult(lod,tablefmt=tablefmt)
            if debug:
                print (qdoc)

并且可以立即将输出粘贴到此处，如下所示。输出还可以以 tabulate library 支持的任何格式提供，例如 mediawiki、latex 等。

实体搜索

查询

SELECT ?s ?sLabel ?item ?itemLabel ?sourceCode ?webSite ?stackexchangeTag  {
    SERVICE wikibase:mwapi {
        bd:serviceParam wikibase:api "EntitySearch".
        bd:serviceParam wikibase:endpoint "www.wikidata.org".
        bd:serviceParam mwapi:search "natural language processing".
        bd:serviceParam mwapi:language "en".
        ?item wikibase:apiOutputItem mwapi:item.
        ?num wikibase:apiOrdinal true.
    }
    ?s wdt:P279|wdt:P31 ?item .
    OPTIONAL { 
      ?s wdt:P1324 ?sourceCode.
    }
    OPTIONAL {    
      ?s wdt:P856 ?webSite.
    }
    OPTIONAL {    
      ?s wdt:P1482 ?stackexchangeTag.
    }
    SERVICE wikibase:label { bd:serviceParam wikibase:language "[AUTO_LANGUAGE],en" }
}
ORDER BY ?itemLabel ?sLabel

结果

s	sLabel	item	itemLabel	stackexchangeTag	sourceCode	webSite
http://www.wikidata.org/entity/Q24841819	Q24841819	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q1898737	Morphological analysis	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q6913444	Morphological parsing	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q104415642	Wikification	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q51751772	biomedical natural language processing	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q46346005	computer-based question classification	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q1513879	natural language generation	http://www.wikidata.org/entity/Q30642	natural language processing	https://stackoverflow.com/tags/nlg
http://www.wikidata.org/entity/Q1078276	natural language understanding	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q1271424	part-of-speech tagging	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q105171570	speaker verification	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q189436	speech recognition	http://www.wikidata.org/entity/Q30642	natural language processing	https://ai.stackexchange.com/tags/speech-recognition
http://www.wikidata.org/entity/Q189436	speech recognition	http://www.wikidata.org/entity/Q30642	natural language processing	https://cs.stackexchange.com/tags/speech-recognition
http://www.wikidata.org/entity/Q189436	speech recognition	http://www.wikidata.org/entity/Q30642	natural language processing	https://dsp.stackexchange.com/tags/speech-recognition
http://www.wikidata.org/entity/Q189436	speech recognition	http://www.wikidata.org/entity/Q30642	natural language processing	https://linguistics.stackexchange.com/tags/speech-recognition
http://www.wikidata.org/entity/Q189436	speech recognition	http://www.wikidata.org/entity/Q30642	natural language processing	https://stackoverflow.com/tags/speech-recognition
http://www.wikidata.org/entity/Q189436	speech recognition	http://www.wikidata.org/entity/Q30642	natural language processing	https://unix.stackexchange.com/tags/speech-recognition
http://www.wikidata.org/entity/Q1948408	text segmentation	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q3484781	text simplification	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q2438971	tokenization	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q105330879	word segmentation	http://www.wikidata.org/entity/Q30642	natural language processing
http://www.wikidata.org/entity/Q7095836	Apache OpenNLP	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit	https://stackoverflow.com/tags/opennlp	http://svn.apache.org/repos/asf/opennlp	http://opennlp.apache.org
http://www.wikidata.org/entity/Q7095836	Apache OpenNLP	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit	https://stackoverflow.com/tags/opennlp	https://github.com/apache/opennlp	http://opennlp.apache.org
http://www.wikidata.org/entity/Q7095836	Apache OpenNLP	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit	https://stackoverflow.com/tags/opennlp	https://github.com/apache/opennlp.git	http://opennlp.apache.org
http://www.wikidata.org/entity/Q7095836	Apache OpenNLP	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit	https://stackoverflow.com/tags/opennlp	http://svn.apache.org/repos/asf/opennlp	https://opennlp.apache.org/
http://www.wikidata.org/entity/Q7095836	Apache OpenNLP	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit	https://stackoverflow.com/tags/opennlp	https://github.com/apache/opennlp	https://opennlp.apache.org/
http://www.wikidata.org/entity/Q7095836	Apache OpenNLP	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit	https://stackoverflow.com/tags/opennlp	https://github.com/apache/opennlp.git	https://opennlp.apache.org/
http://www.wikidata.org/entity/Q975453	General Architecture for Text Engineering	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit		https://github.com/GateNLP	http://gate.ac.uk
http://www.wikidata.org/entity/Q5533567	Gensim	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit		https://github.com/RaRe-Technologies/gensim	https://radimrehurek.com/gensim/
http://www.wikidata.org/entity/Q48688669	LingPipe	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit	https://stackoverflow.com/tags/lingpipe		http://alias-i.com/lingpipe/
http://www.wikidata.org/entity/Q6553971	LinguaStream	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit
http://www.wikidata.org/entity/Q5396532	MALLET	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit			http://mallet.cs.umass.edu/
http://www.wikidata.org/entity/Q6906675	MontyLingua	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit			http://web.media.mit.edu/~hugo/montylingua/
http://www.wikidata.org/entity/Q3630063	Moses	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit		https://github.com/moses-smt/mosesdecoder	http://www.statmt.org/moses
http://www.wikidata.org/entity/Q1635410	Natural Language Toolkit	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit	https://stackoverflow.com/tags/nltk	https://github.com/nltk/nltk	http://nltk.org/
http://www.wikidata.org/entity/Q17071870	NiuTrans	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit			http://www.nlplab.com/NiuPlan/NiuTrans.html
http://www.wikidata.org/entity/Q47405152	Pattern	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit		https://github.com/clips/pattern	http://www.clips.ua.ac.be/pages/pattern
http://www.wikidata.org/entity/Q32998961	Stanford CoreNLP	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit			https://stanfordnlp.github.io/CoreNLP/
http://www.wikidata.org/entity/Q104840874	Stanza	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit		https://stanfordnlp.github.io/stanza/	https://stanfordnlp.github.io/stanza/
http://www.wikidata.org/entity/Q2593410	WordSmith	http://www.wikidata.org/entity/Q21129801	natural language processing toolkit			http://www.lexically.net

test testStackoverflow55961615Query, debug=False took   0.5 s
----------------------------------------------------------------------
Ran 1 test in 0.503s

OK

【讨论】：