【发布时间】:2017-05-13 23:07:16
【问题描述】:
大家!
我正在尝试使用 Apache Spark、Elasticsearch 和 Kibana 实时分析推文流。作为 X 轴,我希望获得推文的“timestamp_ms”属性的值,作为 Y 轴,我希望获得唯一用户 ID 的数量,从而为我提供推文的覆盖范围。
但是,当我尝试创建索引时,选中“索引包含基于时间的事件”复选框并指定下面的属性,我遇到了一个问题,我稍后会解释。
curl -XPUT 'localhost:9200/twitter?pretty' -d'
{
"mappings": {
"tweet": {
"properties": {
"timestamp_ms": {
"type": "date",
"format": "yyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"
}
}
}
}
}'
问题是当我开始将推文流式传输到 ES 时,由于发生此错误,我无法使用 Timelion 显示它们:
org.elasticsearch.transport.RemoteTransportException: [e1S9kLk][127.0.0.1:9300][indices:data/read/search[phase/query]]
Caused by: java.lang.IllegalArgumentException: Field [timestamp_ms] of type [text] does not support custom time zones
at org.elasticsearch.index.mapper.MappedFieldType.docValueFormat(MappedFieldType.java:453) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder.config(ValuesSourceAggregationBuilder.java:374) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder.resolveConfig(ValuesSourceAggregationBuilder.java:305) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder.doBuild(ValuesSourceAggregationBuilder.java:299) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.support.ValuesSourceAggregationBuilder.doBuild(ValuesSourceAggregationBuilder.java:49) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.AbstractAggregationBuilder.build(AbstractAggregationBuilder.java:126) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.AggregatorFactories$Builder.build(AggregatorFactories.java:211) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.AggregatorFactory.<init>(AggregatorFactory.java:187) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.bucket.filters.FiltersAggregatorFactory.<init>(FiltersAggregatorFactory.java:48) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.bucket.filters.FiltersAggregationBuilder.doBuild(FiltersAggregationBuilder.java:176) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.AbstractAggregationBuilder.build(AbstractAggregationBuilder.java:126) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.aggregations.AggregatorFactories$Builder.build(AggregatorFactories.java:211) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:731) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.SearchService.createContext(SearchService.java:553) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:529) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:264) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:300) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.action.search.SearchTransportService$6.messageReceived(SearchTransportService.java:297) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.transport.TransportService$6.doRun(TransportService.java:577) [elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:527) [elasticsearch-5.1.1.jar:5.1.1]
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.1.1.jar:5.1.1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_111]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_111]
at java.lang.Thread.run(Thread.java:745) [?:1.8.0_111]
我试过设置以下,但还是不行:
timelion:es.timefield(默认:@timestamp)-> 设置为 timestamp_ms
timelion:es.default_index(默认值:_all)-> 设置为 twitter
感谢任何有关如何解决此问题的想法,因为我已经研究了很多,但找不到任何适合我的东西。
谢谢!
【问题讨论】:
标签: java elasticsearch apache-spark twitter kibana