【发布时间】：2021-05-17 15:39:00
【问题描述】：
这是我的情况。
- Food_database 在 mysql 中。
- food_database 中有 130 个表
- 我想通过 logstash_jdbc 将 130 个表发送到 elasticsearch。
-> 如何将所有数据库的表发送到elasticsearch？
my conf file (尝试)
input {
jdbc {
clean_run => true
jdbc_driver_library => "C:\ElasticSearch\mysql-connector-java-8.0.23\mysql-connector-java-8.0.23.jar"
jdbc_driver_class => "com.mysql.jdbc.Driver"
jdbc_connection_string => "jdbc:mysql://localhost:3306/food_database?useSSL=false&user=root&password=1234"
jdbc_user => "root"
jdbc_password => "1234"
schedule => "* * * * *"
statement => "select * from ??????"
#use_column_value => true
#tracking_column => "jobid"
}
}
output{
elasticsearch {
hosts => "localhost:9200"
index => "test_indexfile"
}
stdout {
codec => rubydebug
}
}
但我不知道如何发送 food_databases 中的所有 130 个表。
我通过谷歌搜索发现了一个类似的问题，但我无法解决。
->save whole database to elasticsearch using logstash
->https://dzone.com/articles/migrating-mysql-data-to-elasticsearch-using-logsta
请帮帮我。
更新发布（food_database 中的表格）
+--------------------------------------+
| Tables_in_food_database |
+--------------------------------------+
| access_token |
| activity |
| address |
| answer_abuse_reason |
| answer_report_abuse |
| attribute |
| attribute_group |
| banner |
| banner_group |
| banner_image |
| banner_image_description |
| blog |
| blog_related |
| category |
| category_commission |
| category_description |
| category_path |
| contact |
| country |
| coupon |
| coupon_product_category |
| coupon_usage |
| coupon_usage_product |
| currency |
| customer |
| customer_activity |
| customer_cart |
| customer_document |
| customer_group |
| customer_ip |
| customer_transaction |
| customer_wishlist |
| delivery_allocation |
| delivery_location |
| delivery_location_to_location |
| delivery_person |
| delivery_person_to_location |
| delivery_status |
| email_template |
| geo_zone |
| jobs |
| language |
| login_log |
| manufacturer |
| migrations |
| order |
| order_cancel_reason |
| order_history |
| order_log |
| order_product |
| order_product_log |
| order_status |
| order_total |
| page |
| page_group |
| payment |
| payment_archive |
| payment_items |
| payment_items_archive |
| paypal_order |
| paypal_order_transaction |
| permission_module |
| permission_module_group |
| plugins |
| price_update_file_log |
| product |
| product_answer |
| product_answer_like_dislike |
| product_attribute |
| product_description |
| product_discount |
| product_image |
| product_price_log |
| product_question |
| product_rating |
| product_related |
| product_special |
| product_stock_alert |
| product_tag |
| product_tire_price |
| product_to_category |
| product_varient |
| product_varient_option |
| product_varient_option_details |
| product_varient_option_image |
| product_view_log |
| quotation |
| razorpay_order |
| razorpay_order_transaction |
| service |
| service_category |
| service_category_path |
| service_enquiry |
| service_image |
| service_to_category |
| sessions |
| settings |
| settlement |
| settlement_item |
| site_filter |
| site_filter_category |
| site_filter_section |
| site_filter_section_item |
| sku |
| stock_log |
| stock_status |
| stripe_order |
| stripe_order_transaction |
| tax |
| trend |
| trend_image |
| trend_recommend |
| user_group |
| users |
| varients |
| varients_value |
| vendor |
| vendor_category |
| vendor_coupon |
| vendor_coupon_product_category |
| vendor_global_setting |
| vendor_invoice |
| vendor_invoice_item |
| vendor_order_archive |
| vendor_order_archive_log |
| vendor_order_products |
| vendor_order_status |
| vendor_orders |
| vendor_orders_log |
| vendor_payment |
| vendor_payment_archive |
| vendor_product |
| widget |
| widget_item |
| zone |
| zone_to_geo_zone |
+--------------------------------------+
136 rows in set (0.00 sec)
我想
send all the values of my goals 136 tables通过logstash 进行elasticsearch。
【问题讨论】：
-
我的问题是为什么要在 Elasticsearch 中一对一地发送所有表？最终目标是什么？你的用例是什么？
-
@Val 我的问题。如何将所有数据库的表发送到elasticsearch。
my goals I want to save all tables from the schema of the database. -
@Val 这是我的用例
conf file (attempt) -
这并不能回答为什么您需要 Elasticsearch 中的所有这些表的问题。将 RDBMS 1:1 复制到 ES 中是非常罕见的，通常您创建一个或多个您需要的视图并仅将其存储在 ES 中以用于搜索目的。话虽如此，你能解释一下你的用例是什么吗？
-
@Val 用例
store all 130 tables存储在数据库（MYSQL）中，并希望将它们可视化in kibana另外，它是检查数据是否实时存储。in kibana