Implement Elasticsearch plugin #55
Conversation
|
👍 nice. I was working on something similar ... you beat me to it :-) I had a couple of questions about your design.
elastigo, for example, has nice connection functionality. It keeps a pool of ES hosts, runs periodic health checks, and uses the Epsilon Greedy algorithm to mark failed nodes. I'm not sure what's best practice for Telegraf. Some plugins (e.g., kafka, rethinkdb) use clients and others just query a service directly (e.g. memcached, redis).
This plugin might be a good use-case for pass/drop plugin options. Since measurements are prefixed with stat type, if a user only cares about indices stats, they can set |
| } | ||
|
|
||
| for _, n := range esRes.Nodes { | ||
| tags := map[string]string{ |
There was a problem hiding this comment.
node_name should be added to tags. node_uuid perhaps, but I'm not sure if it would be very useful.
There was a problem hiding this comment.
+1. Please do make good use of tags. Node name, hostname if possible, etc.
On Wednesday, July 8, 2015, Alvaro Morales notifications@github.com wrote:
In plugins/elasticsearch/elasticsearch.go
#55 (comment):
return err- }
- if r.StatusCode != http.StatusOK {
return fmt.Errorf("elasticsearch: API responded with status-code %d, expected %d", r.StatusCode, http.StatusOK)- }
- d := json.NewDecoder(r.Body)
- esRes := &struct {
ClusterName string `json:"cluster_name"`Nodes map[string]*node `json:"nodes"`- }{}
- if err = d.Decode(esRes); err != nil {
return err- }
- for _, n := range esRes.Nodes {
tags := map[string]string{node_name should be added to tags. node_uuid perhaps, but I'm not sure if
it would be very useful.—
Reply to this email directly or view it on GitHub
https://github.com/influxdb/telegraf/pull/55/files#r34196293.
There was a problem hiding this comment.
Will add the following too to the tags:
- Node UUID
- Node properties
|
To address your question about not using a client, I think that for an application like Telegraf it is better to keep the number of dependencies as low as possible. In the end, all the libraries (as far as I know) wrap the Elasticsearch REST API. I've implemented the plugin in such a way that it can run on every node within a Elasticsearch cluster (since you might be as well interested in the machine load, disk space, mem usage etc...), or if you prefer it will fetch the stats from all the nodes within the cluster ( To be honest, I'm not 100% sure if all the other stats are relevant, but I think you're right and some will be. This shouldn't be a problem however, since I'm prefixing all stats with |
|
I've updated the tags. Node ID and the node attributes are now added to the tags as well. Will look into implementing the other stats. |
brocaar
changed the title
|
@otoolep @alvaromorales I've updated my pull-request. Please let me know what you think! |
| "github.com/stretchr/testify/assert" | ||
| ) | ||
|
|
||
| type tranportMock struct { |
There was a problem hiding this comment.
minor typo: transport
|
Looks really good! |
|
Thanks for the feedback. I fixed the typo :-) |
Implement Elasticsearch plugin
This implements a plugin for getting the stats from Elasticsearch.
See also: https://www.elastic.co/guide/en/elasticsearch/reference/1.6/cluster-nodes-stats.html