added simple filter on field functionality

[rrichardson]

This is a proposed enhancement to provide basic, low-cost routing and filtering capabilities to logzoom.

It doesn't do anything terribly special, it just adds a new config value to output plugins (currently es and s3) called if_has_field. If the config value is set, then it checks each inbound event for the presence of that Field before allowing it to proceed. It is very low cost, because it is only checking the field name, which is already unpacked into a hash-map. It doesn't need to construct any values from interface{}.

The use case for us is that we want to use logzoom to send business events, debug/error logs, and application profiler events through the same place and have them "routed" based on their type. In many cases a message can be destined for 1, or 2 of the 3 destinations, which is why checking for fields is pretty much the only simple way to accomplish this.

I am certainly open to ideas as to better ways to accomplish this architecturally, or alternate ways to implement this in code.

Thanks for making logzoom!

[rrichardson]

Actually.. I was confused. This is the wrong approach.. I think I just need to add the Route fields filtering loop that S3 and Redis have to the ES, then I should be able to route using the presence of the field, methinks. I'll update and re-submit.

[rrichardson]

I went ahead and cleaned up the route filtering logic to avoid runtime panics for all output types. It also now only does a single check of the fields map per iteration, so it is optimized a tiny bit.

[stanhu]

What happens if the value is not a string (e.g. float, integer, etc.)?

[rrichardson]

What indeed. :)

I was just following the established convention that it routes only by strings. If there is a need to support routing for all JSON datatypes, it could be done with a type switch in each output plugin. IMO that's overkill unless someone has a compelling reason.

[asemt]

I really like this functionality!
+1