Explicitly fetch http payloads for analysis

[jatrost]

Many of the exploit attempts cause ES to attempt to wget a URL. This URL should be fetched, md5'ed, and ideally included in the payload for analysis. Maybe this should be configurable.

[jordan-wright]

I've thought about this exact thing. Kippo does this quite well. My only concern would be how to parse the wget command sent to the honeypot.

[jatrost]

This is how we do it for shockpot: https://github.com/threatstream/shockpot/blob/master/commands.py. We just grabbed a bunch of example exploit attempts and made sure that we covered most of them. The bad guys will be able to evade but I haven't seen many examples of this in the wild.

[cloughrm]

Opened a PR to address this enhancement #6

[jordan-wright]

Thanks for this everyone! Let me review the PR and I'll see what I can do to merge. My biggest holdup is reviewing exactly what's going on with the parsing to make sure there isn't a way to exploit the honeypot.

Give me a week or so and bug me if I haven't merged by then 😄

Thanks again!

[cloughrm]

Thanks @jordan-wright - I have one more commit coming shortly to address handling form-encoded vs json encoded post requests.