$ in lex.models URL problem

[CalHinshaw]

I generated the api file for the lex model service and all of the methods I've tried have worked except for put_bot, which fails with an InvalidSignatureException.

The canonical string it expected had the url /bots/test/versions/%2524LATEST/, but the canonical string generated by the library had the url /bots/test/versions/$LATEST/. When I change the canonical string in the library to match the one aws expects the error changes to <AccessDeniedException><Message>Unable to determine service/operation name to be authorized</Message></AccessDeniedException>.

This has me thinking that the url is getting encoded twice before being sent but I'm really not sure. I looked through HTTPoison and hackney and only saw hackney url encoding string and now I'm out of ideas.

I'm happy to fix this and submit a PR but I'm not sure where to go from here. Any ideas?

[jkakar]

Hmmm, that is a weird one indeed. I wonder if something else is being URL encoded? I'm not sure where to go from here, but I'd probably start by adding debug logging to the AWS SDK for Go and trace the creation of the signature step by step to compare with what's happening here in aws-codegen to see if I could find where the code here diverges from what's known to work in AWS's own code.

[jkakar]

Also, thank you for reporting this issue and taking the time to debug and learn more! ❤️

[CalHinshaw]

I just figured out how to fix the problem. Hackney percent encodes urls by default but AWS expects a non percent encoded $ (which is insane), and the second encoding was happening on the AWS side of things. Passing {:path_encode_fun, fn(a) -> a end} as an option to the Hackney request fixes the problem, but also means nothing will get percent encoded. HTTPotion doesn't expose the hackney :path_encode_fun option, even though it probably should, so there's no good fix without submitting a PR to them.

There's also the issue of how percent encoding dollar signs for other AWS services will affect things, which I'm not sure about.

I'm not sure what the best thing to do is but I'll think about it over the next few days and see if I come up with a good solution.

[jkakar]

Nice work, thanks for debugging that.