global.cfg using flatten is broken, returning list instead of dict

[lingfish]

So this is a weird one, I'm getting this:

TASK [devops-coop.haproxy : Build up the global config] *************************************************************************************************************************************
fatal: [host]: FAILED! => {"changed": false, "msg": "AnsibleUndefinedVariable: 'list object' has no attribute 'items'"}

With much head-scratching, I found this is happening at the end of the template, when dealing with the tune option. I've discovered this:

[anotherhost] TASK: devops-coop.haproxy : Build up the global config (debug)> p task_vars['haproxy_global']
{u'chroot': u'/var/lib/haproxy',
 u'daemon': True,
 u'group': u'haproxy',
 u'log': [{u'address': u'/dev/log', u'facility': u'local0'},
          {u'address': u'/dev/log',
           u'facility': u'local1',
           u'level': u'notice'}],
 u'ssl_default_bind_ciphers': u'{{ _haproxy_ssl_ciphers }}',
 u'ssl_default_bind_options': u'{{ _haproxy_ssl_options }}',
 u'ssl_default_server_ciphers': u'{{ _haproxy_ssl_ciphers }}',
 u'ssl_default_server_options': u'{{ _haproxy_ssl_options }}',
 u'tune': {u'ssl': {u'default-dh-param': 2048}},
 u'user': u'haproxy'}

Ok, tune is a dict of dicts, good. Lets feed this by hand:

(ansible)host [05:22 PM] [j:1] ~/ansible $ cd roles/devops-coop.haproxy/filter_plugins/
(ansible)host [05:22 PM] [j:1] ~/.../roles/devops-coop.haproxy/filter_plugins $ ipython

In [2]: import flatten
In [9]: h = {u'chroot': u'/var/lib/haproxy',
   ...:  u'daemon': True,
   ...:  u'group': u'haproxy',
   ...:  u'log': [{u'address': u'/dev/log', u'facility': u'local0'},
   ...:           {u'address': u'/dev/log',
   ...:            u'facility': u'local1',
   ...:            u'level': u'notice'}],
   ...:  u'ssl_default_bind_ciphers': u'{{ _haproxy_ssl_ciphers }}',
   ...:  u'ssl_default_bind_options': u'{{ _haproxy_ssl_options }}',
   ...:  u'ssl_default_server_ciphers': u'{{ _haproxy_ssl_ciphers }}',
   ...:  u'ssl_default_server_options': u'{{ _haproxy_ssl_options }}',
   ...:  u'tune': {u'ssl': {u'default-dh-param': 2048}},
   ...:  u'user': u'haproxy'}

In [11]: h['tune']
Out[11]: {u'ssl': {u'default-dh-param': 2048}}

In [12]: for param, value in flatten.flatten(h['tune']).items():
    print param, value
   ....:     
ssl.default-dh-param 2048

All good thus far. Now with some debugging in the template:

haproxy_global.tune | flatten | type_debug: list
--
haproxy_global.tune | type_debug: dict
--
haproxy_global.tune | flatten: [u'ssl']
--
    haproxy_global.tune: {u'ssl': {u'default-dh-param': 2048}}
--
haproxy_global pretty print:
{u'chroot': u'/var/lib/haproxy',
 u'daemon': True,
 u'group': u'haproxy',
 u'log': [{u'address': u'/dev/log', u'facility': u'local0'},
          {u'address': u'/dev/log',
           u'facility': u'local1',
           u'level': u'notice'}],
 u'ssl_default_bind_ciphers': u'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS',
 u'ssl_default_bind_options': u'no-sslv3 no-tls-tickets',
 u'ssl_default_server_ciphers': u'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS',                                                     u'ssl_default_server_options': u'no-sslv3 no-tls-tickets',                                                                                                                                   u'tune': {u'ssl': {u'default-dh-param': 2048}},                                                                                                                                              u'user': u'haproxy'}

Very odd! Perhaps custom filters to jinja always return lists or something?!?

I'm running Ansible devel, and only just installed your role.

[lingfish]

So, I've discovered the issue; there's a name collision with your flatten, and a Ansible core one of the same name.

I would submit a PR, but I'll just name it something that probably doesn't agree with you devs.

[benwebber]

Oh, brilliant. Thanks for discovering that. Why not haproxy_flatten to give it a namespace?

[lingfish]

Sure, I'm totally indecisive on it to be honest, and a little shocked I didn't get any warnings from Ansible!

[RebelCodeBase]

As I really need a solution I've created a pull request (#117) in case you want to go along the "simple renaming" route.

[lingfish]

@benwebber ? Thoughts on the PR?

[chrislmayes]

Looks like people are forking to fix.