jinja_to_execution_module.rst 5.0 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164
  1. .. _tutorial-jinja_to_execution-module:
  2. =================================================
  3. How to Convert Jinja Logic to an Execution Module
  4. =================================================
  5. .. versionadded: 2016.???
  6. .. note::
  7. This tutorial assumes a basic knowledge of Salt states and specifically
  8. experience using the `maps.jinja` idiom.
  9. This tutorial was written by a salt user who was told "if your maps.jinja
  10. is too complicated, write an execution module!". If you are experiencing
  11. over-complicated jinja, read on.
  12. The Problem: Jinja Gone Wild
  13. ----------------------------
  14. It is often said in the Salt community that "Jinja is not a Programming Language".
  15. There's an even older saying known as Maslow's hammer.
  16. It goes something like
  17. "if all you have is a hammer, everything looks like a nail".
  18. Jinja is a reliable hammer, and so is the `maps.jinja` idiom.
  19. Unfortunately, it can lead to code that looks like the following.
  20. .. code-block:: jinja
  21. # storage/maps.yaml
  22. {% import_yaml 'storage/defaults.yaml' as default_settings %}
  23. {% set storage = default_settings.storage %}
  24. {% do storage.update(salt['grains.filter_by']({
  25. 'Debian': {
  26. },
  27. 'RedHat': {
  28. }
  29. }, merge=salt['pillar.get']('storage:lookup'))) %}
  30. {% if 'VirtualBox' == grains.get('virtual', None) or 'oracle' == grains.get('virtual', None) %}
  31. {% do storage.update({'depot_ip': '192.168.33.81', 'server_ip': '192.168.33.51'}) %}
  32. {% else %}
  33. {% set colo = pillar.get('inventory', {}).get('colo', 'Unknown') %}
  34. {% set servers_list = pillar.get('storage_servers', {}).get(colo, [storage.depot_ip, ]) %}
  35. {% if opts.id.startswith('foo') %}
  36. {% set modulus = servers_list | count %}
  37. {% set integer_id = opts.id | replace('foo', '') | int %}
  38. {% set server_index = integer_id % modulus %}
  39. {% else %}
  40. {% set server_index = 0 %}
  41. {% endif %}
  42. {% do storage.update({'server_ip': servers_list[server_index]}) %}
  43. {% endif %}
  44. {% for network, _ in salt.pillar.get('inventory:networks', {}) | dictsort %}
  45. {% do storage.ipsets.hash_net.foo_networks.append(network) %}
  46. {% endfor %}
  47. This is an example from the author's salt formulae demonstrating misuse of jinja.
  48. Aside from being difficult to read and maintain,
  49. accessing the logic it contains from a non-jinja renderer
  50. while probably possible is a significant barrier!
  51. Refactor
  52. --------
  53. The first step is to reduce the maps.jinja file to something reasonable.
  54. This gives us an idea of what the module we are writing needs to do.
  55. There is a lot of logic around selecting a storage server ip.
  56. Let's move that to an execution module.
  57. .. code-block:: jinja
  58. # storage/maps.yaml
  59. {% import_yaml 'storage/defaults.yaml' as default_settings %}
  60. {% set storage = default_settings.storage %}
  61. {% do storage.update(salt['grains.filter_by']({
  62. 'Debian': {
  63. },
  64. 'RedHat': {
  65. }
  66. }, merge=salt['pillar.get']('storage:lookup'))) %}
  67. {% if 'VirtualBox' == grains.get('virtual', None) or 'oracle' == grains.get('virtual', None) %}
  68. {% do storage.update({'depot_ip': '192.168.33.81'}) %}
  69. {% endif %}
  70. {% do storage.update({'server_ip': salt['storage.ip']()}) %}
  71. {% for network, _ in salt.pillar.get('inventory:networks', {}) | dictsort %}
  72. {% do storage.ipsets.hash_net.af_networks.append(network) %}
  73. {% endfor %}
  74. And then, write the module.
  75. Note how the module encapsulates all of the logic around finding the storage server IP.
  76. .. code-block:: python
  77. # _modules/storage.py
  78. #!python
  79. """
  80. Functions related to storage servers.
  81. """
  82. import re
  83. def ips():
  84. """
  85. Provide a list of all local storage server IPs.
  86. CLI Example::
  87. salt \* storage.ips
  88. """
  89. if __grains__.get("virtual", None) in ["VirtualBox", "oracle"]:
  90. return [
  91. "192.168.33.51",
  92. ]
  93. colo = __pillar__.get("inventory", {}).get("colo", "Unknown")
  94. return __pillar__.get("storage_servers", {}).get(colo, ["unknown",])
  95. def ip():
  96. """
  97. Select and return a local storage server IP.
  98. This loadbalances across storage servers by using the modulus of the client's id number.
  99. :maintainer: Andrew Hammond <ahammond@anchorfree.com>
  100. :maturity: new
  101. :depends: None
  102. :platform: all
  103. CLI Example::
  104. salt \* storage.ip
  105. """
  106. numerical_suffix = re.compile(r"^.*(\d+)$")
  107. servers_list = ips()
  108. m = numerical_suffix.match(__grains__["id"])
  109. if m:
  110. modulus = len(servers_list)
  111. server_number = int(m.group(1))
  112. server_index = server_number % modulus
  113. else:
  114. server_index = 0
  115. return servers_list[server_index]
  116. Conclusion
  117. ----------
  118. That was... surprisingly straight-forward.
  119. Now the logic is available in every renderer, instead of just Jinja.
  120. Best of all, it can be maintained in Python,
  121. which is a whole lot easier than Jinja.