allow user to include other suricata config YML files

[mmguero]

The suricata.yaml file supports the following:

# Includes:  Files included here will be handled as if they were in-lined
# in this configuration file. Files with relative pathnames will be
# searched for in the same directory as this configuration file. You may
# use absolute pathnames too.
#include:
#  - include1.yaml
#  - include2.yaml

For Malcolm users who want to do extra suricata config that's not handled by the environment variables prefixed by SURICATA_ which we then put in the config file when it gets generated, we could allow them to drop yaml files in another bind-mounted directory and include: them as above.

[mmguero]

With these changes:

suricata@suricata:/tmp$ suricata -v -c /tmp/suricata.yaml -l /tmp/ -Tr
29/11/2023 -- 21:09:16 - <Info> - Running suricata under test mode
29/11/2023 -- 21:09:16 - <Info> - Including configuration file /opt/suricata/include-configs/testing.yaml.
29/11/2023 -- 21:09:16 - <Info> - Including configuration file /opt/suricata/include-configs/testing2.yaml.

For what it's worth, the files in the directory (./suricata/include-configs/) need to have:

%YAML 1.1
---

at the beginning of the YAML file, and have a .yaml extension.

[mmguero]

Tested locally with docker install, need to test on

• Malcolm ISO (should be the same)
• Hedgehog
• Kubernetes

before closing.

[mmguero]

Note this comment I put in the suricata_config_populate.py:

Hackety-hack, don't talk back! Despite the "Including multiple files" section of https://docs.suricata.io/en/latest/configuration/includes.html#including-multiple-files saying this can be set as an array, it does not seem to be working for me.
The sample suricata.yaml file shows it should be done like this:

    include: include1.yaml
    include: include2.yaml

The reason this is a pain is that this is not actually valid YAML. So what we are going to do is remove the 'include' section here, write the YAML to a file, and then append the include:s afterwards just in plain text.

[mmguero]

Tested in Malcolm and on Hedgehog, closing.