Looking for the proper automation solution.

[todd_dsm]

I have a problem. I'm looking for the best (not the easiest) solution. I'm sure this problem was worked out in the 80's, I just don't know how to Google for it.

I need to isolate portions of configuration files to verify they have proper configuration. I would like to keep this confined to bash shell scripting

Some configuration files contain name-value pairs, and some are directives-only. This aspect of the problem is also not an issue for me.

Example: setting postfix recipient restrictions; I need to test if the directive exists, if not sed it into the file:
export zPostfixRecipRestrict="/opt/zimbra/conf/postfix_recipient_restrictions.cf" (simple ASCII file)

# Inject reject_unknown_recipient_domain if not configured
grep reject_unknown_recipient_domain ${zPostfixRecipRestrict}
if [[ $? != 0 ]]; then
sed -i '/reject_non_fqdn_recipient/a\reject_unknown_recipient_domain' ${zPostfixRecipRestrict}
fi

This is not such a big deal until you have to do it many times times. Say you have a samba configuration file with 157 lines of name/value pairs, the tedium would build quick. Of course, it indicates that I'm not approaching this from the correct view-point.

It seems the most scalable method is to read, both (1) target values and (2) actual config file values, into 2 arrays then compare array elements against one-another:

       array1              |              array2
       -----------------------------------------
test   "$target_value[0]"     "$actual_value[0]"

If they are the same then move on to the next array element, else if NOT the same then sed the target value into the actual config file and retest; then move to the next array element.
---

I'd say I'm an intermediate shell scriptor. I haven't had to work with arrays much but most everything else is within reach. Either way, make no assumptions and don't hold back. If I don't get it, I will study the problem till I do get it.

Again, I haven't been down this road before so any input/advice/help is appreciated. Thanks in advance.

[Irithori]

Puppet, augeas and possibly hiera are your friends.

As a quite straight forward showcase, this puppet manifest shows how augeas makes sure, that variables from a hiera datastore are in /etc/ssh/sshd_config

class sshd::data {
  $ssh_permitrootlogin        = 'no'
  $ssh_ciphers                = 'aes128-ctr,aes256-ctr,arcfour256,arcfour,aes128-cbc,aes256-cbc'
  $ssh_listenaddres           = '0.0.0.0'
  $ssh_passwordauthentication = 'no'
}
 
class sshd::server (
  $permitrootlogin        = hiera("ssh_permitrootlogin"),
  $ciphers                = hiera("ssh_ciphers"),
  $listenaddress          = hiera("ssh_listenaddress"),
  $passwordauthentication = hiera("ssh_passwordauthentication")
) {
  include sshd::server::install
  include sshd::server::configure
  include sshd::server::service
}

class sshd::server::install {
  package { 'openssh-server': ensure => 'installed', }
}

class sshd::server::configure {
  augeas { "sshd_config":
    context => "/files/etc/ssh/sshd_config",
    changes => [
      "set PermitRootLogin ${sshd::server::permitrootlogin}",
      "set Ciphers ${sshd::server::ciphers}",
      "set ListenAddress ${sshd::server::listenaddress}",
      "set PasswordAuthentication ${sshd::server::passwordauthentication}",
    ],
    require => Package['openssh-server'],
    notify  => Service['sshd'],
  }
}

class sshd::server::service {
  service { "sshd":
    enable     => 'true',
    hasrestart => 'true',
    ensure     => 'running',
    require    => Package['openssh-server'],
  }
}

Basically:
If the sshd::server class is included (for a node, not shown here), then puppet will
- install the openssh-server package, if not already there
- configure its config file
- and make sure the service sshd is started.
- if you then later change one of the values in hiera, then the config file gets changed as a consequence and the service will be restarted

The fun part:
Hiera is a hierarchical datastore.
Which means, that the hiera lookup can return different values for an env (dev/qa/prod), a domain or even a hostname..

[Irithori]

As for samba.
The smb.conf file has sections.
Say one for each share.

My solution for that was to use the puppet template system together with some data definition:
Here is a snippet of my samba manifest, showing only the ::data (hiera) part.

--snip--
class samba::server::data {
  $smb_conf_shares = {
    example => {
      "comment"              => 'example',
      "path"                 => '/dev/null',
      "hosts allow"          => '',
      "browseable"           => 'no',
      "readable"             => 'no',
      "writable"             => 'no',
      "printable"            => 'no',
      "guest ok"             => 'no',
      "guest only"           => 'no',
      "force create mode"    => '0000',
      "force directory mode" => '0000',
      "force user"           => 'nobody',
      "force group"          => 'nobody',
    },
  }
}
--snap--

It´s a hash of hashes of hashes.
It´s designed to have an arbitrary number of shares, each with own keywords.

The template to expand that looks like this:

--snip--
<% conf_shares.sort.each do |share_name,share_data| -%>
[<%= share_name %>]
<% share_data.sort.each do |opt_name, opt_value| -%>
<%= opt_name -%> = <%= opt_value %>
<% end -%>
--snap--

In case you are wondering: Yes it pays off to know/learn ruby.

[todd_dsm]

Irithori, thanks for the input. I dig puppet for maintaining servers. But, I should have mentioned, I'm building them from scratch and using bash. Part of the automation is installing Zimbra, this package makes puppet somewhat problematic. Either way, I'm invested 99% in bash.

I really like your concept for the other servers I build that do not have Zimbra on them though. Very cool stuff.

[Irithori]

Hmm, I am not quite sure where the roadblock for using puppet with zimbra is.
Imho, the manifests are just a way to write down in a formal and reproduceable way, what you classically would do on the console during a manual deploy.
Granted, puppet has some dependencies and it can be a PITA to develop some parts, that are usually just a bash oneliner away.

Anyway.
To recap:
- You need to isolate portions of configuration files to verify they have proper configuration.
- You would like to keep this confined to bash shell scripting

Would it then be an option for you to use augeas as a bash tool?
Augeas — Documentation

It seems to match your usecase better than sed/awk/etc, as it
- has understanding of the config file grammar
- and offers ways to get/set values for specific keys.

[todd_dsm]

Would it then be an option for you to use augeas as a bash tool?

Yes, I believe it would. augeas is available via EPEL. I've been playing with it for the last day or so. It looks great.

augeas should be required for building this way. Thank you for the help.