Search between two strings?

[jnojr]

I have an XML file with tons of content like:

        <Group id="V-29437">
                <title>Complex passwords must be created Alpha check</title>
                <description>&lt;GroupDescription&gt;&lt;/GroupDescription&gt;</description>
                <Rule id="SV-38603r1_rule" severity="medium" weight="10.0">
                        <version>OSX00036 M6</version>
                        <title>Complex passwords must contain Alphabetic Character.</title>
                        <description>&lt;VulnDiscussion&gt;Configure the local system to verify newly created passwords conform to DoD password complexity policy. Passwords must contain 1 character from the following 4 classes: English uppercase letters, English lowercase letters, Westernized Arabic numerals, and non-alphanumeric characters. Sites are responsible for installing password complexity software complying with the current DoD requirements.&lt;/VulnDiscussion&gt;&lt;FalsePositives&gt;&lt;/FalsePositives&gt;&lt;FalseNegatives&gt;&lt;/FalseNegatives&gt;&lt;Documentable&gt;false&lt;/Documentable&gt;&lt;Mitigations&gt;&lt;/Mitigations&gt;&lt;SecurityOverrideGuidance&gt;&lt;/SecurityOverrideGuidance&gt;&lt;PotentialImpacts&gt;&lt;/PotentialImpacts&gt;&lt;ThirdPartyTools&gt;&lt;/ThirdPartyTools&gt;&lt;MitigationControl&gt;&lt;/MitigationControl&gt;&lt;Responsibility&gt;System Administrator&lt;/Responsibility&gt;&lt;IAControls&gt;IAIA-1&lt;/IAControls&gt;</description>
                        <fixtext fixref="F-33021r1_fix">Open a terminal session and run the following command.

sudo pwpolicy -n -setglobalpolicy "requiresAlpha=1"

For non managed systems the path /Local/Default would need to be added to the command, an example would be: sudo pwpolicy -n /Local/Default -setglobalpolicy "requiresAlpha=1"</fixtext>
                        <fix id="F-33021r1_fix"/>
                        <check system="C-37774r1_chk">
                                <check-content-ref name="M" href="VMS_XCCDF_Benchmark_MACOSX_10.6.xml"/>
                                <check-content>Open a terminal session and run the following command.

pwpolicy -n -getglobalpolicy | tr " " "\n" | grep requiresAlpha

If the value of requiresAlpha is not set to 1, this is a finding.

NOTE: If the command returns a response of: password server is not configured, the system is not managed. Add the path /Local/Default to the above command, an example would be: pwpolicy -n /Local/Default -getglobalpolicy | tr " " "\n" | grep requiresAlpha</check-content>
                        </check>
                </Rule>
        </Group>
        <Group id="V-29439">
                <title>Complex passwords must be created symbol check</title>
                <description>&lt;GroupDescription&gt;&lt;/GroupDescription&gt;</description>
                <Rule id="SV-38607r1_rule" severity="medium" weight="10.0">
                        <version>OSX00038 M6</version>
                        <title>Complex passwords must contain a Symbolic Character.</title>
                        <description>&lt;VulnDiscussion&gt;Configure the local system to verify newly created passwords conform to the DoD password complexity policy. Passwords must contain 1 character from the following 4 classes: English uppercase letters, English lowercase letters, Westernized Arabic numerals, and non-alphanumeric characters. Sites are responsible for installing password complexity software that complies with current DoD requirements.&lt;/VulnDiscussion&gt;&lt;FalsePositives&gt;&lt;/FalsePositives&gt;&lt;FalseNegatives&gt;&lt;/FalseNegatives&gt;&lt;Documentable&gt;false&lt;/Documentable&gt;&lt;Mitigations&gt;&lt;/Mitigations&gt;&lt;SecurityOverrideGuidance&gt;&lt;/SecurityOverrideGuidance&gt;&lt;PotentialImpacts&gt;&lt;/PotentialImpacts&gt;&lt;ThirdPartyTools&gt;&lt;/ThirdPartyTools&gt;&lt;MitigationControl&gt;&lt;/MitigationControl&gt;&lt;Responsibility&gt;System Administrator&lt;/Responsibility&gt;&lt;IAControls&gt;IAIA-1&lt;/IAControls&gt;</description>
                        <fixtext fixref="F-33023r1_fix">Open a terminal session and run the following command.

sudo pwpolicy -n -setglobalpolicy "requiresSymbol=1"

For non managed systems the path /Local/Default would need to be added to the command, an example would be: sudo pwpolicy -n /Local/Default -setglobalpolicy "requiresSymbol=1"</fixtext>
                        <fix id="F-33023r1_fix"/>
                        <check system="C-37776r1_chk">
                                <check-content-ref name="M" href="VMS_XCCDF_Benchmark_MACOSX_10.6.xml"/>
                                <check-content>Open a terminal session and run the following command.

pwpolicy -n -getglobalpolicy | tr " " "\n" | grep requiresSymbol

If the value of requireSymbol is not set to 1, this is a finding.

NOTE: If the command returns a response of password server is not configured, the system is not managed. Add the path /Local/Default to the above commands, an example would be: pwpolicy -n /Local/Default -getglobalpolicy | tr " " "\n" | grep requiresSymbol </check-content>
                        </check>
                </Rule>
        </Group>

I'd like to iterate through it, taking "<Group id=" through "</Group> and performing operations on each chunk in turn. I'm just coming up with a blank...

[atreyu]

I'd like to iterate through it, taking "<Group id=" through "</Group> and performing operations on each chunk in turn. I'm just coming up with a blank...

Hi,

You don't mention if you want to deal with the raw HTML, or if you just care about the plain text. If the latter, then you might want to consider using tidy and html2text, to make things a lot cleaner.

I am assuming you mean the former (dealing w/the raw HTML).

As for searching between two expressions on different lines, i know you can do that w/awk's range expressions, e.g.:

awk '/start/,/stop/{print}'

but for me, this would be easier w/Perl. Below is an example. Save it to a script, say "read-groups.pl". Make it executable, then pass to it your XML file as a command line arg, e.g.:

./read-groups.pl groups.xml

Here is the code:

#!/usr/bin/perl
use strict;
use warnings;

# get the XML file as a command line arg
my $file = shift || die "Usage: $0 <xmlfile>\n";
die $file,": No such file\n" unless(-f$file);

# hash used to store lines, per Group
my %hash;

# index number to keep track of Group keys
my $i = -1;

my $start;

# read the XML file line by line
open(FH,'<',$file) or die "can't read '$file': $!\n";
while(<FH>){
  chomp;
  my $line = $_;
  $_ =~ s/^[ \t]+//; # strip leading whitespace
  if(/^<Group /){
    $i +=1;
    $start = 1;
    push(@{$hash{$i}},$line);
  }elsif(/<\/Group>/){
    $start = 0;
    push(@{$hash{$i}},$line);
  }else{
    push(@{$hash{$i}},$line) if($start);
  }
}
close(FH);

print "Number of Group keys: ",scalar keys %hash,"\n";

# now print the groups
for my $key(sort {$a<=>$b} keys %hash){
  print "\n***** GROUP $key START *****\n";
  for(@{$hash{$key}}){
    print "$_\n";
  }
  print "***** GROUP $key END *****\n";
}