Find the answer to your Linux question:
Results 1 to 9 of 9
Like Tree1Likes
  • 1 Post By Rubberman
Hello folks. I have a bit of a curveball for you today. I have a list of strings that pass through a gambit of "if" statements. It's complicated to explain ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Feb 2013
    Posts
    19

    BASH - Adding new lines into a "for" loop as script runs


    Hello folks. I have a bit of a curveball for you today.

    I have a list of strings that pass through a gambit of "if" statements. It's complicated to explain why, but if it passes through a particular "if" loop, the string needs to be split into 2 pieces -- and each of the 2 new pieces needs to run through the gambit of "if" statements again.

    To do this, I would have to add the 2 substrings back into the list that the "for" loop called.

    Below is a sample of what I'm talking about, and an attempted solution:

    Code:
    #!/usr/bash
    
    testnums=`cat testnums1`
    
    for testnum1 in $testnums
        do
            for testnum2 in `cat testnums2`
                do
                    testnum1=`echo $testnum1 | sed 's/e//'`
                done
            testnums=$testnums$'\n'"line8test1" ##
            echo $testnum1
        done
    
    echo $testnums
    This little script removes the first letter "e" from the strings in the variable called "testnums". The line indicated by the ## appends a new line containing the string "line8test1" to the original variable that was called by the first "for" loop.

    I was hoping that the "for" loop would evaluate a new list containing a new entry every time the loop ran. This would mean that the script would run forever, because a new line is added to the original "for" loop every time. This is the outcome I want (I will break the loop in my real script, this is just a test).

    Unfortuantely, using the attempted solution above, the first "for" loop only uses the original number of lines to assign to the variable "testnum1", ignoring the new ones it creates after every completion of itself. The line "line8test1" is thus added to the end of the string after the "for" loop is completed, as many times as the loop ran.

    Any suggestions? I was going to feed the new list (with the new split strings) back into the top of the script, and then sort | uniq to remove the duplicate results (i.e. from the first time the script was run). This is however very messy and requires I add an additional loop to control how many times the output is fed back into the script.

    Thanks

  2. #2
    Linux Newbie mactruck's Avatar
    Join Date
    Apr 2012
    Location
    City of Salt
    Posts
    187
    have you thought about using a "while read" loop?

  3. #3
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
    Posts
    11,755
    Encode your processing into a shell function, and then call that. I think that bash can handle recursive function calls (not 100% certain of that - haven't needed to yet). If not, then calling an external script with the appropriate arguments would also work.
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

  4. $spacer_open
    $spacer_close
  5. #4
    Just Joined!
    Join Date
    Feb 2013
    Posts
    19
    Thanks mactruck and Rubberman.

    I created a "while" loop that did the same thing as the "for" loop. It evaluates the line:

    Code:
    printf '%b\n' $variable | head -$n | tail -1    ## I use printf '%b\n' because I want the newlines preserved and "echo -e" doesn't work for my build for whatever reason
    With "n" increasing by 1 every cycle, thus giving me the line in question.

    This thing actually works -- I spiked my data with a value that I knew would trigger the splitting of one string into 2, and lo and behold, the additional strings were processed back into the "while" loop, giving me results consistent with the script I used to process the strings.

    Mister Rubberman, do you think my solution is "too messy"?

    EDIT:

    I looked up Shell functions. I think what you're saying is to create a function encompassing the first "for" loop with something that will cause the function to be called within itself, to re-evaluate the list of strings as it expands?

  6. #5
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
    Posts
    11,755
    Quote Originally Posted by deuteros View Post
    Thanks mactruck and Rubberman.

    I created a "while" loop that did the same thing as the "for" loop. It evaluates the line:

    Code:
    printf '%b\n' $variable | head -$n | tail -1    ## I use printf '%b\n' because I want the newlines preserved and "echo -e" doesn't work for my build for whatever reason
    With "n" increasing by 1 every cycle, thus giving me the line in question.

    This thing actually works -- I spiked my data with a value that I knew would trigger the splitting of one string into 2, and lo and behold, the additional strings were processed back into the "while" loop, giving me results consistent with the script I used to process the strings.

    Mister Rubberman, do you think my solution is "too messy"?

    EDIT:

    I looked up Shell functions. I think what you're saying is to create a function encompassing the first "for" loop with something that will cause the function to be called within itself, to re-evaluate the list of strings as it expands?
    You are making great progress! That said, RTBSMP (read the bash shell man page) about shell functions. Not hard, and may be good for what you are trying to do. All of that aside, if it works... who cares?
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

  7. #6
    Just Joined!
    Join Date
    Feb 2013
    Posts
    19
    Oh golly, Rubberman, you're making me blush. I'm worried about making a messy script because I was told bash is pretty high-level (and thus slow) -- I fear if I release the source, real programmers'll make fun of my inefficient code.

    I looked up mactruck's suggestion, using a 'while read' loop. It looks similar to a "for" loop -- I'm not clear as to how it ensures the list of the lines will be re-evaluated.

    And while I'm here, I picked up that | head $n | tail -1 trick off "ask Ubuntu" a while back. Is there a better way to pick up a single line from a list?

  8. #7
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
    Posts
    11,755
    Quote Originally Posted by deuteros View Post
    Oh golly, Rubberman, you're making me blush. I'm worried about making a messy script because I was told bash is pretty high-level (and thus slow) -- I fear if I release the source, real programmers'll make fun of my inefficient code.

    I looked up mactruck's suggestion, using a 'while read' loop. It looks similar to a "for" loop -- I'm not clear as to how it ensures the list of the lines will be re-evaluated.

    And while I'm here, I picked up that | head $n | tail -1 trick off "ask Ubuntu" a while back. Is there a better way to pick up a single line from a list?
    heh, heh... There are times when I revert to C code for this stuff. In some cases, it is actually simpler, let alone faster! Like I said, if it works, and it is fast enough, who cares? For all of that, I write code in C/C++/java/bash/sed/awk/whatever seems most efficacious to solve the problem at hand. My latest tool to extract tsdb/hadoop data for one of my analytic mavens was in bash + sed. Other work I am doing is in C++, capturing real-time systems time-series stats stored in a hadoop database in the "cloud". The bash/sed code? That was to extract that data for SAS and analytics usage.
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

  9. #8
    Just Joined!
    Join Date
    Feb 2013
    Posts
    19
    Quote Originally Posted by Rubberman View Post
    heh, heh... There are times when I revert to C code for this stuff. In some cases, it is actually simpler, let alone faster! Like I said, if it works, and it is fast enough, who cares? For all of that, I write code in C/C++/java/bash/sed/awk/whatever seems most efficacious to solve the problem at hand. My latest tool to extract tsdb/hadoop data for one of my analytic mavens was in bash + sed. Other work I am doing is in C++, capturing real-time systems time-series stats stored in a hadoop database in the "cloud". The bash/sed code? That was to extract that data for SAS and analytics usage.
    Maybe someday I, too, shall become a guru -- and I will preach the gospel of Unix-based languages from the highest mountains!

  10. #9
    Linux Guru Rubberman's Avatar
    Join Date
    Apr 2009
    Location
    I can be found either 40 miles west of Chicago, in Chicago, or in a galaxy far, far away.
    Posts
    11,755
    Quote Originally Posted by deuteros View Post
    Maybe someday I, too, shall become a guru -- and I will preach the gospel of Unix-based languages from the highest mountains!
    Yes, in the beginning there was C, the word according to Kernighan and Ritchie!
    deuteros likes this.
    Sometimes, real fast is almost as good as real time.
    Just remember, Semper Gumbi - always be flexible!

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •