Find the answer to your Linux question:
Results 1 to 5 of 5
I’m trying to use a while/do loop inside a bash script (smap_testing.sh) to read data from a tab-delimited text file (named “test”) into an array. As a trial, my text ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    May 2013
    Posts
    2

    Argument list too long


    I’m trying to use a while/do loop inside a bash script (smap_testing.sh) to read data from a tab-delimited text file (named “test”) into an array. As a trial, my text file has 3456 lines of text. Each line of text is formatted as: <string1> <tab><sting2><tab><string3>. No string is longer than 6 characters. The elements of the array are then defined as variables to be used in a java script. Everything works until it reaches the 628th element. From the 628th element to the 3456th element, I get the following error message referring to line 18 (java -Xmx1200m ...):
    Code:
    /home/user/bin/smap_testing.sh: line18: /usr/bin/java: argument too long
    My code is:
    Code:
    #!/bin/bash
    
    while IFS=$'\t' read -a myArray line 
    do
    
    variable1=${myArray[0]}
    variable2=${myArray[1]}
    variable3=${myArray[2]}
    
    export SMAPROOT=/home/user/bin
    
    # pdb java
    export CLASSPATH=${CLASSPATH}:${SMAPROOT}/classes:${SMAPROOT}/lib/biojava.jar:${SMAPROOT}/lib/pdblibs.jar:${SMAPROOT}/lib/pdbormapping.jar:${SMAPROOT}/lib/mbt.jar:${SMAPROOT}/lib/siteormapping.jar
    
    java -Xmx1200M -cp ${CLASSPATH} org.interactome.siteengine.sitesearch.SMAP -templateChain $templChain -queryChain $queryChain -output $output
    
    done < /gpfs22/home/user/bin/test
    So the script is executing though the while/do loop 627 times; when it comes to the 628th time, it is returning an error that the argument is too long - even though the argument should be no different in length than the previous 627 arguments at this point.
    Last edited by atreyu; 05-05-2013 at 02:52 AM. Reason: clarification (atreyu: added CODE tags)

  2. #2
    Linux Newbie
    Join Date
    Nov 2012
    Posts
    224
    hi,

    there are some mistakes in your code:
    line variable is useless
    if you're using only three words of the line, maybe using an array is useless (while IFS=$'\t' read a b c nm; do :use $a $b $c; done)


    exports should be done outside the loop, only once: at each loop, it's concatenated to itself; that's probably what's causing the failure.

  3. #3
    Trusted Penguin
    Join Date
    May 2011
    Posts
    4,353
    hello and welcome!

    you are needlessly re-exporting an environmental variable, and it keeps growing with every iteration of the loop. to avoid this, do that portion before the loop, e.g.:

    Code:
    #!/bin/bash
    
    export SMAPROOT=/home/user/bin
    
    # pdb java
    export CLASSPATH=${CLASSPATH}:${SMAPROOT}/classes:${SMAPROOT}/lib/biojava.jar:${SMAPROOT}/lib/pdblibs.jar:${SMAPROOT}/lib/pdbormapping.jar:${SMAPROOT}/lib/mbt.jar:${SMAPROOT}/lib/siteormapping.jar
    
    while ... do
      # java code
    done
    Edit: yes, what watael said
    Last edited by atreyu; 05-05-2013 at 03:09 AM. Reason: ditto

  4. #4
    Just Joined!
    Join Date
    May 2013
    Posts
    2
    Thanks - I figured it had to be some error that was causing a resource to grow with each iteration of the loop but was not sure of the cause.



    Quote Originally Posted by watael View Post
    hi,

    there are some mistakes in your code:
    line variable is useless
    if you're using only three words of the line, maybe using an array is useless (while IFS=$'\t' read a b c nm; do :use $a $b $c; done)


    exports should be done outside the loop, only once: at each loop, it's concatenated to itself; that's probably what's causing the failure.

  5. #5
    Trusted Penguin
    Join Date
    May 2011
    Posts
    4,353
    Marking thread as Solved. Note that you can do this yourself at any time using the Thread Tools link at the top of the page.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •