Find the answer to your Linux question:
Page 1 of 2 1 2 LastLast
Results 1 to 10 of 17
Hi all, I am new to shell scripting and I am trying to create a script that counts the number of bytes of files in a folder so I can ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Dec 2013
    Posts
    13

    Need help with shell scripting for loops and arrays


    Hi all,

    I am new to shell scripting and I am trying to create a script that counts the number of bytes of files in a folder so I can later "group" them in 4GB folders.

    My code is:
    Code:
    #!/bin/bash
    IFS='
    '
    
    cd ~/Desktop/1/
    file_bytes=`ls -l | grep ^- | awk '{ print $5 }'`
    file_names=`ls -l | grep ^- | awk '{ print $9 }'`
    
    total_bytes=0
    for bytes in ${file_bytes[@]}
      do
        for names in ${file_names[@]}
          do 
            let total_bytes+=$bytes 
    	  if [ $total_bytes -lt 4000 ]; then
    	    echo $names $bytes
    	  fi
    	  break
          done
      done
    When I run this code, I am getting the output:
    1.sh 243
    1.sh 3385
    1.sh 271

    Can anyone help me understand why the file names doesn't change to 1.sh 2.sh 3.sh ...?

    Thanks in advance.
    Attached Files Attached Files

  2. #2
    Linux Enthusiast
    Join Date
    Jan 2005
    Location
    Saint Paul, MN
    Posts
    668
    1. You have no arrays in your code.
    2. you are walking your lists using the first size for all names then the second size for all the names.


    Change:
    Code:
    file_bytes=`ls -l | grep ^- | awk '{ print $5 }'`
    file_names=`ls -l | grep ^- | awk '{ print $9 }'`
    to be (which builds an array):
    Code:
    file_bytes=( `ls -l | grep ^- | awk '{ print $5 }'` )
    file_names=( `ls -l | grep ^- | awk '{ print $9 }'` )
    It is wise to use:
    Code:
       "${file_names[@]}"
    rather than:
    Code:
        ${file_names[@]}
    Because if you had been allowing files with spaces in the names (which will not work with your 'ls .... awk ....' commands as it will stop on the first space in a name), it expansion will have each value quoted.

    In your case, you wish to use the index of the array and index into both arrays.

    Code:
    for idx in "${!file_bytes[@]}"; do
        echo "${file_names[${idx}]}"  "${file_bytes[${idx}]}"
    done
    Last edited by alf55; 12-07-2013 at 05:59 PM.

  3. #3
    Linux Newbie
    Join Date
    Nov 2012
    Posts
    232
    hi,

    Again, command substitution shoud not be used as is to build an array: filename may have weird (at least space) characters that might break it.

    Also, don't ever try to parse `ls` output: it's versatile, and doesn't have neither fixed length, nor separator;

    instead use `stat` which output can be formatted to fit your needs, and shell pathname expansion.

    Code:
     while read -r s f; do echo "$f"; ts+=$s; done < <(stat -c '%s %n' ~/Desktop/1/*-*)
    two tricks are used here:
    1/ the filename is output last so $f contains the rest of the output
    2/ use of process substitution instead of a pipe so there's no subshell in which variables may be "trapped"
    Last edited by watael; 12-07-2013 at 06:47 PM.

  4. $spacer_open
    $spacer_close
  5. #4
    Just Joined!
    Join Date
    Dec 2013
    Posts
    13
    Thanks a lot!

    I followed your advices and I wrote the following code which is roughly what I wanted:
    Code:
    total_bytes=0
    for idx in "${!file_bytes[@]}"
    do
    	let total_bytes+="${file_bytes[${idx}]}"
    	if [ $total_bytes -lt 4000 ]; then
    		
    		cp "${file_names[${idx}]}" ~/Desktop/2
    	fi
    done
    I didn't know about key and values in an array. I can't find a good tutorial about
    shell scripting arrays, I am getting my information from different sources.

    And I am definitely going to read about the stat command

    Thanks again.
    Last edited by flit; 12-07-2013 at 06:28 PM.

  6. #5
    Linux Enthusiast
    Join Date
    Jan 2005
    Location
    Saint Paul, MN
    Posts
    668
    Quote Originally Posted by watael View Post
    hi,

    Again, command substitution shoud not be used as is to build an array: filename may have weird (at least space) characters that might break it.

    Also, don't ever try to parse `ls` output: it's versatile, and doesn't have neither fixed length, nor separator;

    instead use `stat` which output can be formatted to fit your needs, and shell pathname expansion.

    Code:
     while read -r s f; do echo "$f"; ts+=$s; done < <(stat -c '%s %n' ~/Desktop/1/*-*)
    two tricks are used here:
    1/ the filename is output last so $f contains the rest of the output
    2/ use of process substitution instead of a pipe so there's no subshell in which variables may be "trapped"
    Or chacters such as:
    Code:
    & $ ! | > < * ' " ? { } [ ]  \ ` 
    control characters
    See comments by "watael" at http://www.linuxforums.org/forum/pro...ll-script.html (last entry on page 1)

    I added a comment latter on using some of these bad file names with his solution (top of page 2).
    Last edited by alf55; 12-08-2013 at 03:40 AM.

  7. #6
    Linux Enthusiast
    Join Date
    Jan 2005
    Location
    Saint Paul, MN
    Posts
    668
    The bash manual array section at Bash Reference Manual

    ar=( $(find /var/www/backup -type f -name '*.tar' -print0 | while IFS= read -d '' -r f; do echo "$f" ; done) )
    Last edited by alf55; 12-08-2013 at 06:30 PM.

  8. #7
    Just Joined!
    Join Date
    Dec 2013
    Posts
    13
    Thank you very much, I read it, but I could not find info about
    keys in an array. After I saw you using it yesterday I found out
    about it. I can't find a tutorial that teaches it, I read about it
    in another forum.

    The keys are accessed using an exclamation point: ${!array[@]}, the values are accessed using ${array[@]}.

    You can iterate over the key/value pairs like this:

    for i in "${!array[@]}"
    do
    echo "key : $i"
    echo "value: ${array[$i]}"
    done

  9. #8
    Just Joined!
    Join Date
    Dec 2013
    Posts
    13
    Code:
     while read -r s f; do echo "$f"; ts+=$s; done < <(stat -c '%s %n' ~/Desktop/1/*-*)
    I am trying to understand this code since yesterday, and it is not working on my OS.

    I even tried:
    Code:
    while read -r s f
    do 
      echo "$f"
      ts+=$s
    done < <(stat -f '%N %z' ~/Desktop/1/*)
    But it is still not working, I don't understand the
    Code:
    "ts+=$s"
    part.
    Last edited by flit; 12-08-2013 at 06:47 PM.

  10. #9
    Linux Newbie
    Join Date
    Nov 2012
    Posts
    232
    what OS? why didn't you reuse given `stat`'s syntax?

    źnot working╗ doesn't give much hints.

    `+=` increments `ts` with `$s`'s value.

    I didn't mention the need to echo the result.

  11. #10
    Linux Enthusiast
    Join Date
    Jan 2005
    Location
    Saint Paul, MN
    Posts
    668
    Quote Originally Posted by flit View Post
    Code:
     while read -r s f; do echo "$f"; ts+=$s; done < <(stat -c '%s %n' ~/Desktop/1/*-*)
    I am trying to understand this code since yesterday, and it is not working on my OS.

    I even tried:
    Code:
    while read -r s f
    do 
      echo "$f"
      ts+=$s
    done < <(stat -f '%N %z' ~/Desktop/1/*)
    But it is still not working, I don't understand the
    Code:
    "ts+=$s"
    part.
    The array operator "+=" needs an array on the right hand side as well. Add the '(' and ')' as below
    Code:
    while read -r s f
    do 
      echo "$f"
      ts+=($s)
    done < <(stat -f '%N %z' ~/Desktop/1/*)

Page 1 of 2 1 2 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •