Find the answer to your Linux question:
Results 1 to 7 of 7
Well, the title says it all What I am trying to do is to convert a mass of files from standard Windows text encoding to UNIX encoding(462 .txt files). After ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Nov 2010
    Posts
    6

    Trying to convert mass of textfiles from "Windows" to Unix encoding.


    Well, the title says it all
    What I am trying to do is to convert a mass of files from standard Windows text encoding to UNIX encoding(462 .txt files). After a bit of searching, I found this bit of code, changed the necessary parts, put in "Sleep" so I could see the output and ran it:
    Code:
    #!/bin/bash
    FROM=iso-8859-1
    TO=UTF-8
    ICONV="iconv -f $FROM -t $TO"
    # Convert
    find /home/gmod/adv_duplicator/ -type f -print -name "*" | while read fn;
    do cp ${fn} ${fn}.bak
    $ICONV < ${fn}.bak > ${fn}
    rm ${fn}.bak
    done
    sleep 30
    The problem is that it failed
    This is the output that I found in console:
    Code:
    /home/gmod/Skrivebord/as.sh: line 8: ${fn}.bak: ambiguous redirect
    rm: cannot remove «/home/gmod/adv_duplicator/STEAM_0_1_32704654/bat»: No such file or directory
    rm: cannot remove «mobile»: No such file or directory
    rm: cannot remove «3000.txt.bak»: No such file or directory
    cp: target «complete.txt.bak» is not a catalog
    /home/gmod/Skrivebord/as.sh: line 8: ${fn}.bak: ambiguous redirect
    rm: cannot remove «/home/gmod/adv_duplicator/STEAM_0_1_11360145/space»: No such file or directory
    rm: cannot remove «ship»: No such file or directory
    rm: cannot remove «not»: No such file or directory
    rm: cannot remove «complete.txt.bak»: No such file or directory
    +more of the same
    "ambiguous redirect" is after what Ive found out an error because I am trying to use a command which only works on single files, but is "getting more in its input".
    Can anyone help me with this?

    Thanks in advance

  2. #2
    Just Joined! barriehie's Avatar
    Join Date
    Apr 2008
    Location
    The Desert!
    Posts
    85
    Quote Originally Posted by Anderen2 View Post
    Well, the title says it all
    What I am trying to do is to convert a mass of files from standard Windows text encoding to UNIX encoding(462 .txt files). After a bit of searching, I found this bit of code, changed the necessary parts, put in "Sleep" so I could see the output and ran it:
    Code:
    #!/bin/bash
    FROM=iso-8859-1
    TO=UTF-8
    ICONV="iconv -f $FROM -t $TO"
    # Convert
    find /home/gmod/adv_duplicator/ -type f -print -name "*" | while read fn;
    do cp ${fn} ${fn}.bak
    $ICONV < ${fn}.bak > ${fn}
    rm ${fn}.bak
    done
    sleep 30
    ...snip...
    Try this:
    Code:
    #!/bin/bash
    FROMM=iso-8859-1
    TOO=UTF-8
    # Convert
    find /home/gmod/adv_duplicator/ -type f -print -name "*" | while read fn;
    do cp ${fn} ${fn}.bak
    iconv -f $FROMM -t $TOO $fn.bak > $fn
    # rm ${fn}.bak  - I'ld do this after verifing everything is okay!
    done
    sleep 30
    And if all that fails I know gawk can do it:
    Code:
    gawk '{ sub("\r$", ""); print }' winfile.txt > unixfile.txt

  3. #3
    Just Joined!
    Join Date
    Nov 2010
    Posts
    6
    Hi! Sorry for late response, been busy .
    Okay tried both your methods, but unfortunatly none worked
    #1:
    Code:
    /home/gmod/Skrivebord/sa.sh: line 7: $fn: ambiguous redirect
    cp: target «machine.txt.bak» is not a catalog
    /home/gmod/Skrivebord/sa.sh: line 7: $fn: ambiguous redirect
    cp: target «2.txt.bak» is not a catalog
    /home/gmod/Skrivebord/sa.sh: line 7: $fn: ambiguous redirect
    cp: target «wagon.txt.bak»is not a catalog
    /home/gmod/Skrivebord/sa.sh: line 7: $fn: ambiguous redirect
    cp: target «home.txt.bak» is not a catalog
    /home/gmod/Skrivebord/sa.sh: line 7: $fn: ambiguous redirect
    cp: target «bugs.txt.bak» is not a catalog
    /home/gmod/Skrivebord/sa.sh: line 7: $fn: ambiguous redirect
    cp: target «kart.txt.bak» is not a catalog
    /home/gmod/Skrivebord/sa.sh: line 7: $fn: ambiguous redirect
    cp: target <t.txt.bak» is not a catalog
    /home/gmod/Skrivebord/sa.sh: line 7: $fn: ambiguous redirect
    cp: target «3000.txt.bak» is not a catalog
    /home/gmod/Skrivebord/sa.sh: line 7: $fn: ambiguous redirect
    cp: target «complete.txt.bak»is not a catalog
    /home/gmod/Skrivebord/sa.sh: line 7: $fn: ambiguous redirect
    +++
    #2:
    Code:
    echo /home/anderen2/Skrivebord/convert
    cd /home/anderen2/Skrivebord/convert
    sudo gawk '{ sub("\r$", ""); print }' * > ./unixfile
    sleep 30
    returns:
    Code:
    /home/anderen2/Skrivebord/convert
    [sudo] password for anderen2: 
    gawk: cmd. line:1: fatal: cannot open file `=Public Folder=' for reading (Inappropriate ioctl for device)
    Btw, here is the folder which I am trying to convert:
    Convert folder:
    Code:
    anderen2@ubuntu:~/Skrivebord/convert$ dir
    =Public\ Folder=    STEAM_0_0_23750699    STEAM_0_1_16187122  STEAM_0_1_31251130
    STEAM_0_0_13458808  STEAM_0_0_24605801    STEAM_0_1_17492839  STEAM_0_1_31382123
    STEAM_0_0_15217801  STEAM_0_0_25415433    STEAM_0_1_17897803  STEAM_0_1_32141218
    STEAM_0_0_15973405  STEAM_0_0_27347352    STEAM_0_1_18845917  STEAM_0_1_32159843
    STEAM_0_0_16324415  STEAM_0_0_28364358    STEAM_0_1_20745652  STEAM_0_1_32592959
    STEAM_0_0_18472930  STEAM_0_0_28377716    STEAM_0_1_21982955  STEAM_0_1_32704654
    STEAM_0_0_18729357  STEAM_0_0_28531472    STEAM_0_1_25910477  STEAM_0_1_33125795
    STEAM_0_0_18881452  STEAM_0_0_31886959    STEAM_0_1_26317246  STEAM_0_1_34491390
    STEAM_0_0_19144830  STEAM_0_1_11360145    STEAM_0_1_27243895
    STEAM_0_0_19407299  STEAM_0_1_14097933    STEAM_0_1_27879973
    STEAM_0_0_19907748  STEAM_0_1_14595300    STEAM_0_1_28402009
    STEAM_0_0_21746922  STEAM_0_1_14604192    STEAM_0_1_30904764
    And inside each "Steam..." folder is a bunch of .txt in the wrong encoding, and some has even more folders in it.

    Thanks

  4. $spacer_open
    $spacer_close
  5. #4
    Just Joined! barriehie's Avatar
    Join Date
    Apr 2008
    Location
    The Desert!
    Posts
    85
    Okay, try this, modified a bit. If you know how many dir levels deep they go you can add -maxdepth to the find options to get all of them.

    Code:
    #!/bin/bash
    FROMM=iso-8859-1
    TOO=UTF-8
    # Convert
    find /home/gmod/adv_duplicator/ -type f -print -name "*" | while read fn;
    do
    filename=$fn
    cp $filename $filename.bak
    iconv -f $FROMM -t $TOO $filename.bak > $filename
    # rm $filename.bak  - I'ld do this after verifing everything is okay!
    done
    sleep 5
    Maybe zip a few of the smaller ones and attach. I don't have any windows type anything on here.

  6. #5
    Just Joined!
    Join Date
    Nov 2010
    Posts
    6
    Tried it, but added --verbose to the cp and iconv command to see what they returned:
    Code:
    #!/bin/bash
    FROMM=iso-8859-1
    TOO=UTF-8
    # Convert
    find /home/anderen2/Skrivebord/convert -type f -print -name "*" | while read fn;
    do
    filename=$fn
    cp --verbose $filename $filename.bak
    iconv --verbose -f $FROMM -t $TOO $filename.bak > $filename
    # rm $filename.bak  - I'ld do this after verifing everything is okay!
    done
    sleep 500
    Returns this:
    Code:
    anderen2/Skrivebord/convert/STEAM_0_0_28531472/glados.txt.bak.bak»
    /home/anderen2/Skrivebord/convert/STEAM_0_0_28531472/glados.txt.bak.bak:
    «/home/anderen2/Skrivebord/convert/STEAM_0_0_28531472/glados.txt» -> «/home/anderen2/Skrivebord/convert/STEAM_0_0_28531472/glados.txt.bak»
    /home/anderen2/Skrivebord/convert/STEAM_0_0_28531472/glados.txt.bak:
    cp: målet «wagon.txt.bak» er ikke en katalog
    /home/anderen2/Skrivebord/sa.sh: line 9: $filename: ambiguous redirect
    «/home/anderen2/Skrivebord/convert/STEAM_0_0_28531472/rolerkaster.txt» -> «/home/anderen2/Skrivebord/convert/STEAM_0_0_28531472/rolerkaster.txt.bak»
    /home/anderen2/Skrivebord/convert/STEAM_0_0_28531472/rolerkaster.txt.bak:
    «/home/anderen2/Skrivebord/convert/STEAM_0_0_16324415/faef.txt.bak» -> «/home/anderen2/Skrivebord/convert/STEAM_0_0_16324415/faef.txt.bak.bak»
    /home/anderen2/Skrivebord/convert/STEAM_0_0_16324415/faef.txt.bak.bak:
    «/home/anderen2/Skrivebord/convert/STEAM_0_0_16324415/faef.txt» -> «/home/anderen2/Skrivebord/convert/STEAM_0_0_16324415/faef.txt.bak»
    /home/anderen2/Skrivebord/convert/STEAM_0_0_16324415/faef.txt.bak:
    cp: målet «home.txt.bak» er ikke en katalog
    /home/anderen2/Skrivebord/sa.sh: line 9: $filename: ambiguous redirect
    cp: målet «bugs.txt.bak» er ikke en katalog
    /home/anderen2/Skrivebord/sa.sh: line 9: $filename: ambiguous redirect
    «/home/anderen2/Skrivebord/convert/STEAM_0_0_19144830/helicopter.txt.bak» -> «/home/anderen2/Skrivebord/convert/STEAM_0_0_19144830/helicopter.txt.bak.bak»
    /home/anderen2/Skrivebord/convert/STEAM_0_0_19144830/helicopter.txt.bak.bak:
    «/home/anderen2/Skrivebord/convert/STEAM_0_0_19144830/helicopter.txt» -> «/home/anderen2/Skrivebord/convert/STEAM_0_0_19144830/helicopter.txt.bak»
    /home/anderen2/Skrivebord/convert/STEAM_0_0_19144830/helicopter.txt.bak:
    cp: målet «kart.txt.bak» er ikke en katalog
    /home/anderen2/Skrivebord/sa.sh: line 9: $filename: ambiguous redirect
    cp: målet «jet.txt.bak» er ikke en katalog
    /home/anderen2/Skrivebord/sa.sh: line 9: $filename: ambiguous redirect
    cp: målet «3000.txt.bak» er ikke en katalog
    /home/anderen2/Skrivebord/sa.sh: line 9: $filename: ambiguous redirect
    cp: målet «complete.txt.bak» er ikke en katalog
    /home/anderen2/Skrivebord/sa.sh: line 9: $filename: ambiguous redirect
    «/home/anderen2/Skrivebord/convert/STEAM_0_1_25910477/asus.txt.bak» -> «/home/anderen2/Skrivebord/convert/STEAM_0_1_25910477/asus.txt.bak.bak»
    /home/anderen2/Skrivebord/convert/STEAM_0_1_25910477/asus.txt.bak.bak:
    «/home/anderen2/Skrivebord/convert/STEAM_0_1_25910477/asus.txt» -> «/home/anderen2/Skrivebord/convert/STEAM_0_1_25910477/asus.txt.bak»
    /home/anderen2/Skrivebord/convert/STEAM_0_1_25910477/asus.txt.bak:
    +++
    Heres the folder which I'm trying to convert(.zip):
    2shared.com/file/Y3mPiR5X/convertq.html (Had to remove "h t t p : / /w w w" because I didn't have enough posts...)
    Its an gameserver(GMod) save folder with 588 .txt files with various sizes and folder depth

    Thanks ; )

  7. #6
    Just Joined! barriehie's Avatar
    Join Date
    Apr 2008
    Location
    The Desert!
    Posts
    85
    Gottit, I'll see what I can do.

  8. #7
    Just Joined! barriehie's Avatar
    Join Date
    Apr 2008
    Location
    The Desert!
    Posts
    85
    Umm, we were overthinking this a bit... (run from the top of the dir tree)
    Code:
    #!/bin/bash
    #
    
    find . -type f -iname "*.txt" | while read file
    do
      sed -e 's/\r$//' "$file" > "$file".fixed
    done
    Code:
    05:06:35 /home/barrie/convert/STEAM_0_1_34491390 $ > wc --lines ./*
        63 ./base1.txt
        63 ./base1.txt.fixed
        65 ./car.txt
        65 ./car.txt.fixed
        66 ./lolz.txt
        66 ./lolz.txt.fixed
        98 ./motel.txt
        98 ./motel.txt.fixed
        53 ./pool.txt
        53 ./pool.txt.fixed
        56 ./prototype2.txt
        56 ./prototype2.txt.fixed
        84 ./prototype.txt
        84 ./prototype.txt.fixed
        39 ./rocket tower.txt
        39 ./rocket tower.txt.fixed
      1048 total
    05:06:40 /home/barrie/convert/STEAM_0_1_34491390 $ >
    I checked in ghex2 and the line endings are now 0A.

    HTH

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •