Find the answer to your Linux question:
Results 1 to 9 of 9
Hi All, This is my first post on linuxforums. I did some searching and couldn't find any related posts. I am attempting to insert the contents of a file to ...
Enjoy an ad free experience by logging in. Not a member yet? Register.
  1. #1
    Just Joined!
    Join Date
    Sep 2011
    Posts
    5

    BASH: Append file contents to end of line in another file.


    Hi All,

    This is my first post on linuxforums. I did some searching and couldn't find any related posts.

    I am attempting to insert the contents of a file to the end of a line in another file in a bash script. This is where I am at:

    sed -i 's#^\(.*'${SPR}'.*\)#\1 ,"'${new_content}'"#' file1.txt

    Where $SPR is the string to search for and $new_content is the content of the file to be inserted (e.g. new_content=$(cat file2.txt)).

    Problem is is since the file contains multiple lines sed is freaking out.

    I also tried variations of:

    sed -i 's#^\(.*'${SPR}'.*\)#r file2.txt#' file1.txt

    with no luck.

    any help would be appreciated.

  2. #2
    Just Joined!
    Join Date
    Sep 2011
    Posts
    5
    Really? Nobody has thoughts on this. Maybe I will expand on what I am doing. I have a a CVS liek this...

    column1, column2, column3,

    I want to take the contents of some multi-line text files and insert those as new columns...

    column1, column2, column3, contents of file1, contents of file2, contents of file3

    The files have have commas and regular text.

    Any thoughts would be appreciated.

  3. #3
    Trusted Penguin
    Join Date
    May 2011
    Posts
    4,353
    This sounds like a job for Perl to me, but I don't quite get what you are trying to do. Can you post examples of text files containing data and show what you want the end result to look like?

  4. $spacer_open
    $spacer_close
  5. #4
    Just Joined!
    Join Date
    Sep 2011
    Posts
    5
    File1.CSV looks something like...

    00005196,X44004,Jake,Verified
    00004519,X47404,Brian,Verified
    00009182,X47904,Jason,Verified
    There are several txt files that just have random text in them (like two paragraphs or so of text). It could look something like this...

    Something" is a song by The Beatles, written by lead guitarist George Harrison in 1969. It was featured on the album Abbey Road, and was also the first song written by Harrison to appear on the A-side of a Beatles' single. It was one of the first Beatles' singles to contain tracks already available on a long playing (LP) album, with both "Something" and "Come Together" having appeared on Abbey Road. "Something" was the only Harrison composition to top the American charts while he was in The Beatles.

    John Lennon and Paul McCartney, the band's principal songwriters, both praised "Something" as among the best songs Harrison had written, or the group had to offer. As well as critical acclaim, the single achieved commercial success, topping the Billboard charts in the United States, and entering the top 10 in the United Kingdom. The song has been covered by over 150 artists including Elvis Presley, Shirley Bassey, Frank Sinatra, Tony Bennett, James Brown, Julio Iglesias, The Miracles, Eric Clapton, and Joe Cocker, and is the second-most covered Beatles' song after "Yesterday".[3] Harrison is quoted as saying that his favourite cover of the song was that of James Brown's, keeping Brown's version in his personal jukebox.[4]
    So in the end I want file1.CSV to look like this...

    00005196,X44004,Jake,Verified, [content of X44004.txt]
    00004519,X47404,Brian,Verified, [content of X47404.txt]
    00009182,X47904,Jason,Verified, [content of X47904.txt]
    Make sense?

  6. #5
    Trusted Penguin
    Join Date
    May 2011
    Posts
    4,353
    Okay, making more sense...but what is the format of the string in [content of x44004.txt] - is it just a string of data (with no commas in it, I assume - as they are the field separator)?

    Will this string possibly be appearing multiple times in x44004.txt and if so, what is done then?

  7. #6
    Just Joined!
    Join Date
    Sep 2011
    Posts
    5
    Thanks for the reply. The content of each text file (e.g. x4404.txt) is just a bunch of rhetoric -- plain English text (including standard punctuation like commas). I have no problem matching up the appropriate file with the corresponding line in the CSV. My problem is stuffing the entire contents of a txt file at the end of the line of the CSV.

  8. #7
    Trusted Penguin
    Join Date
    May 2011
    Posts
    4,353
    My problem is stuffing the entire contents of a txt file at the end of the line of the CSV.
    So it is the entire contents of the txt file that goes at the end of the line? I thought it was a subset of it - based upon some matching string.

    Anyway, once you've saved the content you desire to a variable, you can the replace all newline characters with spaces. Then it's a simple matter of appending the modified string to the end of your CVS line.

    For example, to read the txt file in perl and put the contents all on one line:
    Code:
    my $file = shift || die "give me a file\n";
    my $contents;
    {
      local($/,*FH);
      open(FH,'<',$file) or die "can't read $file: $!\n";
      $contents = <FH>;
      close(FH);
    }
    die "Failed to get contents of $file\n" unless($contents);
    chomp($contents);
    $contents =~ s/\n/ /g;
    print "contents of $file: \n$contents\n";
    To now add this as a new column to the CSV file would be another quick perl operation. Does this help, or am I still not on the same page?

  9. #8
    Just Joined!
    Join Date
    Sep 2011
    Posts
    5
    Again, thanks for the reply -- really appreciate it. I think we are getting on the same page. I am not super familiar with Perl -- mainly write Bash scripts. But I am OK with using Perl if it works best. So it looks like your code writes the contents of a file to a variable. Next step is to take that contents (the entire contents of the txt file) and append it to the end of a line in the CSV file such that it adds it as a column to the file by using "," as a delimiter.

  10. #9
    Trusted Penguin
    Join Date
    May 2011
    Posts
    4,353
    Okay, here's a quick and dirty stab at what I think you're trying to do.

    First, some comments:

    The name of the master CSV file is hard-coded at the top of this script, make sure you change it to reflect your filename (or change the name of your CSV file to match it).

    I assume the files containing content you want added to the CSV file are listed in the 2nd column of the CSV example file you posted. If this is not right, just let me know, and we can modify this script.

    The code works by first reading the CSV file and grabbing the names of all files in there (by pulling out the values the 2nd column of every line).

    Then it iterates over each filename found and looks for a file in the current working directory of the same name, with a ".txt" appended to it (e.g. "X44004.txt"). If such a file is found, its contents are slurped in, new lines are removed, and it is added as a 5th column to the CSV file.

    This last part is actually a two-part process: a temporary file is actually written to, and then it replaces the original CSV file when it is done.

    If the text file is not found in the current working directory, then that file is skipped.

    If there is already text in the 5th column of a line entry in the CSV file, then it will be overwritten by any contents of the matching text file, if found.

    Here's the code:
    Code:
    #!/usr/bin/perl
    use strict;
    use warnings;
    use File::Basename;
    use File::Copy;
    
    $| = 1;
    
    # hard-code the path to the CSV file here
    my $csvFile = './text.csv';
    
    # make sure it exists
    die "$csvFile: No such file\n" unless(-f$csvFile);
    print "CSV File: $csvFile\n\n";
    
    my $tmpFile = '/tmp/'.(basename($csvFile)).'.tmp';
    
    # an array (list) of files listed in the 2nd column of the CSV file
    my @files;
    
    # read the CSV file to get the file names
    open(FH,'<',$csvFile) or die "can't read $csvFile: $!\n";
    
    # iterate over each line of the file
    while(<FH>){
      chomp;
    
      # ignore comments
      next if(/^#/);
    
      # split the line on commas
      my($foo,$fname,$name,$status,@rest) = split(/,/,$_);
    
      # save the value in the 2nd column as the text filename
      push(@files,$fname);
    }
    close(FH);
    
    print "Found ",$#files+1, " files in $csvFile\n";
    
    # iterate over the list of files and see if they exist
    for my $file(@files){
    
      # assume the file has a ".txt" file extension
      my $filename = $file.'.txt';
    
      print "\nGetting contents of $filename ... ";
      my $content = &slurp_file($filename);
      if($content){
        print "OK\n";
      }else{
        print "FAILED\n";
        next;
      }
    
      # open the temp file for writing
      open(TMP,'>',$tmpFile) or die "Can't write $tmpFile: $!\n";
    
      # open the CSV file for reading
      open(FH,'<',$csvFile) or die "can't read $csvFile: $!\n";
      while(<FH>){
        chomp;
        if(/^#/){
          print TMP $_,"\n";
        }else{
          # split the line on commas
          my($foo,$fname,$name,$status,@rest) = split(/,/,$_);
          if($fname eq $file){
            print TMP join(',',$foo,$fname,$name,$status,$content),"\n";
          }else{
            print TMP $_,"\n";
          }
        }
      }
    
      # close the CVS file
      close(FH);
    
      # close the temp file
      close(TMP);
    
      # replace the original CSV file with the temp file 
      move($tmpFile,$csvFile);
    }
    
    # subroutine to slurp in the additional text files
    sub slurp_file {
      my($file) = @_;
      unless(-f$file){
        warn $file,": No such file\n";
        return();
      }
    
      my $contents;
      {
        local($/,*FH);
        open(FH,'<',$file) or die "can't read $file: $!\n";
        $contents = <FH>;
        close(FH);
      }
      return() unless($contents);
      chomp($contents);
      $contents =~ s/\n/ /g;
      #print "Contents of $file: \n$contents\n";
      return($contents);
    }

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •