Welcome to Linux Forums!

With a comprehensive Linux Forum, information on various types of Linux software and many Linux Reviews articles, we have all the knowledge you need a click away, or accessible via our knowledgeable members.

Linux Forum ArticlesLinux ForumsLinux Forum DownloadsLinux Hosts
Home|Register|FAQ|Member List|Calendar|Unanswered Posts|Forum Rules|Today's Posts|Advanced Search|
SEARCH FOR IN
Go Back   Linux Forums > GNU Linux Zone > Linux Programming & Scripting
Reload this Page Help. noob needs help with file copy!
Linux Forums
Linux Forums
Welcome To The Linux Forums!
Welcome to Linux Forums. We pride ourselves in being one of the largest Linux communities on the web, we encourage you to REGISTER on our forums and participate in the community. There are over 150,000 members ready to answer your questions. JOINING US today will allow you to make new posts, get support, send messages to other members and submit downloads to our downloads directory and many other great features!

Linux Programming & Scripting C, Perl, PHP, Bash Scripts, anything programming or script related post in here!

Reply
 
Thread Tools Display Modes
Old 07-18-2008   #31 (permalink)
wje_lf
Linux Engineer
 
wje_lf's Avatar
 
Join Date: Sep 2007
Location: Mariposa
Posts: 986
Quote:
Can I just re-run 's again ont he same stream and change the whole string to say FolderB and add that to the end?
Possibly, but it's probably more trouble than it's worth.

And I goofed, big time, on that sed line. In my haste to insert an original string twice instead of once, I completely forgot about stripping off the time and its terminating character "|". The sed line should read (I hope):
Code:
sort astats.txt bstats.txt | uniq -d | sed -e 's/^.*|\(.*\)$/cp \/FolderA\/\1 \/FolderB\/\1/' > copysametime.sh
__________________
--
Bill

Old age and treachery will overcome youth and skill.
wje_lf is offline   Reply With Quote
Old 07-19-2008   #32 (permalink)
ghostdog74
Linux User
 
Join Date: Aug 2006
Posts: 334
@OP,why don't you check for md5 hash instead of checking for equal time? It guarantees 2 identical files are not changed (equal).
ghostdog74 is offline   Reply With Quote
Old 07-21-2008   #33 (permalink)
wildside
Just Joined!
 
Join Date: Jul 2008
Posts: 20
Quote:
Originally Posted by wje_lf View Post
Possibly, but it's probably more trouble than it's worth.

And I goofed, big time, on that sed line. In my haste to insert an original string twice instead of once, I completely forgot about stripping off the time and its terminating character "|". The sed line should read (I hope):
Code:
sort astats.txt bstats.txt | uniq -d | sed -e 's/^.*|\(.*\)$/cp \/FolderA\/\1 \/FolderB\/\1/' > copysametime.sh
I put that in and i get nothing for output to copysametime.sh

i have this halfway working:
Code:
sort astats.txt bstats.txt | uniq -d | sed -e 's:[0-9]*|".:cp -p "/FolderA:'
But it only returns the first part of my 'cp' command. I just need a little help adding the end part to the coammand so that it will copy to FolderB
wildside is offline   Reply With Quote
Old 07-21-2008   #34 (permalink)
wje_lf
Linux Engineer
 
wje_lf's Avatar
 
Join Date: Sep 2007
Location: Mariposa
Posts: 986
Quote:
I just need a little help adding the end part to the coammand so that it will copy to FolderB
The "little help" involves those escaped parentheses in the solution I've been trying to beat you over the head with. :) The reason they're necessary is because you'll be using the same input data at two places on the output line. And that's a major part of where my solution differs from your almost solution. Please bear with me a moment as we recall this:
Quote:
I put that in and i get nothing for output to copysametime.sh
That's very, very strange. Here's a test script with sample data. What do you get when you run it?
Code:
#!/bin/bash

cat > cstats.txt <<EOD
1234567890|./a/b/c
1234567891|./a/b/d
1234567897|./a/same_time_different_name
1234567899|./a/same_name_different_time
EOD
cat > dstats.txt <<EOD
1234567891|./a/b/d
1234567892|./a/b/e
1234567897|./a/same_time_diff_name
1234567898|./a/same_name_different_time
EOD
sort cstats.txt dstats.txt | uniq -d | sed -e 's/^.*|\(.*\)$/cp \/FolderA\/\1 \/FolderB\/\1/' > testsametime.sh
echo output:
cat testsametime.sh
echo end of output
I get:
Code:
output:
cp /FolderA/./a/b/d /FolderB/./a/b/d
end of output
__________________
--
Bill

Old age and treachery will overcome youth and skill.
wje_lf is offline   Reply With Quote
Old 07-21-2008   #35 (permalink)
wildside
Just Joined!
 
Join Date: Jul 2008
Posts: 20
OK. nevermind. im stupid. in my testing. I copied everything form FolderA to FolderB without using -p and none of the files were being returned becuse they were in fact newer.

So im at the point now where I need to either put spaces on the FolderA substitute or somehow pass varibles from the command into the copy.

Code:
ie:  ./copyfiles FolderA FolderB
would end up being something like :
Code:
./copyfiles Volumes/Backup/Backup SAN/CreativeSAN/CURRENT/ABA/ /Volumes/SAN/CURRENT/ABA
I tried this:
Code:
sort astats.txt bstats.txt | uniq -d | sed -e 's/^.*|\(.*\)$/cp -pv /"Volumes/Backup/Backup SAN/CreativeSAN/CURRENT/ABA/"\1 /"Volumes/SAN/CURRENT/ABA/"\1/' > copysametime.sh
But obviously I got errors.
wildside is offline   Reply With Quote
Old 07-21-2008   #36 (permalink)
wje_lf
Linux Engineer
 
wje_lf's Avatar
 
Join Date: Sep 2007
Location: Mariposa
Posts: 986
Quote:
OK. nevermind. im stupid. in my testing. I copied everything form FolderA to FolderB without using -p and none of the files were being returned becuse they were in fact newer.
Now I'm really confused. That's not surprising because of the foggy nature of my brain. (My wife would confirm this, except she's too polite.)

But let me guess:
  1. The live data is not really in "FolderA" and "FolderB"; it's in other places, places which are specified using slashes and spaces. In particular, it's in "Volumes/Backup/Backup SAN/CreativeSAN/CURRENT/ABA/" and "Volumes/SAN/CURRENT/ABA".
  2. You do have test data in FolderA and FolderB.
  3. You've gotten my suggested scripts to work with the data in FolderA and FolderB, because the strings "FolderA" and "FolderB" do not contain spaces.
  4. The actual folder path names do not contain double quotation marks(").
Am I correct on all counts, especially item 3? Because if I am, I now see why you didn't want slashes as the delimiters in your sed command.

Try this. I've gone with a plus sign for the sed string delimiter.
Code:
sort astats.txt bstats.txt | uniq -d | sed -e 's/^.*|\(.*\)$/cp -pv +Volumes/Backup/Backup SAN/CreativeSAN/CURRENT/ABA+\1 +Volumes/SAN/CURRENT/ABA+\1/' > copysametime.sh
Of course, double check the options on that cp command. It's clear that the options on various commands in your environment are different from mine.
__________________
--
Bill

Old age and treachery will overcome youth and skill.
wje_lf is offline   Reply With Quote
Old 07-21-2008   #37 (permalink)
wildside
Just Joined!
 
Join Date: Jul 2008
Posts: 20
First off: Thank you for all your help.

Second: I get this:
Code:
sed: 1: "s/^.*|\(.*\)$/cp -pv +V ...": bad flag in substitute command: 'B'
File created. Check copysametime.sh
I am using this as my script:
Code:
#!/bin/bash
(cd /Volumes/Backup/Backup\ SAN/CreativeSAN/CURRENT/ABA
	find . -type f -print0 | xargs -0 stat -f "%m|\"%N\"") > astats.txt
(cd /Volumes/SAN/CURRENT/ABA
	find . -type f -print0 | xargs -0 stat -f "%m|\"%N\"") > bstats.txt
sort astats.txt bstats.txt | uniq -d | sed -e 's/^.*|\(.*\)$/cp -pv +Volumes/Backup/Backup SAN/CreativeSAN/CURRENT/ABA+\1 +Volumes/SAN/CURRENT/ABA+\1/' > copysametime.sh
chmod +x copysametime.sh
echo File created. Check copysametime.sh
I am assuming I need to turn off quoting with single quotes then use double then turn back on with single?

like so:
Code:
's:[0-9]*|".:cp -pr "'"$1"':'
being that: $1 is something like: /Volumes/Backup/Backup\ SAN/CreativeSAN/CURRENT/ABA

Am I on the right track?

Oh and yes you are correct on all questions.
wildside is offline   Reply With Quote
Old 07-21-2008   #38 (permalink)
wje_lf
Linux Engineer
 
wje_lf's Avatar
 
Join Date: Sep 2007
Location: Mariposa
Posts: 986
This is embarrassing. I have no idea what I've been smoking. (I don't smoke, but you get the idea.)

There was a running gag in the Rocky and Bullwinkle show where Bullwinkle would try the classic magic trick of pulling a rabbit out of his hat. My favorite three-liner?

"Hey, Rocky, watch me pull a rabbit out of my hat."

"That trick never works!"

"This time for sure."

Well, this time it'll work for sure. Let me build it up piece by piece, so you know what I'm doing.

First, we have this:
Code:
sort astats.txt bstats.txt
That combines the lines from both astats.txt and bstats.txt, and outputs them in sorted order. We don't care about the order, except that we want any duplicate lines to appear next to each other. We'll get duplicate lines if we have a line which exists in both input files.

The next step:
Code:
uniq -d
This command outputs nothing at all for each line which only appears once. It outputs exactly one copy of each line which appears at least twice (but the copies must be next to each other, which explains why we did the sort). Since we already knew that the only way we can get duplicate lines was if a line appeared in both input files, we know that these are the lines we are interested in.

Ok, on to sed.
Code:
sed -e 's/fred/barney/'
That would substitute "barney" for the first occurrence of "fred" in each line. If, instead, we said
Code:
sed -e 's/fred/barney/g'
that would mean substitute "barney" for every (not just the first) occurrence of "fred" in each line. But we won't need that "g" option, so we'll leave it out.
Code:
sed -e 's+fred+barney+'
substitutes barney for fred again, but we'll use plus signs for the sed command delimiter, because we'll have slashes in the substitution string.
Code:
sed -e 's+^fred$+barney+'
The ^ means beginning of line, and $ means end of line. So this substitutes barney for fred, but only on those lines which contain exactly fred and nothing else.
Code:
sed -e 's+^.*$+barney+'
In this example, the period (".") means, roughly, any possible character you can imagine. The asterisk ("*") means "as many of the preceding thing that you can find, and zero is an acceptable quantity of that preceding thing". So .*$ means "as many of any character as you can find, up to the end of the line". This command will replace every single line with barney. If the file has 534 text lines containing anything and everything and nothing on its lines, you will end up with 534 lines of barney.

I've put the .*$ in green above for a particular purpose, which I'll revisit later.

Anyway, that sounds useless, but now look at this:
Code:
sed -e 's+^.*|+barney+'
All we did was to add the pipe symbol after the asterisk, and remove the dollar sign, which meant end of line. So we're instructing sed that wherever it finds anything at the beginning of a line, for any number of characters, followed by a pipe symbol, replace all of that with barney. Anything after the pipe symbol remains in the output as it was in the input. If it doesn't find the pipe symbol at all, the line will be output exactly as it was input.

(If there's more than one pipe symbol in a line, sed will be "greedy" and absorb through the final one, and everything up to that will be replaced by barney. But I'm assuming you don't have pipe symbols in your file names, so that won't concern us here.)

What we want to do, of course, is simply to remove everything up to and including the pipe symbol, so we just remove barney from the command:
Code:
sed -e 's+^.*|++'
So if we have a line that looks like this:
Code:
123|abc
we'll get this output:
Code:
abc
We want to grab that data and show it twice on the line, though. That's where the escaped parentheses come in, \( and \). If those appear in the regular expression just once, they mean that anything between them is to be regarded as subexpression 1. If twice, then you have subexpressions 1 and 2. They can be referred to in the replacement part by \1 and \2, respectively. So this command:
Code:
sed -e 's+^.*|\(.*\)$+\1\1+'
says to get rid of the stuff before and including the pipe symbol. The symbols in green (including the dollar sign) mean everything else, up to the end of the line, as we had in a previous example (look for the green there). The characters found in the input line which "fit" between the escaped parentheses are considered to be subexpression 1. So the above sed command, run against this data:
Code:
123|fred barney wilma betty
will give the output
Code:
fred barney wilma bettyfred barney wilma betty
If we have this command:
Code:
sed -e 's+^.*|\(.*\)$+rrr\1eee\1ddd+'
and give it the same input, we'll get:
Code:
rrrfred barney wilma bettyeeefred barney wilma bettyddd
Now, this is where I messed up the last time, in several ways. One of them was misplacing the plus signs; the other was forgetting to place double quotation marks to protect the occurrence of spaces, not just in the first (constant) part of a file's path name, but also spaces in the rest of the file name, which might vary from one line to the next. We want something like this:
Code:
cp -pv "source location" "destination location"
Fortunately, double quotation marks have no special meaning to sed in this situation, so we'll just put them where we want them in the replacement string:
Code:
's+^.*|\(.*\)$+cp -pv "Volumes/Backup/Backup SAN/CreateSAN/CURRENT/ABA\1" "Volumes/SAN/CURRENT/ABA\1"+' > copysametime.sh
Notice the crucial lack of spaces before the \1 in each occurrence. That's one thing I weeded out. If the spaces are in there, you'll have file or directory names beginning with spaces. This is literal stuff we're working with here.

Do you want a slash before each occurrence of "Volumes"? If so, put it there, in both places if you want.

I hope I have it right this time. Are those rabbit ears I see twitching in that hat?
__________________
--
Bill

Old age and treachery will overcome youth and skill.
wje_lf is offline   Reply With Quote
Old 07-21-2008   #39 (permalink)
wildside
Just Joined!
 
Join Date: Jul 2008
Posts: 20
Alright. thank you for that sed tutorial. it really helped me out in understanding the reapeating part. I think thats where all the other documentation fell short.

So after all your help I think I got it.

Its not pretty but it looks like it works.
Code:
#!/bin/bash

(cd "$1"
	find . -type f -print0 | xargs -0 stat -f "%m|\"%N\"") > astats.txt
(cd "$2" 
	find . -type f -print0 | xargs -0 stat -f "%m|\"%N\"") > bstats.txt

sort astats.txt bstats.txt | uniq -d | sed -e 's/^.*|\(.*\)$/cp -pv "\/'"$1"'"\1 "\/'"$2"'"\1/' | sed 's:""./:/:g' > copysametime.sh

chmod +x copysametime.sh

echo File created. Check copysametime.sh
I couldnt figure out how to run sed again so I just piped it again. lol

this is the output I get in the end shell script:
Code:
cp -pv "/Folder A/AnimalToysIcons/Fasticon.com.url" "/Folder B/AnimalToysIcons/Fasticon.com.url"
cp -pv "/Folder A/CODES.txt" "/Folder B/CODES.txt"
cp -pv "/Folder A/new folder/CODES.txt" "/Folder B/new folder/CODES.txt"
cp -pv "/Folder A/new folder/org folder/CODES.txt" "/Folder B/new folder/org folder/CODES.txt"
cp -pv "/Folder A/AnimalToysIcons/License - Read Me.url" "/Folder B/AnimalToysIcons/License - Read Me.url"
cp -pv "/Folder A/AnimalToysIcons/Icons/.DS_Store" "/Folder B/AnimalToysIcons/Icons/.DS_Store"
cp -pv "/Folder A/AnimalToysIcons/Icons/Zebra" "/Folder B/AnimalToysIcons/Icons/Zebra"
cp -pv "/Folder A/AnimalToysIcons/Icons/Gorilla" "/Folder B/AnimalToysIcons/Icons/Gorilla"
cp -pv "/Folder A/AnimalToysIcons/Icons/Lion" "/Folder B/AnimalToysIcons/Icons/Lion"
cp -pv "/Folder A/AnimalToysIcons/Icons/Elephant" "/Folder B/AnimalToysIcons/Icons/Elephant"
cp -pv "/Folder A/AnimalToysIcons/Icons/Giraffe" "/Folder B/AnimalToysIcons/Icons/Giraffe"
cp -pv "/Folder A/C_PROJECTS/conversion.cpp" "/Folder B/C_PROJECTS/conversion.cpp"
cp -pv "/Folder A/.DS_Store" "/Folder B/.DS_Store"
cp -pv "/Folder A/astats.txt" "/Folder B/astats.txt"
cp -pv "/Folder A/copysametime.sh" "/Folder B/copysametime.sh"
So looks good.

Any ideas on refining this?
wildside is offline   Reply With Quote
Old 07-21-2008   #40 (permalink)
wje_lf
Linux Engineer
 
wje_lf's Avatar
 
Join Date: Sep 2007
Location: Mariposa
Posts: 986
Quote:
Any ideas on refining this?
Nah, that's your job. :)

But I will say this: You piped the data through sed twice. Knowing what you knew, that's exactly what you should have done. It shows that you're picking up this stuff pretty well.

Know, though, that the main purpose of the -e option in sed is to allow you to process each line twice. So, for example, the command:
Code:
sed -e 's/a/b/' | sed -e 's/c/d/'
can also be written as
Code:
sed -e 's/a/b/' -e 's/c/d/'
__________________
--
Bill

Old age and treachery will overcome youth and skill.
wje_lf is offline   Reply With Quote
Reply


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are Off
Pingbacks are Off
Refbacks are Off




All times are GMT. The time now is 10:56 AM.




© 2000 - 2008 - All Rights Reserved - Property of  MAS Media

Content Relevant URLs by vBSEO 3.0.0