Faster way to concatenate files of same name in different directories

up vote
0
down vote

favorite

I'm trying to do exactly as in this post and used the accepted answer. My question is, is there a faster alternative? This has literally run for several hours and is still creating the new empty files. I have hundreds of files to concatenate. Thanks!

https://unix.stackexchange.com/questions/435794/create-new-concatenated-files-of-same-name-in-multiple-directories

Here's the code:

find . -type f -path "*/*/*/*/$name" 
-exec cat + >"New_Dir/$name" 
done < list_to_concat.txt

I've read on similar posts that a loop might be quicker, but not able to get that to work properly. I'm testing with echo here:

for i in */*/*/*/*.txt; 
do echo find -name $i -exec bash -c cat $i >>
"New_Dir/$i"; done

edited Oct 17 at 19:00

asked Oct 17 at 13:15

NKN

305

The down-vote to this questions would be helpful if explained why...
– NKN
Nov 11 at 21:54

I have up-voted it for neutralizing it.
– RavinderSingh13
Nov 21 at 2:59

add a comment |

up vote
0
down vote

favorite

https://unix.stackexchange.com/questions/435794/create-new-concatenated-files-of-same-name-in-multiple-directories

Here's the code:

find . -type f -path "*/*/*/*/$name" 
-exec cat + >"New_Dir/$name" 
done < list_to_concat.txt

I've read on similar posts that a loop might be quicker, but not able to get that to work properly. I'm testing with echo here:

for i in */*/*/*/*.txt; 
do echo find -name $i -exec bash -c cat $i >>
"New_Dir/$i"; done

edited Oct 17 at 19:00

asked Oct 17 at 13:15

NKN

305

The down-vote to this questions would be helpful if explained why...
– NKN
Nov 11 at 21:54

I have up-voted it for neutralizing it.
– RavinderSingh13
Nov 21 at 2:59

add a comment |

up vote
0
down vote

favorite

https://unix.stackexchange.com/questions/435794/create-new-concatenated-files-of-same-name-in-multiple-directories

Here's the code:

find . -type f -path "*/*/*/*/$name" 
-exec cat + >"New_Dir/$name" 
done < list_to_concat.txt

I've read on similar posts that a loop might be quicker, but not able to get that to work properly. I'm testing with echo here:

for i in */*/*/*/*.txt; 
do echo find -name $i -exec bash -c cat $i >>
"New_Dir/$i"; done

edited Oct 17 at 19:00

asked Oct 17 at 13:15

NKN

305

https://unix.stackexchange.com/questions/435794/create-new-concatenated-files-of-same-name-in-multiple-directories

Here's the code:

find . -type f -path "*/*/*/*/$name" 
-exec cat + >"New_Dir/$name" 
done < list_to_concat.txt

I've read on similar posts that a loop might be quicker, but not able to get that to work properly. I'm testing with echo here:

for i in */*/*/*/*.txt; 
do echo find -name $i -exec bash -c cat $i >>
"New_Dir/$i"; done

linux performance find concatenation

edited Oct 17 at 19:00

asked Oct 17 at 13:15

NKN

305

edited Oct 17 at 19:00

asked Oct 17 at 13:15

NKN

305

edited Oct 17 at 19:00

asked Oct 17 at 13:15

NKN

305

asked Oct 17 at 13:15

NKN

305

asked Oct 17 at 13:15

NKN

305

The down-vote to this questions would be helpful if explained why...
– NKN
Nov 11 at 21:54

I have up-voted it for neutralizing it.
– RavinderSingh13
Nov 21 at 2:59

add a comment |

The down-vote to this questions would be helpful if explained why...
– NKN
Nov 11 at 21:54

I have up-voted it for neutralizing it.
– RavinderSingh13
Nov 21 at 2:59

The down-vote to this questions would be helpful if explained why...
– NKN
Nov 11 at 21:54

I have up-voted it for neutralizing it.
– RavinderSingh13
Nov 21 at 2:59

add a comment |

1 Answer
1

active

oldest

votes

up vote
0
down vote

accepted

In case this is useful to anyone else searching one day, I learned a much faster way to do this using xargs and ls:

while read -r name
do
ls */*/*/*/$name.txt | xargs cat > $name.combine
done <List_to_combine.txt

answered Nov 11 at 21:53

NKN

305

add a comment |

Your Answer

StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");

StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);

StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);

else
createEditor();

);

function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);

);

draft saved

draft discarded

StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f52855811%2ffaster-way-to-concatenate-files-of-same-name-in-different-directories%23new-answer', 'question_page');

);

Post as a guest

Name

Required, but never shown

1 Answer
1

active

oldest

votes

1 Answer
1

active

oldest

votes

up vote
0
down vote

accepted

In case this is useful to anyone else searching one day, I learned a much faster way to do this using xargs and ls:

while read -r name
do
ls */*/*/*/$name.txt | xargs cat > $name.combine
done <List_to_combine.txt

answered Nov 11 at 21:53

NKN

305

add a comment |

up vote
0
down vote

accepted

In case this is useful to anyone else searching one day, I learned a much faster way to do this using xargs and ls:

while read -r name
do
ls */*/*/*/$name.txt | xargs cat > $name.combine
done <List_to_combine.txt

answered Nov 11 at 21:53

NKN

305

add a comment |

up vote
0
down vote

accepted

In case this is useful to anyone else searching one day, I learned a much faster way to do this using xargs and ls:

while read -r name
do
ls */*/*/*/$name.txt | xargs cat > $name.combine
done <List_to_combine.txt

answered Nov 11 at 21:53

NKN

305

In case this is useful to anyone else searching one day, I learned a much faster way to do this using xargs and ls:

while read -r name
do
ls */*/*/*/$name.txt | xargs cat > $name.combine
done <List_to_combine.txt

answered Nov 11 at 21:53

NKN

305

answered Nov 11 at 21:53

NKN

305

answered Nov 11 at 21:53

NKN

305

answered Nov 11 at 21:53

NKN

305

add a comment |

draft saved

draft discarded

Thanks for contributing an answer to Stack Overflow!

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

Some of your past answers have not been well-received, and you're in danger of being blocked from answering.

Please pay close attention to the following guidance:

Please be sure to answer the question. Provide details and share your research!

But avoid …

Asking for help, clarification, or responding to other answers.

Making statements based on opinion; back them up with references or personal experience.

To learn more, see our tips on writing great answers.

draft saved

draft discarded

Post as a guest

Name

Required, but never shown

Name

Required, but never shown

Name

Required, but never shown

This page is only for reference, If you need detailed information, please check here

搜尋此網誌

Odtnhj