Flatten a Nested Directory & File Hierarchy from Command Line of Mac OS X
Have you ever needed to flatten a directory structure, moving all file contents from a directories child folders into a single folder? While you can do this manually by moving around files and folders from the file system of Mac OS X or Linux, a faster option is to turn to the command line. Maybe at one point you created a nested hierarchy of directories that you now need to undo by moving all files out of those nested folders and back into a single directory, or maybe you’re looking to simplify a directory structure, whatever the reason, this trick works quite well.
Using the command line to accomplish flattening of files and directory structures is obviously best reserved for advanced users who are comfortable with using terminal in general, if that doesn’t describe you, consider doing it manually through Finder, or using the Mac Automator app to accomplish similar automation of file system activities. We’re focusing on directory flattening from the command line here, however.
Example of Flattening a Nested File Directory
To better understand what we’re trying to accomplish, let’s take an example imaginary directory structure called TestDirectory located in a user Home folder. In this example, TestDirectory contains subfolders like SubDirectory1, SubDirectory2, SubDirectory3, etc, each with files in those respective folders. What we’re looking to do here is flatten the directory structure, moving all files from SubDirectory(X) to the parent directory “TestDirectory”. The initial directory and contents shown recursively with the could look something like this:
$ find ~/TestDirectory/ -type f
~/TestDirectory/rooty.jpg
~/TestDirectory/SampleDirectory1/beta-tool-preview.jpg
~/TestDirectory/SampleDirectory1/alphabeta-tool.jpg
~/TestDirectory/SampleDirectory2/test-tools.jpg
~/TestDirectory/SampleDirectory3/test-png.jpg
~/TestDirectory/SampleDirectory3/test1.jpg
~/TestDirectory/SampleDirectory3/test2.jpg
To flatten this directory and subdirectory contents out back into the TestDirectory folder, you would use the following command string:
find TargetDirectory/ -mindepth 2 -type f -exec mv -i '{}' TargetDirectory/ ';'
After the directory contents have been flattened, it should look like this when listed out:
~/TestDirectory/rooty.jpg
~/TestDirectory/beta-tool-preview.jpg
~/TestDirectory/alphabeta-tool.jpg
~/TestDirectory/test-tools.jpg
~/TestDirectory/test-png.jpg
~/TestDirectory/test1.jpg
~/TestDirectory/test2.jpg
Note the subdirectories will still exist, they’ll just be empty. Make sense? If not, or if that doesn’t demonstrate what you want to accomplish, you probably don’t want to flatten a directory at all, maybe you’re looking to merge or use ditto to do a complex copy to elsewhere.
Flattening a Directory Structure & Nested File Hierarchy with the Command Line
Ready to proceed? The command string we’re going to use to flatten a directory structure and move all files from subdirectories to the base of the target directory is as follows:
find [DIRECTORY] -mindepth 2 -type f -exec mv -i '{}' [DIRECTORY] ';'
Replace [DIRECTORY] with the directory of your choice to flatten, as demonstrated in the example above.
Yes, the directory appears twice in the command string, the first time is the directory being searched to flatten subdirectories of, and the second time as the destination for the found items.
Be precise with the specified destination, because this is not reversible (well, at least without a lot of manual work on your part), so only do this if you’re absolutely sure you want to relocate all the files in the target directories child directories back to the target root folder.
As mentioned before, you could also do this in the Finder of OS X, or at least observe the file and folder changes in the Finder. Option+clicking the little arrows in the List view opens up all subdirectories, showing the folder hierarchy like this:
After fiddling with a variety of bash and zsh alternatives, this handy trick was left by a commenter on StackExcange and it ended up being the easiest and most compatible method. If you know of a better way to flatten a nested directory, do let us know in the comments!
I know this can potential delete data but is it possible to automatically confirm these (overwrite y/n) messages?
In my case it does not matter if duplicates are overwritten.
add -f flag to your move command (“mv -i -f” or “mv -if”) to force it
Confused… the description talks about “TestDirectory” but the example code uses “TargetDirectory.” Is this a glaring typo that has gone unnoticed for five years, or am I missing something?
We had a similar requirement, and found escaping spaces would work this way, by putting quotes around the whole file name:
find ./tmp/ -type f | awk ‘{ str=$0; sub(/\.\/tmp\//, “”, str); gsub(/\//, “-“, str); print “cp \”” $0 “\” \”./to/” str “\”” }’ | bash
Is there a way to do this but instead of moving… to copy?
I have it all in folders on my HD but want to move them all to an SD but without the folder structure.
Thanks.
Is it possible to save the files with the names for directories included?
So in the example above the result would be:
~/TestDirectory/rooty.jpg
~/TestDirectory/SampleDirectory1-beta-tool-preview.jpg
~/TestDirectory/SampleDirectory1-alphabeta-tool.jpg
~/TestDirectory/SampleDirectory2-test-tools.jpg
~/TestDirectory/SampleDirectory3-test-png.jpg
~/TestDirectory/SampleDirectory3-test1.jpg
~/TestDirectory/SampleDirectory3-test2.jpg
Thanks, this is the best I’ve found in my search as well. I would suggest some options for improvement:
1. Instead of `mv -i`, use `mv –backup=numbered`. This will prevent over-writing of same name files and instead append numbers to same names as suffixes.
2. Instead of `mv -i`, use `cp –backup=numbered –link –no-dereference –preserve=all`. This merely creates a hard link to the existing files in the new directory (it doesn’t copy, even though we’re using cp). The backup option has the same non-over-writing effect as above.
3. Instead of `find [DIRECTORY] -not -path ‘*/\.DS_Store*’ -mindepth 2 -type f`, use `find [DIRECTORY] -mindepth 2 -type f`. This ignores .DS_Store files. You can also replace ‘*/\.DS_Store*’ with ‘*/\.*’ if you want to ignore all hidden files.
Once we have the flattened directory, we can delete the hierarchical tree with all subdirectories within safely. If the cp command is used, this deleting is still safe as the files themselves are not removed, only the hard links.
Here’s as example of the full command:
find dir/ -not -path ‘*/\.*’ -mindepth 2 -type f -exec cp –backup=numbered –link –no-dereference –preserve=all ‘{}’ dir/ ‘;’
In proper code formatting…:
find dir/ -not -path '*/\.DS_Store*' -mindepth 1 -type f -exec cp --backup=numbered --link --no-dereference --preserve=all '{}' dir/ ';'Great, thank you very much for this post.
Thanks! Great post, anyway to make this action not ask for confirmation to overwrite duplicate files?
Nice post, just what I was looking for.
But, when I do this from terminal all the .DS_Store files require a confirmation. Is there an easy way to exclude those?
Is it possible to move a type of file (jpeg) to the top?
Any idea how to get a working function with this?
function flatten() { find $1 -mindepth 2 -type f -exec mv -i {} $1 }
i.e. flatten blah
This may be a noob question, but how does this command behave in the case where duplicate file names exist in different subdirectories? For example:
~/TestDirectory/rooty.jpg
~/TestDirectory/SampleDirectory1/test-tools.jpg
~/TestDirectory/SampleDirectory1/alphabeta-tool.jpg
~/TestDirectory/SampleDirectory2/test-tools.jpg
~/TestDirectory/SampleDirectory3/test-tools.jpg
In the above, the same filename exists in 3 different subdirectories and the contents may be different in each file.
Are the files skipped? or overwritten?
Terminal asks you whether to owerwrite files of leave them as they are, in each subfolder.
As a programmer I spend half my time in Terminal, so these kinds of tips are always welcome to me. I tried this one but threw a wrench into the works. I started with a directory on the Desktop ‘test’, then made 3 directories under it:
mkdir -p test/one;test/two;test/three
Then added 3 files to each:
cd ~/Desktop
touch one/1;touch one/2.psd;touch one/.3
touch two/4.jpg;touch two/5.txt;touch two/6.bak
touch three/7.old;touch three/8.new;touch three/9.abc
Using the command listed above worked, even with various extensions (and a ‘dot’ file). Excellent! Thanks eh!
~/Desktop user: ls -a1 test
.
..
.3
1
2.psd
4.jpg
5.txt
6.bak
7.old
8.new
9.abc
one
three
two
for the average user who might perform this (and doesn’t have the mv command syntax memorized) there’s a simpler way:
-Instead just open top folder twice in 2 separate browser windows.
-in one, search for ‘jpg’ or ‘.’ (this might exclude files with no extensions so it’s not perfect), select all, and then drag into the other window or new folder.
-done.
also, not sure how the command line will handle items with the same name writing over each other.
Replace ‘;’ with ‘+’ and you’ll invoke me to move loads of files at once instead of one at a time. If you’re only renaming a handful of files it won’t make any appreciable difference, but if you’re renaming a few hundred it will.
Also bear in mind that while directories with tens of thousands of files in them will give you some operational difficulties. This gets really bad if the directory is shared across the network — a simple ‘ls’ can wind up taking several minutes.
Hello, I am really going through some trouble. The command works, but following your advise to replace ‘;’ with ‘+’ in order to move a lot of files at once results in this error: find: -exec: no terminating “;” or “+”