stevepedwards.com/DebianAdmin linux mint IT admin tips info

Exploring Find Cmd Options – file types you may want to find…and go insane trying

This find command exploration nearly got me sectioned! It is the most un-intuitive, cryptic, annoying command I have come across. If you get to the end of this Post doing similar tests you may understand why...if you want some quick simple examples (not all work in Mint) look here:

35 Practical Examples of Linux Find Command

The find command has many complex options, so I wanted to look at some, but with a useful common task in mind. With the VIM editor reading man find, use the “/” search function to search for a parameter you want to explore, e.g: /-mount using the “n” key to find the next occurance of that search term.

Using find generally, the target to be found follows the path to be searched:

find [-H] [-L] [-P] [-D debugopts] [-Olevel] [path...] [expression]

Doing this type of exercise will expose you to some interesting specific options by reading man find, and help to think about and understand more complex system parameters like ASCII key sort order, so (hopefully!) how to use them in building commands to find what you need.

The man page examples are not very clear or comprehensive but a start…

The most basic find option would probably be just a whole or part of a file name that you know exists “somewhere”, so a good catchall is that name with it's file type if you know it – e.g. mp4, docx, pdf etc. and then filter out from there as your memory is jogged (or not!).

Find's defaults include recursive directory search, so no switch required for that, great - but – in Unix tradition - it DOES NOT recognise mixed upper/lower case, so you need to specify correct case in the file name unless you use the -iname switch – e.g.

Example 1 Find file by name and/or doc type

it will find:

find Videos/ -name UFO*

Videos/UFOs - Nazis and Roswell to the 21st Century with Richard Dolan.mp4

Videos/UFO DOCUMENTARY 2015- The UFO Files Presidential Encounters & Underground Bases.mp4

find Videos/ -name *mp4

(finds all mp4's)

but NOT:

find Videos/ -name ufo*

find Videos/ -name *MP4

(finds nothing as none contain lower case “ufo” or upper case “MP4”)

find Videos/ -iname ufo*
Videos/Ufo - The Secret Nasa Transmissions Haunebu Vril.mp4
Videos/UFOs - Nazis and Roswell to the 21st Century with Richard Dolan.mp4
Videos/UFO DOCUMENTARY 2015- The UFO Files Presidential Encounters & Underground Bases.mp4

If you know only the mid part of a file name, you need wildcards front and back as *CV alone won't work i.e:

find . -name *CV*

./LSADocs/Steve_Edwards_CV_2Page.docx

./Desktop/Steve_Edwards_CV_2Page.docx

./Desktop/Steve_Edwards_IT_CV_3p.docx

Another important default listing output is the ASCII char number order, which may confuse you with alphabetical file names if you don't realise this, which is also distribution dependent.

Read p250 TLCL-13.07.pdf for a full explanation.

For example, create a test directory for this example, cd into it, create files, then ls the content:

~/test $ touch file{a..c}

~/test $ touch file{A..C}

~/test $ ls -l file*

-rw-r--r-- 1 stevee stevee 0 Jul 14 19:08 filea
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:09 fileA
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:08 fileb
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:09 fileB
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:08 filec
-rw-r--r-- 1 stevee stevee 0 Jul 14 19:09 fileC

Find seems to list the same ASCII way:

find file*
filea
fileA
fileb
fileB
filec
fileC

This accounts for file names seemingly out of “alphabetical order” at times due to capitalisation, but it is due to a more technical reason explained later:

find Videos/ -name *mp4 | head -3

Videos/David Keith Presents Solar Geoengineering Lecture at Stanford U..mp4

Videos/Masonic Switzerland - Home of the Pharaohs (TPS).mp4

Videos/ALDEBARAN MYSTERY 2 - NAZI UFO SECRETS, THULE, VRIL & THE BLACK SUN.mp4

BUT! If a file begins with a number, it is listed before letters:

~/test $ find *txt

1.txt

2.txt

3.txt

file1.txt

File1.txt

Another important switch to know at first is -mount or -xdev, which stops find searching non local file systems, which could runaway across mounted networked PCs or attached backup drives etc.

DESCRIPTION

This manual page documents the GNU version of find. GNU find searches

the directory tree rooted at each given file name by evaluating the

given expression from left to right, according to the rules of prece‐

dence (see section OPERATORS), until the outcome is known (the left

hand side is false for and operations, true for or), at which point

find moves on to the next file name.

Remember that you need sudo if you are finding files other than your own e.g. on “/” dir.

Example 2 Find most recent file added/changed in a system/dir?

I DLd some you tube vids to Videos, which should be the most recent files known in this directory, as a known quantity check, then ran the below googled find cmd to get a starting point for some options to investigate, which did indeed find the currently downloading file as the most recent in this directory, so how?

It sorted by find's modification time option %T, which is the longest time in seconds since the start of Unix Time.

http://www.stevepedwards.com/DebianAdmin/it-related-numberphile-vids/

find Videos/ -type f -printf "%T@ %p\n"| sort -r | head -3

1468387319.9033764330 Videos/The Human Brain (full documentary) HD.mp4.crdownload

1468386234.9634008040 Videos/RichPlanet TV- Agendas Of The Global Elite.mp4

1468385210.6674238120 Videos/RichPlanet TV- The Falklands Conflict Cover Up.mp4

-type c

File is of type c:

f regular file

-printf format

True; print format on the standard output, interpreting `\' escapes and `%' directives.

Field widths and precisions can be specified as with the `printf' C function. Please note

that many of the fields are printed as %s rather than %d, and this may mean that flags

don't work as you might expect. This also means that the `-' flag does work (it forces

fields to be left-aligned). Unlike -print, -printf does not add a newline at the end of

the string.

\n Newline.

%Tk File's last modification time in the format specified by k, which is the same as

for %A.

%p File's name.

Note the file name by %p decides the find cmd sort order first which would be files beginning with R before T, so the reverse sort by number is used to get the highest time first in secs since Jan 1970 Unix Time. Without %p you just get the last mod time:

find Videos/ -type f -printf "%T@ \n" | sort -r | head -3

1468387319.9033764330

1468386234.9634008040

1468385210.6674238120

If you check this time using the date command you do get the time the download completed shown by ls -lt = 6.21:59am:

date --date='@1468387319'

Wed Jul 13 06:21:59 BST 2016

ls -lt Videos/ | head -3

total 44030732

-rw-r--r-- 1 stevee stevee 376983001 Jul 13 06:21 The Human Brain (full documentary) HD.mp4

-rw-r--r-- 1 stevee stevee 134365402 Jul 13 06:03 RichPlanet TV- Agendas Of The Global Elite.mp4

As the system is dynamic, you will always have recent file changes for many processes as shown by the changes in the last minute (-1) of browser session for example, during this download (imagine what the system/kernel is doing per nanosecond!). This also shows another find default of current directory (pwd) WITH hidden “.” files shown if path not specified.

find -cmin -1

./.config/google-chrome/Default

./.config/google-chrome/Default/Top Sites

./.config/google-chrome/Default/History

./.config/google-chrome/Default/History-journal

./.config/google-chrome/Default/Top Sites-journal

./.config/google-chrome/Default/Preferences

./.config/google-chrome/Default/Current Session

./Videos/The Human Brain (full documentary) HD.mp4.crdownload

-cmin n

File's status was last changed n minutes ago.

This is the ongoing DL status as bytes are added

-mmin n

File's data was last modified n minutes ago.

This is the final name change from .crdownload to .mp4

Example 3 Largest file in a system/dir?

Before experimenting, use the GUI to tell you what IS the biggest file by clicking Size column 1.8GB in this case:

Lots of ways to do this other than find no doubt, (like alias = ducks in Notepad page ), but using find..?

For these types of problems I have to ask “what do I know already about a file size's attributes, via what commands, and how do I get to use those to show me what I want?

I know from the encapsulation Post:

http://www.stevepedwards.com/DebianAdmin/encapsulation-from-bits-to-gigabytes-in-200-years-2/

that ls -ls shows the ext4 block size and IO block size reserved for a file even if it does not completely fill it, so is an indicator of total file size, within 7 block size accuracy, as there are 8 x 512k hard drive block sizes to an ext4 IO sector of 4 x 1024k = 4096k.

ls -ls Videos/ | sort -nr | head -5

1731532 -rw-r--r-- 1 stevee stevee 1773080965 Nov 9 2015 Masonic Switzerland - Home of the Pharaohs (TPS).mp4

1716976 -rw-r--r-- 1 stevee stevee 1758175907 Nov 4 2015 The illuminati Exposed (The Movie).mp4

1669104 -rw-r--r-- 1 stevee stevee 1709156184 Dec 1 2015 UFO DOCUMENTARY 2015- The UFO Files Presidential Encounters & Underground Bases.mp4

1539972 -rw-r--r-- 1 stevee stevee 1576922999 Apr 28 01:56 9-11- Decade of Deception (Full Film NEW 2015).mp4

1487004 -rw-r--r-- 1 stevee stevee 1522684968 Dec 9 2015 The Great Culling- Our Water Official Full Movie.mp4

Seems the GUI uses ls similarly also seen above as it gives the same order?

Does that field have potential use in any of find's options or switches? Yes:

%s File's size in bytes.

Substituting this into Ex 1 above to sort on the file size instead of mod Time, but still print each file's %p name on a new line \n:

find Videos/ -type f -printf "%s%p\n" | sort -nr | head -5

1773080965Videos/Masonic Switzerland - Home of the Pharaohs (TPS).mp4

1758175907Videos/The illuminati Exposed (The Movie).mp4

1709156184Videos/UFO DOCUMENTARY 2015- The UFO Files Presidential Encounters & Underground Bases.mp4

1576922999Videos/9-11- Decade of Deception (Full Film NEW 2015).mp4

1522684968Videos/The Great Culling- Our Water Official Full Movie.mp4

Similarly to above example for mod time, without the %p file name you just get the IO sector size:

1773080965

1758175907

1709156184

1576922999

1522684968

ls -als Videos/The\ Great\ Culling-\ Our\ Water\ Official\ Full\ Movie.mp4

1487004 -rw-r--r-- 1 stevee stevee 1522684968 Dec 9 2015 Videos/The Great Culling- Our Water Official Full Movie.mp4

Do I even need the file -type f? No. Same result.

This is almost identical to ls -ls Videos/ | sort -nr | head -5 but as bs not IO sectors. (1522684968/1487004=1024).

Example 4 finding files set guid +a=s for security reasons

The book 100 Linux Server Hacks mentions disabling binaries that are set uid = s for commands that you probably don't want all users to have root perms to when they run it which is the reason for this s permission historically; as for passwd, it is good for users to be able to change their own at any time without admin permission so best retained as is.

For example, the perm s replaces the x in the User perms section, so is run as root as he is the file owner, and looks like :

ls -l /usr/bin/passwd

-rwsr-xr-x 1 root root 47032 Jan 27 00:50 /usr/bin/passwd

but could also be in the Group or Others depending. You would remove it with chmod u-s.

Some of these can be found in the /bin, /usr/bin and /usr/sbin dirs using the -perms option in 2 forms, and can all be set as search paths at once:

sudo find /bin/ /usr/bin /usr/sbin/ -perm +a=s

/bin/su

/bin/ping6

/bin/umount

/bin/fusermount

/bin/ping

/bin/mount

/usr/bin/chfn

/usr/bin/passwd

/usr/bin/chage

/usr/bin/newgrp

/usr/bin/mail-unlock

/usr/bin/ssh-agent

/usr/bin/crontab

/usr/bin/mail-lock

/usr/bin/dotlockfile

/usr/bin/wall

/usr/bin/pkexec

/usr/bin/traceroute6.iputils

/usr/bin/X

/usr/bin/gpasswd

/usr/bin/mlocate

/usr/bin/lppasswd

/usr/bin/bsd-write

/usr/bin/expiry

/usr/bin/mail-touchlock

/usr/bin/mtr

/usr/bin/chsh

/usr/bin/sudo

/usr/sbin/pppoe

/usr/sbin/uuidd

/usr/sbin/pppd

You could send all these to a text file as usual:

sudo find / -mount -perm +a=s > setuid.txt

Note the list order though…try and work out why it's like that yourself...and let me know! It's different between PCs so I assume it's to do with inode numbers or drive state or some such...?

Not what I thought in example 1 re ASCII format! Is it mod/creation time?

As I've said before, this is why you have to experiment yourself!

Example 5 Find a User's files but skipping their /home dir (Mindbending!)

You may want to find all files belonging to a user that are outside hos /home – say, to delete after an account removal if you forgot to purge fully with deluser –remove-all-files.

This gave me extreme brainache! First, AN answer - but it is not recursing into /750GB - then the how I got to it:

sudo find /* home -prune -user stevee

/750GB
/metest
/Quadra

Good web examples here to understand the AND/OR logical operation:

http://www.theunixschool.com/2012/07/find-command-15-examples-to-exclude.html

The cmd order is critical in Mint, because:

The find command works like this: It starts finding files from the path provided in the command which in this case is the current directory(.). From here, it traverses through all the files in the entire tree and prints those files matching the criteria specified. -prune will not allow the find command to descend into the file any further if it is a directory. Hence, when find starts with the current directory, prune does not allow it to descend the current directory since it itself is a directory, and hence only the current directory gets printed, not the files within the directory. The print happens here because it is the default functionality of find to print anything which is true.

You have to work hard to understand the weirdness..!

A missing * can make all the difference, as can an incorrect logical operator like a “!” (NOT) , or a -prune -o (OR), because the final command must evaluate TRUE to print out. This takes time to get your head around and I still don't get it when constructing a cmd...hmm why I'm a shit programmer also I guess...

Find starts relative to the dir that follows, which is “.” if not otherwise defined, so if at / root to start:

find

...whole root dir contents

find .

...whole root dir contents

find . -name . -prune

.

find . ! -name . -prune

./lib32

./mnt

./sys

./proc

./usr

./home

./initrd.img

./lib64

./boot

./var

./lost+found

./opt

./cdrom

./.rpmdb

./run

./root

./750GB

./media

./Share

./sbin

./lib

./dev

./srv

./etc

./PiImg

./tmp

./vmlinuz

./Quadra

./bobtest.txt

./bin

Also note there is no ASCII alphabetical order here either. The reason is at the Post end. 

There is a big difference between:

find .

find /

and

find /*

later, depending what follows…like logical operator ! {NOT}

So, “find . ! -name . -prune ” means “find in “.” any file NOT named “.” itself, AND don't recurse (prune) into “.” (or print out “.”) – so prints all the files in “.” but not “.” itself.

If you leave out -prune, the whole tree is printed except “.” alone!

This is proved by grepping the output for "." at the line start, with only 1 or no characters after it, which there is not for "." if it is listed, as all other output has more than ./xxx after it:

Anchoring
The caret ^ and the dollar sign $ are meta-characters that respectively
match the empty string at the beginning and end of a line.

Repetition
A regular expression may be followed by one of several repetition
operators:
? The preceding item is optional and matched at most once.
* The preceding item will be matched zero or more times.
+ The preceding item will be matched one or more times.
{n} The preceding item is matched exactly n times.
{n,} The preceding item is matched n or more times.
{,m} The preceding item is matched at most m times. This is a GNU
extension.
{n,m} The preceding item is matched at least n times, but not more
than m times.

find . ! -name . -prune | grep ^.?

or

find . ! -name . -prune | grep ^.$

no output here so "." is NOT listed.

Soooo…by continued logic, to find all the files in the pwd root “.” dir, or “/”, that only belong to user bob, (as “.” belongs to root) without searching his home directory (as it obviously has bob's files in it):

sudo find . ! -name . -prune ! -name home -user bob

./bobtest.txt

This is correct ownership as shown by:

ls -al ./bobtest.txt

-rw-r--r-- 1 bob root 0 Jul 14 00:21 ./bobtest.txt

and the home dir was omitted.

You would not have understood the workings the same if the last command was shown more simply as you would expect it to work from the man page for user file ownership as:

-user uname

File is owned by user uname (numeric user ID allowed).

sudo find -user bob

./bobtest.txt

find: ‘./proc/9683/task/9683/fd/5’: No such file or directory

find: ‘./proc/9683/task/9683/fdinfo/5’: No such file or directory

find: ‘./proc/9683/fd/5’: No such file or directory

find: ‘./proc/9683/fdinfo/5’: No such file or directory

./home/bob

./home/bob/.config….etc.

This is correct for finding ALL files belonging to bob, but along with unwanted StdError, too many lines get shown, not just the desired result of the single known file in root; /bobtest.txt, and his home dir files (because home was NOT pruned!) - so the print out is noisy and confusing as “.” was not pruned, but if it had been, then /home would not have been recursed either. Confused yet?

It showes 73 lines inc. Stderror

You could try to remove the Stderror to clean it up except for bob's files, with:

sudo find -user bob 2> /dev/null

The more intuitive command to show all bob's files under /root, except those in his home directory is:

sudo find /* home -prune -user bob

/bobtest.txt

Let's go through that successful cmd again in English:

sudo find /* home -prune -user bob

sudo required to enter / root dir

find all files/dirs under /, except in /home, that only belong to user bob.

So, you should be able to apply that to other users, and include (by default) any other network or external file systems that are mounted on root / correct...? Wrong!! It does not recurse even though /root was not pruned! This command does my head in...! 

What happens then?

I have an internal partition on my laptop that does not mount at boot, that has a backup of my /stevee directory, owned by stevee; and backup of root directories, owned by root; on it, so I'll mount it:

stevee@AMD / $ sudo mount -t ext4 /dev/sda3 750GB/

stevee@AMD / $ ls /750GB/ -al

total 16

drwxr-xr-x 4 stevee stevee 4096 Jul 14 16:24 .

drwxr-xr-x 29 root root 4096 Jul 14 00:21 ..

drwxr-xr-x 22 root root 4096 Jul 11 20:41 BURoot

drwxr-xr-x 9 stevee stevee 4096 Jul 8 00:21 stevee

If I run the same command for user stevee what happens? No recursion into them! Why not? / was not pruned...??

Is it because the search started from the files under /root, not itself...?

sudo find /* home -prune -user stevee

./750GB

./Quadra

so find shows these dirs of stevee, but not his home dir as required proved with grep. OK so far.

ls -al ./ | grep stevee

drwxr-xr-x 4 stevee stevee 4096 Jul 14 16:24 750GB

drwxrwxr-x 2 stevee root 4096 Jan 18 19:38 Quadra

So how do you get the recursion to happen for these and show just more sub files of these stevee's dirs? First what is on this drive? 2 dirs and one test file.

ls /750GB/ -l
total 8
drwxr-xr-x 22 root root 4096 Jul 11 20:41 BURoot
-rw-r--r-- 1 stevee stevee 0 Jul 14 20:22 metest
drwxr-xr-x 9 stevee stevee 4096 Jul 8 00:21 stevee

I give find the new root reference directory. This works, but it is not clear what is being pruned at first, as no dir name was specified, so apparently nothing was a candidate for pruning leaving only files/dir owned by stevee??:

Maybe it means prune any dir owned by stevee without being specifically named, then print what files are left that are owned by stevee...?

find /750GB/* -prune -user stevee

/750GB/metest
/750GB/stevee

This command is so annoying! I can't see how you would ever know for sure if you got what you need from this command at all...it just does not seem worth the time and effort to study...but to finish I found out how to exclude the stevee dir AND the root owned dir by naming it's full path for pruning, then adding the -user option to find stevee's remaining files. It is about the order of options position...dirs to the left, options to the right in this case...

sudo find /750GB/* /750GB/stevee/ -prune -user stevee
/750GB/metest
/750GB/stevee
/750GB/stevee/

Yet, confusingly, this next seemingly "logical" construction does the opposite of what you may think by still printing all the files belonging to stevee, from ALL the sub dirs I "thought" I told it not to:

sudo find /750GB/* ! -name /750GB/stevee/ -user stevee

/750GB/stevee/Documents/AmpDocs
/750GB/stevee/Documents/AmpDocs/AllValveAmpPosts.pdf....etc.

because it actually reads: Find all files/dirs under /750GB NOT named /750GB/stevee, so prints everything belonging to stevee except the dir /750GB/stevee.

But, this alternative still acts the same way though the name switch is not involved! It's like the logic in Mint is reversed!

sudo find /750GB/* -not \( -path 750GB/stevee/* \) -user stevee

/750GB/stevee/Documents/AmpDocs
/750GB/stevee/Documents/AmpDocs/AllValveAmpPosts.pdf....etc.

The fix is to name the directory completely explicitly using quotes so that NO files that begin with /750GB/stevee at all are printed, so just the dir is:

sudo find /750GB/* ! -wholename "/750GB/stevee/*" -user stevee

/750GB/metest
/750GB/stevee

If you want to omit the directory also, trim the end slash:

sudo find /750GB/* ! -wholename "/750GB/stevee*" -user stevee
/750GB/metest

or use the path switch:

sudo find /750GB/* ! -path "/750GB/stevee*" -user stevee
/750GB/metest

Or from the complete top of / not recursing stevee's dirs:

sudo find /* ! -path home -prune ! -path /750GB -prune -user stevee
/metest

If you want to find files in the root dir / that belong to a user other than root (as they should only be):

sudo find /* -prune -user stevee
/750GB
/metest

sudo find /* -prune -user bob
/bobtest.txt

What a complete brain ache...

http://stackoverflow.com/questions/4210042/exclude-directory-from-find-command

There is clearly some confusion here as to what the preferred syntax for skipping a directory should be. (Really!?)

GNU Opinion

To ignore a directory and the files under it, use -prune
From the GNU find man page

Reasoning

-prune stops find from descending into a directory. Just specifying -not -path will still descend into the skipped directory, but -not -path will be false whenever find tests each file.

Issues with -prune

-prune does what it's intended to, but are still some things you have to take care of when using it.

find prints the pruned directory.
TRUE That's intended behaviour, it just doesn't descend into it. To avoid printing the directory altogether, use a syntax that logically omits it.
-prune only works with -print and no other actions.
NOT TRUE. -prune works with any action except -delete. Why doesn't it work with delete? For -delete to work, find needs to traverse the directory in DFS order, since -delete will first delete the leaves, then the parents of the leaves, etc... But for specifying -prune to make sense, find needs to hit a directory and stop descending it, which clearly makes no sense with -depth or -delete on.


Find command list order. So what is the reason that find appeared to show lists alphabetically initially, then not in the later examples?

From UnixDB book:

"..find does not display a sorted list...info on file characteristics is obtained from the inode as well as the directory entry..sequence..displayed depends on the internal organization of the file system"

Comments are closed.

Post Navigation