Save Gigabytes of Failed Rsync Video Transfers on a Drive

If you watch and keep a lot of YouTube or other video files, and backup regularly from your main laptop to other PCs, you may accumulate a lot of partial videos from interrupted copies and  rsync transfers that have extensions of the type .crdload or .MOV_0002.mp4.B8ObyK which can accumulate to a surprising size.

Running find for mp4 files of these types on my 1.5TB backup drive I found many which I could then approximately sum (ignoring k values as most are in MB, but if you want accuracy, drop the -h to show 1K-blocks) via:

sudo find / -name *.mp4.* -exec du -h {} \; | awk '{sum +=$1} {print $1, sum}'
221M 221
571M 792
539M 1331
91M 1422
....
221M 7218.3
580M 7798.3
422M 8220.3
37M 8257.3

This is about 8GB of useless video files!

Run:

sudo find / -name *.mp4.*

Make sure you check the files before you remove them with -exec rm!!

sudo find / -name *.mp4.* -exec rm --i=no {} \;

Because if you ran the same line for mp3s you may delete important non MP3 files such as:

100K /Quadra/www/wp/wp-includes/ID3/module.audio.mp3.php

Out of interest, as I have a large vid collection, I could measure it approximately as 177GB!:

sudo find /Quadra/ -name *.mp4 -exec du -h {} \; | awk '{sum +=$1} {print $1, sum}'

175M 176824
382M 177206

OR as MB:

sudo find /Quadra/ -name *.mp4 -exec du -Bm {} \; | awk '{sum +=$1} {print $1, sum "M"}'

382M 188597M

If you want MB accuracy converted to GB:

sudo find /Quadra/ -name *.mp4 -exec du -Bm {} \; | awk '{sum +=$1} {print $1MB, sum "MB", sum/1024 "GB"}'

175M 157937MB 154.235GB
382M 158319MB 154.608GB

or

sudo find /Quadra/ -name *.mp3 -exec du -Bk {} \; | awk '{sum +=$1} {print $1KB, sum "KB", sum/1024 "MB", sum/(1024*1024)"GB"}'

If you test what find actually calculates when used by default without with a glob or specific name etc. with -exec du, it includes the parent directory size also.

As a test, create 2 empty text files with touch in an empty dir Public, then add a character as in the Bits to Gb in 200 years Post, which we know is 1 byte in size, but uses a 4k block space on disk:

stevee@T3400 ~/Pictures $ cd
stevee@T3400 ~ $ cd Public/
stevee@T3400 ~/Public $ ls
stevee@T3400 ~/Public $ touch test1.txt test2.txt
stevee@T3400 ~/Public $ ls -l
total 0
-rw-r--r-- 1 stevee stevee 0 Feb 8 22:24 test1.txt
-rw-r--r-- 1 stevee stevee 0 Feb 8 22:24 test2.txt
stevee@T3400 ~/Public $ ls -las
total 8
4 drwxr-xr-x 2 stevee stevee 4096 Feb 8 22:24 .
4 drwxr-xr-x 39 stevee stevee 4096 Feb 8 19:40 ..
0 -rw-r--r-- 1 stevee stevee 0 Feb 8 22:24 test1.txt
0 -rw-r--r-- 1 stevee stevee 0 Feb 8 22:24 test2.txt
stevee@T3400 ~/Public $ echo -n "A" >> test1.txt
stevee@T3400 ~/Public $ ls -las
total 12
4 drwxr-xr-x 2 stevee stevee 4096 Feb 8 22:24 .
4 drwxr-xr-x 39 stevee stevee 4096 Feb 8 19:40 ..
4 -rw-r--r-- 1 stevee stevee 1 Feb 8 22:25 test1.txt
0 -rw-r--r-- 1 stevee stevee 0 Feb 8 22:24 test2.txt
stevee@T3400 ~/Public $ echo -n "A" >> test2.txt
stevee@T3400 ~/Public $ ls -las
total 16
4 drwxr-xr-x 2 stevee stevee 4096 Feb 8 22:24 .
4 drwxr-xr-x 39 stevee stevee 4096 Feb 8 19:40 ..
4 -rw-r--r-- 1 stevee stevee 1 Feb 8 22:25 test1.txt
4 -rw-r--r-- 1 stevee stevee 1 Feb 8 22:25 test2.txt
stevee@T3400 ~/Public $ find -exec du -Bk {} \; | awk '{sum +=$1} {print $1, sum "K"}'
12K 12K
4K 16K
4K 20K

The text files only add to 8k but the total shown by find/exec/du includes the parent dir size.

Without the specific file type search, you get parent dir sizes also:

stevee@T3400 ~/Public $ find *txt -exec du -Bk {} \; | awk '{sum +=$1} {print $1, sum "K"}'
4K 4K
4K 8K
stevee@T3400 ~/Public $ find . -exec du -Bk {} \; | awk '{sum +=$1} {print $1 "k", sum "k"}'
12Kk 12k
4Kk 16k
4Kk 20k 

Comments are closed.

Post Navigation