How to use rsync for very populated directory

On limited devices rsync gets out of RAM while crawling the FileSystem, so I decided to script this and to transfer each directory individually and exclude successfully transferred on second pass.

But 1st I needed to know how to copy a single dir without changing the argument of destitation (ie: not /src/subdir to /dest/subdir , just copy subdir from /src/ to /dest/)

And ended with this solution:

    dir="/tmp/$0.dir/base"
    src="$dir/src"
    dst="$dir/dst"
 
    rm -rfv "$dir"
    mkdir -p "$src" "$dst"
    cd "$dir" 
    pwd
    mkdir -p dir/subdir
    touch dir/subdir/file
    mkdir -p tmp
    mkdir -p dir/ignored-1
    mkdir -p dir/ignored-2
    mkdir -p dir/ignored-3
    mkdir -p dir/ignored-4
 
    dir="dir/subdir"
 
    filters=()
    d="$dir"
    while [ "." != "$d" ] ; do
        filters+=(--filter '+ '$d'/')
        d=$(dirname -- "$d")
    done
    IFS=$'\n'; 
 
    success="$dir/done.lst"
    touch "$success"
 
    sudo rsync \
        --verbose \
        -aAvxXm \
        --exclude-from "$success" \
        --filter "+ $dir/**" \
        "${filters[@]}" \
        --filter '- *' \
        "./" "$dst/" \
        && echo "$dir" | tee -a "$success"
 
    find "$dst"
 
    sudo rsync \
        --verbose \
        -aAvxXm \
        --exclude-from="$success" \
        --filter "+ $dir/**" \
        "${filters[@]}" \
        --filter '- *' \
        "./" "$dst/" \
        && echo "$dir" | tee -a "$success"
 
 
    rm -rf "$dir"

MeeGo

I rebuilt this neat rsync tool for my HandSet … I planned to publish it on ovi store too but I miss a gui, so i created a basic qt using python … but It is not over anyway the package freely installable from shared repo …

NEWS

OpenSource :

FeedBack :

DownLoad :

MISC

MORE

rsync.txt · Last modified: 2022/04/16 12:24 (external edit)
 
Except where otherwise noted, content on this wiki is licensed under the following license: CC Attribution-Share Alike 3.0 Unported
Recent changes RSS feed Donate Powered by PHP Valid XHTML 1.0 Valid CSS Driven by DokuWiki