Synchronization

I synchronize a lot of things. My personal knowledge base, calendar (via vdirsyncer), Qalculate exchange rates, ledger and budget repository, dotfiles, e-mail, tasks… How do I keep up with all of those things?

As a systemd user, it’s natural for me to set up systemd timers for some of those repetitive tasks. Unfortunately creating a timer for each small task is quite frustrating due to very explicit nature of system timers architecture: one have to create a service unit, timer unit, connect them, enable them, start them, think about their dependencies… That’s too much work!

So I created a single shell script (and also set up systemd timer for it) which launches all registered sync functions in parallel, with help of great GNU Parallel. It looks like this:

#!/bin/bash

sync_cal() {
    echo "----- [ ${FUNCNAME[0]} ] -----"
    set -e
    vdirsyncer sync
    khal2rem -o $HOME/.khal.rem
    touch ~/.reminders
    set +e
}

sync_qalc() {
    echo "----- [ ${FUNCNAME[0]} ] -----"
    set -e
    qalc < <(echo "exrates")
    set +e
}

sync_tasks() {
    echo "----- [ ${FUNCNAME[0]} ] -----"
    set -e
    local h="$(hostname)"
    if [[ "$h" == "hostname_behind_a_proxy" ]]; then
        proxychains task sync
    else
        task sync
    fi
    set +e
}

fns=()
register() {
    export -f $1
    fns+=("$1")
}

register sync_cal
register sync_qalc
register sync_tasks

# Handle arguments passed to the script
if [[ $# -gt 0 ]]; then
    fns=()
    for fn in "$@"; do
        fns+=("sync_${fn}")
    done
fi

parallel -j0 '{}' ::: "${fns[@]}"

I called it mg-sync (I prefix all of my personal scripts with mg-, which makes them easier to find with shell tab-completion. Recently I even wrote a wrapper which instead of e.g. mg-sync allows me to write mg sync, the same way as e.g. Git works). Thanks to this script I can run in parallel all registered functions (when it’s called with no arguments) or only selected ones (when it’s called with arguments).

Even though it’s short and concise, there are few things to remember. First, each Bash function must be exported (export -f), otherwise it will be invisible to Parallel. Second, GNU Parallel spawns a separate job for each function (-j0), where full command of each job is a single argument from the argument list following triple colon. I am also explicit about command substitution ({}), but Parallel would work even without it: parallel -j0 ::: "${fns[@]}".

I like this a lot because it handles parallelism for me, waits for each ran command and correctly reports command errors. My systemd timer runs this script every few hours, but I configured a shortcut key which starts systemd service immediately if I need to quickly synchronize some data.