SSH authorized_keys command=

On the host with files that need to be copied

 rsync -v --remove-source-files -r -e "ssh -vi .ssh/id_rsa" src_dir/ user@host:dest_dir/

SSH verbose out will contain something like 

debug1: Sending command: rsync --server -vre.iLsfxCIvu --remove-source-files . dest_dir/

Then constrain authorized_keys with command= and the restrict keyword which disables a variety of things like port forwarding, agents, X11 and PTY allocation.  See man authorized_keys.

command="rsync --server -vre.iLsfxCIvu --remove-source-files . dest_dir/",restrict ssh-rsa ...

Finally drop the -v from the initial rsync command.