Syncing and Updating Multiple Machines

I found that running one system was simple enough, but then I started to use multiple systems.  Then, I realised that each machine syncing with mirrors wouldn't be nice in terms of bandwidth usage for the official mirrors as well as using my own bandwidth multiple times.

So, I started with creating my own rsync mirror.  Note that this is just for the portage package listing, and not downloading all source packages!  This would use way too much hard disk space.

First off, you should make sure you have the rsync package installed - which you should do!  But, let's check using:

equery list --installed rsync

the equery command requires that you already emerged the gentoolkit package.  If you see a result, great, you've got it installed.

Creating the Rsync Server.

Edit /etc/rsyncd.conf, and uncomment these lines:

[gentoo-portage]
       path = /usr/portage
       comment = Gentoo Portage tree
       exclude = /distfiles /packages

then, we need to add the rsync daemon to start when the system boots:

rc-update add rsyncd default
/etc/init.d/rsyncd start

Now, this means you have one machine syncing with the internet when you run emerge --sync.  Now, we need to make sure all the other machines are syncing with the local machine running the rsync daemon.  Edit /etc/make.conf, and add a line like this:

SYNC="rsync://excel.linuxsolutions.org/gentoo-portage"

replace the hostname or FQDN in my case, with the IP address, hostname or FQDN of your rsync server.  Now, each of the other machines will sync with your local machine and not the internet, saving you time and effort as well as your internet connection!

Updating Multiple Machines.

This basically means we will get one machine to store all packages that are downloaded for updates, etc.  Each of the other machines, will then use this, instead of each machine downloading the same packages more than once.  This saves your bandwidth and makes updating multiple machines extremely efficient.

It requires designating a machine as the main one for all the other machines to look at.  In my example, I will use the same machine as my rsync mirror.  I install a package called http-replicator:

emerge http-replicator

then, I edit /etc/conf.d/http-replicator and change a couple of settings.  The ones I changed are below:

GENERAL_OPTS="--dir /usr/portage/distfiles"
DAEMON_OPTS="$DAEMON_OPTS --ip 192.168.1.*"

if you have more than one IP subnet to allow access for, then create a second DAEMON_OPTS line similar to the one above, of course with different IP range!  Then start the http-replicator service.  The machine will then listen on port 8080.

Now, for all the other machines to download and store all their files in one location, edit /etc/make.conf and add the following line:

this only needs to be done on the machines accessing the server.  There is no need to put this line in the make.conf on the server running http-replicator. Now it will work like a proxy, storing all the files on the server as well as of course your own distfiles directory as well.  When one machine downloads the updates, they are ready for the other machines to get at the speed of your LAN instead of the speed of your internet connection for subsequent downloads.