Having a local mirror of Debian provides you more control over how and when your devices are updated. It can also be useful in situations where devices may not have access to public mirrors. In this post we will describe a method to create a local Debian mirror that can be used for Raspberry Pi devices.
Aptly
Aptly is a tool for managing Debian package repositories and mirrors.
There are several steps to setup a mirror using Aptly. These consist of the following:
- Import keys for signature verification
- Create the local mirrors
- Perform updates to fetch the packages from the origin mirror
- Create snapshots
- Create a GPG key for signing
- Publish the snapshot(s) to an endpoint
Import public keys for verifying signatures:
sudo apt install debian-archive-keyring
gpg --no-default-keyring --keyring /usr/share/keyrings/debian-archive-keyring.gpg --export | gpg --no-default-keyring --keyring trustedkeys.gpg --import
gpg --no-default-keyring --keyring trustedkeys.gpg --keyserver keyserver.ubuntu.com --recv-keys 0E98404D386FA1D9 6ED0E7B82643E131 82B129927FA3303E
Note: Some Debian keys were not available and were sourced from the Ubuntu key server instead, as seen in the last line above.
Creating the mirrors
At the time of writing, the current Raspberry Pi sources.list and raspi.list for “bullseye” reference the following repos:
/etc/sources.d
deb http://deb.debian.org/debian bullseye main contrib non-free
deb http://security.debian.org/debian-security bullseye-security main contrib non-free
deb http://deb.debian.org/debian bullseye-updates main contrib non-free
/etc/sources.list.d/raspi.list
deb http://archive.raspberrypi.org/debian/ bullseye main
We therefore need to replicate this within our own repository.
The following demonstrates the commands to create the mirrors using Aptly:
bullseye-main
aptly mirror create -architectures="arm64" --keyring=trustedkeys.gpg bullseye-main http://deb.debian.org/debian bullseye main contrib non-free
bullseye-security
aptly mirror create -architectures="arm64" --keyring=trustedkeys.gpg bullseye-security http://security.debian.org/debian-security bullseye-security updates/main updates/contrib updates/non-free
Note: there is a known Aptly issue that requires the use of “updates/main” “updates/contrib” “updates/non-free” instead of the usual “main”, “contrib”, “non-free” naming of components when creating the “bullseye-security” mirror.
bullseye-updates
aptly mirror create -architectures="arm64" --keyring=trustedkeys.gpg bullseye-security http://security.debian.org/debian-security bullseye-security updates/main updates/contrib updates/non-free
rasp-bullseye-main:
aptly mirror create -architectures="arm64" --keyring=trustedkeys.gpg rasp-bullseye-main http://archive.raspberrypi.org/debian/ bullseye main
Updating the mirrors:
Updating will download the files from the origin and can take some time to complete. I recommend using screen
session for updating to allow the downloads to continue if you are disconnected.
aptly mirror update bullseye-main
aptly mirror update bullseye-security
aptly mirror update bullseye-updates
aptly mirror update rasp-bullseye-main
Creating a snapshot:
Snapshots reflect a instance of the repository at a certain point in time and should be postfixed with a timestamp:
aptly snapshot create bullseye-main-20231221 from mirror bullseye-main
aptly snapshot create bullseye-security-20231221 from mirror bullseye-security
aptly snapshot create bullseye-updates-20231221 from mirror bullseye-updates
aptly snapshot create rasp-bullseye-main-20231221 from mirror rasp-bullseye-main
Merging the snapshots in preparation for publishing:
Snapshots can be published individually or merged into a single final snapshot. A single snapshot can have the advantage that you only need to add a single URL to the client’s sources.list file. But be aware however that it can also cause security warnings because apt thinks the security updates repository is missing, since it is not specified explicitly in the sources.list.
The following will take the individual snapshots and combine them into a single final snapshot that can be used for publishing:
aptly snapshot merge -latest bullseye-final-20231221 bullseye-main-20231221 bullseye-security-20231221 bullseye-updates-20231221 rasp-bullseye-main-20231221
Key generation:
A signing key pair must be created before publishing. The public key is provided to the client (often in the public web root directory) and is used by apt to verify package signatures.
This can be done with the following:
gpg --gen-key
Export the public key to be later provided to the client:
gpg2 --export --armor > signing.pub
Publishing:
Publishing will write all files to ~/.aptly/public, with the expectation to then serve them via a web server:
aptly publish snapshot -distribution=bullseye bullseye-final-20231221
Once publishing is finished, you will be provided with a URL in the output that can be used to update the client device’s sources.list file:
ie. deb http://<example.com>/mirror bullseye main
Re-publishing:
Republishing a new snapshot (with updates from the remote repository) requires updating the local mirror, creating a new snapshot, merging and then performing a aptly publish switch
instead of an aptly publish snapshot
.
Example of performing a publish switch is below:
aptly publish switch bullseye bullseye-final-20231221
Hosting
Once published, a new directory will be created: ~/.aptly/public. You can then use a web server such as NGINX to serve this directory and make it available to client devices.
Client Device Setup
To setup a new device to use the mirror, the following steps must be taken:
- Download and import the signing key (in this example, from the public web server’s root directory)
- Comment or remove existing sources.list and raspi.list entries for the existing Debian and Raspberry Pi repositories and replace them with a reference to the new mirror URL or mirror list file.
- Update apt against the new mirror and upgrade the existing packages
Fetching and installing the public signing key:
First make sure you have placed the signing.pub from above in the ~/.aptly/public directory to be served by your web server.
We then need to run the following from the client device to fetch the new key.
Note: since the deprecation of apt-key add
, the following command is instead recommended to download the key from the mirror public root and install into the local system’s /usr/share/keyrings
folder, where it can be used by apt
:
wget -O - -q https://<example.com>/mirror/signing.key | sudo gpg --dearmor -o /usr/share/keyrings/mirror-keyring.gpg
Of course, replace the above example URL with your mirrror’s URL.
Updating the sources.list files:
As mentioned above the current Raspberry Pi /etc/apt/sources.list and /etc/apt/sources.list.d/raspi.list for “bullseye” reference the following repos which will need to be removed or commented out, like so:
sources.list
# deb http://deb.debian.org/debian bullseye main contrib non-free
# deb http://security.debian.org/debian-security bullseye-security main contrib non-free
# deb http://deb.debian.org/debian bullseye-updates main contrib non-free
raspi.list
# deb http://archive.raspberrypi.org/debian/ bullseye main
Now, add a new entry to the sources.list or create a new file in /etc/sources.list.d/ that contains a reference to the new mirror:
echo "deb [signed-by=/usr/share/keyrings/mirror-keyring.gpg] https://<example.com>/mirror bullseye main" | sudo tee /etc/apt/sources.list.d/mirror-bullseye.list
Again, replace the above example URL with your own mirror’s URL.
Apt update and upgrade:
Once the sources have been updated, then an apt update
will be required to refresh the package information using the new mirror:
sudo apt update
Followed by an upgrade:
sudo apt upgrade