I recently achieved a satisfying level of configuration on my new Helios64 ARM
NAS, therefore I think it’s time to put it into words for my own memory, and for
sharing with the world.
If you didn’t read the previous part about the hardware, it may probably interest you, but it’s not required to follow this post.
TL;DR: Debian Bullseye + ZFS + Docker compose + a bunch of services, everything run fine!
The Helios64 being quite recent, the support is not yet completely upstreamed in
nor in the
That means that only a small set of distributions are currently supporting this NAS. The official ones are currently only Armbian builds of Debian 10 (Buster) and Ubuntu 20.04 (Focal).
For the fun, and because I wanted to give it a shot, I went on trying to put Fedora on it, before realizing at the end of the installation that it was not supported yet. Why the installer booted is still a mystery, but I didn’t take the time to really investigate that, and I suspect that the bootloader I had flashed before helped a lot in that process.
As I’m really comfortable with Debian, I choose Buster for a first real install, and simply followed the official guide which is really easy and clear.
Sadly, as at the time, there were some difficulties for installing ZFS on Buster, I quickly performed a migration to Bullseye (current Debian testing), and got everything running well on this OS. As it’s only for personal use, it’s no problem being a bit bleeding-edge, and so far, this Debian testing installation has been very stable!
As I spoiled in the previous section, the storage runs on ZFS for multiple reasons that are beyond the scope of this article (check out the Internet on this filesystem if you’ve never heard of it!).
Hardware and redundancy
I’m really far from being an expert regarding all the available HDD on the market, so I basically purchased some 4TB Seagate Ironwolf, mainly because I’ve seen some of them running for quite some time at work, and they have some good reviews on the Internet.
It took me quite some time to think about the redundancy layout, but I finally settled on RAID-Z2, because in the case of one disk failure, that would allow me to continue using the NAS with some confidence while waiting for the new replacement disk to arrive. Besides, with 5 4TB HDDs, that grants me about 12TB of usable space, which is far enough for my personal use!
Poor man’s benchmarks
I knew that ZFS had the reputation of eating all the available RAM, and I even
experienced it already on some other machines, so I was a bit afraid of its
performances on this 4GB NAS. Therefore I quickly went on some poor man’s
dd, to get rough results on what I could expect:
- Writing 4GB with 1MB block size
# dd if=/dev/zero of=test bs=1M count=4000 conv=fsync status=progress 4194304000 octets (4,2 GB, 3,9 GiB) copiés, 16,5515 s, 253 MB/s
- Writing 4GB with 10MB block size
# dd if=/dev/zero of=test bs=10M count=400 conv=fsync 4194304000 octets (4,2 GB, 3,9 GiB) copiés, 15,2559 s, 275 MB/s
- Reading 4GB
# dd if=test of=/dev/null 4194304000 octets (4,2 GB, 3,9 GiB) copiés, 89,0503 s, 47,1 MB/s
- Writing 20GB with 1MB block size
# dd if=/dev/zero of=test bs=1M count=20000 conv=fsync 20971520000 octets (21 GB, 20 GiB) copiés, 75,329 s, 278 MB/s
- Reading 20GB
# dd if=test of=/dev/null 20971520000 octets (21 GB, 20 GiB) copiés, 464,902 s, 45,1 MB/s
- Copying 20GB
# dd if=test of=test2 conv=fsync 20971520000 octets (21 GB, 20 GiB) copiés, 1378,97 s, 15,2 MB/s
As always with ZFS, the results are a bit surprising, but can easily be explained once you are aware that ZFS runs checksums at every read. Most importantly, even if the performances are not the best we’ve seen, they will be largely decent enough to get my home services up and running!
Partitionning Voluming Datasetting? Filesysteming?
As it’s fairly easy to roll that back, I went into the extreme by making a
dataset per logical entity I have to store. This gives me a shitload of
“filesystems” (the most common kind of ZFS dataset), but that’s not really a
problem, and besides, that allows me to quickly monitor the used storage without
ncdu. This can also provide some fine quota tuning in the
future if need be.
The icing on the cake is ZFS’s built-in NFS server, that allows a simple
zfs set sharenfs=on storage/data/tvshows to let Kodi access the
TV shows without hassle!
# zfs list NAME USED AVAIL REFER MOUNTPOINT storage 2.05T 8.38T 170K /storage storage/backup 36.3G 8.38T 170K /storage/backup storage/backup/bep 36.3G 8.38T 36.3G /storage/backup/bep storage/config 2.40G 8.38T 3.80M /storage/config storage/config/jackett 4.12M 8.38T 4.12M /storage/config/jackett storage/config/jellyfin 1.43G 8.38T 1.43G /storage/config/jellyfin storage/config/nextcloud 634M 8.38T 634M /storage/config/nextcloud storage/config/pihole 44.6M 8.38T 44.6M /storage/config/pihole storage/config/radarr 236M 8.38T 236M /storage/config/radarr storage/config/sonarr 49.8M 8.38T 49.8M /storage/config/sonarr storage/config/swag 24.3M 8.38T 24.3M /storage/config/swag storage/config/transmission 2.51M 8.38T 2.51M /storage/config/transmission storage/config/wireguard 263K 8.38T 263K /storage/config/wireguard storage/data 2.01T 8.38T 213K /storage/data storage/data/downloads 35.7G 8.38T 35.7G /storage/data/downloads storage/data/movies 627G 8.38T 627G /storage/data/movies storage/data/music 185G 8.38T 185G /storage/data/music storage/data/nextcloud 380G 8.38T 380G /storage/data/nextcloud storage/data/postgres_nextcloud 157M 8.38T 157M /storage/data/postgres_nextcloud storage/data/tvshows 834G 8.38T 834G /storage/data/tvshows
Deploying some services
A new installation is the perfect time to think about reworking how every
service is deployed. That was especially true as my previous server had been
installed more than 6 years ago (2013!), when I was still a second-year student.
I’ve learned a lot since that time!
Long story short: the awesome guys at
Linuxserver.io maintain a bunch of Docker images
for most of the common services. If anything was to be missing, it’s still very
hard to find a project that doesn’t have its own
docker-compose also being a nice and easy way to manage the containers, I would have a very short and consistent configuration in no time.
As the whole configuration is in overall pretty boring,
here is my
docker-compose.yml, that you just have to
/storage/config if you reproduced the filesystems mentioned above.
Every service is basically on its own network, with its own system user, so that
everyone is at least somewhat isolated from each other. It’s pretty basic, but I
think it’s sufficient for my personal usage.
The only really interesting bit is about the seedbox and the VPN, which will make a fine post on its own, but won’t be covered in this article.
With all the currently running services (PiHole,
Nextcloud with its PostgreSQL,
Transmission), I have about 500MB of free RAM
available, and everything is performing quite well.
The load is usually around 1.2, and sometime jumps over 2 when reading a film. Of course, performing big operations in Nextcloud or the media libraries may be slower than on a traditional x86 NAS, but since that remains fairly uncommon and not at all in the daily usage, I’m fine with it!
Hosting a Borg repository too
As a side note, it’s worth mentioning that the NAS also performs nightly
backups for another server, and that the server part of
Borg behaves extremely normally, running its 45GB
archive job usually in less than 10 minutes.
Here is the report of last night, for those who seek more accurate stats:
Time (start): Fri, 2020-12-11 04:38:56 Time (end): Fri, 2020-12-11 04:46:54 Duration: 7 minutes 58.36 seconds Number of files: 204638 Utilization of max. archive size: 0% ------------------------------------------------------------------------------ Original size Compressed size Deduplicated size This archive: 45.59 GB 39.67 GB 239.98 MB All archives: 422.20 GB 366.55 GB 39.10 GB Unique chunks Total chunks Chunk index: 184323 1986320
Speaking about backups, the Helios64 does not yet have some off-site backup, but rest assured that it’s something that is planned, and will be detailed in another future blog post.
The Helios64 has been running fine for two weeks now, without noticeable performance issues (Nextcloud taking a few seconds waking up after some time idling doesn’t count!), and in overall, I’m super happy with it!
Both the hardware and the software please me very much, and the configuration
with a single
docker-compose.yml file is so painless that I won’t go back to a
classic, bloated system anytime soon.
See you in the next post detailing the VPN and seedbox setup!
Oh, and by the way, the IAE was an awesome event, thank you CIG!