SolarNodeOS Managed Deploy Guide
SolarNodeOS is a Debian-based operating system tailored specifically for SolarNode devices. SolarNetwork Foundation maintains a Debian package repository with core packages needed by a SolarNode device. When deploying a fleet of nodes, you will need to consider the following:
- What node plugin(s) will you be using, that you could have installed in each node? For example you might know in advance that all your nodes will need to integrate with modbus devices, and thus would need the Modbus Device plugin installed.
- What standardized plugin configuration would be helpful, that you could have auto-configured in each node? For example you might be integrating with a known modbus device that would require a known configuration.
This guide will describe how you can create a customized SolarNodeOS, possibly with custom OS packages, that you can then easily deploy to as many nodes as needed.
⚠️ Note that when following the recommendations in this guide, you should not use the Plugins page in the SolarNode setup app to install/upgrade plugins. The reason for that is because the Plugins manager won't know that plugins have been installed via OS packages, and you could end up with a broken SolarNode system. In practice this is not much of an inconvenience, as it is easier and more consistent in the long run to maintain nodes using packages as outlined in this guide.
This guide assumes you have access to a Debian-based Linux workstation. This could be an actual Linux workstation, a virtual machine such as VMWare or Virtual Box, or even Windows Subsystem for Linux (WSL) on Windows can work. Any Debian-based Linux distribution should work, such as Ubuntu. This guide has been written with Debian 10 (Buster) in mind. You should be familiar and comfortable with using shell commands.
At a minimum, the following tools are required:
- Git, and Git LFS
- Java version 8 or higher
Typically these can be installed like:
# install general requirements
sudo apt install git git-lfs openjdk-11-jdk-headlessYou will need to set up a build environment for running the SolarNodeOS customization scripts
and/or creating packages. In this guide, we'll create a ~/Documents/SolarNodeOS directory to hold
everything, and all example commands will be based on this directory:
# create build environment base directory
mkdir -p ~/Documents/SolarNodeOSSolarNetwork Foundation provides basic SolarNodeOS device image files that can be deployed to a variety of hardware devices, for example the Raspberry Pi family of devices. This section will show how you can customize one of those images, for example to install/update software or make configuration changes. The outcome of the customization process is a new SolarNodeOS based image that you can then deploy to SolarNode devices.
You need to download the SolarNetwork/solarnode-os-images repository:
# download (clone) the solarnode-os-images repository
cd ~/Documents/SolarNodeOS
git clone https://github.com/SolarNetwork/solarnode-os-images.gitNext you will need to install some required software:
# install customization tools
sudo apt install bc binfmt-support btrfs-progs e2fsprogs qemu qemu-user-static \
rsync systemd-container xz-utilsDownload the image you want to use as the starting point for your custom SolarNodeOS image, and
save that to the ~/Documents/SolarNodeOS directory. In this guide, we'll customize the
solarnodeos-deb10-pi-2GB-20210303.img.xz Raspberry PI image.
The solarnode-os-images/debian/bin/customize.sh script is what will drive the customization
process. This script will take an image file as input and then performs these basic steps:
- Start up a light-weight virtual environment from the input image.
- Either run a customization script or launch an interactive shell for you to customize manually.
- Save the customized environment as a new image.
The customize.sh script has some other helpful tricks up its sleeve, such as the ability to grow
the image filesystem if you need to install a fair bit of custom software.
To get started, you need to create a shell script that performs the customization tasks you need.
This script will be executed in the virtual SolarNodeOS environment, and its job is to make any
changes to SolarNodeOS you want to have in your customized image. For this guide let's start with
a very basic script that simply installs the tmux command, a handy utility to help with remote
node management:
#!/usr/bin/env sh
echo "!!! Begin my SolarNodeOS customizations..."
apt install -qy tmux
echo "!!! Finished my SolarNodeOS customizations."Save this script as ~/Documents/SolarNodeOS/my-cust.sh. Then we can run the customization script,
configuring the output image file to be named my-solarnodeos-DATE.img.xz where DATE is the
current date:
sudo ~/Documents/SolarNodeOS/solarnode-os-images/debian/bin/customize.sh -v -z \
-o ~/Documents/SolarNodeOS/my-solarnodeos-$(date '+%Y%m%d').img \
~/Documents/SolarNodeOS/solarnodeos-deb10-pi-2GB-20210303.img.xz \
~/Documents/SolarNodeOS/my-cust.shThe script produces a lot of output, but somewhere in there you should see the output from your
my-cust.sh script:
!!! Begin my SolarNodeOS customizations...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
libevent-2.1-6 libutempter0
The following NEW packages will be installed:
libevent-2.1-6 libutempter0 tmux
0 upgraded, 3 newly installed, 0 to remove and 27 not upgraded.
Need to get 410 kB of archives.
After this operation, 962 kB of additional disk space will be used.
...
Unpacking tmux (2.8-3) ...
Setting up libevent-2.1-6:armhf (2.1.8-stable-4) ...
Setting up libutempter0:armhf (1.1.6-3) ...
Setting up tmux (2.8-3) ...
Processing triggers for libc-bin (2.28-10+rpi1) ...
!!! Finished my SolarNodeOS customizations.
What you end up with is:
- The custom image file, named
my-solarnodeos-DATE.img - A checksum of the image file, named
my-solarnodeos-DATE.img.sha256 - A compressed image file, named
my-solarnodeos-DATE.img.xz - A checksum of the compressed image file, named
my-solarnodeos-DATE.img.xz.sha256
$ ls -lh my-solarnodeos*
-rw-r--r-- 1 root root 1.3G Apr 11 14:47 my-solarnodeos-20210411.img
-rw-r--r-- 1 root root 94 Apr 11 14:47 my-solarnodeos-20210411.img.sha256
-rw-r--r-- 1 root root 266M Apr 11 14:50 my-solarnodeos-20210411.img.xz
-rw-r--r-- 1 root root 97 Apr 11 14:50 my-solarnodeos-20210411.img.xz.sha256
During the development of your customization script, it can be handy to test out changes in a more
interactive fashion. You can do this by passing the -i argument to customize.sh, which will
launch a shell in the SolarNodeOS virtual environment for you. Once you've completed your changes,
simply exit and customize.sh will continue and generate the customized output image for you.
If you want to throw away your changes and not bother creating the output image just exit 1 to
signal an error condition and customize.sh will clean up without creating the output image.
Note that you must still provide a customization script to customize.sh. That script will
be copied into the SolarNodeOS virtual environment as a script named customize. This makes it
convenient for troubleshooting your script, as you can then run it manually to see what
problems there are. For example:
# run interactively
sudo ~/Documents/SolarNodeOS/solarnode-os-images/debian/bin/customize.sh -v -z \
-o ~/Documents/SolarNodeOS/my-solarnodeos-$(date '+%Y%m%d').img \
-i \
~/Documents/SolarNodeOS/solarnodeos-deb10-pi-2GB-20210303.img.xz \
~/Documents/SolarNodeOS/my-cust.sh
...
'/home/matt/Documents/SolarNodeOS/my-cust.sh' -> '/tmp/sn-CEkN7/var/tmp/sn-iYN9f/customize'
Spawning container solarnode-cust on /tmp/sn-CEkN7.
Press ^] three times within 1s to kill container.
root@solarnode-cust:/var/tmp/sn-iYN9f# ls -l
total 4
-rwxr-xr-x 1 1000 1000 144 Apr 11 14:47 customize
root@solarnode-cust:/var/tmp/sn-iYN9f# ./customize
!!! Begin my SolarNodeOS customizations...
...
!!! Finished my SolarNodeOS customizations.
root@solarnode-cust:/var/tmp/sn-iYN9f# exit 1
exit
Container solarnode-cust failed with error code 1.
!!!
!!! Error with interactive setup in container!
!!!
Restoring original /tmp/sn-CEkN7/etc/resolv.conf
removed '/tmp/sn-CEkN7/var/tmp/sn-iYN9f/customize'
removed directory '/tmp/sn-CEkN7/var/tmp/sn-iYN9f'
Enabling preload shared libs in /tmp/sn-CEkN7/etc/ld.so.preload... OK
Unmounting source SOLARBOOT filesystem /tmp/sn-CEkN7//boot.
Unmounting source SOLARNODE filesystem /tmp/sn-CEkN7.
Closing source image loop device /dev/loop0.
Deleted /tmp/img-tIpFI
A good way to manage SolarNode deployments with a known set of SolarNode plugins on them is to create OS packages out of the set of SolarNode plugins you want to include on your nodes. The SolarNetwork/solarnetwork-build project contains support for you to do just that. How it works is that you'll use the SolarNetwork build system to select the plugins you want to deploy and download them all into a directory structure suitable for copying to a SolarNode device.
From there you can use your OS packaging tool of choice to create the package. For this guide,
we'll use the standard make tool along with fpm to create the package.
Make sure you have make and fpm available like this:
sudo apt install ant ruby ruby-dev build-essential
sudo gem install --no-ri --no-rdoc fpmThen download (clone) the solarnetwork-build project, and we'll switch to the develop branch
to access the latest & greatest:
cd ~/Documents/SolarNodeOS
git clone https://github.com/SolarNetwork/solarnetwork-build.git
cd solarnetwork-build
git checkout developFirst create a directory for our custom package, which we'll name my-solarnode-app-core:
# create package directory
mkdir -p ~/Documents/SolarNodeOS/packages/my-solarnode-app-core/debian
# create symlink to solarnetwork-build dir
cd ~/Documents/SolarNodeOS/packages/my-solarnode-app-core/debian
ln -s ../../../solarnetwork-buildNow we'll create an Ivy XML configuration that defines which plugins we want installed. The available
plugins maintained by SolarNetwork Foundation can be seen here. For this guide,
let's install support for Modbus devices. For that, we need a few plugins, which show up as
<dependency> elements in the XML. Create an ivy-my-main.xml file with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="2.0">
<info organisation="SolarNetwork" module="SolarNode"/>
<configurations>
<conf name="runtime" visibility="public" description="The Runtime"/>
</configurations>
<dependencies defaultconfmapping="runtime->default(runtime)">
<!-- Modbus support -->
<dependency org="net.solarnetwork.external" name="net.solarnetwork.external.jamod.pjc" rev="latest.release"/>
<dependency org="net.solarnetwork.node" name="net.solarnetwork.node.io.modbus" rev="latest.release"/>
<dependency org="net.solarnetwork.node" name="net.solarnetwork.node.io.modbus.jamod" rev="latest.release"/>
<dependency org="net.solarnetwork.node" name="net.solarnetwork.node.datum.modbus" rev="latest.release"/>
<!-- solarnode-app-core excludes -->
<exclude artifact="net.solarnetwork.external.org.rxtx"/>
<exclude artifact="net.solarnetwork.common"/>
<exclude artifact="net.solarnetwork.common.mqtt"/>
<exclude artifact="net.solarnetwork.common.web"/>
<exclude artifact="net.solarnetwork.node"/>
<exclude artifact="net.solarnetwork.node.dao.jdbc"/>
<exclude artifact="net.solarnetwork.node.io.mqtt"/>
<exclude artifact="net.solarnetwork.node.setup.web"/>
<!-- Global excludes provided by the base system -->
<exclude org="commons-(beanutils|codec|collections|digester|fileupload|io)" matcher="regexp"/>
<exclude org="com.fasterxml.jackson.core"/>
<exclude org="com.fasterxml.jackson.dataformat"/>
<exclude org="com.fasterxml.jackson.datatype"/>
<exclude org="javax.measure"/>
<exclude org="javax.servlet"/>
<exclude org="javax.xml.bind"/>
<exclude org="io.netty"/>
<exclude org="joda-time"/>
<exclude org="net.java.dev.jna"/>
<exclude org="net.sf.supercsv"/>
<exclude org="org.apache.commons"/>
<exclude org="org.apache.servicemix.bundles"/>
<exclude org="org.apache.tomcat"/>
<exclude org="org.eclipse.paho"/>
<exclude org="org.eclipse.virgo.mirrored"/>
<exclude org="org.glassfish.tyrus.bundles"/>
<exclude org="org.osgi"/>
<exclude org="org.quartz-scheduler"/>
<exclude org="org.slf4j"/>
<exclude org="org.springframework"/>
<exclude org="org.springframework.security"/>
<exclude org="org.mitre.dsmiley.httpproxy"/>
</dependencies>
</ivy-module>There's a lot of stuff packed in there, but the important lines are the <dependency> ones, which
are dictating which plugins to include in the package.
Now create a Makefile with the following content:
NAME = my-solarnode-app-core
VERSION = 1.0.0-1
SN_BUILD_ROOT = ${CURDIR}/solarnetwork-build
DEB_BUILD_ROOT = $(SN_BUILD_ROOT)/solarnode-deploy/generic/build/deb
IVY_FILE = ${CURDIR}/ivy-my-main.xml
deb : app-core table
fpm -s dir \
-t deb \
-n $(NAME) \
-v $(VERSION) \
--description 'My SolarNode standard device support' \
--chdir $(DEB_BUILD_ROOT) \
-a all \
--maintainer 'Me <me@localhost>' \
--vendor 'Me' \
--license 'Proprietary' \
-f \
-d 'solarnode-app-core (>= 1.12.0)' \
var
clean :
rm $(NAME)_$(VERSION)_all.deb
app-core :
ant -f "$(SN_BUILD_ROOT)/solarnode-deploy/generic/build.xml" \
"-Divy.file=$(IVY_FILE)" \
clean deb-package-assemble
table :
java -jar "$(SN_BUILD_ROOT)/solarnetwork-osgi-lib/lib/bh.jar" \
"$(SN_BUILD_ROOT)/solarnode-deploy/generic/build/deb"
This build script will run the build and download the latest version of each required plugin, and then print out a handy Markdown-formatted table of all the plugins that are included in the package, like this:
| Name | ID | Vers |
|---|---|---|
| Generic Modbus Datum Source | n.s.n.datum.modbus |
2.3.0 |
| Java Modbus Library (PJC) | n.s.external.jamod.pjc |
1.2.0 |
| Modbus Communication Support (Jamod) | n.s.n.io.modbus.jamod |
1.1.0 |
| Modbus Communication Support API | n.s.n.io.modbus |
3.2.0 |
| PureJavaComm | n.s.external.pjc |
1.0.2 |
The package will be named my-solarnode-app-core_1.0.0-1_all.deb and you can inspect its contents
with dpkg -c like this:
$ dpkg -c my-solarnode-app-core_1.0.0-1_all.deb
drwxr-xr-x 0/0 0 2021-04-11 15:35 ./
drwxr-xr-x 0/0 0 2021-04-11 15:35 ./usr/
drwxr-xr-x 0/0 0 2021-04-11 15:35 ./usr/share/
drwxr-xr-x 0/0 0 2021-04-11 15:35 ./usr/share/doc/
drwxr-xr-x 0/0 0 2021-04-11 15:35 ./usr/share/doc/my-solarnode-app-core/
-rw-r--r-- 0/0 148 2021-04-11 15:35 ./usr/share/doc/my-solarnode-app-core/changelog.gz
drwxr-xr-x 0/0 0 2021-04-11 15:35 ./var/
drwxr-xr-x 0/0 0 2021-04-11 15:35 ./var/lib/
drwxr-xr-x 0/0 0 2021-04-11 15:35 ./var/lib/solarnode/
drwxr-xr-x 0/0 0 2021-04-11 15:35 ./var/lib/solarnode/app/
drwxr-xr-x 0/0 0 2021-04-11 15:35 ./var/lib/solarnode/app/main/
-rw-r--r-- 0/0 36879 2021-02-26 15:17 ./var/lib/solarnode/app/main/net.solarnetwork.node.datum.modbus-2.3.0.jar
-rw-r--r-- 0/0 32175 2021-02-02 10:30 ./var/lib/solarnode/app/main/net.solarnetwork.node.io.modbus.jamod-1.1.0.jar
-rw-r--r-- 0/0 155421 2020-08-31 14:09 ./var/lib/solarnode/app/main/net.solarnetwork.external.jamod.pjc-1.2.0.rc1-SN20200831A.jar
-rw-r--r-- 0/0 56392 2021-02-02 10:30 ./var/lib/solarnode/app/main/net.solarnetwork.node.io.modbus-3.2.0.jar
-rw-r--r-- 0/0 127161 2020-08-31 14:08 ./var/lib/solarnode/app/main/net.solarnetwork.external.pjc-1.0.2.SN20200831A.jar
This section shows how you can combine the steps you took in the previous two sections to customize
a SolarNodeOS image by installing your own custom plugin package. This time when we run the
customize.sh script we'll make our ~/Documents/SolarNodeOS/packages directory available to the
virtual environment as /tmp/packages. That is done by passing an additional
~/Documents/SolarNodeOS/packages:/tmp/packages argument. Knowing that directory will be available,
we can then update our my-cust.sh script to install the my-solarnode-app-core_1.0.0-1_all.deb
package for us. Change the my-cust.sh script to this:
#!/usr/bin/env sh
echo "!!! Begin my SolarNodeOS customizations..."
dpkg -i /tmp/packages/my-solarnode-app-core/debian/my-solarnode-app-core_1.0.0-1_all.deb
echo "!!! Finished my SolarNodeOS customizations."Now run customize.sh like this:
sudo ~/Documents/SolarNodeOS/solarnode-os-images/debian/bin/customize.sh -v -z \
-o ~/Documents/SolarNodeOS/my-solarnodeos-$(date '+%Y%m%d').img \
~/Documents/SolarNodeOS/solarnodeos-deb10-pi-2GB-20210303.img.xz \
~/Documents/SolarNodeOS/my-cust.sh \
~/Documents/SolarNodeOS/packages:/tmp/packagesThis time, notice the my-solarnode-app-core package is installed:
!!! Begin my SolarNodeOS customizations...
Selecting previously unselected package my-solarnode-app-core.
(Reading database ... 22383 files and directories currently installed.)
Preparing to unpack .../my-solarnode-app-core_1.0.0-1_all.deb ...
Unpacking my-solarnode-app-core (1.0.0-1) ...
Setting up my-solarnode-app-core (1.0.0-1) ...
!!! Finished my SolarNodeOS customizations.
This section outlines how you can customize SolarNodeOS to include support for your own custom
Debian package repository hosted on AWS S3 (or compatible service). This is a very nice way of
maintaining the software on your nodes over time, because the nodes can access software updates via
standard OS tools like apt update && apt upgrade.
Maintaining an S3 Debian package repository is outside the scope of this guide. There are good tools available for doing this, such as aptly. Once you have the package repository available on S3, you'll need:
- An S3 access token and secret with read-only access to the S3 bucket hosting the repository.
- The public GPG key used to sign the packages on the S3 repo, in a GPG keyring file.
To make it easy to configure the S3 package repository into your customized SolarNodeOS image,
we'll create a custom package with all the necessary configuration included, and that package
can be installed via the customize.sh process. Let's call this package my-s3-repo:
# create package directory
mkdir -p ~/Documents/SolarNodeOS/packages/my-s3-repo/debian
cd ~/Documents/SolarNodeOS/packages/my-s3-repo/debian
# create package directories
mkdir -p etc/apt/sources.list.d
mkdir -p etc/apt/trusted.gpg.d
# just to be safe, ignore our s3 credentials file from being added to git
echo '/etc/apt/s3auth.conf' >.gitignoreNow create a etc/apt/sources.list.d/private-s3.list file with content similar to this,
but with your S3 bucket name instead of my-debian-repo:
deb s3://my-debian-repo/ buster main
Now copy the public GPG keyring that holds the public GPG key you use to sign the packages in
the S3 repository to etc/apt/trusted.gpg.d/my-packaging.gpg.
Finally, create a etc/apt/s3auth.conf file with the appropriate S3 credentials, similar to
this:
AccessKeyId = MY_ACCESS_KEY
SecretAccessKey = MY_ACCESS_KEY_SECRET
Region = MY_S3_BUCKET_REGION
Token = ''
Finally, create a Makefile like this:
NAME = my-s3-repo
VERSION = 1.0.0-1
PATHS = etc
deb :
fpm \
--input-type dir \
--output-type deb \
--name $(NAME) \
--version $(VERSION) \
--architecture all \
--maintainer 'Me <me@localhost>' \
--vendor 'Me' \
--description 'My private S3 package repo' \
--license 'Proprietary' \
--force \
--depends 'apt-transport-s3 (>= 1.2.1)' \
--deb-no-default-config-files \
$(PATHS)
clean :
rm $(NAME)_$(VERSION)_all.deb
Now you can build your package, via make.
Using the same process as outlined previously,
you can now update the my-cust.sh script to install the new my-s3-repo_1.0.0-1_all.deb package,
and assuming you've published your my-solarnode-app-core package there, can then install that
afterwards, like this:
#!/usr/bin/env sh
echo "!!! Begin my SolarNodeOS customizations..."
# Install S3 repo support
dpkg -i /tmp/packages/my-solarnode-app-core/debian/my-s3-repo_1.0.0-1_all.deb
# Now install my-solarnode-app-core, which will come from the S3 repo
apt install -qy my-solarnode-app-core
echo "!!! Finished my SolarNodeOS customizations."This section will outline how you can supply custom SolarNode plugin configuration that is applied when SolarNode first starts up. This allows you to have components configured with settings that you know in advance will be used by your nodes. For example, you might know that your nodes will be integrated with a modbus-based temperature sensor via a specific serial port.
SolarNode plugins with configurable properties, called settings, can be set via the SolarNode setup
web application, using web forms to configure the settings. Those settings can also be configured
via "auto settings" when SolarNode starts up, by placing CSV files using a specific format in the
/etc/solarnode/auto-settings.d directory. See the SolarNode Settings page
for more details. Auto settings files can then be incorporated into the SolarNodeOS customization
process, for example by including them in a custom package.
Here's an example auto settings file that could be saved as
/etc/solarnode/auto-settings.d/db-sensor.csv and would serve to configure the following:
- A Modbus serial port
/dev/ttyS2named db Sensor Port. - A Modbus Device data source using the modbus port named db Sensor Port that
collects a datum stream
/DB/1from the default unit ID (1) with a single propertyloudnessread from the default register (0) as a 16-bit unsigned integer value multiplied by0.1.
net.solarnetwork.node.io.modbus.1,serialParams.portName,/dev/ttyS2,0,2020-08-27 00:00:00
net.solarnetwork.node.io.modbus.1,uid,db Sensor Port,0,2020-08-27 00:00:00
net.solarnetwork.node.io.modbus.FACTORY,1,1,0,2020-08-27 00:00:00
net.solarnetwork.node.datum.modbus.1,jobService.datumDataSource.modbusNetwork.propertyFilters['uid'],db Sensor Port,0,2020-08-27 00:00:00
net.solarnetwork.node.datum.modbus.1,jobService.datumDataSource.sourceId,/DB/1,0,2020-08-27 00:00:00
net.solarnetwork.node.datum.modbus.1,jobService.datumDataSource.propConfigsCount,1,0,2020-08-27 00:00:00
net.solarnetwork.node.datum.modbus.1,jobService.datumDataSource.propConfigs[0].name,loudness,0,2020-08-27 00:00:00
net.solarnetwork.node.datum.modbus.1,jobService.datumDataSource.propConfigs[0].dataTypeKey,u16,0,2020-08-27 00:00:00
net.solarnetwork.node.datum.modbus.1,jobService.datumDataSource.propConfigs[0].unitMultiplier,0.1,0,2020-08-27 00:00:00
net.solarnetwork.node.datum.modbus.FACTORY,1,1,0,2020-08-27 00:00:00
It isn't exactly obvious how to construct a settings CSV file. Often the easiest way to see the format is to use the SolarNode setup app to configure the components exactly the way you need them, and then export the SolarNode settings. That will generate a CSV file of all SolarNode settings, from which you can extract out just the relevant lines. If you're not sure which lines are relevant, then save a copy of the exported settings before you configure your components. Then after you configure your components and export the settings again, you can compare the difference between the before/after settings files and see what lines were added in the "after" settings file. The added lines are the only ones you need to put in your auto settings file.
SolarNode includes a service to perform backups of the node's configuration. By default this service creates backup archive files on the node itself. You can instead install the S3 Backup plugin to enable performing backups to any S3-compatible service. This makes for a robust backup solution where a failed node can be brought back to life by starting from a new, unassociated SolarNodeOS image and restoring the failed node's backup directly from S3.
S3 Backup requires some configuration to set up in SolarNode, so the recommended way to achieve this is to create another custom OS package with the relevant files so SolarNode can start up with S3 Backup all ready to go from the start.
The /etc/solarnode/services/net.solarnetwork.node.backup.s3.S3BackupService.cfg file can be used
to configure the S3 Backup settings. See the full plugin documentation for
details, but here's an example file to give you a good idea what is required:
accessToken = 123abc
accessSecret = 234bcd
regionName = us-east-1
bucketName = mybucket
storageClass = STANDARD_IA
objectKeyPrefix = solarnode-backups/basic/
Finally the /etc/solarnode/services/net.solarnetwork.node.backup.DefaultBackupManager.cfg file
can be used to make S3 Backup the default backup service. The contents of this file should be:
preferredBackupServiceKey = net.solarnetwork.node.backup.s3.S3BackupService
⚠️ Note that all nodes that include the same S3 Backup configuration will have access to all backups created by itself and all other nodes sharing the same configuration. You must use differentbucketNameand/orobjectKeyPrefixvalues to partition sets of nodes from one another, if needed.
You can create another custom package with the configuration files listed in the previous sections,
for example a package named my-solarnode-s3-backup could be created with a Makefile like:
NAME = my-solarnode-s3-backup
VERSION = 1.0.0-1
PATHS = etc
deb :
fpm \
--input-type dir \
--output-type deb \
--name $(NAME) \
--version $(VERSION) \
--architecture all \
--maintainer 'Me <me@localhost>' \
--vendor 'Me' \
--description 'My SolarNode S3 Backup configuration' \
--license 'Proprietary' \
--force \
--deb-no-default-config-files \
$(PATHS)
where you had the configuration files arranged in an etc directory, like:
.
├── Makefile
└── etc
└── solarnode
└── services
├── net.solarnetwork.node.backup.DefaultBackupManager.cfg
└── net.solarnetwork.node.backup.s3.S3BackupService.cfg
Once you create this package, you can integrate the package into your custom SolarNodeOS image in the same way as outlined previously.
When you have to manage more than just a few nodes, then keeping all the nodes updated with the same set of software can be time consuming. The S3 Setup SolarNode plugin can help here: it provides a way to define versioned "setup packages" that consist of SolarNode plugins, configuration, or most helpful OS packages. The configuration and resources are stored on S3 (or compatible service) and nodes can be configured to
- automatically download and install the newest setup package version the first time the node starts up
- respond to UpdatePlatform SolarNetwork instructions to download and install specific setup package versions
Conceptually, you can think of a setup package as a set of resources to be installed together.
For example, you could define a version 1 setup package to contain:
- the
solarnode-baseOS package version1.8.1 - the
solarnode-app-coreOS package version1.14.0 - the
my-solarnode-app-coreOS package version1.0.0
When a new node configured to use S3 Setup starts up the first time, it would find that setup
package and download/install the 3 OS packages specified there. If a node was sent an
UpdatePlatform instruction with a Version=1 parameter, it would likewise download/install those
same 3 OS packages.
S3 Setup requires S3 Backup to also be configured, and it shares the S3 credentials defined there.
The S3 object key prefix setting required, which can be set in the
/etc/solarnode/services/net.solarnetwork.node.setup.s3.S3SetupManager.cfg file. For example:
objectKeyPrefix = solarnode-setup/basic/
This defines the S3 path solarnode-setup/basic/ as the root of where S3 Setup metadata files and
resource files will be stored. The metadata files are created in a setup-meta/ sub-path, and the
names must be numbers that define the setup package version, with a .json suffix . It can be
convenient to include leading zeros in the names for sorting purposes. The content is a JSON object
that defines all attributes of the package. See the S3 Setup configuration and
metadata structure docs for more details.
Here is an example version 1 metadata file solarnode-setup/basic/setup-meta/000001.json, that
installs the my-solarnode-app-core OS package available in S3 and the solarnode-base and
solarnode-app-core OS packages from OS-configured package repositories:
{
"cleanPaths":[
"{osgi.configuration.area}/config.ini"
],
"objects":[
"solarnode-setup/basic/setup-data/my-solarnode-app-core_1.0.0-1_all.deb"
],
"packages":[
{"action":"Install","name":"solarnode-base","version":"1.8.1_1"},
{"action":"Install","name":"solarnode-app-core","version":"1.14.0_1"}
],
"restartRequired":true
}If you have a custom OS package repository configured, then all the OS packages could be installed from repositories like:
{
"cleanPaths":[
"{osgi.configuration.area}/config.ini"
],
"packages":[
{"action":"Install","name":"solarnode-base","version":"1.8.1_1"},
{"action":"Install","name":"solarnode-app-core","version":"1.14.0_1"},
{"action":"Install","name":"my-solarnode-app-core","version":"1.0.0_1"}
],
"restartRequired":true
}S3 Setup also supports upgrading all OS packages from the configured OS package repositories by
using the Upgrade action, like this:
{
"cleanPaths":[
"{osgi.configuration.area}/config.ini"
],
"packages":[
{"action":"Upgrade"}
],
"restartRequired":true
}