External DNS needs to support multiple external addresses

[bnaecker]

External DNS operates on exactly one external IPv4 address today. We need to support multiple external addresses, to allow dual-stack and IPv6-only services. This includes the actual binary, the SMF service, and sled-agent populating the SMF properties required.

[bnaecker]

This program gets its addresses through SMF properties, e.g., here:

omicron/smf/external-dns/manifest.xml

Line 31 in c0f4725

<propval name='dns_address' type='astring' value='unknown' />

which are set by the sled-agent here:

omicron/sled-agent/src/services.rs

Lines 2053 to 2057 in c0f4725

	.add_property(
	"dns_address",
	"astring",
	private_dns_address,
	);

We can pass multiple values for an SMF property, but I'm not sure how those show up to the binary. E.g., the shell may separate them on whitespace, so that --dns-address A B shows up in the argv as --dns-address, A and B. That would break things. We could work around that by comma-separating the values, like A,B, and then handling that in the binary. Not sure what's best yet.

[bnaecker]

I wrote a small SMF service and binary to test this. The summary is that, AFAICT, setting multiple properties with something like svccfg addpropvalue passes the multiple arguments directly to the start method unquoted. I.e., we get "A" "B" as two arguments, rather than "A B" as one. I think that means we need to go the second route above, passing a delimited list as a single SMF property and having the binary split and parse that.

An example is below:

SMF manifest

<?xml version="1.0"?>
<!DOCTYPE service_bundle SYSTEM "/usr/share/lib/xml/dtd/service_bundle.dtd.1">

<service_bundle type='manifest' name='smf-test'>

  <service name='test/smf_test' type='service' version='1'>
  <create_default_instance enabled='true' />

  <exec_method type='method' name='start'
    exec='ctrun -l child -o noorphan,regent /home/bnaecker/smf-test/target/release/smf-test --dns-address %{config/dns_address} &amp;'
    timeout_seconds='0' />
  <exec_method type='method' name='stop' exec=':kill' timeout_seconds='0' />

  <property_group name='config' type='application'>
    <propval name='dns_address' type='astring' value='unknown' />
  </property_group>

  <property_group name='startd' type='framework'>
    <propval name='duration' type='astring' value='contract' />
  </property_group>

  <stability value='Unstable' />

  <template>
    <common_name>
      <loctext xml:lang='C'>SMF Test</loctext>
    </common_name>
    <description>
      <loctext xml:lang='C'>A dumb SMF test</loctext>
    </description>
  </template>
</service>

</service_bundle>

Rust binary

use std::net::IpAddr;

use anyhow::Context as _;
use clap::Parser as _;

#[derive(Debug, Clone)]
struct Addrs(Vec<IpAddr>);

impl std::str::FromStr for Addrs {
    type Err = anyhow::Error;

    fn from_str(input: &str) -> anyhow::Result<Self> {
        input
            .split(',')
            .map(|s| s.parse().with_context(|| format!("parsing {s} as IP")))
            .collect::<Result<Vec<_>, _>>()
            .map(Self)
    }
}

#[derive(Debug, clap::Parser)]
struct Args {
    #[arg(long)]
    dns_address: Addrs,
}

fn main() -> anyhow::Result<()> {
    let args = Args::try_parse()?;
    println!("{args:#?}");
    Ok(())
}

Then importing the configuration and setting the property with:

$ svccfg import manifest.xml
$ svccfg -s smf_test:default setprop config/dns_address = "fd00::1,fd00::2"
$ svccfg -s smf_test:default refresh

Shows this in the log file:

bnaecker@obsidian : ~/smf-test $ tail -15 /var/svc/log/test-smf_test:default.log
[ Aug 21 04:16:25 Stopping because all processes in service exited. ]
[ Aug 21 04:16:25 Executing stop method (:kill). ]
[ Aug 21 04:16:25 Executing start method ("ctrun -l child -o noorphan,regent /home/bnaecker/smf-test/target/release/smf-test --dns-address fd00::1,fd00::2 &"). ]
[ Aug 21 04:16:25 Method "start" exited with status 0. ]
Args {
    dns_address: Addrs(
        [
            fd00::1,
            fd00::2,
        ],
    ),
}
[ Aug 21 04:16:25 Stopping because all processes in service exited. ]
[ Aug 21 04:16:25 Executing stop method (:kill). ]
[ Aug 21 04:16:25 Restarting too quickly, changing state to maintenance. ]