[illumos-Developer] networking question - local address when virtual IP aliases exist?
danmcd at nexenta.com
Mon Jun 27 11:52:15 PDT 2011
On Mon, Jun 27, 2011 at 02:28:47PM -0400, Gordon Ross wrote:
> I'm seeing something odd when a system has virtual NICs, i.e. an "IP alias"
> in addition to the system's primary IP address. This situation often exists
> with cluster setups where all cluster nodes have a "physical" IP that never
> changes or moves, plus some set of "virtual" IP addresses that may move
> to any physical host based on cluster fail-over logic, etc.
> I don't have all the details yet, but it appears that outbound connections
> are sometimes using the "physical" IP address as the local address,
> and other times using a "virtual" IP as the local address. Who cares?
> Well, when the SMB service makes a connection to an AD server, the
> AD server appears to decide we're an "impostor" trying to use some
> other machine's AD account, and denies our logon request.
> So... Other than making the application bind the local endpoint using
> an explicit IP address, is there any way to mark some interfaces as
> more or less preferred for anonymous (local) IP bind requests?
You can use IFF_DEPRECATED on the less-important addresses.
points to the source-address selection for IPv4. This is where these
addresses get decided. Look at the comment block:
* Pick a source address based on the destination ill and an optional setsrc
* The result is stored in srcp. If generation is set, then put the source
* generation number there before we look for the source address (to avoid
* missing changes in the set of source addresses.
* If flagsp is set, then us it to pass back ipif_flags.
* If the caller wants to cache the returned source address and detect when
* that might be stale, the caller should pass in a generation argument,
* which the caller can later compare against ips_src_generation
* The precedence order for selecting an IPv4 source address is:
* - RTF_SETSRC on the offlink ire always wins.
* - If usrsrc is set, swap the ill to be the usesrc one.
* - If IPMP is used on the ill, select a random address from the most
* preferred ones below:
* 1. If onlink destination, same subnet and not deprecated, not ALL_ZONES
* 2. Not deprecated, not ALL_ZONES
* 3. If onlink destination, same subnet and not deprecated, ALL_ZONES
* 4. Not deprecated, ALL_ZONES
* 5. If onlink destination, same subnet and deprecated
* 6. Deprecated.
* We have lower preference for ALL_ZONES IP addresses,
* as they pose problems with unlabeled destinations.
* Note that when multiple IP addresses match e.g., #1 we pick
* the first one if IPMP is not in use. With IPMP we randomize.
So there's factors to consider:
- Any ALL_ZONES IP addresses in a zoned system?
- Any use of IPMP?
- As you pointed on IRC, is the desired local address the first one
in a list (e.g. e1000g0:0)?
Someone else may need to chime in here in case I'm missing something, but
please watch out for those. A simple test running connect() with unspecified
local addresses and differing remotes can help shake things out as well.
More information about the Developer