CCIE Security Notes: ASA Clustering (9.6.1)

Unsupported features with Clustering -

Unified communications that reply on TLS proxy
RA VPN (SSL VPN and Ipsec VPN)
Following application inspections:
- CTIQBE
- H323, H225, and RAS
- IPSec passthrough
- MGCP
- MMP
- RTSP
- SCCP
- WAAS
- WCCP
Botnet traffic filter
Auto Update Server
DHCP client, server and proxy - relay is supported though
VPN load balancing
Failover
ASA CX Module

Features only supported through centralized model on the Primary ASA only -

Site-to-site VPN
Following application inspections:
- DCERPC
- ESMTP
- IM
- NetBIOS
- PPTP
- RADIUS
- RSH
- SNMP
- SQLNET
- SUNRPC
- TFTP
- XDMCP
Dynamic routing (Spanned EtherChannel mode only)
Multicast routing (Individual Interface mode only)
Static route monitoring
IGMP multicast control plane protocol processing (data plan forwarding still distributed across the cluster)
PIM multicast control plane protocol processing (data plane forwarding still distributed across the cluster)
Authentication and Authorization for network access - Accounting is decentralized
Filtering Services

Features Applied to Individual ASAs -

QoS - Synced across the cluster as part of configuration replication but it's enforced on each ASA independently
Threat Detection - Works on each ASA independently
Resource management - This is for multicontext mode and enforced on each ASA separately
LISP Traffic - Each ASA adds to the EID table that's shared across the cluster but it doesn't participate in cluster state sharing
ASA Firepower module - There's no config or state sync between the ASA firepower modules when clustering. Don't use different ASA-interface-based zone definitions on the devices in the cluster
ASA IPS modules - No state or config sync between them

Other features and caveats -

AAA for network access - Authentication and authorization is centralized, accounting is distributed on a per-flow basis
FTP - If you use AAA for FTP access, it's centralized on the primary ASA. If not, the data channel and control channel can be owned by different members of the cluster.
Identity Firewall - Only the primary ASA retrieves user-group from the AD and user-ip mapping from the AD agent but it will populate that information to secondaries
Multicast Routing - Depends on the interface mode:
- Spanned EtherChannel Mode - Primary ASA handles all multicast routing packets and data packets until fast-path forward is established and then each secondary can forward multicast data packets
- Individual Interfaces Mode - All data and routing packets are processed and forwarded by the primary ASA to avoid packet replication
NAT - This can be a performance hit. If you're going to use NAT and returning packets come back to a non-owner, that packet is going to have to be forwarded to the owner over the cluster control link. Important NAT guidelines:
- No Proxy ARP - Not much of an issue with Spanned EtherChannel Mode but with Individual Interface mode, you have to consider that the reply is never set for mapped addresses. It prevents the adjacent router from maintaining a peer relationship with an ASA that may not be in the cluster anymore. Upstream router needs a static route or PBR with object tracking for the mapped addresses that points to the Main cluster IP address.
- Interface PAT is not supported for Individual Interface mode
- No PAT with port block allocation is not supported for the cluster
- PAT with Port Block Allocation
  - Maximum-per-host limit is not a cluster-wide limit and enforced on each ASA individually
  - Port blocks created on the backup ASA from the backup pools are not counted when enforcing the maximum-per-host limit
  - When PAT IP address owner goes down, backup owner will own the PAT IP address, corresponding port blocks and xlates but it won't use these to service new request. As the connections time out, the blocks get freed
  - On-the-fly PAT modifications will result in xlate backup creation failures for the xlate backup requests that were still in transit while the new pool became effective.
- NAT pool address distribution for dynamic PAT - the primary ASA will evenly pre-distribute addresses across the cluster and if a member receives a connection and has no addresses left, the connection is dropped whether or not other members still have addresses available.
- No round-robin for PAT with clustering
- Dynamic NAT xlate is managed by the primary ASA and it'll replicate the xlate table to the secondary ASAs.
- Per-session PAT allows each secondary to own the PAT connections. Some traffic will not support this such as ICMP
- No static PAT for the following inspections:
  - FTP
  - PPTP
  - RSH
  - SQLNET
  - TFTP
  - XDMCP
  - SIP
SIP Inspection - The control flow can be created on any of the ASA but it's child data flows must reside on the same ASA. TLS proxy configuration isn't supported.
SNMP - SNMP agents poll each individual ASA by it's Local IP address. Can't poll consolidated data for the cluster. If the SNMP agent polls the Main Cluster IP and a new primary ASA is elected, the poll to the new primary ASA will fail.
STUN - Supported in failover and cluster modes as pinholes are replicated but transaction ID is not replicated among the ASAs. If an ASA fails after receiving a STUN Request and another ASA receives it, the STUN request will be dropped.
Syslog and Netflow - Each ASA in the cluster generates it's own syslog messages and can be configured to use the same or a different device ID in the syslog message header. With Netflow, each cluster also generates it's own Netflow stream and the collector will treat each ASA as a separate Netflow exporter.
TrustSec - The primary ASAs learns the SGT and then populages that information to the secondary ASAs so they can make match decisions on their own
VPN - Site-to-site VPN is a centralized feature and only the primary will support it. If you're using Spanned EtherChannel, the connections are automatically going to be forwarded to the primary ASA.
VXLAN is not supported on Individual Interfaces but it is supported on Spanned EtherChannel.
IS-IS is not supported in Spanned EtherChannel but it is with Individual Interfaces.

Primary and Secondary Roles -

In any cluster, there is going to be one single primary ASA and the rest of the ASAs will be secondary
Primary is determined by priority which can be set between 1 and 100. 1 is the highest priority.
If nothing is altered with priority from their default, it's the first ASA that comes up that will be the primary
Configurations are done on the primary ASA only and replicated to other ASAs after initial setup
Election of primary:
- Initially, the ASA that is newly configured for clustering will broadcast an election every 3 seconds
- ASAs with a higher priority will respond to that election request.
- If there isn't a response from any other ASAs within 45 seconds, that ASA will become the primary. Note: If a member joins with a higher priority later, it doesn't necessarily become the primary until the primary stops responding or you manually force the ASA to become primary
When an ASA in a cluster fails, the traffic is transferred to other ASAs. If the primary fails, the ASA with the second highest priority becomes the primary. Depending on the type of failure, the ASA that failed may automatically try to rejoin.

Management -

Management interface - It's recommended to use the dedicated management interfaces for the ASAs as individual interfaces. This allows you to connect to the individual ASA for certain things such as TFTP, get syslog, etc.
Primary - Management and monitoring for the entire cluster usually takes place on the primary. You can also do file management on secondary ASAs from the primary if you want. The features that aren't available from the primary are:
- Monitoring per-unit cluster-specific states
- Syslog monitoring per unit with the exception of syslogs sent to the console when console replication is enable
- SNMP
- Netflow

Interfaces -

Can only configure one type of interface mode for clustering: Spanned EtherChannel Mode or Individual Interface Mode. You can't mix interface types in a cluster.
Spanned EtherChannel Mode - This is the recommended mode to use. You can group one or more interfaces per an ASA into a bundle that spans across all the units in the cluster. You can use Spanned EtherChannel in both routed or transparent mode. With transparent mode, you have to assign the IP address to the BVI, not a bridge group member interface.
Individual Interface Mode - This is only available in routed mode as these are routed interfaces with their own local IP address. This is less preferrable over port channels since this method requires routing protocols to load balance the traffic and convergence is slower. Since you have to configure everything from the primary ASA, you can configure a pool of IP addresses to be used on a given interface on the cluster members and one for the primary. The primary would have a Main Cluster IP address which would be a fixed address for the cluster that always belongs to the primary and acts as a secondary IP address for that primary. The local IP address is what is used as the address for routing while the Main Cluster IP address provides an IP address for consistent management access - that way if the primary changes, the Main Cluster IP will just move along with it.
Cluster Control Link (CCL) - Controls both control traffic for primary election, configuration replication, and health monitoring and data traffic for state replication and connection ownership queries and data packet forwarding. Needs to be layer 2 adjacent and have an IP on the same subnet. You want to make sure that the links you use for the CCL can handle a LOT of traffic. The amount of traffic sent on it can vary but it definitely can spike. Another thing to know is that a higher bandwidth CCL will help the cluster converge faster when changes are made. The best practice is to use a port channel for the CCL and connect it to other switches that utilize multi-chassis port channels switch as switches that use VSS or vPC and remember that the port channel on the ASA is not spanned between the other members since this is the ASA's cluster control link and has to be treated somewhat unique per ASA. There's a few caveats for the type of link you can use for the CCL:
- Can't use VLAN subinterface
- Can't use management interface
Other things to be aware of with the CCL:
- Need to have a RTT of 20ms or less for latency
- Can't have any out-of-order or dropped packets
If CCL goes down for the ASA, clustering is disabled and data interfaces shut down. Manually rejoin the cluster and re-enable clustering to bring it up. The management interface will remain up during this time

With clustering, you have the following available options for load balancing -

Spanned EtherChannel - It's not only the recommended way with clustering but it also provides faster convergence and failure discovery. Recommended to use the same kind of line cards in the bundle and the use the same load balancing hash algorithm on both sides of the bundle.
PBR from upstream and downstream routers to the routed firewalls - Recommended if you're already using PBR. It'll make decisions based on the route-map and an ACL. This is a more static option. It's recommended that you create a PBR policy that forwards and returns packets of a connection to the same physical ASA.
ECMP routing between the upstream and downstream routers to the routed firewalls - This is when you're using individual interfaces. IT can be slower to converge though that just using port channels. Definitely not recommended to use static routing as failures can cause issues when this happens unless you're using object tracking. Dynamic routing protocols are better overall.

Replication -

Data Path Connection State - Connections have one owner and backup owner in the cluster. The backup owner only takes over if the owner has a failure by looking at the TCP/UDP state information and the connection is transferred there.
Note: SNMP ID and site-to-site VPN state information is not replicated. Features replicated across cluster:
- Up time
- ARP table (Transparent only)
- MAC address table (Transparent only)
- User identity (AAA rules for uauth and identity firewall)
- IPv6 neighbor database
- Dynamic routing
Configuration - All the ASAs in a cluster will share a single configuration and you can only make changes to this configuration on the primary. After the change is made, it's replicated across the cluster.
RSA Key Replication - When you create a RSA key on the primary unit, it's replicated to the other units

Health Monitoring for the Cluster -

ASA Health - The primary sends keepalives to every secondary over the CCL at an interval to make sure they're up. If it fails that keepalive health check, the primary will remove it from the cluster
Interface - Each ASA monitors the link status of their interfaces and reports it to the primary. When health monitoring is enabled, it's enabled for all of the interfaces by default but this can be adjusted and you can turn it off per interface. If a monitored interface goes down, that member is removed from the cluster so this is probably one of those things you want to control a bit.

ASA Connection Management -

This management is important in load balancing to members of the cluster and how the connections are handled once received
Can be 3 different ASA roles defined for the connections:
- Owner - Initial ASA that receives the connection & maintains the TCP state. A single connection should only have one owner. ASA owns both directions of the connection. To have the best performance, ensure that your ASA receives both directions of the flow.
- Director - This ASA handles the owner lookup requests from forwarders and maintains the connection state. It'll be the backup if the owner fails. This is not a static assignment - the owner ASA will choose a director based on a hack of the source/dest IP address and TCP ports and will send a message to the new director. There is only one director per connection
- Forwarder - This ASA might receive a packet for a connection and is neither the director of the owner. Instead, it'll forward the packets director who forwards the flow to the owner. Sometimes the SYN packets can have the owner in it so it's not a hard and fast rule that the ASA will forward always to the director - if it's somehow able to derive the owner from the packets, it'll directly forward them to the owner. The best throughput for clustering is accomplished through load-balancing methods where all packets of a connection are received by an owner.
Overloaded units can be configured to redirect new TCP flows to other units
PAT can influence who becomes the member of a new connection:
- Per-session PAT - Owner is the ASA that receives the initial connection
- Multi-session PAT - Owner is always the primary ASA. You can change the per-session PAT defaults for TCP/UDP so connects are handled per-session or multi-session depending on the config. ICMP cannot be changed from the default multi-session PAT.

Dynamic Routing and Clustering -

Spanned EtherChannel Mode -
- IS-IS not supported
- Routing protocol only run on the primary ASA and routes are replicated to the secondary ASAs
- Routing packets arriving to the secondary ASAs are redirected to the primary ASA
- After routes are learned from the primary ASA, the individual ASAs will make their forwarding decisions independently
- OSPF LSA database is not synchronized from the primary unit to the secondary units so if there is a primary ASA switchover, the neighboring router will detect a restart - this is not a transparent failover. You would want to look into non-stop forwarding to address this.
Individual Interface Mode-
- Each ASA runs the routing protocol as a standalone router and the routes are learned independently
- You should configure a cluster pool for the router IDs so that each unit has a separate router ID
- To avoid asymmetric routing and possible traffic loss, group all the ASA interfaces into the same traffic zone

Inter-site Clustering -

The ASA clusters can be utilized for inter-site clusters but you have to make sure that the DCI can accommodate the bandwidth for the cluster control link. Depending on the amount of ASAs and the sites, this can be an important design consideration. The calculation should be something along these lines:

For this, you can configure each cluster member to belong to a separate site ID which will work with site-specific MAC addresses and IP addresses. The packets that will be sourced from the cluster will use the site-specific MAC and IP while packets received by the cluster will use a global MAC and IP address to prevent the switches from learning the same MAC address from both sites on different ports and cause MAC flapping.
Flow mobility can be enabled using LISP inspection with the Site IDs:
- VMware may migrate between sites and to support this LISP can update this to make sure that the traffic is being send to the right site.
- The ASA won't run LISP itself but it can inspect LISP to monitor for location changes and use this information for the clustering operation so the traffic and flows don't go to the wrong cluster member in a different site than the ASA that's closest.
- In order for this to work, the ASA cluster members have to reside between the first hop router and the ITR or ETR for the site and the ASA cluster cannot be the first hop router for an extended segment. This applies to only fully-distributed flows and the cluster only moves layer 3 and 4 flow states. Short-lived or non-critical flows might not be worth moving and it can be adjusted so you're not wasting resources on this.
- ASA inspects LISP traffic on UDP port 4342 for EID-notify messages sent from the first hop router and the ITR or ETR.
When using individual interface mode and ECMP towards a multicast RP, it's recommended to use a static route for the RP IP address next hop since it's prevents sending unicast PIM register packets to secondary units that might drop it.
Not recommended to configure connection rebalancing since this might rebalance members at different sites
There are some interesting caveats for transparent mode:
- If cluster is between a pair of inside and outside routers, ensure that both inside routers share a MAC address and both outside routers share a MAC address.
- If placed between data networks for east-west traffic, each gateway router should use a FHRP to provide identical virtual IP and MAC address destinations at each site.

Failures -

If an ASA leaves the cluster and fails to rejoin, it's data interfaces will remain down but it's management interface will remain up and able to communicate
Depending on how it's removed from the cluster, it can be rejoined. Here's the behavior based on failure:
- CCL down upon initial join- Needs to be rejoined manually with cluster group name and then enable
- Failed CCL after joining - ASA automatically tried to rejoin every 5 minutes by default unless configured otherwise
- Failed data interface - ASA automatically tries to rejoin at 5 minutes, then 10 minutes, and then 20 minutes. If it's not able to rejoin in that time, you'll have to manually rejoin with cluster group name and then enable
- Failed ASA due to health check - It should rejoin when power back up or the failure is resolved. It'll try to rejoin the cluster every 5 seconds automatically at that point.

Recommendations for clustering-

For the cluster control link interfaces on the switch side, enable Spanning Tree PortFast to speed up the join process with new units
For Spanned EtherChannel, you can enable LACP rate fast for the individual interfaces on the switches to speed up the bundling
On the switch, it's recommended to use source-dest-ip or source-dest-ip-port algorithm and do not use the vlan keyword.
Port-channel bundling downtime shouldn't except the configured keepalive interval if you can avoid it
When topology changes occur where you find yourself adding or removing an EtherChannel interface, disable the health check feature and disable interface monitoring for those disabled interfaces while you're working on it.

Defaults for Clustering -

With Spanned EtherChannels, cLACP system ID is auto-generated with system priority of 1
Cluster health check enabled by default with holdtime of 3 seconds. Interface health monitoring is enabled on all interfaces
Cluster auto-rejoin feature for failed cluster control link is unlimited attempts every 5 minutes.
Cluster auto-rejoin features for failed data interface is 3 attempts every 5 minutes with increasing interval set to 2
Connection replication delay of 5 seconds is enabled by default for HTTP traffic

In order for clustering to be set up -

ASAs must be the same model with the same amount of DRAM
Must have identical software at the time of the image upgrade but hitless upgrading is supported
Must be the same security context mode (single or multiple)
Must be same firewall mode if it's in single context mode - either routed or transparent
New members must use the same SSL encryption settings
Cluster control link - which is an isolated high-speed backplane network
Management access to each ASA which is essential for all the configuration and monitoring
Cluster control link will only support IPv4

Configuring Clustering -

To view any incompatible configuration so you can fix the configuration, use the following command:
cluster interface-mode {individual | spanned} check-details
Configure the interface mode first. There's no default for this, it has to be explicitly chosen:
cluster interface-mode {individual | spanned} force
You can also remove the interface mode with the no cluster interface-mode command.
Configure the interfaces on the primary:
- Individual Interface Mode - These are normal routed interfaces taken from a pool of IP addresses while the Main cluster IP address is a fixed IP that belongs to the cluster primary. If you have multiple contexts, you'll need to configure each context as you're going.
  - Configure a pool of Local IP addresses:
    IPv4: ip local pool poolname first-address last-address [mask mask]
    IPv6: ipv6 local pool poolname ipv6-address/prefix-length number-of-addresses
  - Or enter interface config mode and configure the nameif, ip address, security level, etc and configure the cluster IP with cluster-pool there:
    interface interface
    (optional) management-only
    nameif name
    security-level num
    IPv4: ip address ip-address [mask] cluster-pool poolname
    IPv6: ipv6 address ipv6-address/prefix-length cluster-pool poolname
    no shutdown
- Spanning EtherChannel Mode - If this is in multicontext mode, you'll have to configure it in every context. If you're in transparent mode, you'll have to configure the bridge group. It's not recommended to configure the maximum or minimum links in the EtherChannel on either the ASA or switch unless you absolutely need to. It's not recommended to change the load-balancing algorithm from the default on the ASA. On the switch, it's recommended to change it on the switch to source-dest-ip or source-dest-ip-port. The lacp port-priority and lacp system-priority commands are not used for Spanned EtherChannel.
  - Configure the interface you want to add to the channel group:
    interface interface
    channel-group id mode active [vss-id {1|2}] <- The id is between 1 and 48. If you haven't configured it yet, it'll be created when you configure it on the interface.
    no shutdown
  - Go to the port channel and configure it as a Spanned EtherChannel:
    interface port-channel id
    port-channel span-cluster [vss-load-balance] - If you're connecting it to multiple switches in VSS or vPC, enable VSS load balancing using the vss-load-balance keyword. You must configure the vss-id keyword in the channel-group command for each member before enabling load balancing.
    Note: Setting ethernet properties on the port channel will override the properties set on the individual interfaces. You can also create VLAN subinterfaces on the port channel.
  - If you're in multicontext mode, make sure to allocate the interface to a context:
    context name
    allocate-interface port-channel id
  - Configure the interface:
    - Routed mode:
      interface port-channel id
      nameif name
      security-level num
      IPv4: ip address ip-address [mask]
      IPv6: ipv6 address ipv6-prefix/prefix-length
    - Transparent mode:
      interface port-channel id
      nameif name
      security-level num
      bridge-group num
      Note: The bridge group can be between 1 and 100. You can have up to 64 interfaces per bridge group.
  - Configure a global MAC address for the Spanned EtherChannel:
    mac-address mac-address
    This MAC address stays with the current primary ASA. If you're using multi-context, you should enable auto-generation of MAC address.
  - For inter-site clustering, configure a site-specific MAC address and IP address for each site:
    mac-address mac-address site-id num
Configure the Primary ASA bootstrap settings:
- Note: If you are in multicontext mode, you want to configure the settings in the system space.
- It's recommended to enable jumbo frame reservation for use with the cluster control link
- You have to be consoled into the ASA to enable clustering - you can't do it remotely.
- Before you join the cluster, enable the cluster control link. Some things to note: It must be enabled separately on each ASA since this configuration isn't replicated. You cannot use a VLAN subinterface or the management interface as the Cluster Control Link.
  interface interface
  channel-group id mode on - This is optional if you're using a port channel for your CCL. If you do, you would configure the rest under the port channel interface.
- Optionally, if you would like to configure it past the default of 1500. It's recommended to be 1600 bytes or more.
  jumbo-frame reservation
  mtu cluster bytes
- This names the cluster group and enters cluster configuration mode
  cluster group name
  - Name the member of the cluster
    local-unit unit-name
  - Specify the cluster control link interface. The interface can't have a nameif configured on it.
    cluster-interface interface ip ip-address
  - If it's an inter-site cluster, set the site ID:
    site-id number
  - Set the priority for the ASA for primary unit elections:
    priority num
  - Optionally, you can configure an authentication key
    key key
  - Optionally, you can configure dynamic port priority in LACP
    clacp static-port-priority
  - Optionally, you can manually specify the cLACP system ID and system priority:
    clacp system-mac {mac-address | auto} [system-priority num]
  - Enable the cluster:
    enable [noconfirm]
Configure the secondary ASA bootstrap settings:
- Configure the same cluster control link interface as you did on the primary.
- Specify the same MTU you configured on the primary
- Configure the same cluster name as before:
  cluster group name
  - Configure the local unit name with a unique setting per ASA:
    local-unit name
  - Specify the same CCL but with a different IP address as the primary:
    cluster-interface interface ip ip-address mask
  - Set the site ID:
    site-id num
  - Set the priority:
    priority num
  - Set the optional key:
    key key
  - Enable clustering as a slave if you want it to join as a secondary ASA:
    enable as-slave

Cluster operation configuration -

Cluster parameters inside cluster configuration mode:
- console-replicate - If you want to enable console replication from secondary units to primary units. Disabled by default.
- trace-level level - Sets the minimum trace level for clustering events.
- health-check [holdtime timeout] [vss-enabled] - Customize the cluster unit health check feature. The vss-enabled knob allows the ASA to flood keepalive messages on all Etherchannel interfaces in the CCL to ensure that at least one of the switches can receive them.
- no health-check monitor-interface [interface | service-module] - Disable interface health check on that interface or module
- health-check {data-interface | cluster-interface} auto-rejoin {unlimited | rejoin-max} [auto-rejoin-interface [auto-rejoin-interval-variation]] - If you want to customize the auto-rejoin cluster settings after a health check failure. The number of attempts can be anyways between 0 and 65535. The default is unlimited and 3 for a data interface. The interval duration in minutes between rejoin attempts can be between 2 and 60 with the default being 5. The maximum total time that the unit attempts to rejoin the cluster is limited to 14400 minutes or 10 days from the time of the last failure. For the interval duration increases by setting, it can be between 1 and 3. The default value is 1 for the cluster interface and 2 for the data interface.
- conn-rebalance [frequency seconds] - Enable connection rebalancing for TCP traffic
cluster replication delay seconds {http | match tcp {host ip-address | ip-address mask | any | any4 | any6} [{eq | lt | gt} port] {host ip-address | ip-address mask | any | any4 | any6} [eq | lt | gt} port ]} - Enable cluster replication delay for TCP connections to cut back on the unnecessary work on short lived flows
To configure cluster flow mobility utilizing LISP traffic inspection:
- Create an ACL where the destination IP address is matched to the EID embedded address:
  access-list acl-name extended permit ip source-address mask dest-address mask
- Create a LISP inspection map:
  policy-map type inspect lisp map-name
  parameters
  allowed-eid access-list acl-name
  validate-key key - If needed
- Configure LISP inspection for UDP traffic between the first hop router and the ITR or ETR on port 4342:
  - Configure an extended ACL to identity LISP traffic:
    access-list inspect-acl-name extended permit udp source-address mask dest-address mask eq 4342
  - Create a class map for the ACL:
    class-map inspect-class-name
    match access-list inspect-acl-name
  - Specify the policy map, class map, enable inspection using optional LISP inspection map, and apply the service policy to the interface:
    policy-map policy-map-name
    class inspect-class-name
    inspect lisp inspect-map-name
    service-policy policy-map-name {global | interface interface}
- Enable flow mobility for the traffic class:
  - Configure the extended ACL to identify business critical traffic that you want to re-assign to the most optimal site when servers change sites:
    access-list flow-acl-name extended permit udp source-address mask dest-address mask eq port
  - Create a class map for the ACL
    class-map flow-map-name
    match access-list flow-acl-name
  - Specify the policy map on which you enabled LISP, the flow class map, and enable flow mobility:
    policy-map policy-map-name
    class flow-map-name
    cluster flow-mobility lisp
- Enter cluster group configuration mode and enable the flow mobility for the cluster:
  cluster group name
  flow-mobility lisp
You can disable a cluster member while leaving the whole configuration intact with the following:
cluster group name
no enable
To inactivate a member:
cluster remote unit unit-name
To leave the cluster and remove the entire cluster bootstrap configuration:
- For a secondary ASA, disable clustering since you can't make changes while clustering is enabled:
  cluster group name no enable
- Clear the cluster configuration:
  clear configure cluster
- Disable cluster interface mode:
  no cluster interface-mode
- write mem
Change the primary unit:
cluster master unit name
Execute a command cluster-wide:
cluster exec [unit name] command
Configure a logging ID for clustering on each device for syslog:
logging device-id

Other useful commands -

cluster exec capture - For troubleshooting, this enables a capture of cluster-specific traffic on the primary unit.
debug route cluster - Debug routing cluster information
debug lacp cluster [all | ccp | misc | protocol] - Shows debug messages for cLACP
debug cluster [ccp | datapath | fsm | general | hc | license | rpc | transport] - Shows debug messages for clustering
debug cluster flow-mobility - Shows events related to clustering flow mobiltiy
debug lisp eid-notifiy-intercept - Shows events when eid-notify message is intercepted.

Show commands -

show cluster info [health] - Shows the status of all the members in a cluster. With the health switch, it'll show the current health of interfaces, units, and the cluster overall
show cluster history - Shows the cluster history
show cluster {cpu | memory | resource} [options] - Displays aggregated data for the entire cluster on the resource or options you choose
show conn [detail] or cluster exec show conn - The first shows whether a flow is a directory, backup or forwarder flow. The second command will show all connections on any unit.
show cluster info [conn-distribution | packet-distribution | loadbalance | flow-mobility counters] - Shows traffic distributed across all cluster units and to help troubleshoot and adjust external load balancing
show cluster info loadbalance - Shows connection rebalance statistics
show cluster info flow-mobility counters - shows EID movement and flow owner movement information
show cluster {access-list | conn | traffic | user-identity | xlate} [options] - Displays aggregated data for the entire cluster
show asp cluster counter - Command is useful for datapath troubleshooting
show lisp eigp - Shows ASA EID table showing EIDs and site IDs
show route cluster - Shows cluster information for routing
show asp table classify domain inspect-lisp - Good troubleshooting command
show cluster interface-mode
show port-channel
show lacp cluster {system-mac | system-id} - Shows the cLACP system ID and priority
show interface - Should show the use of the site MAC address when in use
show cluster info trace - Shows the debug information for further troubleshooting