CONFIGURING NETWORK INTERFACE ADAPTERS
In our example, we have two NICs, one used as cluster interconnects and other as bootable adapter. The service IP address will be activated on the bootable adapter after cluster services are started on the nodes. Following IP addresses are used in the setup:
NODE1: hostname – btcppesrv5
Boot IP address - 10.73.70.155 btcppesrv5
Netmask - 255.255.254.0
Interconnect IP address - 192.168.73.100 btcppesrv5i
Netmask - 255.255.255.0
Service IP address - 10.73.68.222 btcppesrv5sv
Netmask - 255.255.254.0
NODE2: hostname – btcppesrv6
Boot IP address - 10.73.70.156 btcppesrv6
Netmask - 255.255.254.0
Interconnect IP address - 192.168.73.101 btcppesrv6i
Netmask - 255.255.255.0
Service IP address - 10.73.68.223 btcppesrv6sv
Netmask - 255.255.254.0
EDITING CONFIGURATION FILES FOR HACMP
1. /usr/sbin/cluster/netmon.cf – All the IP addresses present in the network need to be entered in this file. Refer to Appendix for sample file.
2. /usr/sbin/cluster/etc/clhosts – All the IP addresses present in the network need to be entered in this file. Refer to Appendix for sample file.
3. /usr/sbin/cluster/etc/rhosts - All the IP addresses present in the network need to be entered in this file. Refer to Appendix for sample file.
4. /.rhosts – All the IP addresses present in the network with username (i.e. root) need to be entered in this file. Refer to Appendix for sample file.
5. /etc/hosts – All the IP addresses with their IP labels present in network need to be entered in this file. Refer to Appendix for sample file.
Note: All the above mentioned files need to be configured on both the nodes of cluster.
CREATING CLUSTER USING SMIT
This is a sample HACMP configuration that might require customization for your environment. This section demonstrates how to configure two AIX nodes, btcppesrv5 and btcppesrv6, into a HACMP cluster.
1. Configure two AIX nodes to allow the user root to use the rcp and remsh commands between themselves without having to specify a password.
2. Log in as user root on AIX node btcppesrv5.
3. Enter the following command to create an HACMP cluster.
# smit hacmp
Perform the following steps. These instructions assume that you are using the graphical user interface to SMIT (that is, smit –M). If you are using the ASCII interface to SMIT (that is, smit –C), modify these instructions accordingly.
a) Click Initialization and Standard Configuration.
b) Click Configure an HACMP Cluster and Nodes.
c) In the Cluster Name field, enter netapp.
d) In the New Nodes (via selected communication paths) field, enter btcppesrv5 and btcppesrv6.
e) Click OK.
f) Click Done.
g) Click Cancel.
h) Select the Exit > Exit SMIT Menu option.
4. Enter the following command to configure the heartbeat networks as private networks.
# smit hacmp
Perform the following steps.
a) Click Extended Configuration.
b) Click Extended Topology Configuration.
c) Click Configure HACMP Networks.
d) Click Change/Show a Network in the HACMP cluster.
e) Select net_ether_01 (192.168.73.0/24).
f) In the Network Attribute field, select private.
g) Click OK.
h) Click Done.
i) Click Cancel.
j) Select the Exit > Exit SMIT Menu option.
5. Enter the following command to configure Service IP Labels/Addresses.
# smit hacmp
Perform the following steps.
a) Click Initialization and Standard Configuration.
b) Click Configure Resources to Make Highly Available.
c) Click Configure Service IP Labels / Addresses.
d) Click Add a Service IP Label / Address.
e) In the IP Label / Address field, enter btcppsrv5sv.
f) In the Network Name field, select net_ether_02 (10.73.70.0/23). The Service IP label will be activated on network interface 10.73.70.0/23 after cluster service starts.
g) Click OK.
h) Click Done.
i) Click Cancel.
j) Similarly follow steps d) to h) for adding second service IP label btcppsrv6sv.
k) Select the Exit > Exit SMIT Menu option.
6. Enter the following command to create Empty Resource Groups with Node priorities.
# smit hacmp
Perform the following steps.
a) Click Initialization and Standard Configuration.
b) Click Configure HACMP Resource Groups.
c) Click Add a Resource Group.
d) In the Resource Group Name field, enter RG1.
e) In the Participating Nodes (Default Node Priority) field, enter btcppesrv5 and btcppesrv6. The Resource Group RG1 will be online on btcppsrv5 first when cluster service starts; in the event of failure RG1 will be taken over by btcppesrv6 as the node priority for RG1 is assigned to btcppesrv5 first.
f) Click OK.
g) Click Done.
h) Click Cancel.
i) Similarly follow steps d) to h) for adding second Resource group RG2 with node priority first assigned to btcppesrv6.
j) Select the Exit > Exit SMIT Menu option.
7. Enter the following command to make Service IP labels part of Resource Groups.
# smit hacmp
Perform the following steps.
a) Click Initialization and Standard Configuration.
b) Click Configure HACMP Resource Groups.
c) Click Change/Show Resources for a Resource Group (standard).
d) Select a resource Group from pick list as RG1.
e) In the Service IP Labels / Addresses field, enter btcppesrv5sv. As btcppesrv5sv service IP label has to be activated on first node btcppesrv5.
f) Click OK.
g) Click Done.
h) Click Cancel.
i) Similarly follow steps c) to h) for adding second Service IP Label btcppesrv6sv in Resource Group RG2. A btcppesrv6sv service IP label has to be activated on second node btcppesrv6.
j) Select the Exit > Exit SMIT Menu option.
VERIFYING AND SYNCHRONIZING CLUSTER USING SMIT
This section demonstrates how to verify and synchronize the nodes in an HACMP cluster. This process of verification and synchronization actually verifies the HACMP configuration done from one node and then synchronizes to other node in the cluster. So whenever there are any changes to be done in the HACMP cluster, they are required to be done from a single node and to be synchronized with other nodes.
1. Log in as user root on AIX node btcppesrv5.
2. Enter following command to verify and synchronize all nodes in HACMP cluster.
# smit hacmp
Perform the following steps.
a) Click Initialization and Standard Configuration.
b) Click Verify and Synchronize HACMP Configuration.
c) Click Done.
d) Select the Exit > Exit SMIT Menu option.
STARTING CLUSTER SERVICES
This section demonstrates how to start an HACMP cluster on both the participating nodes.
1. Log in as user root on AIX node btcppesrv5.
2. Enter following command to start HACMP cluster.
# smit cl_admin
Perform the following steps.
a) Click Manage HACMP services.
b) Click Start Cluster Services.
c) In the Start Now, on System Restart or Both fields, select now.
d) In the Start Cluster Services on these nodes field, enter btcppesrv5 and btcppesrv6. The cluster services can be started on both the nodes simultaneously.
e) In the Startup Cluster Information Daemon field, select true.
f) Click OK.
g) Click Done.
h) Click Cancel.
i) Select the Exit > Exit SMIT Menu option.
STOPPING CLUSTER SERVICES
This section demonstrates how to stop an HACMP cluster on both the participating nodes.
1. Log in as user root on AIX node btcppesrv5.
2. Enter following command to stop HACMP cluster.
# smit cl_admin
Perform the following steps.
a) Click Manage HACMP services.
b) Click Stop Cluster Services.
c) In the Stop Now, on System Restart or Both fields, select now.
d) In the Stop Cluster Services on these nodes field, enter btcppesrv5 and btcppesrv6. The cluster services can be stopped on both the nodes simultaneously.
e) Click OK.
f) In the Are You Sure? Dialog box, click OK.
g) Click Done.
h) Click Cancel.
i) Select the Exit > Exit SMIT Menu option.
CONFIGURING DISK HEARTBEAT
For configuring Disk Heartbeating, it is required to create the Enhanced Concurrent Capable Volume group on both the AIX nodes.To be able to use HACMP C-SPC successfully, it is required that some basic IP based topology already exists, and that the storage devices have their PVIDs on both systems’ ODMs. This can be verified by running lspv command on each AIX node. If a PVID does not exist on any AIX node, it is necessary to run
chdev –l -a pv=yes command on each AIX node.
btcppesrv5#> chdev –l hdisk3 –a pv=yes
btcppesrv6#> chdev –l hdisk3 –a pv=yes
This will allow C-SPOC to match up the device(s) as known shared storage devices.
This demonstrates how to create Enhanced Concurrent Volume Group:
1. Log in as user root on AIX node btcppesrv5.
2. Enter following command to create Enhanced concurrent VG.
# smit vg
Perform the following steps.
a) Click Add Volume Group.
b) Click Add an Original Group.
c) In the Volume group name field, enter heartbeat.
d) In the Physical Volume Names field, enter hdisk3.
e) In the Volume Group Major number field, enter 59. This number is the number available for a particular AIX node; it can be found out from the available list in the field.
f) In the Create VG concurrent capable field, enter YES.
g) Click OK.
h) Click Done.
i) Click Cancel.
j) Select the Exit > Exit SMIT Menu option.
On btcppesrv5 AIX node check the newly created volume group using command lsvg.
On second AIX node enter importvg –V -y command to import the volume group:
btcppesrv6#> importvg -V 59 -y heartbeat hdisk3
Since the enhanced concurrent volume groups are available for both the AIX nodes, we will use discovery method of HACMP to find the disks available for Heartbeat.
This demonstrates how to configure Disk heartbeat in HACMP:
1. Log in as user root on AIX node btcppesrv5.
2. Enter following command to configure Disk heartbeat.
# smit hacmp
Perform the following steps.
a) Click Extended Configuration.
b) Click Discover HACMP-related information from configured Nodes. This will run automatically and create /usr/es/sbin/cluster/etc/config/clvg_config file that contains the information it has discovered.
c) Click Done.
d) Click Cancel.
e) Click Extended Configuration.
f) Click Extended Topology Configuration.
g) Click Configure HACMP communication Interfaces/Devices.
h) Click Add Communication Interfaces/Devices.
i) Click Add Discovered Communication Interfaces and Devices.
j) Click Communication Devices.
k) Select both the Devices listed in the list.
l) Click Done.
m) Click Cancel.
n) Select the Exit > Exit SMIT Menu option.
It is necessary to add the Volume group into HACMP Resource Group and synchronize the cluster.
Enter the following command to create Empty Resource Groups with different policies than what we created earlier.
# smit hacmp
Perform the following steps.
a) Click Initialization and Standard Configuration.
b) Click Configure HACMP Resource Groups.
c) Click Add a Resource Group.
d) In the Resource Group Name field, enter RG3.
e) In the Participating Nodes (Default Node Priority) field, enter btcppesrv5 and btcppesrv6.
f) In the Startup policy field, enter Online On All Available Nodes.
g) In the Fallover Policy field, enter Bring Offline (On Error Node Only).
h) In the Fallback Policy field, enter never Fallback.
i) Click OK.
j) Click Done.
k) Click Cancel.
l) Click Change/Show Resources for a Resource Group (Standard).
m) Select RG3 from the list.
n) In the Volume Groups field, enter heartbeat. The concurrent capable volume group which was created earlier.
o) Click OK.
p) Click Done.
q) Click Cancel.
r) Select the Exit > Exit SMIT Menu option.
In our example, we have two NICs, one used as cluster interconnects and other as bootable adapter. The service IP address will be activated on the bootable adapter after cluster services are started on the nodes. Following IP addresses are used in the setup:
NODE1: hostname – btcppesrv5
Boot IP address - 10.73.70.155 btcppesrv5
Netmask - 255.255.254.0
Interconnect IP address - 192.168.73.100 btcppesrv5i
Netmask - 255.255.255.0
Service IP address - 10.73.68.222 btcppesrv5sv
Netmask - 255.255.254.0
NODE2: hostname – btcppesrv6
Boot IP address - 10.73.70.156 btcppesrv6
Netmask - 255.255.254.0
Interconnect IP address - 192.168.73.101 btcppesrv6i
Netmask - 255.255.255.0
Service IP address - 10.73.68.223 btcppesrv6sv
Netmask - 255.255.254.0
EDITING CONFIGURATION FILES FOR HACMP
1. /usr/sbin/cluster/netmon.cf – All the IP addresses present in the network need to be entered in this file. Refer to Appendix for sample file.
2. /usr/sbin/cluster/etc/clhosts – All the IP addresses present in the network need to be entered in this file. Refer to Appendix for sample file.
3. /usr/sbin/cluster/etc/rhosts - All the IP addresses present in the network need to be entered in this file. Refer to Appendix for sample file.
4. /.rhosts – All the IP addresses present in the network with username (i.e. root) need to be entered in this file. Refer to Appendix for sample file.
5. /etc/hosts – All the IP addresses with their IP labels present in network need to be entered in this file. Refer to Appendix for sample file.
Note: All the above mentioned files need to be configured on both the nodes of cluster.
CREATING CLUSTER USING SMIT
This is a sample HACMP configuration that might require customization for your environment. This section demonstrates how to configure two AIX nodes, btcppesrv5 and btcppesrv6, into a HACMP cluster.
1. Configure two AIX nodes to allow the user root to use the rcp and remsh commands between themselves without having to specify a password.
2. Log in as user root on AIX node btcppesrv5.
3. Enter the following command to create an HACMP cluster.
# smit hacmp
Perform the following steps. These instructions assume that you are using the graphical user interface to SMIT (that is, smit –M). If you are using the ASCII interface to SMIT (that is, smit –C), modify these instructions accordingly.
a) Click Initialization and Standard Configuration.
b) Click Configure an HACMP Cluster and Nodes.
c) In the Cluster Name field, enter netapp.
d) In the New Nodes (via selected communication paths) field, enter btcppesrv5 and btcppesrv6.
e) Click OK.
f) Click Done.
g) Click Cancel.
h) Select the Exit > Exit SMIT Menu option.
4. Enter the following command to configure the heartbeat networks as private networks.
# smit hacmp
Perform the following steps.
a) Click Extended Configuration.
b) Click Extended Topology Configuration.
c) Click Configure HACMP Networks.
d) Click Change/Show a Network in the HACMP cluster.
e) Select net_ether_01 (192.168.73.0/24).
f) In the Network Attribute field, select private.
g) Click OK.
h) Click Done.
i) Click Cancel.
j) Select the Exit > Exit SMIT Menu option.
5. Enter the following command to configure Service IP Labels/Addresses.
# smit hacmp
Perform the following steps.
a) Click Initialization and Standard Configuration.
b) Click Configure Resources to Make Highly Available.
c) Click Configure Service IP Labels / Addresses.
d) Click Add a Service IP Label / Address.
e) In the IP Label / Address field, enter btcppsrv5sv.
f) In the Network Name field, select net_ether_02 (10.73.70.0/23). The Service IP label will be activated on network interface 10.73.70.0/23 after cluster service starts.
g) Click OK.
h) Click Done.
i) Click Cancel.
j) Similarly follow steps d) to h) for adding second service IP label btcppsrv6sv.
k) Select the Exit > Exit SMIT Menu option.
6. Enter the following command to create Empty Resource Groups with Node priorities.
# smit hacmp
Perform the following steps.
a) Click Initialization and Standard Configuration.
b) Click Configure HACMP Resource Groups.
c) Click Add a Resource Group.
d) In the Resource Group Name field, enter RG1.
e) In the Participating Nodes (Default Node Priority) field, enter btcppesrv5 and btcppesrv6. The Resource Group RG1 will be online on btcppsrv5 first when cluster service starts; in the event of failure RG1 will be taken over by btcppesrv6 as the node priority for RG1 is assigned to btcppesrv5 first.
f) Click OK.
g) Click Done.
h) Click Cancel.
i) Similarly follow steps d) to h) for adding second Resource group RG2 with node priority first assigned to btcppesrv6.
j) Select the Exit > Exit SMIT Menu option.
7. Enter the following command to make Service IP labels part of Resource Groups.
# smit hacmp
Perform the following steps.
a) Click Initialization and Standard Configuration.
b) Click Configure HACMP Resource Groups.
c) Click Change/Show Resources for a Resource Group (standard).
d) Select a resource Group from pick list as RG1.
e) In the Service IP Labels / Addresses field, enter btcppesrv5sv. As btcppesrv5sv service IP label has to be activated on first node btcppesrv5.
f) Click OK.
g) Click Done.
h) Click Cancel.
i) Similarly follow steps c) to h) for adding second Service IP Label btcppesrv6sv in Resource Group RG2. A btcppesrv6sv service IP label has to be activated on second node btcppesrv6.
j) Select the Exit > Exit SMIT Menu option.
VERIFYING AND SYNCHRONIZING CLUSTER USING SMIT
This section demonstrates how to verify and synchronize the nodes in an HACMP cluster. This process of verification and synchronization actually verifies the HACMP configuration done from one node and then synchronizes to other node in the cluster. So whenever there are any changes to be done in the HACMP cluster, they are required to be done from a single node and to be synchronized with other nodes.
1. Log in as user root on AIX node btcppesrv5.
2. Enter following command to verify and synchronize all nodes in HACMP cluster.
# smit hacmp
Perform the following steps.
a) Click Initialization and Standard Configuration.
b) Click Verify and Synchronize HACMP Configuration.
c) Click Done.
d) Select the Exit > Exit SMIT Menu option.
STARTING CLUSTER SERVICES
This section demonstrates how to start an HACMP cluster on both the participating nodes.
1. Log in as user root on AIX node btcppesrv5.
2. Enter following command to start HACMP cluster.
# smit cl_admin
Perform the following steps.
a) Click Manage HACMP services.
b) Click Start Cluster Services.
c) In the Start Now, on System Restart or Both fields, select now.
d) In the Start Cluster Services on these nodes field, enter btcppesrv5 and btcppesrv6. The cluster services can be started on both the nodes simultaneously.
e) In the Startup Cluster Information Daemon field, select true.
f) Click OK.
g) Click Done.
h) Click Cancel.
i) Select the Exit > Exit SMIT Menu option.
STOPPING CLUSTER SERVICES
This section demonstrates how to stop an HACMP cluster on both the participating nodes.
1. Log in as user root on AIX node btcppesrv5.
2. Enter following command to stop HACMP cluster.
# smit cl_admin
Perform the following steps.
a) Click Manage HACMP services.
b) Click Stop Cluster Services.
c) In the Stop Now, on System Restart or Both fields, select now.
d) In the Stop Cluster Services on these nodes field, enter btcppesrv5 and btcppesrv6. The cluster services can be stopped on both the nodes simultaneously.
e) Click OK.
f) In the Are You Sure? Dialog box, click OK.
g) Click Done.
h) Click Cancel.
i) Select the Exit > Exit SMIT Menu option.
CONFIGURING DISK HEARTBEAT
For configuring Disk Heartbeating, it is required to create the Enhanced Concurrent Capable Volume group on both the AIX nodes.To be able to use HACMP C-SPC successfully, it is required that some basic IP based topology already exists, and that the storage devices have their PVIDs on both systems’ ODMs. This can be verified by running lspv command on each AIX node. If a PVID does not exist on any AIX node, it is necessary to run
chdev –l
btcppesrv5#> chdev –l hdisk3 –a pv=yes
btcppesrv6#> chdev –l hdisk3 –a pv=yes
This will allow C-SPOC to match up the device(s) as known shared storage devices.
This demonstrates how to create Enhanced Concurrent Volume Group:
1. Log in as user root on AIX node btcppesrv5.
2. Enter following command to create Enhanced concurrent VG.
# smit vg
Perform the following steps.
a) Click Add Volume Group.
b) Click Add an Original Group.
c) In the Volume group name field, enter heartbeat.
d) In the Physical Volume Names field, enter hdisk3.
e) In the Volume Group Major number field, enter 59. This number is the number available for a particular AIX node; it can be found out from the available list in the field.
f) In the Create VG concurrent capable field, enter YES.
g) Click OK.
h) Click Done.
i) Click Cancel.
j) Select the Exit > Exit SMIT Menu option.
On btcppesrv5 AIX node check the newly created volume group using command lsvg.
On second AIX node enter importvg –V
btcppesrv6#> importvg -V 59 -y heartbeat hdisk3
Since the enhanced concurrent volume groups are available for both the AIX nodes, we will use discovery method of HACMP to find the disks available for Heartbeat.
This demonstrates how to configure Disk heartbeat in HACMP:
1. Log in as user root on AIX node btcppesrv5.
2. Enter following command to configure Disk heartbeat.
# smit hacmp
Perform the following steps.
a) Click Extended Configuration.
b) Click Discover HACMP-related information from configured Nodes. This will run automatically and create /usr/es/sbin/cluster/etc/config/clvg_config file that contains the information it has discovered.
c) Click Done.
d) Click Cancel.
e) Click Extended Configuration.
f) Click Extended Topology Configuration.
g) Click Configure HACMP communication Interfaces/Devices.
h) Click Add Communication Interfaces/Devices.
i) Click Add Discovered Communication Interfaces and Devices.
j) Click Communication Devices.
k) Select both the Devices listed in the list.
l) Click Done.
m) Click Cancel.
n) Select the Exit > Exit SMIT Menu option.
It is necessary to add the Volume group into HACMP Resource Group and synchronize the cluster.
Enter the following command to create Empty Resource Groups with different policies than what we created earlier.
# smit hacmp
Perform the following steps.
a) Click Initialization and Standard Configuration.
b) Click Configure HACMP Resource Groups.
c) Click Add a Resource Group.
d) In the Resource Group Name field, enter RG3.
e) In the Participating Nodes (Default Node Priority) field, enter btcppesrv5 and btcppesrv6.
f) In the Startup policy field, enter Online On All Available Nodes.
g) In the Fallover Policy field, enter Bring Offline (On Error Node Only).
h) In the Fallback Policy field, enter never Fallback.
i) Click OK.
j) Click Done.
k) Click Cancel.
l) Click Change/Show Resources for a Resource Group (Standard).
m) Select RG3 from the list.
n) In the Volume Groups field, enter heartbeat. The concurrent capable volume group which was created earlier.
o) Click OK.
p) Click Done.
q) Click Cancel.
r) Select the Exit > Exit SMIT Menu option.
No comments:
Post a Comment