Oracle 12c clusterware will actually let you rehearse with SRVCTL & CRSCTL commands using "predict" & "eval" options. Exploring the predict option in this post.
[oracle@ol6-121-rac1 bin]$ ./srvctl predict -h
The SRVCTL predict command evaluates the consequences of resource failure.
Usage: srvctl predict database -db <database_name> [-verbose]
Usage: srvctl predict service -db <database_name> -service <service_name> [-verbose]
Usage: srvctl predict asm [-node <node_name>] [-verbose]
Usage: srvctl predict diskgroup -diskgroup <diskgroup_name> [-verbose]
Usage: srvctl predict filesystem -device <volume_device> [-verbose]
Usage: srvctl predict vip -vip <vip_name> [-verbose]
Usage: srvctl predict network [-netnum <network_number>] [-verbose]
Usage: srvctl predict listener -listener <listener_name> [-verbose]
Usage: srvctl predict scan -scannumber <scan_ordinal_number> [-netnum <network_number>] [-verbose]
Usage: srvctl predict scan_listener -scannumber <scan_ordinal_number> [-netnum <network_number>] [-verbose]
Usage: srvctl predict oc4j [-verbose]
Lets take a snap of the current resources and its status before executing the predict commands so that we ll understand why it predicts so..
[oracle@ol6-121-rac1 bin]$ ./crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE ol6-121-rac1 STABLE
ONLINE ONLINE ol6-121-rac2 STABLE
ora.LISTENER.lsnr
ONLINE ONLINE ol6-121-rac1 STABLE
ONLINE ONLINE ol6-121-rac2 STABLE
ora.asm
ONLINE ONLINE ol6-121-rac1 Started,STABLE
ONLINE ONLINE ol6-121-rac2 Started,STABLE
ora.net1.network
ONLINE ONLINE ol6-121-rac1 STABLE
ONLINE ONLINE ol6-121-rac2 STABLE
ora.ons
ONLINE ONLINE ol6-121-rac1 STABLE
ONLINE ONLINE ol6-121-rac2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE ol6-121-rac2 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE ol6-121-rac1 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE ol6-121-rac1 STABLE
ora.MGMTLSNR
1 ONLINE ONLINE ol6-121-rac1 169.254.149.190 192.
168.1.101,STABLE
ora.cdbrac.db
1 ONLINE ONLINE ol6-121-rac1 Open,STABLE
2 ONLINE ONLINE ol6-121-rac2 Open,STABLE
ora.cvu
1 ONLINE ONLINE ol6-121-rac1 STABLE
ora.mgmtdb
1 ONLINE ONLINE ol6-121-rac1 Open,STABLE
ora.oc4j
1 ONLINE ONLINE ol6-121-rac2 STABLE
ora.ol6-121-rac1.vip
1 ONLINE ONLINE ol6-121-rac1 STABLE
ora.ol6-121-rac2.vip
1 ONLINE ONLINE ol6-121-rac2 STABLE
ora.scan1.vip
1 ONLINE ONLINE ol6-121-rac2 STABLE
ora.scan2.vip
1 ONLINE ONLINE ol6-121-rac1 STABLE
ora.scan3.vip
1 ONLINE ONLINE ol6-121-rac1 STABLE
--------------------------------------------------------------------------------
Now lets see the usage in detail.The output will summarize the behavior in case of the resource failure for which the "predict" commands are run.
[oracle@ol6-121-rac1 bin]$ srvctl predict database -db cdbrac -verbose
Database cdbrac will be stopped on nodes ol6-121-rac1,ol6-121-rac2
[oracle@ol6-121-rac1 bin]$ srvctl predict asm -node ol6-121-rac1 -verbose
Resource ora.asm will be stopped
Resource ora.mgmtdb will be stopped
Resource ora.DATA.dg will be stopped
Resource ora.MGMTLSNR will be started on node ol6-121-rac2
Database cdbrac will be stopped on node ol6-121-rac1
Resource ora.mgmtdb will be stopped
Resource ora.mgmtdb will be started on node ol6-121-rac2
[oracle@ol6-121-rac1 bin]$ srvctl predict diskgroup -diskgroup DATA -verbose
Resource ora.DATA.dg will be stopped
[oracle@ol6-121-rac1 bin]$ srvctl predict vip -vip ol6-121-rac1
VIP ol6-121-rac1 will be started on node ol6-121-rac2
Listener LISTENER will be stopped
[oracle@ol6-121-rac1 bin]$ srvctl predict network -netnum 1
Listener LISTENER_SCAN3 will be started on node ol6-121-rac2
Listener LISTENER will be stopped
Listener LISTENER_SCAN1 will be stopped
Listener LISTENER_SCAN2 will be stopped
Listener LISTENER_SCAN3 will be stopped
Resource ora.cvu will be started on nodes ol6-121-rac2,ol6-121-rac1
Network number 1 will be started on node ol6-121-rac1
Network number 1 will be stopped
Resource ora.cvu will be stopped
VIP ol6-121-rac1 will be started on nodes ol6-121-rac2,ol6-121-rac1
VIP ol6-121-rac2 will be started on node ol6-121-rac1
Resource ora.ons will be stopped
VIP scan1 will be started on node ol6-121-rac1
VIP scan2 will be started on nodes ol6-121-rac2,ol6-121-rac1
VIP scan3 will be started on node ol6-121-rac2
Listener LISTENER_SCAN1 will be started on node ol6-121-rac1
Listener LISTENER_SCAN2 will be started on nodes ol6-121-rac2,ol6-121-rac1
VIP scan1 will be stopped
VIP scan2 will be stopped
VIP scan3 will be stopped
Listener LISTENER will be started on node ol6-121-rac1
Resource ora.ons will be started on node ol6-121-rac1
[oracle@ol6-121-rac1 bin]$ srvctl predict listener -listener LISTENER
Listener LISTENER will be stopped
[oracle@ol6-121-rac1 bin]$ srvctl predict listener -listener MGMTLSNR
Resource ora.MGMTLSNR will be started on node ol6-121-rac2
Resource ora.mgmtdb will be started on node ol6-121-rac2
[oracle@ol6-121-rac1 bin]$ srvctl predict scan -scannumber 1
VIP scan1 will be started on node ol6-121-rac1
Listener LISTENER_SCAN1 will be started on node ol6-121-rac1
[oracle@ol6-121-rac1 bin]$ srvctl predict scan -scannumber 2
VIP scan2 will be started on node ol6-121-rac2
Listener LISTENER_SCAN2 will be started on node ol6-121-rac2
[oracle@ol6-121-rac1 bin]$ srvctl predict scan_listener -scannumber 1
Listener LISTENER_SCAN1 will be started on node ol6-121-rac1
VIP scan1 will be started on node ol6-121-rac1
[oracle@ol6-121-rac1 bin]$ srvctl predict scan_listener -scannumber 2
Listener LISTENER_SCAN2 will be started on node ol6-121-rac2
VIP scan2 will be started on node ol6-121-rac2
[oracle@ol6-121-rac1 bin]$ srvctl predict oc4j
Resource ora.oc4j will be started on node ol6-121-rac1
From the above o/p except few exceptions the rest makes sense. For example, in case of the diskgroup failure the o/p says the diskgroup resource will be stopped. But it did not mention anything about the failure of the instances running the in the same node which is not correct.
[oracle@ol6-121-rac1 bin]$ ./srvctl predict -h
The SRVCTL predict command evaluates the consequences of resource failure.
Usage: srvctl predict database -db <database_name> [-verbose]
Usage: srvctl predict service -db <database_name> -service <service_name> [-verbose]
Usage: srvctl predict asm [-node <node_name>] [-verbose]
Usage: srvctl predict diskgroup -diskgroup <diskgroup_name> [-verbose]
Usage: srvctl predict filesystem -device <volume_device> [-verbose]
Usage: srvctl predict vip -vip <vip_name> [-verbose]
Usage: srvctl predict network [-netnum <network_number>] [-verbose]
Usage: srvctl predict listener -listener <listener_name> [-verbose]
Usage: srvctl predict scan -scannumber <scan_ordinal_number> [-netnum <network_number>] [-verbose]
Usage: srvctl predict scan_listener -scannumber <scan_ordinal_number> [-netnum <network_number>] [-verbose]
Usage: srvctl predict oc4j [-verbose]
Lets take a snap of the current resources and its status before executing the predict commands so that we ll understand why it predicts so..
[oracle@ol6-121-rac1 bin]$ ./crsctl stat res -t
--------------------------------------------------------------------------------
Name Target State Server State details
--------------------------------------------------------------------------------
Local Resources
--------------------------------------------------------------------------------
ora.DATA.dg
ONLINE ONLINE ol6-121-rac1 STABLE
ONLINE ONLINE ol6-121-rac2 STABLE
ora.LISTENER.lsnr
ONLINE ONLINE ol6-121-rac1 STABLE
ONLINE ONLINE ol6-121-rac2 STABLE
ora.asm
ONLINE ONLINE ol6-121-rac1 Started,STABLE
ONLINE ONLINE ol6-121-rac2 Started,STABLE
ora.net1.network
ONLINE ONLINE ol6-121-rac1 STABLE
ONLINE ONLINE ol6-121-rac2 STABLE
ora.ons
ONLINE ONLINE ol6-121-rac1 STABLE
ONLINE ONLINE ol6-121-rac2 STABLE
--------------------------------------------------------------------------------
Cluster Resources
--------------------------------------------------------------------------------
ora.LISTENER_SCAN1.lsnr
1 ONLINE ONLINE ol6-121-rac2 STABLE
ora.LISTENER_SCAN2.lsnr
1 ONLINE ONLINE ol6-121-rac1 STABLE
ora.LISTENER_SCAN3.lsnr
1 ONLINE ONLINE ol6-121-rac1 STABLE
ora.MGMTLSNR
1 ONLINE ONLINE ol6-121-rac1 169.254.149.190 192.
168.1.101,STABLE
ora.cdbrac.db
1 ONLINE ONLINE ol6-121-rac1 Open,STABLE
2 ONLINE ONLINE ol6-121-rac2 Open,STABLE
ora.cvu
1 ONLINE ONLINE ol6-121-rac1 STABLE
ora.mgmtdb
1 ONLINE ONLINE ol6-121-rac1 Open,STABLE
ora.oc4j
1 ONLINE ONLINE ol6-121-rac2 STABLE
ora.ol6-121-rac1.vip
1 ONLINE ONLINE ol6-121-rac1 STABLE
ora.ol6-121-rac2.vip
1 ONLINE ONLINE ol6-121-rac2 STABLE
ora.scan1.vip
1 ONLINE ONLINE ol6-121-rac2 STABLE
ora.scan2.vip
1 ONLINE ONLINE ol6-121-rac1 STABLE
ora.scan3.vip
1 ONLINE ONLINE ol6-121-rac1 STABLE
--------------------------------------------------------------------------------
Now lets see the usage in detail.The output will summarize the behavior in case of the resource failure for which the "predict" commands are run.
- For a local resource it will list out what are the associated resources that will be stopped along with it.
 - For a cluster resource it will list out how the fail-over will happen to a remote node(very helpful for 3+ node cluster). This will also be a very easy way to understand the dependency among st the resources.
 
[oracle@ol6-121-rac1 bin]$ srvctl predict database -db cdbrac -verbose
Database cdbrac will be stopped on nodes ol6-121-rac1,ol6-121-rac2
[oracle@ol6-121-rac1 bin]$ srvctl predict asm -node ol6-121-rac1 -verbose
Resource ora.asm will be stopped
Resource ora.mgmtdb will be stopped
Resource ora.DATA.dg will be stopped
Resource ora.MGMTLSNR will be started on node ol6-121-rac2
Database cdbrac will be stopped on node ol6-121-rac1
Resource ora.mgmtdb will be stopped
Resource ora.mgmtdb will be started on node ol6-121-rac2
[oracle@ol6-121-rac1 bin]$ srvctl predict diskgroup -diskgroup DATA -verbose
Resource ora.DATA.dg will be stopped
[oracle@ol6-121-rac1 bin]$ srvctl predict vip -vip ol6-121-rac1
VIP ol6-121-rac1 will be started on node ol6-121-rac2
Listener LISTENER will be stopped
[oracle@ol6-121-rac1 bin]$ srvctl predict network -netnum 1
Listener LISTENER_SCAN3 will be started on node ol6-121-rac2
Listener LISTENER will be stopped
Listener LISTENER_SCAN1 will be stopped
Listener LISTENER_SCAN2 will be stopped
Listener LISTENER_SCAN3 will be stopped
Resource ora.cvu will be started on nodes ol6-121-rac2,ol6-121-rac1
Network number 1 will be started on node ol6-121-rac1
Network number 1 will be stopped
Resource ora.cvu will be stopped
VIP ol6-121-rac1 will be started on nodes ol6-121-rac2,ol6-121-rac1
VIP ol6-121-rac2 will be started on node ol6-121-rac1
Resource ora.ons will be stopped
VIP scan1 will be started on node ol6-121-rac1
VIP scan2 will be started on nodes ol6-121-rac2,ol6-121-rac1
VIP scan3 will be started on node ol6-121-rac2
Listener LISTENER_SCAN1 will be started on node ol6-121-rac1
Listener LISTENER_SCAN2 will be started on nodes ol6-121-rac2,ol6-121-rac1
VIP scan1 will be stopped
VIP scan2 will be stopped
VIP scan3 will be stopped
Listener LISTENER will be started on node ol6-121-rac1
Resource ora.ons will be started on node ol6-121-rac1
[oracle@ol6-121-rac1 bin]$ srvctl predict listener -listener LISTENER
Listener LISTENER will be stopped
[oracle@ol6-121-rac1 bin]$ srvctl predict listener -listener MGMTLSNR
Resource ora.MGMTLSNR will be started on node ol6-121-rac2
Resource ora.mgmtdb will be started on node ol6-121-rac2
[oracle@ol6-121-rac1 bin]$ srvctl predict scan -scannumber 1
VIP scan1 will be started on node ol6-121-rac1
Listener LISTENER_SCAN1 will be started on node ol6-121-rac1
[oracle@ol6-121-rac1 bin]$ srvctl predict scan -scannumber 2
VIP scan2 will be started on node ol6-121-rac2
Listener LISTENER_SCAN2 will be started on node ol6-121-rac2
[oracle@ol6-121-rac1 bin]$ srvctl predict scan_listener -scannumber 1
Listener LISTENER_SCAN1 will be started on node ol6-121-rac1
VIP scan1 will be started on node ol6-121-rac1
[oracle@ol6-121-rac1 bin]$ srvctl predict scan_listener -scannumber 2
Listener LISTENER_SCAN2 will be started on node ol6-121-rac2
VIP scan2 will be started on node ol6-121-rac2
[oracle@ol6-121-rac1 bin]$ srvctl predict oc4j
Resource ora.oc4j will be started on node ol6-121-rac1
From the above o/p except few exceptions the rest makes sense. For example, in case of the diskgroup failure the o/p says the diskgroup resource will be stopped. But it did not mention anything about the failure of the instances running the in the same node which is not correct.
Planning the same with "eval" in a different post.
Happy Reading!!
Happy Reading!!