Saturday, December 19, 2020

Oracle 19c (19.9) with ASMFD, Kernel 5.4.17 & Oracle Linux 8.3 (2-Node RAC)

Oracle has released the patches for Oracle 19c (19.8) and 19c (19.9) for ACFS/AFD drivers.

1. 30590023 UEK6 SUPPORT FOR ACFS/AFD (System Patch) - > 19.9.0.0.0ACFSRU
2. 30590023 UEK6 SUPPORT FOR ACFS/AFD (System Patch) - > 19.8.0.0.0ACFSRU

Please refer for more details:
ACFS Support On OS Platforms (Certification Matrix). (Doc ID 1369107.1)

Click Additional notes: 
OL8 UEK6: 30590023 OL8 UEK6 SUPPORT FOR ACFS/AFD (NOTE: 19.x and later Grid only)

Environment:

1. Installed OEL 8.3
2. Upgraded Kernel "kernel-uek-4.18.0-193.13.2.el8_2.x86_64" to kernel-uek-5.4.17-2036.100.6.1.el8uek.x86_64
3. Installed GI 19.3 and applied patch 19.7 and applied Oracle 19c (19.9)
4. Applied "p30590023_199000ACFSRU_Linux-x86-64.zip" patch on Oracle 19c (19.9) for GRID

[oracle@rac2-oel77-123 ~]$ . oraenv
ORACLE_SID = [orcldb2] ? +ASM2
ORACLE_HOME = [/home/oracle] ? /u01/app/19.7.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac2-oel77-123 ~]$
[oracle@rac2-oel77-123 ~]$ cd $ORACLE_HOME/OPatch
[oracle@rac2-oel77-123 OPatch]$ ./opatch lspatches
30898856;TOMCAT RELEASE UPDATE 19.0.0.0.0 (30898856)
30894985;OCW RELEASE UPDATE 19.7.0.0.0 (30894985)
30869304;ACFS RELEASE UPDATE 19.7.0.0.0 (30869304)
30869156;Database Release Update : 19.7.0.0.200414 (30869156)

OPatch succeeded.
[oracle@rac2-oel77-123 OPatch]$


[oracle@rac1-oel77-122 ~]$ su - root
Password:
[root@rac1-oel77-122 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/19.7.0/grid
The Oracle base has been set to /u01/app/oracle
[root@rac1-oel77-122 ~]#

[root@rac1-oel77-122 ~]# ps -ef | grep pmon
oracle      6506       1  0 15:21 ?        00:00:00 asm_pmon_+ASM1
oracle      8961       1  0 15:22 ?        00:00:00 mdb_pmon_-MGMTDB
oracle      9092       1  0 15:22 ?        00:00:00 ora_pmon_orcldb1
root       38171   37950  0 16:23 pts/1    00:00:00 grep --color=auto pmon
[root@rac1-oel77-122 ~]#

[root@rac1-oel77-122 ~]# sh /u01/app/19.7.0/grid/OPatch/opatchauto apply /home/oracle/31750108/

OPatchauto session is initiated at Sat Dec 19 16:24:20 2020

System initialization log file is /u01/app/19.7.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-19_04-24-23PM.log.

Session log file is /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-19_04-24-58PM.log
The id for this session is SKYE

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.7.0/grid

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.3.0/db_1
Patch applicability verified successfully on home /u01/app/19.7.0/grid

Patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1

Verifying SQL patch applicability on home /u01/app/oracle/product/19.3.0/db_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1

Preparing to bring down database service on home /u01/app/oracle/product/19.3.0/db_1
Successfully prepared home /u01/app/oracle/product/19.3.0/db_1 to bring down database service

Bringing down CRS service on home /u01/app/19.7.0/grid
CRS service brought down successfully on home /u01/app/19.7.0/grid

Performing prepatch operation on home /u01/app/oracle/product/19.3.0/db_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/db_1

Start applying binary patch on home /u01/app/oracle/product/19.3.0/db_1
Binary patch applied successfully on home /u01/app/oracle/product/19.3.0/db_1

Performing postpatch operation on home /u01/app/oracle/product/19.3.0/db_1
Postpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/db_1

Start applying binary patch on home /u01/app/19.7.0/grid
Binary patch applied successfully on home /u01/app/19.7.0/grid

Starting CRS service on home /u01/app/19.7.0/grid
CRS service started successfully on home /u01/app/19.7.0/grid

Preparing home /u01/app/oracle/product/19.3.0/db_1 after database service restarted
No step execution required.........

Trying to apply SQL patch on home /u01/app/oracle/product/19.3.0/db_1
SQL patch applied successfully on home /u01/app/oracle/product/19.3.0/db_1

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:rac1-oel77-122
RAC Home:/u01/app/oracle/product/19.3.0/db_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/oracle/31750108/31773437
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/oracle/31750108/31780966
Reason: This patch is not applicable to this specified target type - "rac_database"

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/31750108/31771877
Log: /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_16-30-30PM_1.log

Patch: /home/oracle/31750108/31772784
Log: /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_16-30-30PM_1.log

Host:rac1-oel77-122
CRS Home:/u01/app/19.7.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/31750108/31771877
Log: /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_16-39-54PM_1.log

Patch: /home/oracle/31750108/31772784
Log: /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_16-39-54PM_1.log

Patch: /home/oracle/31750108/31773437
Log: /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_16-39-54PM_1.log

Patch: /home/oracle/31750108/31780966
Log: /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_16-39-54PM_1.log


OPatchauto session completed at Sat Dec 19 16:54:51 2020
Time taken to complete the session 30 minutes, 31 seconds
[root@rac1-oel77-122 ~]# 


[root@rac2-oel77-123 ~]# . oraenv
ORACLE_SID = [+ASM2] ?
ORACLE_HOME = [/home/oracle] ? /u01/app/19.7.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[root@rac2-oel77-123 ~]#

[root@rac2-oel77-123 ~]# sh /u01/app/19.7.0/grid/OPatch/opatchauto apply /home/oracle/31750108/

OPatchauto session is initiated at Sat Dec 19 17:19:31 2020

System initialization log file is /u01/app/19.7.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-19_05-19-35PM.log.

Session log file is /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-19_05-19-56PM.log
The id for this session is BKPL

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.7.0/grid

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.3.0/db_1
Patch applicability verified successfully on home /u01/app/19.7.0/grid

Patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1

Verifying SQL patch applicability on home /u01/app/oracle/product/19.3.0/db_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1

Preparing to bring down database service on home /u01/app/oracle/product/19.3.0/db_1
Successfully prepared home /u01/app/oracle/product/19.3.0/db_1 to bring down database service

Bringing down CRS service on home /u01/app/19.7.0/grid
CRS service brought down successfully on home /u01/app/19.7.0/grid

Performing prepatch operation on home /u01/app/oracle/product/19.3.0/db_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/db_1

Start applying binary patch on home /u01/app/oracle/product/19.3.0/db_1
Binary patch applied successfully on home /u01/app/oracle/product/19.3.0/d                                                                                       b_1
Performing postpatch operation on home /u01/app/oracle/product/19.3.0/db_1
Postpatch operation completed successfully on home /u01/app/oracle/product                                    /19.3.0/db_1

Start applying binary patch on home /u01/app/19.7.0/grid
Binary patch applied successfully on home /u01/app/19.7.0/grid

Starting CRS service on home /u01/app/19.7.0/grid
CRS service started successfully on home /u01/app/19.7.0/grid

Preparing home /u01/app/oracle/product/19.3.0/db_1 after database service restarted
No step execution required.........

Trying to apply SQL patch on home /u01/app/oracle/product/19.3.0/db_1
SQL patch applied successfully on home /u01/app/oracle/product/19.3.0/db_1

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:rac2-oel77-123
RAC Home:/u01/app/oracle/product/19.3.0/db_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/oracle/31750108/31773437
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/oracle/31750108/31780966
Reason: This patch is not applicable to this specified target type - "rac_database"

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/31750108/31771877
Log: /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_17-28-51PM_1.log

Patch: /home/oracle/31750108/31772784
Log: /u01/app/oracle/product/19.3.0/db_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_17-28-51PM_1.log

Host:rac2-oel77-123
CRS Home:/u01/app/19.7.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/31750108/31771877
Log: /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_17-38-02PM_1.log

Patch: /home/oracle/31750108/31772784
Log: /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_17-38-02PM_1.log

Patch: /home/oracle/31750108/31773437
Log: /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_17-38-02PM_1.log

Patch: /home/oracle/31750108/31780966
Log: /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_17-38-02PM_1.log

OPatchauto session completed at Sat Dec 19 18:04:34 2020
Time taken to complete the session 45 minutes, 3 seconds
[root@rac2-oel77-123 ~]#


[oracle@rac1-oel77-122 ~]$ . oraenv
ORACLE_SID = [+ASM1] ?
ORACLE_HOME = [/home/oracle] ? /u01/app/19.7.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac1-oel77-122 ~]$

[oracle@rac1-oel77-122 ~]$ cd $ORACLE_HOME/OPatch

[oracle@rac1-oel77-122 OPatch]$ ./opatch lspatches
31780966;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31780966)
31773437;ACFS RELEASE UPDATE 19.9.0.0.0 (31773437)
31772784;OCW RELEASE UPDATE 19.9.0.0.0 (31772784)
31771877;Database Release Update : 19.9.0.0.201020 (31771877)

OPatch succeeded.
[oracle@rac1-oel77-122 OPatch]$


[oracle@rac1-oel77-122 ~]$ su - root
Password:
[root@rac1-oel77-122 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/19.7.0/grid
The Oracle base has been set to /u01/app/oracle
[root@rac1-oel77-122 ~]#

[root@rac1-oel77-122 ~]# sh /u01/app/19.7.0/grid/OPatch/opatchauto apply /home/oracle/30590023/

OPatchauto session is initiated at Sat Dec 19 18:29:38 2020

System initialization log file is /u01/app/19.7.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-19_06-29-40PM.log.

Session log file is /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-19_06-30-18PM.log
The id for this session is 1WBA

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.7.0/grid

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.3.0/db_1
Patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1

Patch applicability verified successfully on home /u01/app/19.7.0/grid


Verifying SQL patch applicability on home /u01/app/oracle/product/19.3.0/db_1
No step execution required.........


Preparing to bring down database service on home /u01/app/oracle/product/19.3.0/db_1
Successfully prepared home /u01/app/oracle/product/19.3.0/db_1 to bring down database service


Bringing down CRS service on home /u01/app/19.7.0/grid
CRS service brought down successfully on home /u01/app/19.7.0/grid


Start applying binary patch on home /u01/app/19.7.0/grid
Binary patch applied successfully on home /u01/app/19.7.0/grid


Starting CRS service on home /u01/app/19.7.0/grid
CRS service started successfully on home /u01/app/19.7.0/grid


Preparing home /u01/app/oracle/product/19.3.0/db_1 after database service restarted
No step execution required.........

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:rac1-oel77-122
RAC Home:/u01/app/oracle/product/19.3.0/db_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/oracle/30590023/30590023
Reason: This patch is not applicable to this specified target type - "rac_database"


Host:rac1-oel77-122
CRS Home:/u01/app/19.7.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/30590023/30590023
Log: /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_18-32-14PM_1.log

OPatchauto session completed at Sat Dec 19 18:37:26 2020
Time taken to complete the session 7 minutes, 49 seconds
[root@rac1-oel77-122 ~]#

[root@rac2-oel77-123 ~]# sh /u01/app/19.7.0/grid/OPatch/opatchauto apply /home/oracle/30590023/

OPatchauto session is initiated at Sat Dec 19 18:42:26 2020

System initialization log file is /u01/app/19.7.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-19_06-42-30PM.log.

Session log file is /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-19_06-42-51PM.log
The id for this session is 1VGP

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.7.0/grid

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.3.0/db_1
Patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/db_1

Patch applicability verified successfully on home /u01/app/19.7.0/grid

Verifying SQL patch applicability on home /u01/app/oracle/product/19.3.0/db_1
No step execution required.........

Preparing to bring down database service on home /u01/app/oracle/product/19.3.0/db_1
Successfully prepared home /u01/app/oracle/product/19.3.0/db_1 to bring down database service

Bringing down CRS service on home /u01/app/19.7.0/grid
CRS service brought down successfully on home /u01/app/19.7.0/grid

Start applying binary patch on home /u01/app/19.7.0/grid
Binary patch applied successfully on home /u01/app/19.7.0/grid

Starting CRS service on home /u01/app/19.7.0/grid
CRS service started successfully on home /u01/app/19.7.0/grid

Preparing home /u01/app/oracle/product/19.3.0/db_1 after database service restarted
No step execution required.........

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:rac2-oel77-123
RAC Home:/u01/app/oracle/product/19.3.0/db_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/oracle/30590023/30590023
Reason: This patch is not applicable to this specified target type - "rac_database"


Host:rac2-oel77-123
CRS Home:/u01/app/19.7.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/30590023/30590023
Log: /u01/app/19.7.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-19_18-45-41PM_1.log


OPatchauto session completed at Sat Dec 19 18:54:07 2020
Time taken to complete the session 11 minutes, 41 seconds
[root@rac2-oel77-123 ~]#

[oracle@rac1-oel77-122 ~]$ . oraenv
ORACLE_SID = [+ASM1] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/19.7.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac1-oel77-122 ~]$

[oracle@rac1-oel77-122 ~]$ cd $ORACLE_HOME/OPatch
[oracle@rac1-oel77-122 OPatch]$

[oracle@rac1-oel77-122 OPatch]$ ./opatch lspatches
30590023;ACFS Interim patch for 30590023
31780966;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31780966)
31772784;OCW RELEASE UPDATE 19.9.0.0.0 (31772784)
31771877;Database Release Update : 19.9.0.0.201020 (31771877)

OPatch succeeded.

[oracle@rac1-oel77-122 ~]$ su - root
Password:

[root@rac1-oel77-122 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/19.7.0/grid
The Oracle base has been set to /u01/app/oracle
[root@rac1-oel77-122 ~]#

[root@rac1-oel77-122 ~]# cd $ORACLE_HOME/bin
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# ./afddriverstate supported
AFD-9200: Supported
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid installed
[WARNING] 09389: ORACLE_HOME is not set to the location of the Grid Infrastructure home.
ACFS-9203: true
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid installed
ACFS-9203: true
[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid loaded
ACFS-9203: true
[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid supported
ACFS-9200: Supported
[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid version
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.9.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.9.0.0.0).
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid version
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.9.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.9.0.0.0).
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# asmcmd afd_configure
ASMCMD-9523: command cannot be used when Oracle Clusterware stack is up
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac1-oel77-122'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.chad' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.orcldb.db' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.orcldb.db' on 'rac1-oel77-122' succeeded
CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.DATADG.dg' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.OCRVD.dg' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.RECODG.dg' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.cvu' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.DATADG.dg' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.OCRVD.dg' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.RECODG.dg' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.rac1-oel77-122.vip' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.scan3.vip' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.rac1-oel77-122.vip' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.scan2.vip' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.scan3.vip' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.cvu' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on 'rac1-oel77-122' succeeded
CRS-33677: Stop of resource group 'ora.asmgroup' on server 'rac1-oel77-122' succeeded.
CRS-2677: Stop of 'ora.chad' on 'rac1-oel77-122' succeeded
CRS-2672: Attempting to start 'ora.scan2.vip' on 'rac2-oel77-123'
CRS-2672: Attempting to start 'ora.scan3.vip' on 'rac2-oel77-123'
CRS-2672: Attempting to start 'ora.cvu' on 'rac2-oel77-123'
CRS-2672: Attempting to start 'ora.rac1-oel77-122.vip' on 'rac2-oel77-123'
CRS-2676: Start of 'ora.cvu' on 'rac2-oel77-123' succeeded
CRS-2676: Start of 'ora.rac1-oel77-122.vip' on 'rac2-oel77-123' succeeded
CRS-2676: Start of 'ora.scan2.vip' on 'rac2-oel77-123' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'rac2-oel77-123'
CRS-2676: Start of 'ora.scan3.vip' on 'rac2-oel77-123' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on 'rac2-oel77-123'
CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'rac2-oel77-123' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.ons' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.net1.network' on 'rac1-oel77-122' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac1-oel77-122' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.crf' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.crf' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.ctssd' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.cssd' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac1-oel77-122'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac1-oel77-122'
CRS-2677: Stop of 'ora.gipcd' on 'rac1-oel77-122' succeeded
CRS-2677: Stop of 'ora.gpnpd' on 'rac1-oel77-122' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac1-oel77-122' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# asmcmd afd_configure
AFD-627: AFD distribution files found.
AFD-634: Removing previous AFD installation.
AFD-635: Previous AFD components successfully removed.
AFD-9294: updating file /etc/sysconfig/oracledrivers.conf
AFD-636: Installing requested AFD software.
AFD-637: Loading installed AFD drivers.
AFD-9321: Creating udev for AFD.
AFD-9323: Creating module dependencies - this may take some time.
AFD-9154: Loading 'oracleafd.ko' driver.
AFD-649: Verifying AFD devices.
AFD-9156: Detecting control device '/dev/oracleafd/admin'.
AFD-638: AFD installation correctness verified.
Modifying resource dependencies - this may take some time.
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# asmcmd afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'rac1-oel77-122'
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@rac1-oel77-122 bin]#
[root@rac1-oel77-122 bin]# ls -lrt afd*
-rwxr-xr-x 1 root oinstall     1000 Jun 20  2016 afdroot
-rwxr-xr-x 1 root oinstall 87863448 Nov 25 00:33 afdboot
-rwxr-xr-x 1 root oinstall   284992 Nov 25 00:33 afdtool.bin
-rwxr-xr-x 1 root oinstall      760 Dec 19 19:01 afdload
-rwxr-xr-x 1 root oinstall     3084 Dec 19 19:01 afddriverstate
-rwxr-xr-x 1 root oinstall     2862 Dec 19 19:01 afdtool
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# lsmod|grep oracle
oracleafd             221184  1
oracleacfs           5001216  0
oracleadvm            745472  0
oracleoks             729088  2 oracleadvm,oracleacfs
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# cat /etc/os-release | head -n2
NAME="Oracle Linux Server"
VERSION="8.3"
[root@rac1-oel77-122 bin]#

[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid version
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.9.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.9.0.0.0).
[root@rac1-oel77-122 bin]#

[oracle@rac2-oel77-123 ~]$ . oraenv
ORACLE_SID = [+ASM2] ?
ORACLE_HOME = [/home/oracle] ? /u01/app/19.7.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac2-oel77-123 ~]$

[oracle@rac2-oel77-123 ~]$ cd $ORACLE_HOME/OPatch
[oracle@rac2-oel77-123 OPatch]$ ./opatch lspatches
30590023;ACFS Interim patch for 30590023
31780966;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31780966)
31772784;OCW RELEASE UPDATE 19.9.0.0.0 (31772784)
31771877;Database Release Update : 19.9.0.0.201020 (31771877)

OPatch succeeded.

[oracle@rac2-oel77-123 ~]$ su - root
Password:
[root@rac2-oel77-123 ~]#

[root@rac2-oel77-123 ~]# . oraenv
ORACLE_SID = [root] ? +ASM2
ORACLE_HOME = [/home/oracle] ? /u01/app/19.7.0/grid
The Oracle base has been set to /u01/app/oracle
[root@rac2-oel77-123 ~]#

[root@rac2-oel77-123 ~]# cd $ORACLE_HOME/OPatch
[root@rac2-oel77-123 OPatch]#
[root@rac2-oel77-123 OPatch]# ./opatch lspatches
The user is root. OPatch cannot continue if the user is root.

OPatch failed with error code 255
[root@rac2-oel77-123 OPatch]#
[root@rac2-oel77-123 OPatch]# cd
[root@rac2-oel77-123 ~]#
[root@rac2-oel77-123 ~]# cd $ORACLE_HOME/bin
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]# ./afddriverstate supported
AFD-9200: Supported
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid installed
ACFS-9203: true
[root@rac2-oel77-123 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid loaded
ACFS-9203: true
[root@rac2-oel77-123 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid supported
ACFS-9200: Supported
[root@rac2-oel77-123 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid version
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.9.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.9.0.0.0).
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]# asmcmd afd_configure
ASMCMD-9523: command cannot be used when Oracle Clusterware stack is up
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.crsd' on 'rac2-oel77-123'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.mgmtdb' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.qosmserver' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.orcldb.db' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.chad' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.orcldb.db' on 'rac2-oel77-123' succeeded
CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.DATADG.dg' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.OCRVD.dg' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.RECODG.dg' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.cvu' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.DATADG.dg' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.RECODG.dg' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.OCRVD.dg' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.scan2.vip' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.cvu' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.scan3.vip' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.scan2.vip' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.scan3.vip' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.qosmserver' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on 'rac2-oel77-123' succeeded
CRS-33677: Stop of resource group 'ora.asmgroup' on server 'rac2-oel77-123' succeeded.
CRS-2677: Stop of 'ora.mgmtdb' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.MGMTLSNR' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.MGMTLSNR' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.rac2-oel77-123.vip' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.rac2-oel77-123.vip' on 'rac2-oel77-123' succeeded
CRS-2675: Stop of 'ora.chad' on 'rac2-oel77-123' failed
CRS-2679: Attempting to clean 'ora.chad' on 'rac2-oel77-123'
CRS-2681: Clean of 'ora.chad' on 'rac2-oel77-123' succeeded
CRS-2672: Attempting to start 'ora.MGMTLSNR' on 'rac1-oel77-122'
CRS-2672: Attempting to start 'ora.qosmserver' on 'rac1-oel77-122'
CRS-2672: Attempting to start 'ora.scan2.vip' on 'rac1-oel77-122'
CRS-2672: Attempting to start 'ora.scan3.vip' on 'rac1-oel77-122'
CRS-2672: Attempting to start 'ora.cvu' on 'rac1-oel77-122'
CRS-2672: Attempting to start 'ora.rac2-oel77-123.vip' on 'rac1-oel77-122'
CRS-2676: Start of 'ora.cvu' on 'rac1-oel77-122' succeeded
CRS-2676: Start of 'ora.rac2-oel77-123.vip' on 'rac1-oel77-122' succeeded
CRS-2676: Start of 'ora.MGMTLSNR' on 'rac1-oel77-122' succeeded
CRS-2672: Attempting to start 'ora.mgmtdb' on 'rac1-oel77-122'
CRS-2676: Start of 'ora.scan2.vip' on 'rac1-oel77-122' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on 'rac1-oel77-122'
CRS-2676: Start of 'ora.scan3.vip' on 'rac1-oel77-122' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on 'rac1-oel77-122'
CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on 'rac1-oel77-122' succeeded
CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on 'rac1-oel77-122' succeeded
CRS-2676: Start of 'ora.qosmserver' on 'rac1-oel77-122' succeeded
CRS-2676: Start of 'ora.mgmtdb' on 'rac1-oel77-122' succeeded
CRS-2673: Attempting to stop 'ora.ons' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.ons' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.net1.network' on 'rac2-oel77-123' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on 'rac2-oel77-123' has completed
CRS-2677: Stop of 'ora.crsd' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.asm' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.crf' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.mdnsd' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.crf' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.mdnsd' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.asm' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.evmd' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.ctssd' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.evmd' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.cssd' on 'rac2-oel77-123' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on 'rac2-oel77-123'
CRS-2673: Attempting to stop 'ora.gpnpd' on 'rac2-oel77-123'
CRS-2677: Stop of 'ora.gpnpd' on 'rac2-oel77-123' succeeded
CRS-2677: Stop of 'ora.gipcd' on 'rac2-oel77-123' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on 'rac2-oel77-123' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@rac2-oel77-123 bin]#

[root@rac2-oel77-123 bin]# asmcmd afd_configure
AFD-627: AFD distribution files found.
AFD-634: Removing previous AFD installation.
AFD-635: Previous AFD components successfully removed.
AFD-9294: updating file /etc/sysconfig/oracledrivers.conf
AFD-636: Installing requested AFD software.
AFD-637: Loading installed AFD drivers.
AFD-9321: Creating udev for AFD.
AFD-9323: Creating module dependencies - this may take some time.
AFD-9154: Loading 'oracleafd.ko' driver.
AFD-649: Verifying AFD devices.
AFD-9156: Detecting control device '/dev/oracleafd/admin'.
AFD-638: AFD installation correctness verified.
Modifying resource dependencies - this may take some time.
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]# asmcmd afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'rac2-oel77-123'
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]# ls -lrt afd*
-rwxr-xr-x 1 root oinstall     1000 Aug  1 11:59 afdroot
-rwxr-xr-x 1 root oinstall 87863448 Nov 25 00:33 afdboot
-rwxr-xr-x 1 root oinstall   284992 Nov 25 00:33 afdtool.bin
-rwxr-xr-x 1 root oinstall      760 Dec 19 20:39 afdload
-rwxr-xr-x 1 root oinstall     3084 Dec 19 20:39 afddriverstate
-rwxr-xr-x 1 root oinstall     2862 Dec 19 20:39 afdtool
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]# lsmod|grep oracle
oracleafd             221184  1
oracleacfs           5001216  0
oracleadvm            745472  0
oracleoks             729088  2 oracleadvm,oracleacfs
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid version
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.9.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.9.0.0.0).
[root@rac2-oel77-123 bin]#
[root@rac2-oel77-123 bin]#

Final Check:

[root@rac1-oel77-122 ~]# uname -r
5.4.17-2036.100.6.1.el8uek.x86_64
[root@rac1-oel77-122 ~]#
[root@rac1-oel77-122 ~]# cat /etc/oracle-release
Oracle Linux Server release 8.3
[root@rac1-oel77-122 ~]#
[root@rac1-oel77-122 ~]# cd $ORACLE_HOME/bin
[root@rac1-oel77-122 bin]#
[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid installed
ACFS-9203: true
[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid loaded
ACFS-9203: true
[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid supported
ACFS-9200: Supported
[root@rac1-oel77-122 bin]# ./acfsdriverstate -orahome /u01/app/19.7.0/grid version
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.9.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.9.0.0.0).
[root@rac1-oel77-122 bin]#
[root@rac1-oel77-122 bin]# asmcmd afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host 'rac1-oel77-122'
[root@rac1-oel77-122 bin]#
[root@rac1-oel77-122 bin]# exit
logout
[oracle@rac1-oel77-122 ~]$ . oraenv
ORACLE_SID = [+ASM1] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/19.7.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac1-oel77-122 ~]$
[oracle@rac1-oel77-122 ~]$ cd $ORACLE_HOME/OPatch
[oracle@rac1-oel77-122 OPatch]$
[oracle@rac1-oel77-122 OPatch]$ ./opatch lspatches
30590023;ACFS Interim patch for 30590023
31780966;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31780966)
31772784;OCW RELEASE UPDATE 19.9.0.0.0 (31772784)
31771877;Database Release Update : 19.9.0.0.201020 (31771877)

OPatch succeeded.
[oracle@rac1-oel77-122 OPatch]$

Posted by at 1 comment:

Friday, December 18, 2020

Oracle 19c (19.8) with ASMFD, Kernel 5.4.17 & Oracle Linux 8.2 (3-Node RAC)

Oracle has released the patches for Oracle 19c (19.8) and 19c (19.9) for ACFS/AFD drivers.

1. 30590023 UEK6 SUPPORT FOR ACFS/AFD (System Patch) - > 19.9.0.0.0ACFSRU
2. 30590023 UEK6 SUPPORT FOR ACFS/AFD (System Patch) - > 19.8.0.0.0ACFSRU

Please refer for more details:
ACFS Support On OS Platforms (Certification Matrix). (Doc ID 1369107.1)

Click Additional notes: 
OL8 UEK6: 30590023 OL8 UEK6 SUPPORT FOR ACFS/AFD (NOTE: 19.x and later Grid only)

Environment:

1. Installed OEL 8.2
2. Upgraded Kernel "kernel-uek-4.18.0-193.13.2.el8_2.x86_64" to kernel-uek-5.4.17-2036.100.6.1.el8uek.x86_64
3. Installed GI 19.3 (while GI installation applied Oracle 19c GI (19.8) RU)
4. Applied "p30590023_198000ACFSRU_Linux-x86-64.zip" patch on Oracle 19c (19.8) for GRID

19c-rac1:

[oracle@19c-rac1 ~]$ ps -ef | grep pmon
oracle     12651       1  0 18:30 ?        00:00:00 asm_pmon_+ASM1
oracle     14408       1  0 18:31 ?        00:00:00 mdb_pmon_-MGMTDB
oracle     35737   35364  0 18:39 pts/1    00:00:00 grep --color=auto pmon
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ uname -r
5.4.17-2036.100.6.1.el8uek.x86_64
[oracle@19c-rac1 ~]$
[oracle@19c-rac1 ~]$ uname -a
Linux 19c-rac1.localdomain 5.4.17-2036.100.6.1.el8uek.x86_64 #2 SMP Thu Oct 29 17:06:00 PDT 2020 x86_64 x86_64 x86_64 GNU/Linux
[oracle@19c-rac1 ~]$
[oracle@19c-rac1 ~]$ cat /etc/oracle-release
Oracle Linux Server release 8.2
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ?
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ echo $ORACLE_HOME
/u01/app/19.3.0/grid
[oracle@19c-rac1 ~]$ cd $ORACLE_HOME/OPatch
[oracle@19c-rac1 OPatch]$ ./opatch lspatches
31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)

OPatch succeeded.
[oracle@19c-rac1 OPatch]$

[oracle@19c-rac1 ~]$ chmod 777 p30590023_198000ACFSRU_Linux-x86-64.zip
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ scp p30590023_198000ACFSRU_Linux-x86-64.zip oracle@19c-rac2:/home/oracle/
p30590023_198000ACFSRU_Linux-x86-64.zip        100%  686MB  36.5MB/s   00:18
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ scp p30590023_198000ACFSRU_Linux-x86-64.zip oracle@19c-rac3:/home/oracle/
p30590023_198000ACFSRU_Linux-x86-64.zip        100%  686MB  37.3MB/s   00:18
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ unzip p30590023_198000ACFSRU_Linux-x86-64.zip
Archive:  p30590023_198000ACFSRU_Linux-x86-64.zip
   creating: 30590023/
   creating: 30590023/automation/
  inflating: 30590023/automation/messages.properties
  inflating: 30590023/automation/bp1-out-of-place-non-rolling-switchback.xml
  inflating: 30590023/automation/bp1-out-of-place-switchback.xml

[oracle@19c-rac1 ~]$ echo $ORACLE_HOME
/u01/app/19.3.0/grid
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ cd $ORACLE_HOME/OPatch
[oracle@19c-rac1 OPatch]$ ./opatch lspatches
31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)

OPatch succeeded.
[oracle@19c-rac1 OPatch]$

[oracle@19c-rac1 OPatch]$ ./opatch version
OPatch Version: 12.2.0.1.21
OPatch succeeded.
[oracle@19c-rac1 OPatch]$
[oracle@19c-rac1 OPatch]$ cd
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ su - root
Password:
[root@19c-rac1 ~]#

[root@19c-rac1 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base has been set to /u01/app/oracle
[root@19c-rac1 ~]#

[root@19c-rac1 ~]# sh /u01/app/19.3.0/grid/OPatch/opatchauto apply /home/oracle/30590023/

OPatchauto session is initiated at Fri Dec 18 19:24:30 2020

System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-18_07-24-35PM.log.

Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-18_07-25-12PM.log
The id for this session is P258

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid
Patch applicability verified successfully on home /u01/app/19.3.0/grid

Bringing down CRS service on home /u01/app/19.3.0/grid
CRS service brought down successfully on home /u01/app/19.3.0/grid

Start applying binary patch on home /u01/app/19.3.0/grid
Binary patch applied successfully on home /u01/app/19.3.0/grid

Starting CRS service on home /u01/app/19.3.0/grid
CRS service started successfully on home /u01/app/19.3.0/grid

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:19c-rac1
CRS Home:/u01/app/19.3.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/30590023/30590023
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-18_19-27-43PM_1.log

OPatchauto session completed at Fri Dec 18 19:31:02 2020
Time taken to complete the session 6 minutes, 32 seconds
[root@19c-rac1 ~]#
[root@19c-rac1 ~]# exit
logout
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ?
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ cd $ORACLE_HOME/OPatch
[oracle@19c-rac1 OPatch]$ ./opatch lspatches
30590023;ACFS Interim patch for 30590023
31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
OPatch succeeded.
[oracle@19c-rac1 OPatch]$ cd
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ ps -ef | grep pmon
oracle     87342       1  0 19:30 ?        00:00:00 asm_pmon_+ASM1
oracle     97416   35364  0 19:48 pts/1    00:00:00 grep --color=auto pmon
[oracle@19c-rac1 ~]$

19c-rac2:

[oracle@19c-rac2 ~]$ . oraenv
ORACLE_SID = [cdbrac2] ? +ASM2
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac2 ~]$

[oracle@19c-rac2 ~]$ cd $ORACLE_HOME/OPatch
[oracle@19c-rac2 OPatch]$

[oracle@19c-rac2 OPatch]$ ./opatch lspatches
31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)

OPatch succeeded.
[oracle@19c-rac2 OPatch]$

[oracle@19c-rac2 OPatch]$ ./opatch version
OPatch Version: 12.2.0.1.21
OPatch succeeded.
[oracle@19c-rac2 OPatch]$

[oracle@19c-rac2 OPatch]$ cd
[oracle@19c-rac2 ~]$

[oracle@19c-rac2 ~]$ ps -ef | grep pmon
oracle     15057       1  0 18:32 ?        00:00:00 asm_pmon_+ASM2
oracle     48512   29638  0 19:24 pts/0    00:00:00 grep --color=auto pmon
[oracle@19c-rac2 ~]$

[oracle@19c-rac2 ~]$ su - root
Password:
[root@19c-rac2 ~]# . oraenv
ORACLE_SID = [root] ? +ASM2
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base has been set to /u01/app/oracle
[root@19c-rac2 ~]#

[root@19c-rac2 ~]# sh /u01/app/19.3.0/grid/OPatch/opatchauto apply /home/oracle/30590023/

OPatchauto session is initiated at Fri Dec 18 19:47:58 2020

System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-18_07-48-02PM.log.

Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-18_07-48-14PM.log
The id for this session is ZJLN

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid
Patch applicability verified successfully on home /u01/app/19.3.0/grid

Bringing down CRS service on home /u01/app/19.3.0/grid
CRS service brought down successfully on home /u01/app/19.3.0/grid

Start applying binary patch on home /u01/app/19.3.0/grid
Binary patch applied successfully on home /u01/app/19.3.0/grid

Starting CRS service on home /u01/app/19.3.0/grid
CRS service started successfully on home /u01/app/19.3.0/grid

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:19c-rac2
CRS Home:/u01/app/19.3.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/30590023/30590023
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-18_19-49-48PM_1.log

OPatchauto session completed at Fri Dec 18 19:53:11 2020
Time taken to complete the session 5 minutes, 13 seconds
[root@19c-rac2 ~]#

19c-rac3:

[oracle@19c-rac3 ~]$ . oraenv
ORACLE_SID = [cdbrac3] ? +ASM3
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac3 ~]$

[oracle@19c-rac3 ~]$ cd $ORACLE_HOME/OPatch
[oracle@19c-rac3 OPatch]$

[oracle@19c-rac3 OPatch]$ ./opatch lspatches
31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
OPatch succeeded.
[oracle@19c-rac3 OPatch]$

[oracle@19c-rac3 ~]$ ps -ef | grep pmon
oracle     14810       1  0 18:32 ?        00:00:00 asm_pmon_+ASM3
oracle     48694   29385  0 19:24 pts/0    00:00:00 grep --color=auto pmon
[oracle@19c-rac3 ~]$

[oracle@19c-rac3 ~]$ su - root
Password:
[root@19c-rac3 ~]# . oraenv
ORACLE_SID = [root] ? +ASM3
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base has been set to /u01/app/oracle
[root@19c-rac3 ~]#

[root@19c-rac3 ~]# sh /u01/app/19.3.0/grid/OPatch/opatchauto apply /home/oracle/30590023/

OPatchauto session is initiated at Fri Dec 18 19:54:35 2020

System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-12-18_07-54-39PM.log.

Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-12-18_07-54-50PM.log
The id for this session is KS92

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid
Patch applicability verified successfully on home /u01/app/19.3.0/grid

Bringing down CRS service on home /u01/app/19.3.0/grid
CRS service brought down successfully on home /u01/app/19.3.0/grid

Start applying binary patch on home /u01/app/19.3.0/grid
Binary patch applied successfully on home /u01/app/19.3.0/grid

Starting CRS service on home /u01/app/19.3.0/grid
CRS service started successfully on home /u01/app/19.3.0/grid

OPatchAuto successful.

--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:19c-rac3
CRS Home:/u01/app/19.3.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/30590023/30590023
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-12-18_19-57-06PM_1.log

OPatchauto session completed at Fri Dec 18 20:03:37 2020
Time taken to complete the session 9 minutes, 2 seconds
[root@19c-rac3 ~]#

Checking drivers in 19c-rac1:

[oracle@19c-rac1 ~]$ su - root
Password:
[root@19c-rac1 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base has been set to /u01/app/oracle
[root@19c-rac1 ~]#

[root@19c-rac1 ~]# cd $ORACLE_HOME/bin
[root@19c-rac1 bin]# ./afddriverstate supported
AFD-9200: Supported

[root@19c-rac1 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid installed
ACFS-9203: true

[root@19c-rac1 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid loaded
ACFS-9203: true

[root@19c-rac1 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid supported
ACFS-9200: Supported

[root@19c-rac1 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid version
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.8.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.8.0.0.0).
[root@19c-rac1 bin]#

[root@19c-rac1 bin]#  asmcmd afd_configure
ASMCMD-9523: command cannot be used when Oracle Clusterware stack is up
[root@19c-rac1 bin]#

[root@19c-rac1 bin]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '19c-rac1'
CRS-2673: Attempting to stop 'ora.crsd' on '19c-rac1'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server '19c-rac1'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on '19c-rac1'
CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server '19c-rac1'
CRS-2673: Attempting to stop 'ora.OCRVD.dg' on '19c-rac1'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on '19c-rac1'
CRS-2673: Attempting to stop 'ora.cvu' on '19c-rac1'
CRS-2673: Attempting to stop 'ora.chad' on '19c-rac1'
CRS-2677: Stop of 'ora.OCRVD.dg' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on '19c-rac1'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.19c-rac1.vip' on '19c-rac1'
CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.scan3.vip' on '19c-rac1'
CRS-2677: Stop of 'ora.asm' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac1'
CRS-2677: Stop of 'ora.19c-rac1.vip' on '19c-rac1' succeeded
CRS-2677: Stop of 'ora.cvu' on '19c-rac1' succeeded
CRS-2677: Stop of 'ora.scan3.vip' on '19c-rac1' succeeded
CRS-2677: Stop of 'ora.chad' on '19c-rac1' succeeded
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on '19c-rac1'
CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on '19c-rac1' succeeded
CRS-33677: Stop of resource group 'ora.asmgroup' on server '19c-rac1' succeeded.
CRS-2672: Attempting to start 'ora.19c-rac1.vip' on '19c-rac3'
CRS-2672: Attempting to start 'ora.scan3.vip' on '19c-rac2'
CRS-2672: Attempting to start 'ora.cvu' on '19c-rac2'
CRS-2676: Start of 'ora.cvu' on '19c-rac2' succeeded
CRS-2676: Start of 'ora.19c-rac1.vip' on '19c-rac3' succeeded
CRS-2676: Start of 'ora.scan3.vip' on '19c-rac2' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on '19c-rac2'
CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.ons' on '19c-rac1'
CRS-2677: Stop of 'ora.ons' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on '19c-rac1'
CRS-2677: Stop of 'ora.net1.network' on '19c-rac1' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on '19c-rac1' has completed
CRS-2677: Stop of 'ora.crsd' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.asm' on '19c-rac1'
CRS-2673: Attempting to stop 'ora.crf' on '19c-rac1'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on '19c-rac1'
CRS-2673: Attempting to stop 'ora.mdnsd' on '19c-rac1'
CRS-2677: Stop of 'ora.crf' on '19c-rac1' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on '19c-rac1' succeeded
CRS-2677: Stop of 'ora.mdnsd' on '19c-rac1' succeeded
CRS-2677: Stop of 'ora.asm' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on '19c-rac1'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on '19c-rac1'
CRS-2673: Attempting to stop 'ora.evmd' on '19c-rac1'
CRS-2677: Stop of 'ora.ctssd' on '19c-rac1' succeeded
CRS-2677: Stop of 'ora.evmd' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on '19c-rac1'
CRS-2677: Stop of 'ora.cssd' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on '19c-rac1'
CRS-2673: Attempting to stop 'ora.gpnpd' on '19c-rac1'
CRS-2677: Stop of 'ora.gpnpd' on '19c-rac1' succeeded
CRS-2677: Stop of 'ora.gipcd' on '19c-rac1' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '19c-rac1' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@19c-rac1 bin]#

[root@19c-rac1 bin]# asmcmd afd_configure
AFD-627: AFD distribution files found.
AFD-634: Removing previous AFD installation.
AFD-635: Previous AFD components successfully removed.
AFD-9294: updating file /etc/sysconfig/oracledrivers.conf
AFD-636: Installing requested AFD software.
AFD-637: Loading installed AFD drivers.
AFD-9321: Creating udev for AFD.
AFD-9323: Creating module dependencies - this may take some time.
AFD-9154: Loading 'oracleafd.ko' driver.
AFD-649: Verifying AFD devices.
AFD-9156: Detecting control device '/dev/oracleafd/admin'.
AFD-638: AFD installation correctness verified.
Modifying resource dependencies - this may take some time.
[root@19c-rac1 bin]#

[root@19c-rac1 bin]# asmcmd afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host '19c-rac1.localdomain'
[root@19c-rac1 bin]#

[root@19c-rac1 bin]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@19c-rac1 bin]#
[root@19c-rac1 bin]#
[root@19c-rac1 bin]# ls -lrt afd*
-rwxr-xr-x 1 root oinstall     1000 Jun 20  2016 afdroot
-rwxr-xr-x 1 root oinstall   280896 Nov 15 09:17 afdtool.bin
-rwxr-xr-x 1 root oinstall 87596064 Nov 15 09:17 afdboot
-rwxr-xr-x 1 root oinstall     2860 Dec 18 20:07 afdtool
-rwxr-xr-x 1 root oinstall      760 Dec 18 20:07 afdload
-rwxr-xr-x 1 root oinstall     3084 Dec 18 20:07 afddriverstate
[root@19c-rac1 bin]#

[root@19c-rac1 bin]# lsmod|grep oracle
oracleafd             221184  1
oracleacfs           4997120  0
oracleadvm            745472  0
oracleoks             729088  2 oracleadvm,oracleacfs
[root@19c-rac1 bin]#

[root@19c-rac1 bin]# cat /etc/os-release | head -n2
NAME="Oracle Linux Server"
VERSION="8.2"
[root@19c-rac1 bin]#

[root@19c-rac1 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid version
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.8.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.8.0.0.0).
[root@19c-rac1 bin]#

Checking drivers in 19c-rac2:

[root@19c-rac2 ~]# cd $ORACLE_HOME/bin
[root@19c-rac2 bin]# ./afddriverstate supported
AFD-9200: Supported
[root@19c-rac2 bin]#

[root@19c-rac2 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid installed
ACFS-9203: true

[root@19c-rac2 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid loaded
ACFS-9203: true

[root@19c-rac2 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid supported
ACFS-9200: Supported

[root@19c-rac2 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid version
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.8.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.8.0.0.0).
[root@19c-rac2 bin]#

[root@19c-rac2 bin]# asmcmd afd_configure
ASMCMD-9523: command cannot be used when Oracle Clusterware stack is up

[root@19c-rac2 bin]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '19c-rac2'
CRS-2673: Attempting to stop 'ora.crsd' on '19c-rac2'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server '19c-rac2'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on '19c-rac2'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN3.lsnr' on '19c-rac2'
CRS-2673: Attempting to stop 'ora.cvu' on '19c-rac2'
CRS-2673: Attempting to stop 'ora.qosmserver' on '19c-rac2'
CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server '19c-rac2'
CRS-2673: Attempting to stop 'ora.OCRVD.dg' on '19c-rac2'
CRS-2673: Attempting to stop 'ora.chad' on '19c-rac2'
CRS-2677: Stop of 'ora.OCRVD.dg' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on '19c-rac2'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.19c-rac2.vip' on '19c-rac2'
CRS-2677: Stop of 'ora.LISTENER_SCAN3.lsnr' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.scan3.vip' on '19c-rac2'
CRS-2677: Stop of 'ora.cvu' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.19c-rac2.vip' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.scan3.vip' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.asm' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac2'
CRS-2677: Stop of 'ora.chad' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.qosmserver' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on '19c-rac2'
CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on '19c-rac2' succeeded
CRS-33677: Stop of resource group 'ora.asmgroup' on server '19c-rac2' succeeded.
CRS-2672: Attempting to start 'ora.qosmserver' on '19c-rac1'
CRS-2672: Attempting to start 'ora.19c-rac2.vip' on '19c-rac3'
CRS-2672: Attempting to start 'ora.scan3.vip' on '19c-rac1'
CRS-2672: Attempting to start 'ora.cvu' on '19c-rac1'
CRS-2676: Start of 'ora.cvu' on '19c-rac1' succeeded
CRS-2676: Start of 'ora.19c-rac2.vip' on '19c-rac3' succeeded
CRS-2676: Start of 'ora.scan3.vip' on '19c-rac1' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN3.lsnr' on '19c-rac1'
CRS-2676: Start of 'ora.LISTENER_SCAN3.lsnr' on '19c-rac1' succeeded
CRS-2676: Start of 'ora.qosmserver' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on '19c-rac2'
CRS-2677: Stop of 'ora.ons' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on '19c-rac2'
CRS-2677: Stop of 'ora.net1.network' on '19c-rac2' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on '19c-rac2' has completed
CRS-2677: Stop of 'ora.crsd' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.storage' on '19c-rac2'
CRS-2673: Attempting to stop 'ora.crf' on '19c-rac2'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on '19c-rac2'
CRS-2673: Attempting to stop 'ora.mdnsd' on '19c-rac2'
CRS-2677: Stop of 'ora.crf' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.storage' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.asm' on '19c-rac2'
CRS-2677: Stop of 'ora.mdnsd' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.asm' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on '19c-rac2'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on '19c-rac2'
CRS-2673: Attempting to stop 'ora.evmd' on '19c-rac2'
CRS-2677: Stop of 'ora.ctssd' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.evmd' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on '19c-rac2'
CRS-2677: Stop of 'ora.cssd' on '19c-rac2' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on '19c-rac2'
CRS-2673: Attempting to stop 'ora.gpnpd' on '19c-rac2'
CRS-2677: Stop of 'ora.gpnpd' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.gipcd' on '19c-rac2' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '19c-rac2' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@19c-rac2 bin]#

[root@19c-rac2 bin]# asmcmd afd_configure
AFD-627: AFD distribution files found.
AFD-634: Removing previous AFD installation.
AFD-635: Previous AFD components successfully removed.
AFD-9294: updating file /etc/sysconfig/oracledrivers.conf
AFD-636: Installing requested AFD software.
AFD-637: Loading installed AFD drivers.
AFD-9321: Creating udev for AFD.
AFD-9323: Creating module dependencies - this may take some time.
AFD-9154: Loading 'oracleafd.ko' driver.
AFD-649: Verifying AFD devices.
AFD-9156: Detecting control device '/dev/oracleafd/admin'.
AFD-638: AFD installation correctness verified.
Modifying resource dependencies - this may take some time.
[root@19c-rac2 bin]#

[root@19c-rac2 bin]# asmcmd afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host '19c-rac2.localdomain'
[root@19c-rac2 bin]#

[root@19c-rac2 bin]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@19c-rac2 bin]#
[root@19c-rac2 bin]# ls -lrt afd*
-rwxr-xr-x 1 root oinstall     1000 Nov 12 21:27 afdroot
-rwxr-xr-x 1 root oinstall   280896 Nov 15 09:17 afdtool.bin
-rwxr-xr-x 1 root oinstall 87596064 Nov 15 09:17 afdboot
-rwxr-xr-x 1 root oinstall      760 Dec 18 20:16 afdload
-rwxr-xr-x 1 root oinstall     2860 Dec 18 20:16 afdtool
-rwxr-xr-x 1 root oinstall     3084 Dec 18 20:16 afddriverstate
[root@19c-rac2 bin]#

[root@19c-rac2 bin]# lsmod|grep oracle
oracleafd             221184  1
oracleacfs           4997120  0
oracleadvm            745472  0
oracleoks             729088  2 oracleadvm,oracleacfs

[root@19c-rac2 bin]# cat /etc/os-release | head -n2
NAME="Oracle Linux Server"
VERSION="8.2"
[root@19c-rac2 bin]#

Checking drivers in 19c-rac3:

[root@19c-rac3 ~]# cd $ORACLE_HOME/bin
[root@19c-rac3 bin]#

[root@19c-rac3 bin]# ./afddriverstate supported
AFD-9200: Supported

[root@19c-rac3 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid installed
ACFS-9203: true

[root@19c-rac3 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid loaded
ACFS-9203: true

[root@19c-rac3 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid supported
ACFS-9200: Supported

[root@19c-rac3 bin]# ./acfsdriverstate -orahome /u01/app/19.3.0/grid version
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.8.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.8.0.0.0).
[root@19c-rac3 bin]#

[root@19c-rac3 bin]# asmcmd afd_configure
ASMCMD-9523: command cannot be used when Oracle Clusterware stack is up

[root@19c-rac3 bin]# crsctl stop crs
CRS-2791: Starting shutdown of Oracle High Availability Services-managed resources on '19c-rac3'
CRS-2673: Attempting to stop 'ora.crsd' on '19c-rac3'
CRS-2790: Starting shutdown of Cluster Ready Services-managed resources on server '19c-rac3'
CRS-33673: Attempting to stop resource group 'ora.asmgroup' on server '19c-rac3'
CRS-2673: Attempting to stop 'ora.OCRVD.dg' on '19c-rac3'
CRS-2673: Attempting to stop 'ora.LISTENER.lsnr' on '19c-rac3'
CRS-2673: Attempting to stop 'ora.LISTENER_SCAN2.lsnr' on '19c-rac3'
CRS-2677: Stop of 'ora.OCRVD.dg' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.asm' on '19c-rac3'
CRS-2677: Stop of 'ora.LISTENER_SCAN2.lsnr' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.scan2.vip' on '19c-rac3'
CRS-2677: Stop of 'ora.LISTENER.lsnr' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.chad' on '19c-rac1'
CRS-2673: Attempting to stop 'ora.chad' on '19c-rac2'
CRS-2673: Attempting to stop 'ora.chad' on '19c-rac3'
CRS-2677: Stop of 'ora.scan2.vip' on '19c-rac3' succeeded
CRS-2677: Stop of 'ora.asm' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac3'
CRS-2677: Stop of 'ora.chad' on '19c-rac1' succeeded
CRS-2677: Stop of 'ora.chad' on '19c-rac2' succeeded
CRS-2677: Stop of 'ora.chad' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.mgmtdb' on '19c-rac3'
CRS-2677: Stop of 'ora.ASMNET1LSNR_ASM.lsnr' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.asmnet1.asmnetwork' on '19c-rac3'
CRS-2677: Stop of 'ora.asmnet1.asmnetwork' on '19c-rac3' succeeded
CRS-33677: Stop of resource group 'ora.asmgroup' on server '19c-rac3' succeeded.
CRS-2677: Stop of 'ora.mgmtdb' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.MGMTLSNR' on '19c-rac3'
CRS-2677: Stop of 'ora.MGMTLSNR' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.19c-rac3.vip' on '19c-rac3'
CRS-2672: Attempting to start 'ora.MGMTLSNR' on '19c-rac1'
CRS-2677: Stop of 'ora.19c-rac3.vip' on '19c-rac3' succeeded
CRS-2676: Start of 'ora.MGMTLSNR' on '19c-rac1' succeeded
CRS-2672: Attempting to start 'ora.mgmtdb' on '19c-rac1'
CRS-2676: Start of 'ora.mgmtdb' on '19c-rac1' succeeded
CRS-2672: Attempting to start 'ora.chad' on '19c-rac1'
CRS-2672: Attempting to start 'ora.chad' on '19c-rac2'
CRS-2676: Start of 'ora.chad' on '19c-rac2' succeeded
CRS-2676: Start of 'ora.chad' on '19c-rac1' succeeded
CRS-2672: Attempting to start 'ora.19c-rac3.vip' on '19c-rac2'
CRS-2672: Attempting to start 'ora.scan2.vip' on '19c-rac1'
CRS-2676: Start of 'ora.19c-rac3.vip' on '19c-rac2' succeeded
CRS-2676: Start of 'ora.scan2.vip' on '19c-rac1' succeeded
CRS-2672: Attempting to start 'ora.LISTENER_SCAN2.lsnr' on '19c-rac1'
CRS-2676: Start of 'ora.LISTENER_SCAN2.lsnr' on '19c-rac1' succeeded
CRS-2673: Attempting to stop 'ora.ons' on '19c-rac3'
CRS-2677: Stop of 'ora.ons' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.net1.network' on '19c-rac3'
CRS-2677: Stop of 'ora.net1.network' on '19c-rac3' succeeded
CRS-2792: Shutdown of Cluster Ready Services-managed resources on '19c-rac3' has completed
CRS-2677: Stop of 'ora.crsd' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.asm' on '19c-rac3'
CRS-2673: Attempting to stop 'ora.crf' on '19c-rac3'
CRS-2673: Attempting to stop 'ora.drivers.acfs' on '19c-rac3'
CRS-2673: Attempting to stop 'ora.mdnsd' on '19c-rac3'
CRS-2677: Stop of 'ora.crf' on '19c-rac3' succeeded
CRS-2677: Stop of 'ora.drivers.acfs' on '19c-rac3' succeeded
CRS-2677: Stop of 'ora.mdnsd' on '19c-rac3' succeeded
CRS-2677: Stop of 'ora.asm' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.cluster_interconnect.haip' on '19c-rac3'
CRS-2677: Stop of 'ora.cluster_interconnect.haip' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.ctssd' on '19c-rac3'
CRS-2673: Attempting to stop 'ora.evmd' on '19c-rac3'
CRS-2677: Stop of 'ora.ctssd' on '19c-rac3' succeeded
CRS-2677: Stop of 'ora.evmd' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.cssd' on '19c-rac3'
CRS-2677: Stop of 'ora.cssd' on '19c-rac3' succeeded
CRS-2673: Attempting to stop 'ora.gipcd' on '19c-rac3'
CRS-2673: Attempting to stop 'ora.gpnpd' on '19c-rac3'
CRS-2677: Stop of 'ora.gpnpd' on '19c-rac3' succeeded
CRS-2677: Stop of 'ora.gipcd' on '19c-rac3' succeeded
CRS-2793: Shutdown of Oracle High Availability Services-managed resources on '19c-rac3' has completed
CRS-4133: Oracle High Availability Services has been stopped.
[root@19c-rac3 bin]#

[root@19c-rac3 bin]# asmcmd afd_configure
AFD-627: AFD distribution files found.
AFD-634: Removing previous AFD installation.
AFD-635: Previous AFD components successfully removed.
AFD-9294: updating file /etc/sysconfig/oracledrivers.conf
AFD-636: Installing requested AFD software.
AFD-637: Loading installed AFD drivers.
AFD-9321: Creating udev for AFD.
AFD-9323: Creating module dependencies - this may take some time.
AFD-9154: Loading 'oracleafd.ko' driver.
AFD-649: Verifying AFD devices.
AFD-9156: Detecting control device '/dev/oracleafd/admin'.
AFD-638: AFD installation correctness verified.
Modifying resource dependencies - this may take some time.
[root@19c-rac3 bin]#

[root@19c-rac3 bin]# asmcmd afd_state
ASMCMD-9526: The AFD state is 'LOADED' and filtering is 'ENABLED' on host '19c-rac3.localdomain'
[root@19c-rac3 bin]#

[root@19c-rac3 bin]# crsctl start crs
CRS-4123: Oracle High Availability Services has been started.
[root@19c-rac3 bin]#
[root@19c-rac3 bin]# ls -lrt afd*
-rwxr-xr-x 1 root oinstall     1000 Nov 12 21:27 afdroot
-rwxr-xr-x 1 root oinstall   280896 Nov 15 09:17 afdtool.bin
-rwxr-xr-x 1 root oinstall 87596064 Nov 15 09:17 afdboot
-rwxr-xr-x 1 root oinstall     2860 Dec 18 20:25 afdtool
-rwxr-xr-x 1 root oinstall      760 Dec 18 20:25 afdload
-rwxr-xr-x 1 root oinstall     3084 Dec 18 20:25 afddriverstate
[root@19c-rac3 bin]#

[root@19c-rac3 bin]# lsmod|grep oracle
oracleafd             221184  1
oracleacfs           4997120  0
oracleadvm            745472  0
oracleoks             729088  2 oracleadvm,oracleacfs
[root@19c-rac3 bin]#

[root@19c-rac3 bin]# cat /etc/os-release | head -n2
NAME="Oracle Linux Server"
VERSION="8.2"
[root@19c-rac3 bin]#

Creating diskgroup ACFSDG using asmca:

[oracle@19c-rac1 ~]$ . oraenv
ORACLE_SID = [cdbrac1] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ asmcmd
ASMCMD> lsdg
State    Type    Rebal  Sector  Logical_Sector  Block       AU  Total_MB  Free_MB  Req_mir_free_MB  Usable_file_MB  Offline_disks  Voting_files  Name
MOUNTED  EXTERN  N         512             512   4096  4194304     49136    25104                0           25104              0             Y  OCRVD/
ASMCMD>
ASMCMD> exit
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ asmca &
[1] 166449
[oracle@19c-rac1 ~]$

Screenshots: 19cr8-ASMFD-1.JPG to 19cr8-ASMFD-4.JPG


















































[oracle@19c-rac1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ?
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ cd $ORACLE_HOME/bin
[oracle@19c-rac1 bin]$
[oracle@19c-rac1 bin]$ pwd
/u01/app/19.3.0/grid/bin
[oracle@19c-rac1 bin]$

[oracle@19c-rac1 bin]$ asmcmd dsget
parameter:/dev/sd*
profile:/dev/sd*
[oracle@19c-rac1 bin]$

[oracle@19c-rac1 bin]$ asmcmd dsset /dev/sd*,'AFD:*'
[oracle@19c-rac1 bin]$

[oracle@19c-rac1 bin]$ asmcmd dsget
parameter:/dev/sd*, AFD:*
profile:/dev/sd*,AFD:*
[oracle@19c-rac1 bin]$

[oracle@19c-rac1 ~]$ su - root
Password:
[root@19c-rac1 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base has been set to /u01/app/oracle
[root@19c-rac1 ~]#

[root@19c-rac1 ~]# cd $ORACLE_HOME/bin
[root@19c-rac1 bin]#

[root@19c-rac1 bin]#  ls -alrt /dev/oracleafd/disks/*
ls: cannot access '/dev/oracleafd/disks/*': No such file or directory

[root@19c-rac1 bin]# ./acfsload start
ACFS-9391: Checking for existing ADVM/ACFS installation.
ACFS-9392: Validating ADVM/ACFS installation files for operating system.
ACFS-9393: Verifying ASM Administrator setup.
ACFS-9308: Loading installed ADVM/ACFS drivers.
ACFS-9325:     Driver OS kernel version = 5.4.17-2011.0.7.el8uek.x86_64.
ACFS-9326:     Driver build number = RELEASE.
ACFS-9212:     Driver build version = 19.0.0.0.0 (19.8.0.0.0).
ACFS-9547:     Driver available build number = RELEASE.
ACFS-9548:     Driver available build version = 19.0.0.0.0 (19.8.0.0.0).
ACFS-9549:     Kernel and command versions.
Kernel:
    Build version: 19.0.0.0.0
    Build full version: 19.8.0.0.0
    Build hash:    9256567290
    Bug numbers:   NoTransactionInformation
Commands:
    Build version: 19.0.0.0.0
    Build full version: 19.8.0.0.0
    Build hash:    9256567290
    Bug numbers:   NoTransactionInformation
ACFS-9327: Verifying ADVM/ACFS devices.
ACFS-9156: Detecting control device '/dev/asm/.asm_ctl_spec'.
ACFS-9156: Detecting control device '/dev/ofsctl'.
ACFS-9294: updating file /etc/sysconfig/oracledrivers.conf
ACFS-9322: completed
[root@19c-rac1 bin]#

[root@19c-rac1 bin]# ls -alrt /dev/oracleafd/disks/*
ls: cannot access '/dev/oracleafd/disks/*': No such file or directory
[root@19c-rac1 bin]#

[root@19c-rac1 bin]# ./asmcmd afd_label ACFSDG01 /dev/sdp1
[root@19c-rac1 bin]# ./asmcmd afd_label ACFSDG02 /dev/sdq1

[root@19c-rac1 bin]# ls -alrt /dev/oracleafd/disks/
total 8
drwxrwx--- 3 oracle dba      80 Dec 18 20:07 ..
-rw-rw-r-- 1 oracle oinstall 10 Dec 18 20:51 ACFSDG01
-rw-rw-r-- 1 oracle oinstall 10 Dec 18 20:51 ACFSDG02
drwxrwxr-x 2 oracle dba      80 Dec 18 20:51 .
[root@19c-rac1 bin]#

Note: Again executed ASMCA and created ACFSDG


















































[root@19c-rac1 bin]# exit
logout
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ?
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Dec 18 20:52:11 2020
Version 19.8.0.0.0
Copyright (c) 1982, 2019, Oracle.  All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

SQL>  SELECT group_number, name, compatibility, database_compatibility FROM v$asm_diskgroup;

GROUP_NUMBER NAME
------------ ------------------------------
COMPATIBILITY
------------------------------------------------------------
DATABASE_COMPATIBILITY
------------------------------------------------------------
           1 OCRVD
19.0.0.0.0
10.1.0.0.0

SQL> exit
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Dec 18 20:59:54 2020
Version 19.8.0.0.0
Copyright (c) 1982, 2019, Oracle.  All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

SQL> set lines 100 pages 100
SQL> col GROUP_NUMBER format 99
SQL> col NAME format a10
SQL> col COMPATIBILITY format a10
SQL> col DATABASE_COMPATIBILITY format a20
SQL> SELECT group_number, name, compatibility, database_compatibility FROM v$asm_diskgroup;

GROUP_NUMBER NAME       COMPATIBIL DATABASE_COMPATIBILI
------------ ---------- ---------- --------------------
           1 OCRVD      19.0.0.0.0 10.1.0.0.0
           2 ACFSDG     19.0.0.0.0 10.1.0.0.0

SQL>  alter diskgroup ACFSDG set attribute 'compatible.rdbms'='19.0.0.0.0';
Diskgroup altered.

SQL> SELECT group_number, name, compatibility, database_compatibility FROM v$asm_diskgroup;

GROUP_NUMBER NAME       COMPATIBIL DATABASE_COMPATIBILI
------------ ---------- ---------- --------------------
           1 OCRVD      19.0.0.0.0 10.1.0.0.0
           2 ACFSDG     19.0.0.0.0 19.0.0.0.0

SQL> exit
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ asmcmd
ASMCMD> volcreate -G ACFSDG -s 19G acfs_vol

ASMCMD> volinfo --all
Diskgroup Name: ACFSDG

         Volume Name: ACFS_VOL
         Volume Device: /dev/asm/acfs_vol-296
         State: ENABLED
         Size (MB): 19456
         Resize Unit (MB): 64
         Redundancy: UNPROT
         Stripe Columns: 8
         Stripe Width (K): 1024
         Usage:
         Mountpath:

ASMCMD> exit
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ su - root
Password:
[root@19c-rac1 ~]#

[root@19c-rac1 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base has been set to /u01/app/oracle
[root@19c-rac1 ~]#

[root@19c-rac1 ~]# modprobe  oracleacfs
[root@19c-rac1 ~]# modprobe  oracleadvm
[root@19c-rac1 ~]# modprobe  oracleoks
[root@19c-rac1 ~]# lsmod | grep oracle
oracleafd             221184  1
oracleacfs           4997120  1
oracleadvm            745472  7
oracleoks             729088  2 oracleadvm,oracleacfs
[root@19c-rac1 ~]#

[root@19c-rac1 ~]# mkfs -t acfs /dev/asm/acfs_vol-296
mkfs.acfs: version                   = 19.0.0.0.0
mkfs.acfs: on-disk version           = 46.0
mkfs.acfs: volume                    = /dev/asm/acfs_vol-296
mkfs.acfs: volume size               = 20401094656  (  19.00 GB )
mkfs.acfs: Format complete.
[root@19c-rac1 ~]#

[root@19c-rac1 ~]# chown -R oracle:oinstall /acfs_vol
chown: cannot access '/acfs_vol': No such file or directory
[root@19c-rac1 ~]#
[root@19c-rac1 ~]# cd /
[root@19c-rac1 /]# mkdir acfs_vol
[root@19c-rac1 /]#
[root@19c-rac1 /]# chown -R oracle:oinstall /acfs_vol
[root@19c-rac1 /]#
[root@19c-rac1 /]# chmod -R 777 /acfs_vol
[root@19c-rac1 /]#
[root@19c-rac1 /]# acfsutil registry -a /dev/asm/acfs_vol-296 /acfs_vol
acfsutil registry: mount point /acfs_vol successfully added to Oracle Registry
[root@19c-rac1 /]#
[root@19c-rac1 /]# acfsutil registry -c /dev/asm/acfs_vol-296 /acfs_vol -u oracle
acfsutil registry: successfully modified ACFS registration for '/dev/asm/acfs_vol-296'
[root@19c-rac1 /]#
[root@19c-rac1 /]# mount -t acfs /dev/asm/acfs_vol-296 /acfs_vol
[root@19c-rac1 /]#

[root@19c-rac1 /]# df -h
Filesystem             Size  Used Avail Use% Mounted on
devtmpfs               4.8G  8.0K  4.8G   1% /dev
tmpfs                  4.8G  845M  3.9G  18% /dev/shm
tmpfs                  4.8G   18M  4.8G   1% /run
tmpfs                  4.8G     0  4.8G   0% /sys/fs/cgroup
/dev/mapper/ol-root     50G   26G   25G  51% /
/dev/mapper/ol-home     42G  7.4G   35G  18% /home
/dev/sda1             1014M  291M  724M  29% /boot
tmpfs                  968M  1.2M  967M   1% /run/user/42
tmpfs                  968M  4.6M  964M   1% /run/user/54321
/dev/asm/acfs_vol-296   19G  828M   19G   5% /acfs_vol
[root@19c-rac1 /]#

[root@19c-rac1 /]# mount | grep acfs
/dev/asm/acfs_vol-296 on /acfs_vol type acfs (rw,relatime,device,rootsuid,ordered)
[root@19c-rac1 /]#

[root@19c-rac1 /]# cat /etc/fstab

#
# /etc/fstab
# Created by anaconda on Sun May 17 21:14:39 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/ol-root     /                       xfs     defaults        0 0
UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot                   xfs     defaults        0 0
/dev/mapper/ol-home     /home                   xfs     defaults        0 0
/dev/mapper/ol-swap     swap                    swap    defaults        0 0
[root@19c-rac1 /]#
[root@19c-rac1 /]# vi /etc/fstab
[root@19c-rac1 /]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Sun May 17 21:14:39 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/ol-root     /                       xfs     defaults        0 0
UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot xfs     defaults        0 0
/dev/mapper/ol-home     /home                   xfs     defaults        0 0
/dev/mapper/ol-swap     swap                    swap    defaults        0 0
/dev/asm/acfs_vol-296   /acfs_vol               acfs    defaults        0 0
[root@19c-rac1 /]#

[root@19c-rac1 /]# crsctl stat res -t | grep acfs
ora.acfsdg.acfs_vol.acfs
               ONLINE  ONLINE       19c-rac1                 mounted on /acfs_vol
               ONLINE  ONLINE       19c-rac2                 mounted on /acfs_vol
               ONLINE  ONLINE       19c-rac3                 mounted on /acfs_vol
[root@19c-rac1 /]#

[root@19c-rac1 /]# crsctl stat res -t | grep ACFS
ora.ACFSDG.ACFS_VOL.advm
ora.ACFSDG.dg(ora.asmgroup)
[root@19c-rac1 /]#

[root@19c-rac1 /]# srvctl status diskgroup -diskgroup ACFSDG -verbose
Disk Group ACFSDG is running on 19c-rac1,19c-rac3,19c-rac2
[root@19c-rac1 /]#
[root@19c-rac1 /]# exit
logout

[oracle@19c-rac1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ?
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Dec 18 21:10:08 2020
Version 19.8.0.0.0
Copyright (c) 1982, 2019, Oracle.  All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

SQL> set lines 100 pages 100
SQL>  col name for a20
col path for a18
col library for a60
set lines 160

SQL> select name, path, library from v$asm_disk where group_number <> 0;

NAME                 PATH               LIBRARY
-------------------- ------------------ ------------------------------------------------------------
OCRVD_0002           /dev/sdl1          System
OCRVD_0003           /dev/sdm1          System
OCRVD_0000           /dev/sdj1          System
OCRVD_0001           /dev/sdk1          System
ACFSDG01             AFD:ACFSDG01       AFD Library - Generic , version 3 (KABI_V3)
ACFSDG02             AFD:ACFSDG02       AFD Library - Generic , version 3 (KABI_V3)

6 rows selected.

SQL> exit
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

[oracle@19c-rac1 ~]$ cd $ORACLE_HOME/bin
[oracle@19c-rac1 bin]$ ./asmcmd afd_lsdsk
--------------------------------------------------------------------------------
Label                     Filtering   Path
=========================================================================ACFSDG01                    ENABLED   /dev/sdp1
ACFSDG02                    ENABLED   /dev/sdq1
[oracle@19c-rac1 bin]$

[oracle@19c-rac2 ~]$ . oraenv
ORACLE_SID = [+ASM2] ?
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac2 ~]$

[oracle@19c-rac2 ~]$ cd $ORACLE_HOME/bin
[oracle@19c-rac2 bin]$ asmcmd dsget
parameter:/dev/sd*, AFD:*
profile:/dev/sd*,AFD:*
[oracle@19c-rac2 bin]$

[oracle@19c-rac2 bin]$ cd
[oracle@19c-rac2 ~]$ su - root
Password:
[root@19c-rac2 ~]#

[root@19c-rac2 /]# df -h
Filesystem             Size  Used Avail Use% Mounted on
devtmpfs               4.8G  8.0K  4.8G   1% /dev
tmpfs                  4.8G  845M  3.9G  18% /dev/shm
tmpfs                  4.8G   26M  4.7G   1% /run
tmpfs                  4.8G     0  4.8G   0% /sys/fs/cgroup
/dev/mapper/ol-root     50G   25G   26G  49% /
/dev/sda1             1014M  290M  725M  29% /boot
/dev/mapper/ol-home     42G  1.8G   40G   5% /home
tmpfs                  968M  1.2M  967M   1% /run/user/42
tmpfs                  968M     0  968M   0% /run/user/54321
/dev/asm/acfs_vol-296   19G  828M   19G   5% /acfs_vol
[root@19c-rac2 /]#

[root@19c-rac2 /]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Sun May 17 21:14:39 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/ol-root     /                       xfs     defaults        0 0
UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot                   xfs     defaults        0 0
/dev/mapper/ol-home     /home                   xfs     defaults        0 0
/dev/mapper/ol-swap     swap                    swap    defaults        0 0
[root@19c-rac2 /]#

[root@19c-rac2 /]# vi /etc/fstab
[root@19c-rac2 /]# cat /etc/fstab

#
# /etc/fstab
# Created by anaconda on Sun May 17 21:14:39 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/ol-root     /                       xfs     defaults        0 0
UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot                   xfs     defaults        0 0
/dev/mapper/ol-home     /home                   xfs     defaults        0 0
/dev/mapper/ol-swap     swap                    swap    defaults        0 0
/dev/asm/acfs_vol-296   /acfs_vol               acfs    defaults        0 0
[root@19c-rac2 /]#

[oracle@19c-rac3 ~]$ . oraenv
ORACLE_SID = [+ASM3] ?
ORACLE_HOME = [/home/oracle] ? /u01/app/19.3.0/grid
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac3 ~]$

[oracle@19c-rac3 ~]$ cd $ORACLE_HOME/bin
[oracle@19c-rac3 bin]$

[oracle@19c-rac3 bin]$ asmcmd dsget
parameter:/dev/sd*, AFD:*
profile:/dev/sd*,AFD:*

[oracle@19c-rac3 ~]$ df -h
Filesystem             Size  Used Avail Use% Mounted on
devtmpfs               4.8G  8.0K  4.8G   1% /dev
tmpfs                  4.8G  845M  3.9G  18% /dev/shm
tmpfs                  4.8G   34M  4.7G   1% /run
tmpfs                  4.8G     0  4.8G   0% /sys/fs/cgroup
/dev/mapper/ol-root     50G   25G   26G  49% /
/dev/mapper/ol-home     42G  1.8G   40G   5% /home
/dev/sda1             1014M  290M  725M  29% /boot
tmpfs                  968M  1.2M  967M   1% /run/user/42
tmpfs                  968M     0  968M   0% /run/user/54321
/dev/asm/acfs_vol-296   19G  828M   19G   5% /acfs_vol
[oracle@19c-rac3 ~]$

[oracle@19c-rac3 ~]$ su - root
Password:
[root@19c-rac3 ~]#

[root@19c-rac3 ~]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Sun May 17 21:14:39 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/ol-root     /                       xfs     defaults        0 0
UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot xfs     defaults        0 0
/dev/mapper/ol-home     /home                   xfs     defaults        0 0
/dev/mapper/ol-swap     swap                    swap    defaults        0 0
[root@19c-rac3 ~]#

[root@19c-rac3 ~]# vi /etc/fstab
[root@19c-rac3 ~]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Sun May 17 21:14:39 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk/'.
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info.
#
# After editing this file, run 'systemctl daemon-reload' to update systemd
# units generated from this file.
#
/dev/mapper/ol-root     /                       xfs     defaults        0 0
UUID=539eae26-4e39-4888-8697-cb027503ef1b /boot xfs     defaults        0 0
/dev/mapper/ol-home     /home                   xfs     defaults        0 0
/dev/mapper/ol-swap     swap                    swap    defaults        0 0
/dev/asm/acfs_vol-296   /acfs_vol               acfs    defaults        0 0
[root@19c-rac3 ~]#

Final Status





























Hope it helps...Thank You for referring my blog.
Posted by at 5 comments:
Subscribe to: Posts (Atom)