Oracle 12c (12.1.0.2) on Oracle Linux 8.2 and Oracle GI 19c (19.6.0) - - > 3 Node RAC

Oracle 12c (12.1.0.2) on Oracle Linux 8.2 and Oracle GI 19c (19.6.0) - - > 3 Node RAC

Overall Steps:

1. Installed Oracle Enterprise Linux 8.2
2. Installed GRID Infrastructure Oracle 19c (19.3.0).
3. While installation GI, applied Oracle 19c (19.6.0) RU
   ./gridSetup.sh -applyPSU /home/oracle/30501910/
4. Installed RDBMS Oracle 12c (12.1.0.2.0).
5. Applied patch "p21626377_121020_Linux-x86-64.zip"
   (BUG 21626377 - 12.2_150812: DBCA FAILS TO CREATE 12102 DB OVER 12.2 GI/ASM). 
6. Created and Configured Oracle 12c (12.1.0.2) on Oracle Linux 8.2 & Oracle GI 19c (19.6.0) RU

Required Softwares:

1. Oracle Linux 8.2
2. Oracle 19c GRID (19.3.0) binaries
   LINUX.X64_193000_grid_home.zip
3. Oracle 19c GRID (19.6.0) Release Update RU)
   p30501910_190000_Linux-x86-64.zip
4. Oracle 12c RDBMS (12.1.0.2) binaries
5. Latest OPatch   
   p6880880_190000_Linux-x86-64.zip
6. Patch for Oracle 12c RDBMS (12.1.0.2) for GI 12.2 or Above
   p21626377_121020_Linux-x86-64.zip

Step-1: Apply latest OPatch

[oracle@19c-rac1 ~]$ unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/19.3.0/grid/
Archive:  p6880880_190000_Linux-x86-64.zip
replace /u01/app/19.3.0/grid/OPatch/emdpatch.pl? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
  inflating: /u01/app/19.3.0/grid/OPatch/emdpatch.pl
  inflating: /u01/app/19.3.0/grid/OPatch/oracle_common/modules/com.oracle.glcm.common-logging_1.6.5.0.jar

Step-2: Unzip the softwares - Oracle 19c GI (19.6.0) and Oracle 19c GI (19.6.0) RU 

[oracle@19c-rac1 ~]$ unzip p30501910_190000_Linux-x86-64.zip
Archive:  p30501910_190000_Linux-x86-64.zip
   creating: 30501910/
   creating: 30501910/30557433/
  inflating: 30501910/30557433/README.txt
   creating: 30501910/30557433/files/
   creating: 30501910/30557433/files/network/

Step-3: Install 19c GI with 19c (19.6.0) RU

[oracle@19c-rac1 grid]$ cd /u01/app/19.3.0/grid/
[oracle@19c-rac1 grid]$ ./gridSetup.sh -applyPSU /home/oracle/30501910/
Preparing the home to patch...
Applying the patch /home/oracle/30501910/...
Successfully applied the patch.
The log can be found at: /tmp/GridSetupActions2020-07-25_02-43-53PM/installerPatchActions_2020-07-25_02-43-53PM.log











Step-4: Execute scripts as a 'root' user

[root@19c-rac1 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.

[root@19c-rac2 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@19c-rac2 ~]#

[root@19c-rac3 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@19c-rac3 ~]#

[root@19c-rac1 ~]# /u01/app/19.3.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/19.3.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.3.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/19c-rac1/crsconfig/rootcrs_19c-rac1_2020-07-25_07-12-38PM.log
2020/07/25 19:12:49 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2020/07/25 19:12:49 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2020/07/25 19:12:49 CLSRSC-363: User ignored prerequisites during installation
2020/07/25 19:12:49 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2020/07/25 19:12:51 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2020/07/25 19:12:53 CLSRSC-594: Executing installation step 5 of 19: 'SetupOSD'.
2020/07/25 19:12:53 CLSRSC-594: Executing installation step 6 of 19: 'CheckCRSConfig'.
2020/07/25 19:12:53 CLSRSC-594: Executing installation step 7 of 19: 'SetupLocalGPNP'.
2020/07/25 19:13:24 CLSRSC-594: Executing installation step 8 of 19: 'CreateRootCert'.
2020/07/25 19:13:27 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2020/07/25 19:13:31 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2020/07/25 19:13:48 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2020/07/25 19:13:48 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2020/07/25 19:13:53 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2020/07/25 19:13:54 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2020/07/25 19:14:18 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2020/07/25 19:14:23 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2020/07/25 19:14:36 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2020/07/25 19:14:41 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
Redirecting to /bin/systemctl restart rsyslog.service

ASM has been created and started successfully.

[DBT-30001] Disk groups created successfully. Check /u01/app/oracle/cfgtoollogs/asmca/asmca-200725PM071514.log for details.

2020/07/25 19:16:30 CLSRSC-482: Running command: '/u01/app/19.3.0/grid/bin/ocrconfig -upgrade oracle oinstall'
CRS-4256: Updating the profile
Successful addition of voting disk ebeea6dcabca4f1ebfd5c96606a31426.
Successfully replaced voting disk group with +OCRVD.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   ebeea6dcabca4f1ebfd5c96606a31426 (/dev/sdj1) [OCRVD]
Located 1 voting disk(s).
2020/07/25 19:17:59 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2020/07/25 19:19:19 CLSRSC-343: Successfully started Oracle Clusterware stack
2020/07/25 19:19:20 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2020/07/25 19:20:58 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2020/07/25 19:21:22 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@19c-rac1 ~]#


[root@19c-rac2 ~]# /u01/app/19.3.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/19.3.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.3.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/19c-rac2/crsconfig/rootcrs_19c-rac2_2020-07-25_07-22-01PM.log
2020/07/25 19:22:08 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2020/07/25 19:22:08 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2020/07/25 19:22:09 CLSRSC-363: User ignored prerequisites during installation
2020/07/25 19:22:09 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2020/07/25 19:22:10 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2020/07/25 19:22:10 CLSRSC-594: Executing installation step 5 of 19: 'SetupOSD'.
2020/07/25 19:22:10 CLSRSC-594: Executing installation step 6 of 19: 'CheckCRSConfig'.
2020/07/25 19:22:11 CLSRSC-594: Executing installation step 7 of 19: 'SetupLocalGPNP'.
2020/07/25 19:22:13 CLSRSC-594: Executing installation step 8 of 19: 'CreateRootCert'.
2020/07/25 19:22:13 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2020/07/25 19:22:31 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2020/07/25 19:22:31 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2020/07/25 19:22:33 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2020/07/25 19:22:34 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2020/07/25 19:22:37 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2020/07/25 19:22:54 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2020/07/25 19:22:56 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2020/07/25 19:22:57 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2020/07/25 19:22:59 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
Redirecting to /bin/systemctl restart rsyslog.service
2020/07/25 19:23:07 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2020/07/25 19:24:01 CLSRSC-343: Successfully started Oracle Clusterware stack
2020/07/25 19:24:02 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2020/07/25 19:24:16 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2020/07/25 19:24:23 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@19c-rac2 ~]#

[root@19c-rac3 ~]# /u01/app/19.3.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/19.3.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.3.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/19c-rac3/crsconfig/rootcrs_19c-rac3_2020-07-25_07-25-01PM.log
2020/07/25 19:25:08 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2020/07/25 19:25:09 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2020/07/25 19:25:09 CLSRSC-363: User ignored prerequisites during installation
2020/07/25 19:25:09 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2020/07/25 19:25:10 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2020/07/25 19:25:10 CLSRSC-594: Executing installation step 5 of 19: 'SetupOSD'.
2020/07/25 19:25:10 CLSRSC-594: Executing installation step 6 of 19: 'CheckCRSConfig'.
2020/07/25 19:25:11 CLSRSC-594: Executing installation step 7 of 19: 'SetupLocalGPNP'.
2020/07/25 19:25:13 CLSRSC-594: Executing installation step 8 of 19: 'CreateRootCert'.
2020/07/25 19:25:13 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2020/07/25 19:25:33 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2020/07/25 19:25:33 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2020/07/25 19:25:36 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2020/07/25 19:25:36 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2020/07/25 19:25:39 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2020/07/25 19:25:57 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2020/07/25 19:25:59 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2020/07/25 19:26:00 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2020/07/25 19:26:01 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
Redirecting to /bin/systemctl restart rsyslog.service
2020/07/25 19:26:11 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2020/07/25 19:27:05 CLSRSC-343: Successfully started Oracle Clusterware stack
2020/07/25 19:27:06 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2020/07/25 19:27:22 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2020/07/25 19:27:29 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@19c-rac3 ~]#

Step-5: Installed of Oracle 12c (12.1.0.2) binaries

Step-6: Check the compatible of RDBMS on ASM Instance


[oracle@19c-rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Sat Jul 25 22:08:28 2020
Version 19.6.0.0.0
Copyright (c) 1982, 2019, Oracle.  All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.6.0.0.0

SQL> set lines 180 pages 200
SQL> col name format a10
SQL> col COMPATIBILITY format a10
SQL> col DATABASE_COMPATIBILITY format a10
SQL> select NAME,STATE,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;

NAME       STATE       COMPATIBIL DATABASE_C
---------- ----------- ---------- ----------
DATADG     MOUNTED     19.0.0.0.0 12.1.0.2.0
OCRVD      MOUNTED     19.0.0.0.0 10.1.0.0.0
RECODG     MOUNTED     19.0.0.0.0 12.1.0.2.0
SQL>

SQL> alter diskgroup OCRVD set attribute 'compatible.rdbms'='12.1.0.2.0';
Diskgroup altered.

SQL> select NAME,STATE,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;

NAME       STATE       COMPATIBIL DATABASE_C
---------- ----------- ---------- ----------
DATADG     MOUNTED     19.0.0.0.0 12.1.0.2.0
OCRVD      MOUNTED     19.0.0.0.0 12.1.0.2.0
RECODG     MOUNTED     19.0.0.0.0 12.1.0.2.0
SQL>

Step-7: Got the following error while creation of Oracle 12c (12.1.0.2) Database:


Please refer for more details:
"Recovery Manager Failed to Restore Datafiles" - Using DBCA To Create Database 12.1.0.2.0 
 on Grid Infrastructure / ASM 12.2.0.1.0 (Doc ID 2260473.1)



Step-8: Applied patch on cluster nodes

[oracle@19c-rac1 OPatch]$ export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/db_1
[oracle@19c-rac1 OPatch]$ ./opatch apply -local /home/oracle/21626377/
Oracle Interim Patch Installer version 12.1.0.1.3
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/oracle/product/12.1.0.2/db_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/12.1.0.2/db_1/oraInst.loc
OPatch version    : 12.1.0.1.3
OUI version       : 12.1.0.2.0
Log file location : /u01/app/oracle/product/12.1.0.2/db_1/cfgtoollogs/opatch/opatch2020-07-25_23-34-11PM_1.log

Applying interim patch '21626377' to OH '/u01/app/oracle/product/12.1.0.2/db_1'
Verifying environment and performing prerequisite checks...
All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/12.1.0.2/db_1')

Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...

Patching component oracle.rdbms, 12.1.0.2.0...

Verifying the update...
Patch 21626377 successfully applied
Log file location: /u01/app/oracle/product/12.1.0.2/db_1/cfgtoollogs/opatch/opatch2020-07-25_23-34-11PM_1.log

OPatch succeeded.
[oracle@19c-rac1 OPatch]$

Step-9: Check the status of ASM and RDBMS instances and version of Oracle Linux


[oracle@19c-rac1 ~]$ cat /etc/os-release | grep Linux | grep 8.2
PRETTY_NAME="Oracle Linux Server 8.2"
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sqlplus sys@db12c as sysdba

SQL*Plus: Release 12.1.0.2.0 Production on Sat Jul 25 23:58:15 2020
Copyright (c) 1982, 2014, Oracle.  All rights reserved.
Enter password:
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select instance_name,instance_number from gv$instance;

INSTANCE_NAME    INSTANCE_NUMBER
---------------- ---------------
db12c3                         3
db12c2                         2
db12c1                         1

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDB1                           READ WRITE NO
         4 PDB2                           READ WRITE NO
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ grid_env
[oracle@19c-rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Sat Jul 25 23:58:50 2020
Version 19.6.0.0.0
Copyright (c) 1982, 2019, Oracle.  All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.6.0.0.0

SQL> select instance_name,instance_number from gv$instance;

INSTANCE_NAME    INSTANCE_NUMBER
---------------- ---------------
+ASM1                          1
+ASM3                          3
+ASM2                          2
SQL>

Hope it helps......