Saturday, July 25, 2020

Oracle 12c (12.1.0.2) on Oracle Linux 8.2 and Oracle GI 19c (19.6.0) - - > 3 Node RAC


Oracle 12c (12.1.0.2) on Oracle Linux 8.2 and Oracle GI 19c (19.6.0) - - > 3 Node RAC

Overall Steps:

1. Installed Oracle Enterprise Linux 8.2
2. Installed GRID Infrastructure Oracle 19c (19.3.0).
3. While installation GI, applied Oracle 19c (19.6.0) RU
   ./gridSetup.sh -applyPSU /home/oracle/30501910/
4. Installed RDBMS Oracle 12c (12.1.0.2.0).
5. Applied patch "p21626377_121020_Linux-x86-64.zip"
   (BUG 21626377 - 12.2_150812: DBCA FAILS TO CREATE 12102 DB OVER 12.2 GI/ASM). 
6. Created and Configured Oracle 12c (12.1.0.2) on Oracle Linux 8.2 & Oracle GI 19c (19.6.0) RU

Required Softwares:

1. Oracle Linux 8.2
2. Oracle 19c GRID (19.3.0) binaries
   LINUX.X64_193000_grid_home.zip
3. Oracle 19c GRID (19.6.0) Release Update RU)
   p30501910_190000_Linux-x86-64.zip
4. Oracle 12c RDBMS (12.1.0.2) binaries
5. Latest OPatch   
   p6880880_190000_Linux-x86-64.zip
6. Patch for Oracle 12c RDBMS (12.1.0.2) for GI 12.2 or Above
   p21626377_121020_Linux-x86-64.zip

Step-1: Apply latest OPatch

[oracle@19c-rac1 ~]$ unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/19.3.0/grid/
Archive:  p6880880_190000_Linux-x86-64.zip
replace /u01/app/19.3.0/grid/OPatch/emdpatch.pl? [y]es, [n]o, [A]ll, [N]one, [r]ename: A
  inflating: /u01/app/19.3.0/grid/OPatch/emdpatch.pl
  inflating: /u01/app/19.3.0/grid/OPatch/oracle_common/modules/com.oracle.glcm.common-logging_1.6.5.0.jar

Step-2: Unzip the softwares - Oracle 19c GI (19.6.0) and Oracle 19c GI (19.6.0) RU 

[oracle@19c-rac1 ~]$ unzip p30501910_190000_Linux-x86-64.zip
Archive:  p30501910_190000_Linux-x86-64.zip
   creating: 30501910/
   creating: 30501910/30557433/
  inflating: 30501910/30557433/README.txt
   creating: 30501910/30557433/files/
   creating: 30501910/30557433/files/network/

Step-3: Install 19c GI with 19c (19.6.0) RU

[oracle@19c-rac1 grid]$ cd /u01/app/19.3.0/grid/
[oracle@19c-rac1 grid]$ ./gridSetup.sh -applyPSU /home/oracle/30501910/
Preparing the home to patch...
Applying the patch /home/oracle/30501910/...
Successfully applied the patch.
The log can be found at: /tmp/GridSetupActions2020-07-25_02-43-53PM/installerPatchActions_2020-07-25_02-43-53PM.log











Step-4: Execute scripts as a 'root' user

[root@19c-rac1 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.

[root@19c-rac2 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@19c-rac2 ~]#

[root@19c-rac3 ~]# /u01/app/oraInventory/orainstRoot.sh
Changing permissions of /u01/app/oraInventory.
Adding read,write permissions for group.
Removing read,write,execute permissions for world.

Changing groupname of /u01/app/oraInventory to oinstall.
The execution of the script is complete.
[root@19c-rac3 ~]#

[root@19c-rac1 ~]# /u01/app/19.3.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/19.3.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.3.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/19c-rac1/crsconfig/rootcrs_19c-rac1_2020-07-25_07-12-38PM.log
2020/07/25 19:12:49 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2020/07/25 19:12:49 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2020/07/25 19:12:49 CLSRSC-363: User ignored prerequisites during installation
2020/07/25 19:12:49 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2020/07/25 19:12:51 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2020/07/25 19:12:53 CLSRSC-594: Executing installation step 5 of 19: 'SetupOSD'.
2020/07/25 19:12:53 CLSRSC-594: Executing installation step 6 of 19: 'CheckCRSConfig'.
2020/07/25 19:12:53 CLSRSC-594: Executing installation step 7 of 19: 'SetupLocalGPNP'.
2020/07/25 19:13:24 CLSRSC-594: Executing installation step 8 of 19: 'CreateRootCert'.
2020/07/25 19:13:27 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2020/07/25 19:13:31 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2020/07/25 19:13:48 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2020/07/25 19:13:48 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2020/07/25 19:13:53 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2020/07/25 19:13:54 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2020/07/25 19:14:18 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2020/07/25 19:14:23 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2020/07/25 19:14:36 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2020/07/25 19:14:41 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
Redirecting to /bin/systemctl restart rsyslog.service

ASM has been created and started successfully.

[DBT-30001] Disk groups created successfully. Check /u01/app/oracle/cfgtoollogs/asmca/asmca-200725PM071514.log for details.

2020/07/25 19:16:30 CLSRSC-482: Running command: '/u01/app/19.3.0/grid/bin/ocrconfig -upgrade oracle oinstall'
CRS-4256: Updating the profile
Successful addition of voting disk ebeea6dcabca4f1ebfd5c96606a31426.
Successfully replaced voting disk group with +OCRVD.
CRS-4256: Updating the profile
CRS-4266: Voting file(s) successfully replaced
##  STATE    File Universal Id                File Name Disk group
--  -----    -----------------                --------- ---------
 1. ONLINE   ebeea6dcabca4f1ebfd5c96606a31426 (/dev/sdj1) [OCRVD]
Located 1 voting disk(s).
2020/07/25 19:17:59 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2020/07/25 19:19:19 CLSRSC-343: Successfully started Oracle Clusterware stack
2020/07/25 19:19:20 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2020/07/25 19:20:58 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2020/07/25 19:21:22 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@19c-rac1 ~]#


[root@19c-rac2 ~]# /u01/app/19.3.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/19.3.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.3.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/19c-rac2/crsconfig/rootcrs_19c-rac2_2020-07-25_07-22-01PM.log
2020/07/25 19:22:08 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2020/07/25 19:22:08 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2020/07/25 19:22:09 CLSRSC-363: User ignored prerequisites during installation
2020/07/25 19:22:09 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2020/07/25 19:22:10 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2020/07/25 19:22:10 CLSRSC-594: Executing installation step 5 of 19: 'SetupOSD'.
2020/07/25 19:22:10 CLSRSC-594: Executing installation step 6 of 19: 'CheckCRSConfig'.
2020/07/25 19:22:11 CLSRSC-594: Executing installation step 7 of 19: 'SetupLocalGPNP'.
2020/07/25 19:22:13 CLSRSC-594: Executing installation step 8 of 19: 'CreateRootCert'.
2020/07/25 19:22:13 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2020/07/25 19:22:31 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2020/07/25 19:22:31 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2020/07/25 19:22:33 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2020/07/25 19:22:34 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2020/07/25 19:22:37 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2020/07/25 19:22:54 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2020/07/25 19:22:56 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2020/07/25 19:22:57 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2020/07/25 19:22:59 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
Redirecting to /bin/systemctl restart rsyslog.service
2020/07/25 19:23:07 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2020/07/25 19:24:01 CLSRSC-343: Successfully started Oracle Clusterware stack
2020/07/25 19:24:02 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2020/07/25 19:24:16 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2020/07/25 19:24:23 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@19c-rac2 ~]#

[root@19c-rac3 ~]# /u01/app/19.3.0/grid/root.sh
Performing root user operation.

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /u01/app/19.3.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Relinking oracle with rac_on option
Using configuration parameter file: /u01/app/19.3.0/grid/crs/install/crsconfig_params
The log of current session can be found at:
  /u01/app/oracle/crsdata/19c-rac3/crsconfig/rootcrs_19c-rac3_2020-07-25_07-25-01PM.log
2020/07/25 19:25:08 CLSRSC-594: Executing installation step 1 of 19: 'SetupTFA'.
2020/07/25 19:25:09 CLSRSC-594: Executing installation step 2 of 19: 'ValidateEnv'.
2020/07/25 19:25:09 CLSRSC-363: User ignored prerequisites during installation
2020/07/25 19:25:09 CLSRSC-594: Executing installation step 3 of 19: 'CheckFirstNode'.
2020/07/25 19:25:10 CLSRSC-594: Executing installation step 4 of 19: 'GenSiteGUIDs'.
2020/07/25 19:25:10 CLSRSC-594: Executing installation step 5 of 19: 'SetupOSD'.
2020/07/25 19:25:10 CLSRSC-594: Executing installation step 6 of 19: 'CheckCRSConfig'.
2020/07/25 19:25:11 CLSRSC-594: Executing installation step 7 of 19: 'SetupLocalGPNP'.
2020/07/25 19:25:13 CLSRSC-594: Executing installation step 8 of 19: 'CreateRootCert'.
2020/07/25 19:25:13 CLSRSC-594: Executing installation step 9 of 19: 'ConfigOLR'.
2020/07/25 19:25:33 CLSRSC-594: Executing installation step 10 of 19: 'ConfigCHMOS'.
2020/07/25 19:25:33 CLSRSC-594: Executing installation step 11 of 19: 'CreateOHASD'.
2020/07/25 19:25:36 CLSRSC-594: Executing installation step 12 of 19: 'ConfigOHASD'.
2020/07/25 19:25:36 CLSRSC-330: Adding Clusterware entries to file 'oracle-ohasd.service'
2020/07/25 19:25:39 CLSRSC-4002: Successfully installed Oracle Trace File Analyzer (TFA) Collector.
2020/07/25 19:25:57 CLSRSC-594: Executing installation step 13 of 19: 'InstallAFD'.
2020/07/25 19:25:59 CLSRSC-594: Executing installation step 14 of 19: 'InstallACFS'.
2020/07/25 19:26:00 CLSRSC-594: Executing installation step 15 of 19: 'InstallKA'.
2020/07/25 19:26:01 CLSRSC-594: Executing installation step 16 of 19: 'InitConfig'.
Redirecting to /bin/systemctl restart rsyslog.service
2020/07/25 19:26:11 CLSRSC-594: Executing installation step 17 of 19: 'StartCluster'.
2020/07/25 19:27:05 CLSRSC-343: Successfully started Oracle Clusterware stack
2020/07/25 19:27:06 CLSRSC-594: Executing installation step 18 of 19: 'ConfigNode'.
2020/07/25 19:27:22 CLSRSC-594: Executing installation step 19 of 19: 'PostConfig'.
2020/07/25 19:27:29 CLSRSC-325: Configure Oracle Grid Infrastructure for a Cluster ... succeeded
[root@19c-rac3 ~]#

Step-5: Installed of Oracle 12c (12.1.0.2) binaries

Step-6: Check the compatible of RDBMS on ASM Instance


[oracle@19c-rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Sat Jul 25 22:08:28 2020
Version 19.6.0.0.0
Copyright (c) 1982, 2019, Oracle.  All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.6.0.0.0

SQL> set lines 180 pages 200
SQL> col name format a10
SQL> col COMPATIBILITY format a10
SQL> col DATABASE_COMPATIBILITY format a10
SQL> select NAME,STATE,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;

NAME       STATE       COMPATIBIL DATABASE_C
---------- ----------- ---------- ----------
DATADG     MOUNTED     19.0.0.0.0 12.1.0.2.0
OCRVD      MOUNTED     19.0.0.0.0 10.1.0.0.0
RECODG     MOUNTED     19.0.0.0.0 12.1.0.2.0
SQL>

SQL> alter diskgroup OCRVD set attribute 'compatible.rdbms'='12.1.0.2.0';
Diskgroup altered.

SQL> select NAME,STATE,COMPATIBILITY,DATABASE_COMPATIBILITY from v$asm_diskgroup;

NAME       STATE       COMPATIBIL DATABASE_C
---------- ----------- ---------- ----------
DATADG     MOUNTED     19.0.0.0.0 12.1.0.2.0
OCRVD      MOUNTED     19.0.0.0.0 12.1.0.2.0
RECODG     MOUNTED     19.0.0.0.0 12.1.0.2.0
SQL>

Step-7: Got the following error while creation of Oracle 12c (12.1.0.2) Database:


Please refer for more details:
"Recovery Manager Failed to Restore Datafiles" - Using DBCA To Create Database 12.1.0.2.0 
 on Grid Infrastructure / ASM 12.2.0.1.0 (Doc ID 2260473.1)



Step-8: Applied patch on cluster nodes

[oracle@19c-rac1 OPatch]$ export ORACLE_HOME=/u01/app/oracle/product/12.1.0.2/db_1
[oracle@19c-rac1 OPatch]$ ./opatch apply -local /home/oracle/21626377/
Oracle Interim Patch Installer version 12.1.0.1.3
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

Oracle Home       : /u01/app/oracle/product/12.1.0.2/db_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/12.1.0.2/db_1/oraInst.loc
OPatch version    : 12.1.0.1.3
OUI version       : 12.1.0.2.0
Log file location : /u01/app/oracle/product/12.1.0.2/db_1/cfgtoollogs/opatch/opatch2020-07-25_23-34-11PM_1.log

Applying interim patch '21626377' to OH '/u01/app/oracle/product/12.1.0.2/db_1'
Verifying environment and performing prerequisite checks...
All checks passed.

Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = '/u01/app/oracle/product/12.1.0.2/db_1')

Is the local system ready for patching? [y|n]
y
User Responded with: Y
Backing up files...

Patching component oracle.rdbms, 12.1.0.2.0...

Verifying the update...
Patch 21626377 successfully applied
Log file location: /u01/app/oracle/product/12.1.0.2/db_1/cfgtoollogs/opatch/opatch2020-07-25_23-34-11PM_1.log

OPatch succeeded.
[oracle@19c-rac1 OPatch]$

Step-9: Check the status of ASM and RDBMS instances and version of Oracle Linux


[oracle@19c-rac1 ~]$ cat /etc/os-release | grep Linux | grep 8.2
PRETTY_NAME="Oracle Linux Server 8.2"
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sqlplus sys@db12c as sysdba

SQL*Plus: Release 12.1.0.2.0 Production on Sat Jul 25 23:58:15 2020
Copyright (c) 1982, 2014, Oracle.  All rights reserved.
Enter password:
Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options

SQL> select instance_name,instance_number from gv$instance;

INSTANCE_NAME    INSTANCE_NUMBER
---------------- ---------------
db12c3                         3
db12c2                         2
db12c1                         1

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDB1                           READ WRITE NO
         4 PDB2                           READ WRITE NO
SQL> exit
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management, OLAP,
Advanced Analytics and Real Application Testing options
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ grid_env
[oracle@19c-rac1 ~]$ sqlplus / as sysasm

SQL*Plus: Release 19.0.0.0.0 - Production on Sat Jul 25 23:58:50 2020
Version 19.6.0.0.0
Copyright (c) 1982, 2019, Oracle.  All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.6.0.0.0

SQL> select instance_name,instance_number from gv$instance;

INSTANCE_NAME    INSTANCE_NUMBER
---------------- ---------------
+ASM1                          1
+ASM3                          3
+ASM2                          2
SQL>

Hope it helps......

Sunday, July 19, 2020

Database In-Memory Base Level Feature Now Available in Oracle 19c (19.8.0.0.0) Release Update (RU)


Database In-Memory Base Level Feature Now Available in Oracle 19c (19.8.0.0.0) RU



The Database In-Memory Base Level feature is now available for Oracle Database 19c! Support 
for the Base Level feature was added to 19c in the just released 19.8 RU. 
As a brief review, with the Base Level feature you can use up to a 16GB column store 
without having to license the Database In-Memory option. 

This now gives anyone running Oracle Database 19c Enterprise Edition the ability to try 
out Database In-Memory for in-production workloads, without having to license the 
full Database In-Memory option first.

Simply apply the 19.8 RU and set the INMEMORY_FORCE initialization parameter to BASE_LEVEL and 
set the INMEMORY_SIZE parameter to a value up to 16GB. When you restart your database, 
the Base Level feature will then be enabled. 
That's really all there is to it. 

Oracle Database In-Memory will now have a "Base Level" feature. 
This will allow the use of Database In-Memory with up to a 16GB column store for no extra cost. 
That's not a typo. Database In-Memory will be free to use with up to a 16GB column store!

Note: Apply July 2020 RU to get Oracle 19c (19.8.0.0.0)

Required Softwares:
-------------------

Latest OPatch:                       p6880880_190000_Linux-x86-64.zip
GI Release Update (July 2020):       p31305339_190000_Linux-x86-64.zip
Database Release Update (July 2020): p31281355_190000_Linux-x86-64.zip

Step-1: Check the cluster instances
------------------------------------
[oracle@rac1-oel72 ~]$ ps -ef | grep pmon
oracle    6818     1  0 20:35 ?        00:00:00 asm_pmon_+ASM1
oracle   13729     1  0 20:38 ?        00:00:00 ora_pmon_orcl1
oracle   21836 12071  0 20:46 pts/0    00:00:00 grep --color=auto pmon
[oracle@rac1-oel72 ~]$

[oracle@rac2-oel72 ~]$ ps -ef | grep pmon
oracle    2127     1  0 20:43 ?        00:00:00 asm_pmon_+ASM2
oracle    2692     1  0 20:44 ?        00:00:00 ora_pmon_orcl2
oracle    3033     1  0 20:45 ?        00:00:00 mdb_pmon_-MGMTDB
oracle    3455 20208  0 20:46 pts/0    00:00:00 grep --color=auto pmon
[oracle@rac2-oel72 ~]$

[oracle@rac1-oel72 ~]$ srvctl status database -d orcl
Instance orcl1 is running on node rac1-oel72
Instance orcl2 is running on node rac2-oel72
[oracle@rac1-oel72 ~]$

Step-2: Check the patches 
--------------------------
[oracle@rac1-oel72 ~]$ . oraenv
ORACLE_SID = [orcl1] ? orcl
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac1-oel72 ~]$
[oracle@rac1-oel72 ~]$ sh /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/opatch lspatches
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399)
OPatch succeeded.
[oracle@rac1-oel72 ~]$

[oracle@rac1-oel72 ~]$ ssh rac2-oel72
Last login: Sun Jul 19 21:08:10 2020
[oracle@rac2-oel72 ~]$
[oracle@rac2-oel72 ~]$ . oraenv
ORACLE_SID = [orcl2] ? orcl
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac2-oel72 ~]$

[oracle@rac2-oel72 ~]$ sh /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/opatch lspatches
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399)
OPatch succeeded.
[oracle@rac2-oel72 ~]$

Step-3: Increase the SGA ans restart the cluster instances  
----------------------------------------------------------
[oracle@rac1-oel72 ~]$ . oraenv
ORACLE_SID = [orcl] ? orcl
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac1-oel72 ~]$

[oracle@rac1-oel72 ~]$ sqlplus sys/oracle@orcl as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Sun Jul 19 20:14:00 2020
Version 19.8.0.0.0
Copyright (c) 1982, 2020, Oracle.  All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

SQL> show parameter sga_

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
sga_max_size                         big integer 5952M
sga_min_size                         big integer 0
sga_target                           big integer 5952M
unified_audit_sga_queue_size         integer     1048576
SQL>

SQL> show parameter inmemory

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
inmemory_adg_enabled                 boolean     TRUE
inmemory_automatic_level             string      OFF
inmemory_clause_default              string
inmemory_expressions_usage           string      ENABLE
inmemory_force                       string      DEFAULT
inmemory_max_populate_servers        integer     0
inmemory_optimized_arithmetic        string      DISABLE
inmemory_prefer_xmem_memcompress     string
inmemory_prefer_xmem_priority        string
inmemory_query                       string      ENABLE
inmemory_size                        big integer 0
inmemory_trickle_repopulate_servers_ integer     1
percent
inmemory_virtual_columns             string      MANUAL
inmemory_xmem_size                   big integer 0
optimizer_inmemory_aware             boolean     TRUE
SQL>

SQL> select instance_name from v$instance;

INSTANCE_NAME
----------------
orcl1

SQL> create pfile='/home/oracle/initorcl_07192020.ora' from spfile;
File created.

SQL> alter system set sga_target=20480M scope=spfile;
System altered.

SQL> alter system set sga_max_size=20480M scope=spfile;
System altered.

SQL> exit
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0
[oracle@rac1-oel72 ~]$

[oracle@rac1-oel72 ~]$ srvctl status database -d orcl
Instance orcl1 is running on node rac1-oel72
Instance orcl2 is running on node rac2-oel72
[oracle@rac1-oel72 ~]$

[oracle@rac1-oel72 ~]$ srvctl stop database -d orcl

[oracle@rac1-oel72 ~]$ srvctl start database -d orcl
PRCR-1079 : Failed to start resource ora.orcl.db
CRS-5017: The resource action "ora.orcl.db start" encountered the following error:
ORA-27104: system-defined limits for shared memory was misconfigured
. For details refer to "(:CLSN00107:)" in "/u01/app/oracle/diag/crs/rac2-oel72/crs/trace/crsd_oraagent_oracle.trc".

CRS-2674: Start of 'ora.orcl.db' on 'rac2-oel72' failed
CRS-2632: There are no more servers to try to place resource 'ora.orcl.db' on that would satisfy its placement policy
CRS-5017: The resource action "ora.orcl.db start" encountered the following error:
ORA-27104: system-defined limits for shared memory was misconfigured
. For details refer to "(:CLSN00107:)" in "/u01/app/oracle/diag/crs/rac1-oel72/crs/trace/crsd_oraagent_oracle.trc".

CRS-2674: Start of 'ora.orcl.db' on 'rac1-oel72' failed

[oracle@rac1-oel72 ~]$ cat /etc/sysctl.conf | grep kernel | grep shm | grep -v "#"
kernel.shmmni = 4096
kernel.shmall = 1073741824
kernel.shmmax = 4398046511104
[oracle@rac1-oel72 ~]$


Note: Set the 'Kernel' parameters and SGA values according to the InMemory settings

Step-4: Change the values 'inmemory_force' and 'inmemory_size'
-------------------------------------------------------------

[oracle@rac1-oel72 ~]$ srvctl status database -d orcl
Instance orcl1 is running on node rac1-oel72
Instance orcl2 is running on node rac2-oel72
[oracle@rac1-oel72 ~]$

[oracle@rac1-oel72 ~]$ . oraenv
ORACLE_SID = [orcl1] ? orcl1
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac1-oel72 ~]$

[oracle@rac1-oel72 ~]$ sqlplus sys@orcl as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Sun Jul 19 20:49:13 2020
Version 19.8.0.0.0
Copyright (c) 1982, 2020, Oracle.  All rights reserved.
Enter password:
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

SQL> show parameter sga_

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
sga_max_size                         big integer 20G
sga_min_size                         big integer 0
sga_target                           big integer 20G
unified_audit_sga_queue_size         integer     1048576
SQL>

SQL> show parameter inmemory

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
inmemory_adg_enabled                 boolean     TRUE
inmemory_automatic_level             string      OFF
inmemory_clause_default              string
inmemory_expressions_usage           string      ENABLE
inmemory_force                       string      DEFAULT
inmemory_max_populate_servers        integer     0
inmemory_optimized_arithmetic        string      DISABLE
inmemory_prefer_xmem_memcompress     string
inmemory_prefer_xmem_priority        string
inmemory_query                       string      ENABLE
inmemory_size                        big integer 0
inmemory_trickle_repopulate_servers_ integer     1
percent
inmemory_virtual_columns             string      MANUAL
inmemory_xmem_size                   big integer 0
optimizer_inmemory_aware             boolean     TRUE
SQL>

SQL> show sga

Total System Global Area 2.1475E+10 bytes
Fixed Size                  9146112 bytes
Variable Size            1879048192 bytes
Database Buffers         1.9529E+10 bytes
Redo Buffers               57962496 bytes
SQL>

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDB1                           READ WRITE NO
         4 PDB2                           READ WRITE NO
SQL>

SQL> alter system set inmemory_force = 'BASE_LEVEL' scope=spfile;
System altered.

SQL> alter system set inmemory_size = 16G scope=spfile;
System altered.

SQL> exit
Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0
[oracle@rac1-oel72 ~]$

[oracle@rac1-oel72 ~]$ srvctl stop database -d orcl
[oracle@rac1-oel72 ~]$
[oracle@rac1-oel72 ~]$ srvctl start database -d orcl
[oracle@rac1-oel72 ~]$

[oracle@rac1-oel72 ~]$ srvctl status database -d orcl
Instance orcl1 is running on node rac1-oel72
Instance orcl2 is running on node rac2-oel72
[oracle@rac1-oel72 ~]$

[oracle@rac1-oel72 ~]$ . oraenv
ORACLE_SID = [orcl1] ? orcl1
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@rac1-oel72 ~]$

[oracle@rac1-oel72 ~]$ sqlplus sys@orcl as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Sun Jul 19 20:56:08 2020
Version 19.8.0.0.0
Copyright (c) 1982, 2020, Oracle.  All rights reserved.
Enter password:
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.8.0.0.0

SQL> show pdbs

    CON_ID CON_NAME                       OPEN MODE  RESTRICTED
---------- ------------------------------ ---------- ----------
         2 PDB$SEED                       READ ONLY  NO
         3 PDB1                           READ WRITE NO
         4 PDB2                           READ WRITE NO
SQL>

SQL> show parameter sga_

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
sga_max_size                         big integer 20G
sga_min_size                         big integer 0
sga_target                           big integer 20G
unified_audit_sga_queue_size         integer     1048576
SQL>

SQL> show parameter inmemory_force

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
inmemory_force                       string      BASE_LEVEL
SQL>

SQL> show parameter inmemory_size

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
inmemory_size                        big integer 16G
SQL>

SQL> select instance_name, instance_number from gv$instance order by instance_number;

INSTANCE_NAME    INSTANCE_NUMBER
---------------- ---------------
orcl1                          1
orcl2                          2

SQL>

Please refer for more details:
https://blogs.oracle.com/in-memory/base-level-198-ru
https://blogs.oracle.com/in-memory/oracle-database-in-memory-base-level-feature

Co-Author of the Oracle Database Books


Co-Author of three Oracle Database Books




Oracle Database Upgrade and Migration Methods
https://mikedietrichde.com/2017/12/19/book-recommendation-oracle-database-upgrade-migration-methods/

Oracle High Availability, Disaster Recovery, and Cloud Services
https://www.apress.com/in/book/9781484243503#otherversion=9781484243510

Oracle GoldenGate With Microservices: (Real-Time Scenarios with Oracle GoldenGate)
https://www.amazon.com/dp/9389328489/ref=sr_1_fkmr1_2?keywords=oracle+golden+gate+ravikumar+bpb+publications&qid=1582262272&sr=8-2-fkmr1

Friday, July 17, 2020

Applying GRID RU (19.8.0.0.200714) & Database RU (19.8.0.0.0) to Oracle 19c (19.3.0.0.0) - 3-Node RAC


Applying GI Release Update 19.8.0.0.200714 & Database Release Update 19.8.0.0.0 to Oracle 19c (19.3.0.0.0) - 3-Node RAC
-------------------------------------------------------------------------------------------------------------------------

Required Softwares:
-------------------

Latest OPatch:                          p6880880_190000_Linux-x86-64.zip
GI Release Update (July 2020):       p31305339_190000_Linux-x86-64.zip
Database Release Update (July 2020): p31281355_190000_Linux-x86-64.zip

Check the status of cluster instances:
--------------------------------------

[oracle@19c-rac1 ~]$ srvctl status database -d cdbrac
Instance cdbrac1 is running on node 19c-rac1
Instance cdbrac2 is running on node 19c-rac2
Instance cdbrac3 is running on node 19c-rac3
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ . oraenv
ORACLE_SID = [cdbrac] ? cdbrac
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sqlplus sys/oracle@cdbrac as sysdba

SQL*Plus: Release 19.0.0.0.0 - Production on Fri Jul 17 11:58:16 2020
Version 19.7.0.0.0
Copyright (c) 1982, 2020, Oracle.  All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.7.0.0.0

SQL> select instance_name,instance_number from gv$instance order by instance_number;

INSTANCE_NAME    INSTANCE_NUMBER
---------------- ---------------
cdbrac1                        1
cdbrac2                        2
cdbrac3                        3
SQL>

Copy latest OPatch (12.2.0.1.21), GRID RU (p31305339_190000_Linux-x86-64.zip) and RDBMS RU (p31281355_190000_Linux-x86-64.zip)
-------------------------------------------------------------------------------------------------------------------------------

[oracle@19c-rac1 ~]$ scp *.zip oracle@192.168.2.20:/home/oracle/
p31281355_190000_Linux-x86-64.zip        100% 1295MB  56.3MB/s   00:23
p31305339_190000_Linux-x86-64.zip        100% 2490MB  53.0MB/s   00:47
p6880880_190000_Linux-x86-64.zip         100%  113MB  56.4MB/s   00:02
[oracle@19c-rac1 ~]$

[oracle@19c-rac2 ~]$ scp *.zip oracle@192.168.2.30:/home/oracle/
p31281355_190000_Linux-x86-64.zip        100% 1295MB  51.8MB/s   00:25
p31305339_190000_Linux-x86-64.zip        100% 2490MB  49.8MB/s   00:50
p6880880_190000_Linux-x86-64.zip         100%  113MB  37.6MB/s   00:03
[oracle@19c-rac2 ~]$

Update latest OPatch in GI and RDBMS homes:
-------------------------------------------

[oracle@19c-rac1 ~]$ unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/19.3.0/grid/
[oracle@19c-rac1 ~]$ unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/oracle/product/19.3.0/dbhome_1/

[oracle@19c-rac2 ~]$ unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/19.3.0/grid/
[oracle@19c-rac2 ~]$ unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/oracle/product/19.3.0/dbhome_1/

[oracle@19c-rac3 ~]$ unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/19.3.0/grid/
[oracle@19c-rac3 ~]$ unzip p6880880_190000_Linux-x86-64.zip -d /u01/app/oracle/product/19.3.0/dbhome_1/


Run OPatch Conflict Check
=========================

[oracle@19c-rac1 ~]$ sh /u01/app/19.3.0/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -phBaseDir /home/oracle/31
31281355/ 31305339/

[oracle@19c-rac1 ~]$ sh /u01/app/19.3.0/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail 
-phBaseDir /home/oracle/31305339/31281355/

Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/19.3.0/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/19.3.0/grid/oraInst.loc
OPatch version    : 12.2.0.1.21
OUI version       : 12.2.0.7.0
Log file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/opatch2020-07-17_23-13-59PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"
Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sh /u01/app/19.3.0/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail 

-phBaseDir /home/oracle/31305339/31305087

Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/19.3.0/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/19.3.0/grid/oraInst.loc
OPatch version    : 12.2.0.1.21
OUI version       : 12.2.0.7.0
Log file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/opatch2020-07-17_23-15-35PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sh /u01/app/19.3.0/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail 
-phBaseDir /home/oracle/31305339/31304218

Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/19.3.0/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/19.3.0/grid/oraInst.loc
OPatch version    : 12.2.0.1.21
OUI version       : 12.2.0.7.0
Log file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/opatch2020-07-17_23-15-51PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sh /u01/app/19.3.0/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail 
-phBaseDir /home/oracle/31305339/

Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/19.3.0/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/19.3.0/grid/oraInst.loc
OPatch version    : 12.2.0.1.21
OUI version       : 12.2.0.7.0
Log file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/opatch2020-07-17_23-15-59PM_1.log

This command doesn't support System Patch.

OPatch failed with error code 21
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sh /u01/app/19.3.0/grid/OPatch/opatch prereq CheckConflictAgainstOHWithDetail 
-phBaseDir /home/oracle/31305339/31335188

Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/19.3.0/grid
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/19.3.0/grid/oraInst.loc
OPatch version    : 12.2.0.1.21
OUI version       : 12.2.0.7.0
Log file location : /u01/app/19.3.0/grid/cfgtoollogs/opatch/opatch2020-07-17_23-16-11PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@19c-rac1 ~]$

As a Oracle User:
=================

[oracle@19c-rac1 ~]$ sh /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/opatch prereq CheckConflictAgainstOHWithDetail 
-phBaseDir /home/oracle/31305339/31281355

Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/oracle/product/19.3.0/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/19.3.0/dbhome_1/oraInst.loc
OPatch version    : 12.2.0.1.21
OUI version       : 12.2.0.7.0
Log file location : /u01/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatch/opatch2020-07-17_23-18-06PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sh /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/opatch prereq CheckConflictAgainstOHWithDetail 
-phBaseDir /home/oracle/31305339/31305087

Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/oracle/product/19.3.0/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/19.3.0/dbhome_1/oraInst.loc
OPatch version    : 12.2.0.1.21
OUI version       : 12.2.0.7.0
Log file location : /u01/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatch/opatch2020-07-17_23-18-34PM_1.log

Invoking prereq "checkconflictagainstohwithdetail"

Prereq "checkConflictAgainstOHWithDetail" passed.

OPatch succeeded.
[oracle@19c-rac1 ~]$

Run OPatch SystemSpace Check:
==============================

[oracle@19c-rac1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ? cdbrac
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac1 ~]$

[oracle@19c-rac1 ~]$ sh /u01/app/19.3.0/grid/OPatch/opatch prereq CheckSystemSpace -phBaseFile /tmp/patch_list_gihome.txt
Oracle Interim Patch Installer version 12.2.0.1.21
Copyright (c) 2020, Oracle Corporation.  All rights reserved.

PREREQ session

Oracle Home       : /u01/app/oracle/product/19.3.0/dbhome_1
Central Inventory : /u01/app/oraInventory
   from           : /u01/app/oracle/product/19.3.0/dbhome_1/oraInst.loc
OPatch version    : 12.2.0.1.21
OUI version       : 12.2.0.7.0
Log file location : /u01/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatch/opatch2020-07-17_23-26-45PM_1.log

Invoking prereq "checksystemspace"
Prereq "checkSystemSpace" passed.
OPatch succeeded.
[oracle@19c-rac1 ~]$

To patch the GI home and all Oracle RAC database homes of the same version:
---------------------------------------------------------------------------

[root@19c-rac1 ~]# . oraenv
ORACLE_SID = [root] ? +ASM1
The Oracle base has been set to /u01/app/oracle
[root@19c-rac1 ~]#

[root@19c-rac1 ~]# sh /u01/app/19.3.0/grid/OPatch/opatchauto apply /home/oracle/31305339


OPatchauto session is initiated at Fri Jul 17 23:28:25 2020

System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-07-17_11-28-29PM.log.

Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-07-17_11-29-09PM.log
The id for this session is 46DR

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.3.0/dbhome_1
Patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Patch applicability verified successfully on home /u01/app/19.3.0/grid

Verifying SQL patch applicability on home /u01/app/oracle/product/19.3.0/dbhome_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Preparing to bring down database service on home /u01/app/oracle/product/19.3.0/dbhome_1
Successfully prepared home /u01/app/oracle/product/19.3.0/dbhome_1 to bring down database service

Bringing down CRS service on home /u01/app/19.3.0/grid
CRS service brought down successfully on home /u01/app/19.3.0/grid

Performing prepatch operation on home /u01/app/oracle/product/19.3.0/dbhome_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Start applying binary patch on home /u01/app/oracle/product/19.3.0/dbhome_1
Binary patch applied successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Performing postpatch operation on home /u01/app/oracle/product/19.3.0/dbhome_1
Postpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Start applying binary patch on home /u01/app/19.3.0/grid
Binary patch applied successfully on home /u01/app/19.3.0/grid

Starting CRS service on home /u01/app/19.3.0/grid
CRS service started successfully on home /u01/app/19.3.0/grid

Preparing home /u01/app/oracle/product/19.3.0/dbhome_1 after database service restarted
No step execution required.........

Trying to apply SQL patch on home /u01/app/oracle/product/19.3.0/dbhome_1
SQL patch applied successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:

Host:19c-rac1
RAC Home:/u01/app/oracle/product/19.3.0/dbhome_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/oracle/31305339/31304218
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/oracle/31305339/31335188
Reason: This patch is not applicable to this specified target type - "rac_database"

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/31305339/31281355
Log: /u01/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-17_23-36-23PM_1.log

Patch: /home/oracle/31305339/31305087
Log: /u01/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-17_23-36-23PM_1.log

Host:19c-rac1
CRS Home:/u01/app/19.3.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/31305339/31281355
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-17_23-41-50PM_1.log

Patch: /home/oracle/31305339/31304218
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-17_23-41-50PM_1.log

Patch: /home/oracle/31305339/31305087
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-17_23-41-50PM_1.log

Patch: /home/oracle/31305339/31335188
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-17_23-41-50PM_1.log

OPatchauto session completed at Fri Jul 17 23:57:46 2020
Time taken to complete the session 29 minutes, 21 seconds
[root@19c-rac1 ~]#

Second Node: 19c-rac2
----------------------

[oracle@19c-rac2 ~]$ su - root
Password:
[root@19c-rac2 ~]# . oraenv
ORACLE_SID = [root] ? +ASM2
The Oracle base has been set to /u01/app/oracle
[root@19c-rac2 ~]#

[root@19c-rac2 ~]# sh /u01/app/19.3.0/grid/OPatch/opatchauto apply /home/oracle/31305339


OPatchauto session is initiated at Sat Jul 18 00:00:53 2020

System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-07-18_12-00-57AM.log.

Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-07-18_12-01-21AM.log
The id for this session is QIXV

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.3.0/dbhome_1
Patch applicability verified successfully on home /u01/app/19.3.0/grid

Patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Verifying SQL patch applicability on home /u01/app/oracle/product/19.3.0/dbhome_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Preparing to bring down database service on home /u01/app/oracle/product/19.3.0/dbhome_1
Successfully prepared home /u01/app/oracle/product/19.3.0/dbhome_1 to bring down database service

Bringing down CRS service on home /u01/app/19.3.0/grid
CRS service brought down successfully on home /u01/app/19.3.0/grid

Performing prepatch operation on home /u01/app/oracle/product/19.3.0/dbhome_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Start applying binary patch on home /u01/app/oracle/product/19.3.0/dbhome_1
Binary patch applied successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Performing postpatch operation on home /u01/app/oracle/product/19.3.0/dbhome_1
Postpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Start applying binary patch on home /u01/app/19.3.0/grid
Binary patch applied successfully on home /u01/app/19.3.0/grid

Starting CRS service on home /u01/app/19.3.0/grid
CRS service started successfully on home /u01/app/19.3.0/grid

Preparing home /u01/app/oracle/product/19.3.0/dbhome_1 after database service restarted
No step execution required.........

Trying to apply SQL patch on home /u01/app/oracle/product/19.3.0/dbhome_1
SQL patch applied successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

OPatchAuto successful.
--------------------------------Summary--------------------------------
Patching is completed successfully. Please find the summary as follows:

Host:19c-rac2
RAC Home:/u01/app/oracle/product/19.3.0/dbhome_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/oracle/31305339/31304218
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/oracle/31305339/31335188
Reason: This patch is not applicable to this specified target type - "rac_database"

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/31305339/31281355
Log: /u01/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-05-12AM_1.log

Patch: /home/oracle/31305339/31305087
Log: /u01/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-05-12AM_1.log

Host:19c-rac2
CRS Home:/u01/app/19.3.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/31305339/31281355
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-10-52AM_1.log

Patch: /home/oracle/31305339/31304218
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-10-52AM_1.log

Patch: /home/oracle/31305339/31305087
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-10-52AM_1.log

Patch: /home/oracle/31305339/31335188
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-10-52AM_1.log

OPatchauto session completed at Sat Jul 18 00:31:50 2020
Time taken to complete the session 30 minutes, 57 seconds
[root@19c-rac2 ~]#

Third Node: 19c-rac3:
----------------------

[oracle@19c-rac3 ~]$ su - root
Password:
[root@19c-rac3 ~]# . oraenv
ORACLE_SID = [root] ? +ASM3
The Oracle base has been set to /u01/app/oracle
[root@19c-rac3 ~]#

[root@19c-rac3 ~]# sh /u01/app/19.3.0/grid/OPatch/opatchauto apply /home/oracle/31305339

OPatchauto session is initiated at Sat Jul 18 00:33:22 2020

System initialization log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchautodb/systemconfig2020-07-18_12-33-26AM.log.

Session log file is /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/opatchauto2020-07-18_12-33-51AM.log
The id for this session is SJV7

Executing OPatch prereq operations to verify patch applicability on home /u01/app/19.3.0/grid

Executing OPatch prereq operations to verify patch applicability on home /u01/app/oracle/product/19.3.0/dbhome_1
Patch applicability verified successfully on home /u01/app/19.3.0/grid

Patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Verifying SQL patch applicability on home /u01/app/oracle/product/19.3.0/dbhome_1
SQL patch applicability verified successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Preparing to bring down database service on home /u01/app/oracle/product/19.3.0/dbhome_1
Successfully prepared home /u01/app/oracle/product/19.3.0/dbhome_1 to bring down database service

Bringing down CRS service on home /u01/app/19.3.0/grid
CRS service brought down successfully on home /u01/app/19.3.0/grid

Performing prepatch operation on home /u01/app/oracle/product/19.3.0/dbhome_1
Perpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Start applying binary patch on home /u01/app/oracle/product/19.3.0/dbhome_1
Binary patch applied successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Performing postpatch operation on home /u01/app/oracle/product/19.3.0/dbhome_1
Postpatch operation completed successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

Start applying binary patch on home /u01/app/19.3.0/grid
Binary patch applied successfully on home /u01/app/19.3.0/grid

Starting CRS service on home /u01/app/19.3.0/grid
CRS service started successfully on home /u01/app/19.3.0/grid

Preparing home /u01/app/oracle/product/19.3.0/dbhome_1 after database service restarted
No step execution required.........

Trying to apply SQL patch on home /u01/app/oracle/product/19.3.0/dbhome_1
SQL patch applied successfully on home /u01/app/oracle/product/19.3.0/dbhome_1

OPatchAuto successful.
--------------------------------Summary--------------------------------

Patching is completed successfully. Please find the summary as follows:

Host:19c-rac3
RAC Home:/u01/app/oracle/product/19.3.0/dbhome_1
Version:19.0.0.0.0
Summary:

==Following patches were SKIPPED:

Patch: /home/oracle/31305339/31304218
Reason: This patch is not applicable to this specified target type - "rac_database"

Patch: /home/oracle/31305339/31335188
Reason: This patch is not applicable to this specified target type - "rac_database"

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/31305339/31281355
Log: /u01/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-40-38AM_1.log

Patch: /home/oracle/31305339/31305087
Log: /u01/app/oracle/product/19.3.0/dbhome_1/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-40-38AM_1.log

Host:19c-rac3
CRS Home:/u01/app/19.3.0/grid
Version:19.0.0.0.0
Summary:

==Following patches were SUCCESSFULLY applied:

Patch: /home/oracle/31305339/31281355
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-46-06AM_1.log

Patch: /home/oracle/31305339/31304218
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-46-06AM_1.log

Patch: /home/oracle/31305339/31305087
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-46-06AM_1.log

Patch: /home/oracle/31305339/31335188
Log: /u01/app/19.3.0/grid/cfgtoollogs/opatchauto/core/opatch/opatch2020-07-18_00-46-06AM_1.log

OPatchauto session completed at Sat Jul 18 01:41:24 2020
Time taken to complete the session 68 minutes, 2 seconds
[root@19c-rac3 ~]#

Grid Home:
----------

[oracle@19c-rac1 ~]$ . oraenv
ORACLE_SID = [cdbrac1] ? +ASM1
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac1 ~]$ sh /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/opatch lspatches
31335188;TOMCAT RELEASE UPDATE 19.0.0.0.0 (31335188)
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31304218;ACFS RELEASE UPDATE 19.8.0.0.0 (31304218)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
OPatch succeeded.
[oracle@19c-rac1 ~]$

RDBMS Home:
-----------

[oracle@19c-rac1 ~]$ sh /u01/app/oracle/product/19.3.0/dbhome_1/OPatch/opatch lspatches
31305087;OCW RELEASE UPDATE 19.8.0.0.0 (31305087)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
OPatch succeeded.
[oracle@19c-rac1 ~]$


[oracle@19c-rac1 ~]$ . oraenv
ORACLE_SID = [+ASM1] ? cdbrac
The Oracle base remains unchanged with value /u01/app/oracle
[oracle@19c-rac1 ~]$ srvctl status database -d cdbrac
Instance cdbrac1 is running on node 19c-rac1
Instance cdbrac2 is running on node 19c-rac2
Instance cdbrac3 is running on node 19c-rac3
[oracle@19c-rac1 ~]$

Saturday, June 6, 2020

Kubernetes Cluster Security Primitives - Managing Master Node from Workstation


Kubernetes Cluster Security Primitives - Managing Master Node from Workstation

We will create a workstation for you to administer your cluster without logging in to the Kubernetes master server.

List the service accounts in your cluster:
Master Node:
------------
[root@centos7 ~]# kubectl config view
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: DATA+OMITTED
    server: https://192.168.2.130:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: REDACTED
    client-key-data: REDACTED
[root@centos7 ~]#

[root@centos7 ~]# cat ~/.kube/config
apiVersion: v1
clusters:
- cluster:
    certificate-authority-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUN5RENDQWJDZ0F3SUJBZ0lCQURBTkJna3Foa2lHOXcwQkFRc0ZBREFWTVJNd0VRWURWUVFERXdwcmRXSmwKY201bGRHVnpNQjRYRFRJd01EWXdOREF3TWprMU4xb1hEVE13TURZd01qQXdNamsxTjFvd0ZURVRNQkVHQTFVRQpBeE1LYTNWaVpYSnVaWFJsY3pDQ0FTSXdEUVlKS29aSWh2Y05BUUVCQlFBRGdnRVBBRENDQVFvQ2dnRUJBTHlKCjZLY0N6QktJUEczb2trSXk0RGF2OUJZc0ZEd3UxR2dmZmZXVnFyWENieU5wbUFmOFF0SkhRMGRjV2FNWGYrU1cKT1RtenJxekdlM1l4YU5DWU8yQW1ENHNDbTd6emE3SlMxcExtNjJkZmhCRUlRcU1RUE40SzY5L1RiLzJzd2VrTwo2emNJMUtCY3JZVEFsbnRvWXlmc2JiK1BuWmJhc2g2QWdQMjA2VjlnUzM4dDJPMXdVcmtKZEhQSmlKdXF2TmpWCmZDd1dsSEthU0x1OGhBM1VWeHdMQ001dnBodVdDa0tOQ3RGU3QrNHRsQjdyaGE3cThCS3dJMTBMUldxMU9uVTYKc0VyM3NEZE16eGsrclAxTHk1UlMzUk1nN0VFV3NMRDIrWkFCYStieWpJRjZTYWhKUG8xVTBuNUlnYUgwNS9wYwozUWJNdm16Y240ZGhQWnBrb3pVQ0F3RUFBYU1qTUNFd0RnWURWUjBQQVFIL0JBUURBZ0trTUE4R0ExVWRFd0VCCi93UUZNQU1CQWY4d0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFCUzlhcE9pakV6RERUcU00b2JJb2ZLMkdVckYKYmx4U2ZwUEJnYzlWa3lEZk9pQTZLUzBLeUx2d292TlUrbmZlZ09kdnRVVTRLajdDWk01U2dVY2ZrWnlJeElVVApuNnkzTXd0RjFqZzJUYk5TbE5wY3pMNHJaNmRBaFpRYXp2R01vb2t6TXNKUnhNWjVjT1JrYkJOdHRIbmVWTWRJCkVNMFB4MVRmeTdSdkhGTXVVN0lqSzhuT2FsbXlCZXRRRThFanN3SmYwclpOYmlPUExQbWpyYktyWWtCd3hKT0MKUEtxcXUvMG9sbEVvaVpHYi9TZ0lObUVaWmRJaUIxd25RbDlFcGdhVGJoOHllcUg5QWx2WHhZRjhzRkJGcFVpZgpGQSs2bk5Cd0tMZHlMbmhBTy9MN2MrRktmT01kM1VZbG9ZVWdheWpIV2pTQlNTUFdPc1hqS2NqeXJlcz0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    server: https://192.168.2.130:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: kubernetes-admin
  name: kubernetes-admin@kubernetes
current-context: kubernetes-admin@kubernetes
kind: Config
preferences: {}
users:
- name: kubernetes-admin
  user:
    client-certificate-data: LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUM4akNDQWRxZ0F3SUJBZ0lJWVhBNW9oWWJnbWN3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TURBMk1EUXdNREk1TlRkYUZ3MHlNVEEyTURRd01ETXdNRE5hTURReApGekFWQmdOVkJBb1REbk41YzNSbGJUcHRZWE4wWlhKek1Sa3dGd1lEVlFRREV4QnJkV0psY201bGRHVnpMV0ZrCmJXbHVNSUlCSWpBTkJna3Foa2lHOXcwQkFRRUZBQU9DQVE4QU1JSUJDZ0tDQVFFQXNWMW9qRmEyVGcvanVKb2sKZjBUZ0FGUFhYUEI5eUdwOTFDWWluRUFiVzBsa20vTTF3KzkwUVcvaHFKdmVwZENkNk9yYWk1WURUbmZhVGF2UQp0aHVKNUp3REF0N1kyQXpGaFJQR2xVVjJzWUpPd2xXNkNZNTQ4Y3V3Q2pPRGVDTDlXc09tcFVDZlI2T0psOVlCCitMWnlxcTFlWHc2ZExoVmFseTl4OFBUcFI1OGovamEyVGdPSTF5UEZDUmNhYy8vakhiZDhkMzd4b1hxQjAwY2kKRW12ZXBVSWhyOUp6bTdmRDNnRW5YaWF4UUxBSkhyUE9mTDQ1aWMzeERuUzFmYmp6anpjS1lNcUFPWDNEdld6QQpDdVdWZWNoZGZ0WVdNTkFVN0RZbW5MZlJscU4wYjU0em04R3IvMFJhUnNTL0ZVdHVQQi9JK3dxeFB2dFQwMWtkCjJvaFhvUUlEQVFBQm95Y3dKVEFPQmdOVkhROEJBZjhFQkFNQ0JhQXdFd1lEVlIwbEJBd3dDZ1lJS3dZQkJRVUgKQXdJd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dFQkFBdGNPNnRTTkNMRWs3UEtqclQ1RnpaYVVnQlg5bTlSZnVoSQp3WVM3YTZTeHhxRDNmWWFYU0g2NU9PdzllVUYwcGRYWFlTV3NZZHVGc2NmekJseG5mcTJuam05RzVkdC9jMVdSClZUZzZKZmNuMTBxaXlCeHlyNWgzOEUwQjMzV2FLV0ljbjZUUjE1bFprTVJKb0tYSHVBTmFtejUxSkNHSDBIUXoKSFVhV09YSTZwUnJIM0J6Z0ZLZjFkcUpjbEt5b2h5V2ZST3NIZGJKb3E3YnpPMjV0U21XVVJ3NDVHZmJNaGhiMgplZWQwL2tvTFpxZ2tGTExDM04vS3U4QUsxUkpDcGUvTnl4bDYzMldESi8rRktrNTJ3RHZ1OEJ2d2RRNjlJZ1JjCm1QcE5BNTN1M0puTEp4aXkxYlhRNFVZaWpyRkh3c1Q3SDc1bkJQcXJ5OVp2djhkbGZ0RT0KLS0tLS1FTkQgQ0VSVElGSUNBVEUtLS0tLQo=
    client-key-data: LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFcEFJQkFBS0NBUUVBc1Yxb2pGYTJUZy9qdUpva2YwVGdBRlBYWFBCOXlHcDkxQ1lpbkVBYlcwbGttL00xCncrOTBRVy9ocUp2ZXBkQ2Q2T3JhaTVZRFRuZmFUYXZRdGh1SjVKd0RBdDdZMkF6RmhSUEdsVVYyc1lKT3dsVzYKQ1k1NDhjdXdDak9EZUNMOVdzT21wVUNmUjZPSmw5WUIrTFp5cXExZVh3NmRMaFZhbHk5eDhQVHBSNThqL2phMgpUZ09JMXlQRkNSY2FjLy9qSGJkOGQzN3hvWHFCMDBjaUVtdmVwVUlocjlKem03ZkQzZ0VuWGlheFFMQUpIclBPCmZMNDVpYzN4RG5TMWZianpqemNLWU1xQU9YM0R2V3pBQ3VXVmVjaGRmdFlXTU5BVTdEWW1uTGZSbHFOMGI1NHoKbThHci8wUmFSc1MvRlV0dVBCL0krd3F4UHZ0VDAxa2Qyb2hYb1FJREFRQUJBb0lCQVFDd1Y0V2wybVRCTWkwZwpFaWx0WEFZMmJTVjNveWY4Qmx6NmU3djNqQWxtOW82UU92eGV4WW5ac2Fvem9GMTZrUE1mYTdsRUJTOWVhZTFqClFndHRzKzJpbkxhbHVQanh2YWo0aWRQci83ZGwvMy9jMmF6dzZYVTRPbUxaa2FRSjR3ZndvUXZqeTE5UnM3Z08KZlZsU2RkdUY5SDhhbFlZVEZtb09YTDYxYWxzeHQ5RXRDclF6UXluR0RvQlJ1RVYva2VBeCs3SVExZXJoclVZZgpvUk1pbFpwR3kvRlYxTnY4OTJvUGtTNlpWREowaTh5RG1UbC9ERkxobk5VS1VnSm1sZ2J4a1RHWXg1T2tjdTRxCmtPRUFuNzFydkZrQndweW1LaWJ6R2orWHYxd0xtWGc2V2RnU3dkTlVXUUk2Vnc3VmdwaC93T3lVWFF5RTMxekIKV0ZxRXVPR2RBb0dCQU5NQ01hVGN4ZURRU21qdldlQWlzWHRkS0dQbFhDUGk3TitEMW05SVFvYVVlVElYZm52OAp0bXR6cmhwT2o2Unl6emM2Z2Y0QVN6RkdJb09DOTU2bnd4TjV6c0pkL0hxOEgvWEZBSk5WNTdPcVpwNEd2VURlCnBSS2w2YW1GenFENGs3UU5YNy8vcFFyVlNvS2VSbzlLbkhMYzZFZmVyUERWM2xjVmxab1YyTHMzQW9HQkFOY3UKeWZRbElUVVlYK1RZNWlaZEZGNkZHcVlJN3NSbE1PRHB1cmNHdmRIYnZwdEFXRFFLSjd6dFhNZzV0VWVKb1E3dQpzSWdBYnBQR0x1cUhaUUtucFQzTzFqZSs0TkMrcW1VbGpwalV0OXltVG4xZHBFRTdUb3A4bzFCQjVJajROd0JiCm9xVzlWYWdPOUxhSFd4bnk4dGxPczhMRHc2U1VOSkRXSmxGb2NsL25Bb0dBWkM4S2V3Z0hPM0lwb2lEaVB6UzUKcDZUWFpIYWxoTFJkV3RJeG9heGhTWng4M25laEtpVVdSM1lwRjN4dFh0clQzNWo3MXlrMGlqU09kbnBjTHd2Zgo1d05TVTU1a0hiSHNTcmc2U2JuanhMNEc4c2lvV29CYlI3dFdjWktDUkpOQkxaK2I4b0RpVDM1dFhnN2YyWERIClR4SHJoeFFHY0dKYXFtWXBIWHQ1NTRrQ2dZQnBCTlBaalBsOUFVMWowaHZaSTFwYW14bWJhUWFtNFVBT3RPUEUKaWN6QWpEK2xZNnpva1FBOTY1WjV2ZmhrVnA0OVBHNzVvVVJXYU1YaS9udGs1VTczTUdoS1FxVlVHNG5yM1gyUAp4MW1JbmIzMjMyblgwRXNQdWE2Mjc4SEt1MkdVa2lSNlFVSjV4L3JVenBDK3hGeVRoQjZsY29lY1hoeWRoM2FpCkNwaXZod0tCZ1FDNCtEWE9IcERWSi9lM2NRNThsZDh4TmNNQmFzeGs4YWxZM2NNOWh6dTYxTDRCSjRydVp4THYKalphN2tYWTBEUGkvdEhsQ0ZYTTFsbTJCVlBhUklSU2pBdExmRDYxeTVOMzJaL0NaTHhJem04SVZsVFVGekZVawpkOFM4UHk1ak16UjRpS3IvSTFBTjJnZlkzQlBEZzMrUndKV2xmZHg0NGNjOWhNNXhJeXovc0E9PQotLS0tLUVORCBSU0EgUFJJVkFURSBLRVktLS0tLQo=
[root@centos7 ~]#

[root@centos7 ~]# kubectl config set-credentials admin --username=admin --password=password
User "admin" set.
[root@centos7 ~]#

[root@centos7 ~]# kubectl create clusterrolebinding cluster-system-anonymous --clusterrole=cluster-admin --user=system:anonymous
clusterrolebinding.rbac.authorization.k8s.io/cluster-system-anonymous created
[root@centos7 ~]#

[root@centos7 ~]# cd /etc/kubernetes/pki/
[root@centos7 pki]#
[root@centos7 pki]# ls -lrth
total 56K
-rw------- 1 root root 1.7K Jun  3 19:29 ca.key
-rw-r--r-- 1 root root 1.1K Jun  3 19:29 ca.crt
-rw------- 1 root root 1.7K Jun  3 19:29 apiserver.key
-rw-r--r-- 1 root root 1.2K Jun  3 19:29 apiserver.crt
-rw------- 1 root root 1.7K Jun  3 19:29 apiserver-kubelet-client.key
-rw-r--r-- 1 root root 1.1K Jun  3 19:29 apiserver-kubelet-client.crt
-rw------- 1 root root 1.7K Jun  3 19:29 front-proxy-ca.key
-rw-r--r-- 1 root root 1.1K Jun  3 19:29 front-proxy-ca.crt
-rw------- 1 root root 1.7K Jun  3 19:29 front-proxy-client.key
-rw-r--r-- 1 root root 1.1K Jun  3 19:29 front-proxy-client.crt
drwxr-xr-x 2 root root  162 Jun  3 19:30 etcd
-rw------- 1 root root 1.7K Jun  3 19:30 apiserver-etcd-client.key
-rw-r--r-- 1 root root 1.1K Jun  3 19:30 apiserver-etcd-client.crt
-rw------- 1 root root  451 Jun  3 19:30 sa.pub
-rw------- 1 root root 1.7K Jun  3 19:30 sa.key
[root@centos7 pki]#

Copy to WorkStation (192.168.2.133)
-------------------------------------

[root@centos7 pki]# scp ca.crt admin@192.168.2.133:~/
The authenticity of host '192.168.2.133 (192.168.2.133)' can't be established.
ECDSA key fingerprint is SHA256:uImodzAY8g3iu7IcbnQhbNo1J5WFZZciTDzWqmi5d08.
ECDSA key fingerprint is MD5:02:7d:55:43:91:99:3a:d5:06:68:25:28:9e:db:05:66.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.2.133' (ECDSA) to the list of known hosts.
admin@192.168.2.133's password:
ca.crt                                      100% 1025   593.6KB/s   00:00
[root@centos7 pki]#

Login to WorkStation (192.168.2.133) and Install Kubectl software
-------------------------------------------------------------------
[root@centos7-ws ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.2.133 centos7-ws.localdomain centos7-ws
192.168.2.130 centos7.localdomain    centos7
[root@centos7-ws ~]#

[root@centos7-ws ~]# setenforce 0
[root@centos7-ws ~]# sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
[root@centos7-ws ~]# firewall-cmd --state
not running
[root@centos7-ws ~]# modprobe br_netfilter
[root@centos7-ws ~]# echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

[root@centos7-ws ~]# cat < /etc/yum.repos.d/kubernetes.repo
> [kubernetes]
> name=Kubernetes
> baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
> enabled=1
> gpgcheck=1
> repo_gpgcheck=1
> gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
>         https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
> EOF
[root@centos7-ws ~]#

[root@centos7-ws ~]# yum install -y kubectl
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.steadfastnet.com
 * extras: mirror.dal.nexril.net
 * updates: repos-tx.psychz.net
kubernetes/signature                                                                                                                             |  454 B  00:00:00
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key "
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
kubernetes/signature                                                                                                                             | 1.4 kB  00:00:00 !!!
kubernetes/primary                                                                                                                               |  69 kB  00:00:00
kubernetes                                                                                                                                                      505/505
Resolving Dependencies
--> Running transaction check
---> Package kubectl.x86_64 0:1.18.3-0 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================================================
 Package                                Arch                                  Version                                   Repository                                 Size
========================================================================================================================================================================
Installing:
 kubectl                                x86_64                                1.18.3-0                                  kubernetes                                9.5 M

Transaction Summary
========================================================================================================================================================================
Install  1 Package

Total download size: 9.5 M
Installed size: 42 M
Downloading packages:
warning: /var/cache/yum/x86_64/7/kubernetes/packages/cd5d6980c3e1b15de222db08729eff40f7031b7fa56c71ae3e28e420ba9678cd-kubectl-1.18.3-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY
Public key for cd5d6980c3e1b15de222db08729eff40f7031b7fa56c71ae3e28e420ba9678cd-kubectl-1.18.3-0.x86_64.rpm is not installed
cd5d6980c3e1b15de222db08729eff40f7031b7fa56c71ae3e28e420ba9678cd-kubectl-1.18.3-0.x86_64.rpm                                                     | 9.5 MB  00:00:05
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key "
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Importing GPG key 0x3E1BA8D5:
 Userid     : "Google Cloud Packages RPM Signing Key "
 Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5
 From       : https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : kubectl-1.18.3-0.x86_64                                                                                                                              1/1
  Verifying  : kubectl-1.18.3-0.x86_64                                                                                                                              1/1

Installed:
  kubectl.x86_64 0:1.18.3-0

Complete!
[root@centos7-ws ~]# su - admin
[admin@centos7-ws ~]$ ls -lrth
total 4.0K
-rw-r--r--. 1 admin admin 1.1K Jun  5 12:47 ca.crt
[admin@centos7-ws ~]$

Check the Kubectl version:
--------------------------
[admin@centos7-ws ~]$ kubectl version
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.3", GitCommit:"2e7996e3e2712684bc73f0dec0200d64eec7fe40", GitTreeState:"clean", BuildDate:"2020-05-20T12:52:00Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
The connection to the server localhost:8080 was refused - did you specify the right host or port?
[admin@centos7-ws ~]$

[admin@centos7-ws ~]$ kubectl config set-cluster kubernetes --server=https://192.168.2.130:6443 --certificate-authority=ca.crt --embed-certs=true
Cluster "kubernetes" set.
[admin@centos7-ws ~]$

[admin@centos7-ws ~]$ kubectl config set-credentials admin --username=admin --password=password
User "admin" set.
[admin@centos7-ws ~]$

[admin@centos7-ws ~]$ kubectl config set-context kubernetes --cluster=kubernetes --user=admin --namespace=default
Context "kubernetes" created.
[admin@centos7-ws ~]$

[admin@centos7-ws ~]$ kubectl config use-context kubernetes
Switched to context "kubernetes".
[admin@centos7-ws ~]$

Check the Kubernetes Cluster:
-----------------------------

[admin@centos7-ws ~]$ kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
centos7      Ready    master   40h   v1.18.3
centos7-w1   Ready       40h   v1.18.3
centos7-w2   Ready       40h   v1.18.3
[admin@centos7-ws ~]$

[admin@centos7-ws ~]$ kubectl get nodes -o wide
NAME         STATUS   ROLES    AGE   VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE                KERNEL-VERSION           CONTAINER-RUNTIME
centos7      Ready    master   40h   v1.18.3   192.168.2.130           CentOS Linux 7 (Core)   3.10.0-1062.el7.x86_64   docker://19.3.11
centos7-w1   Ready       40h   v1.18.3   192.168.2.131           CentOS Linux 7 (Core)   3.10.0-1062.el7.x86_64   docker://19.3.11
centos7-w2   Ready       40h   v1.18.3   192.168.2.132           CentOS Linux 7 (Core)   3.10.0-1062.el7.x86_64   docker://19.3.11
[admin@centos7-ws ~]$

[admin@centos7-ws ~]$ kubectl get pods -o wide
NAME                         READY   STATUS    RESTARTS   AGE   IP            NODE         NOMINATED NODE   READINESS GATES
busybox                      1/1     Running   0          24m   10.244.2.52   centos7-w2              
kubeserve-6b65f9d76d-9k8jf   1/1     Running   1          11h   10.244.2.50   centos7-w2              
kubeserve-6b65f9d76d-hjvq7   1/1     Running   1          11h   10.244.1.48   centos7-w1              
kubeserve-6b65f9d76d-lr6qp   1/1     Running   1          11h   10.244.2.47   centos7-w2              
myreplicaset-66h6j           1/1     Running   1          11h   10.244.2.51   centos7-w2              
myreplicaset-jllkz           1/1     Running   1          11h   10.244.2.49   centos7-w2              
myreplicaset-qktmw           1/1     Running   1          11h   10.244.1.50   centos7-w1              
nginx-f89759699-85wrf        1/1     Running   1          11h   10.244.2.48   centos7-w2              
nginx-f89759699-dbht5        1/1     Running   1          11h   10.244.1.55   centos7-w1              
pref-646c88c576-5tzfc        1/1     Running   1          11h   10.244.1.54   centos7-w1              
pref-646c88c576-fgl8h        1/1     Running   1          11h   10.244.1.52   centos7-w1              
pref-646c88c576-nxjmw        1/1     Running   1          11h   10.244.1.51   centos7-w1              
pref-646c88c576-wr8vw        1/1     Running   1          11h   10.244.1.49   centos7-w1              
pref-646c88c576-x6vrp        1/1     Running   1          11h   10.244.1.53   centos7-w1              
web-0                        0/1     Pending   0          15h                             
[admin@centos7-ws ~]$


Configuring Kubernetes Cluster with Master Node, Two Worker Nodes on CENTOS7


Configuring Kubernetes Cluster With Master Node, Two Worker Nodes on CENTOS7

Master Node	: 192.168.2.130 (centos7)
Worker Node-1	: 192.168.2.131 (centos7-w1)
Worker Node-2	: 192.168.2.132 (centos7-w2)
Work Station	: 192.168.2.133 (centos7-ws)

Master Node	: 192.168.2.130 (centos7)
-----------------------------------------

1. Check the Status Of Docker:
-----------------------------

[root@centos7 ~]# systemctl status docker
? docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2020-05-31 12:08:08 CDT; 4h 4min ago
     Docs: https://docs.docker.com
 Main PID: 5561 (dockerd)
    Tasks: 18
   Memory: 82.9M
   CGroup: /system.slice/docker.service
           +-5561 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

May 31 16:04:31 centos7 dockerd[5561]: time="2020-05-31T16:04:31.921506028-05:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic...askDelete"
May 31 16:05:32 centos7 dockerd[5561]: time="2020-05-31T16:05:32.234346689-05:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic...askDelete"
May 31 16:06:32 centos7 dockerd[5561]: time="2020-05-31T16:06:32.553116712-05:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic...askDelete"
May 31 16:07:32 centos7 dockerd[5561]: time="2020-05-31T16:07:32.875037752-05:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic...askDelete"
May 31 16:08:33 centos7 dockerd[5561]: time="2020-05-31T16:08:33.192984415-05:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic...askDelete"
May 31 16:09:28 centos7 dockerd[5561]: time="2020-05-31T16:09:28.727330062-05:00" level=info msg="NetworkDB stats centos7(9e86f33306e3) - netID:fhzqs13pt2x...etMsg/s:0"
May 31 16:09:33 centos7 dockerd[5561]: time="2020-05-31T16:09:33.499272211-05:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic...askDelete"
May 31 16:10:33 centos7 dockerd[5561]: time="2020-05-31T16:10:33.847703789-05:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic...askDelete"
May 31 16:11:34 centos7 dockerd[5561]: time="2020-05-31T16:11:34.182730856-05:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic...askDelete"
May 31 16:12:34 centos7 dockerd[5561]: time="2020-05-31T16:12:34.488821680-05:00" level=info msg="ignoring event" module=libcontainerd namespace=moby topic...askDelete"
Hint: Some lines were ellipsized, use -l to show in full.
[root@centos7 ~]#

2. Login to your kubernetes master node and set the hostname and disable selinux using following commands
---------------------------------------------------------------------------------------------------------
[root@centos7 ~]# hostnamectl set-hostname centos7

[root@centos7 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.2.130 centos7.localdomain centos7
192.168.2.131 centos7-w1.localdomain centos7-w1
192.168.2.132 centos7-w2.localdomain centos7-w2
192.168.2.133 centos7-ws.localdomain centos7-ws
[root@centos7 ~]#

[root@centos7 ~]# ping centos7-w1
PING centos7-w1.localdomain (192.168.2.131) 56(84) bytes of data.
64 bytes from centos7-w1.localdomain (192.168.2.131): icmp_seq=1 ttl=64 time=0.361 ms
64 bytes from centos7-w1.localdomain (192.168.2.131): icmp_seq=1 ttl=63 time=0.534 ms (DUP!)
64 bytes from centos7-w1.localdomain (192.168.2.131): icmp_seq=1 ttl=64 time=0.537 ms (DUP!)
64 bytes from centos7-w1.localdomain (192.168.2.131): icmp_seq=1 ttl=63 time=0.746 ms (DUP!)
^C
--- centos7-w1.localdomain ping statistics ---
1 packets transmitted, 1 received, +3 duplicates, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.361/0.544/0.746/0.138 ms
[root@centos7 ~]#

[root@centos7 ~]# ping centos7-w2
PING centos7-w2.localdomain (192.168.2.132) 56(84) bytes of data.
64 bytes from centos7-w2.localdomain (192.168.2.132): icmp_seq=1 ttl=64 time=0.130 ms
64 bytes from centos7-w2.localdomain (192.168.2.132): icmp_seq=1 ttl=64 time=0.188 ms (DUP!)
64 bytes from centos7-w2.localdomain (192.168.2.132): icmp_seq=1 ttl=63 time=0.192 ms (DUP!)
64 bytes from centos7-w2.localdomain (192.168.2.132): icmp_seq=1 ttl=63 time=0.298 ms (DUP!)
64 bytes from centos7-w2.localdomain (192.168.2.132): icmp_seq=2 ttl=64 time=0.442 ms
64 bytes from centos7-w2.localdomain (192.168.2.132): icmp_seq=2 ttl=63 time=0.566 ms (DUP!)
64 bytes from centos7-w2.localdomain (192.168.2.132): icmp_seq=2 ttl=64 time=0.575 ms (DUP!)
64 bytes from centos7-w2.localdomain (192.168.2.132): icmp_seq=2 ttl=63 time=0.647 ms (DUP!)
^C
--- centos7-w2.localdomain ping statistics ---
2 packets transmitted, 2 received, +6 duplicates, 0% packet loss, time 1000ms
rtt min/avg/max/mdev = 0.130/0.379/0.647/0.191 ms
[root@centos7 ~]#

[root@centos7 ~]# setenforce 0
[root@centos7 ~]# sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
[root@centos7 ~]# modprobe br_netfilter
[root@centos7 ~]# echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

3. Update latest Docker Binaries
--------------------------------
[root@master-node ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: repos-tx.psychz.net
 * extras: mirror.mobap.edu
 * updates: mirror.umd.edu
Resolving Dependencies
--> Running transaction check
---> Package device-mapper-persistent-data.x86_64 0:0.8.5-1.el7 will be updated
---> Package device-mapper-persistent-data.x86_64 0:0.8.5-2.el7 will be an update
---> Package lvm2.x86_64 7:2.02.185-2.el7 will be updated
--> Processing Dependency: lvm2 = 7:2.02.185-2.el7 for package: 7:lvm2-cluster-2.02.185-2.el7.x86_64
---> Package lvm2.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Processing Dependency: lvm2-libs = 7:2.02.186-7.el7_8.2 for package: 7:lvm2-2.02.186-7.el7_8.2.x86_64
---> Package yum-utils.noarch 0:1.1.31-52.el7 will be updated
---> Package yum-utils.noarch 0:1.1.31-54.el7_8 will be an update
--> Running transaction check
---> Package lvm2-cluster.x86_64 7:2.02.185-2.el7 will be updated
---> Package lvm2-cluster.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Processing Dependency: device-mapper = 7:1.02.164 for package: 7:lvm2-cluster-2.02.186-7.el7_8.2.x86_64
---> Package lvm2-libs.x86_64 7:2.02.185-2.el7 will be updated
--> Processing Dependency: lvm2-libs = 7:2.02.185-2.el7 for package: 7:lvm2-python-libs-2.02.185-2.el7.x86_64
---> Package lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Processing Dependency: device-mapper-event = 7:1.02.164-7.el7_8.2 for package: 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64
--> Running transaction check
---> Package device-mapper.x86_64 7:1.02.158-2.el7 will be updated
--> Processing Dependency: device-mapper = 7:1.02.158-2.el7 for package: 7:device-mapper-libs-1.02.158-2.el7.x86_64
---> Package device-mapper.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-event.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-event.x86_64 7:1.02.164-7.el7_8.2 will be an update
--> Processing Dependency: device-mapper-event-libs = 7:1.02.164-7.el7_8.2 for package: 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64
---> Package lvm2-python-libs.x86_64 7:2.02.185-2.el7 will be updated
---> Package lvm2-python-libs.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Running transaction check
---> Package device-mapper-event-libs.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-libs.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================================================
 Package                                              Arch                          Version                                        Repository                      Size
========================================================================================================================================================================
Updating:
 device-mapper-persistent-data                        x86_64                        0.8.5-2.el7                                    base                           422 k
 lvm2                                                 x86_64                        7:2.02.186-7.el7_8.2                           updates                        1.3 M
 yum-utils                                            noarch                        1.1.31-54.el7_8                                updates                        122 k
Updating for dependencies:
 device-mapper                                        x86_64                        7:1.02.164-7.el7_8.2                           updates                        295 k
 device-mapper-event                                  x86_64                        7:1.02.164-7.el7_8.2                           updates                        191 k
 device-mapper-event-libs                             x86_64                        7:1.02.164-7.el7_8.2                           updates                        190 k
 device-mapper-libs                                   x86_64                        7:1.02.164-7.el7_8.2                           updates                        324 k
 lvm2-cluster                                         x86_64                        7:2.02.186-7.el7_8.2                           updates                        750 k
 lvm2-libs                                            x86_64                        7:2.02.186-7.el7_8.2                           updates                        1.1 M
 lvm2-python-libs                                     x86_64                        7:2.02.186-7.el7_8.2                           updates                        188 k

Transaction Summary
========================================================================================================================================================================
Upgrade  3 Packages (+7 Dependent packages)

Total size: 4.8 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : 7:device-mapper-libs-1.02.164-7.el7_8.2.x86_64                                                                                                      1/20
  Updating   : 7:device-mapper-1.02.164-7.el7_8.2.x86_64                                                                                                           2/20
  Updating   : 7:device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64                                                                                                3/20
  Updating   : 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64                                                                                                     4/20
  Updating   : 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64                                                                                                               5/20
  Updating   : device-mapper-persistent-data-0.8.5-2.el7.x86_64                                                                                                    6/20
  Updating   : 7:lvm2-2.02.186-7.el7_8.2.x86_64                                                                                                                    7/20
  Updating   : 7:lvm2-cluster-2.02.186-7.el7_8.2.x86_64                                                                                                            8/20
  Updating   : 7:lvm2-python-libs-2.02.186-7.el7_8.2.x86_64                                                                                                        9/20
  Updating   : yum-utils-1.1.31-54.el7_8.noarch                                                                                                                   10/20
  Cleanup    : 7:lvm2-cluster-2.02.185-2.el7.x86_64                                                                                                               11/20
  Cleanup    : 7:lvm2-2.02.185-2.el7.x86_64                                                                                                                       12/20
  Cleanup    : 7:lvm2-python-libs-2.02.185-2.el7.x86_64                                                                                                           13/20
  Cleanup    : yum-utils-1.1.31-52.el7.noarch                                                                                                                     14/20
  Cleanup    : 7:lvm2-libs-2.02.185-2.el7.x86_64                                                                                                                  15/20
  Cleanup    : 7:device-mapper-event-1.02.158-2.el7.x86_64                                                                                                        16/20
  Cleanup    : 7:device-mapper-event-libs-1.02.158-2.el7.x86_64                                                                                                   17/20
  Cleanup    : 7:device-mapper-libs-1.02.158-2.el7.x86_64                                                                                                         18/20
  Cleanup    : 7:device-mapper-1.02.158-2.el7.x86_64                                                                                                              19/20
  Cleanup    : device-mapper-persistent-data-0.8.5-1.el7.x86_64                                                                                                   20/20
  Verifying  : 7:device-mapper-1.02.164-7.el7_8.2.x86_64                                                                                                           1/20
  Verifying  : device-mapper-persistent-data-0.8.5-2.el7.x86_64                                                                                                    2/20
  Verifying  : 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64                                                                                                               3/20
  Verifying  : 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64                                                                                                     4/20
  Verifying  : 7:lvm2-python-libs-2.02.186-7.el7_8.2.x86_64                                                                                                        5/20
  Verifying  : 7:lvm2-2.02.186-7.el7_8.2.x86_64                                                                                                                    6/20
  Verifying  : 7:device-mapper-libs-1.02.164-7.el7_8.2.x86_64                                                                                                      7/20
  Verifying  : 7:lvm2-cluster-2.02.186-7.el7_8.2.x86_64                                                                                                            8/20
  Verifying  : yum-utils-1.1.31-54.el7_8.noarch                                                                                                                    9/20
  Verifying  : 7:device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64                                                                                               10/20
  Verifying  : 7:device-mapper-event-1.02.158-2.el7.x86_64                                                                                                        11/20
  Verifying  : device-mapper-persistent-data-0.8.5-1.el7.x86_64                                                                                                   12/20
  Verifying  : yum-utils-1.1.31-52.el7.noarch                                                                                                                     13/20
  Verifying  : 7:lvm2-python-libs-2.02.185-2.el7.x86_64                                                                                                           14/20
  Verifying  : 7:device-mapper-event-libs-1.02.158-2.el7.x86_64                                                                                                   15/20
  Verifying  : 7:lvm2-cluster-2.02.185-2.el7.x86_64                                                                                                               16/20
  Verifying  : 7:device-mapper-1.02.158-2.el7.x86_64                                                                                                              17/20
  Verifying  : 7:device-mapper-libs-1.02.158-2.el7.x86_64                                                                                                         18/20
  Verifying  : 7:lvm2-libs-2.02.185-2.el7.x86_64                                                                                                                  19/20
  Verifying  : 7:lvm2-2.02.185-2.el7.x86_64                                                                                                                       20/20

Updated:
  device-mapper-persistent-data.x86_64 0:0.8.5-2.el7                 lvm2.x86_64 7:2.02.186-7.el7_8.2                 yum-utils.noarch 0:1.1.31-54.el7_8

Dependency Updated:
  device-mapper.x86_64 7:1.02.164-7.el7_8.2            device-mapper-event.x86_64 7:1.02.164-7.el7_8.2       device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2
  device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2       lvm2-cluster.x86_64 7:2.02.186-7.el7_8.2              lvm2-libs.x86_64 7:2.02.186-7.el7_8.2
  lvm2-python-libs.x86_64 7:2.02.186-7.el7_8.2

Complete!
[root@master-node ~]#

[root@master-node ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Loaded plugins: fastestmirror, langpacks
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
[root@master-node ~]#

Install Docker Community Edition
----------------------------------
[root@master-node ~]# yum install -y docker-ce
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: repos-tx.psychz.net
 * extras: repos-lax.psychz.net
 * updates: mirror.arizona.edu
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 3:19.03.8-3.el7 will be updated
---> Package docker-ce.x86_64 3:19.03.10-3.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================================================
 Package                              Arch                              Version                                       Repository                                   Size
========================================================================================================================================================================
Updating:
 docker-ce                            x86_64                            3:19.03.10-3.el7                              docker-ce-stable                             24 M

Transaction Summary
========================================================================================================================================================================
Upgrade  1 Package

Total download size: 24 M
Downloading packages:
No Presto metadata available for docker-ce-stable
docker-ce-19.03.10-3.el7.x86_64.rpm                                                                                                              |  24 MB  00:00:03
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : 3:docker-ce-19.03.10-3.el7.x86_64                                                                                                                    1/2
  Cleanup    : 3:docker-ce-19.03.8-3.el7.x86_64                                                                                                                     2/2
  Verifying  : 3:docker-ce-19.03.10-3.el7.x86_64                                                                                                                    1/2
  Verifying  : 3:docker-ce-19.03.8-3.el7.x86_64                                                                                                                     2/2

Updated:
  docker-ce.x86_64 3:19.03.10-3.el7

Complete!
[root@master-node ~]#

[root@master-node ~]# systemctl status docker
? docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
   Active: active (running) since Sun 2020-05-31 16:35:11 CDT; 10s ago
     Docs: https://docs.docker.com
 Main PID: 9801 (dockerd)
    Tasks: 18
   Memory: 72.6M
   CGroup: /system.slice/docker.service
           +-9801 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock

May 31 16:35:11 master-node dockerd[9801]: time="2020-05-31T16:35:11.255973127-05:00" level=error msg="fatal task error" error="No such container: nginx.2....kp54sahn5y
May 31 16:35:11 master-node dockerd[9801]: time="2020-05-31T16:35:11.523002017-05:00" level=info msg="Daemon has completed initialization"
May 31 16:35:11 master-node dockerd[9801]: time="2020-05-31T16:35:11.540338030-05:00" level=info msg="API listen on /var/run/docker.sock"
May 31 16:35:11 master-node systemd[1]: Started Docker Application Container Engine.
May 31 16:35:12 master-node dockerd[9801]: time="2020-05-31T16:35:12.438487241-05:00" level=warning msg="failed to deactivate service binding for container...j25q2kesay
Hint: Some lines were ellipsized, use -l to show in full.
[root@master-node ~]#

[root@centos7 ~]# systemctl restart docker && systemctl enable docker

[root@master-node ~]# docker version
Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b
 Built:             Wed Mar 11 01:27:04 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.10
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.10
  Git commit:       9424aeaee9
  Built:            Thu May 28 22:16:43 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
[root@master-node ~]#

4. Kubernetes packages are not available in the default CentOS 7 & RHEL 7 repositories, Use below command to configure its package repositories.
------------------------------------------------------------------------------------------------------------------------------------------------
cat < /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
        https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
EOF

Output:
-------
[root@master-node ~]# cat < /etc/yum.repos.d/kubernetes.repo
> [kubernetes]
> name=Kubernetes
> baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
> enabled=1
> gpgcheck=1
> repo_gpgcheck=1
> gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
>         https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
> EOF
[root@master-node ~]#

5. Once the package repositories are configured, run the beneath command to install kubeadm and docker packages.
===============================================================================================================
[root@master-node ~]# yum install -y kubelet kubeadm kubectl –disableexcludes=kubernetes
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: repos-tx.psychz.net
 * extras: mirror.mobap.edu
 * updates: repos-va.psychz.net
kubernetes/signature                                                                                                                             |  454 B  00:00:00
kubernetes/signature                                                                                                                             | 1.4 kB  00:00:00 !!!
Resolving Dependencies
--> Running transaction check
---> Package kubeadm.x86_64 0:1.18.3-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.7.5 for package: kubeadm-1.18.3-0.x86_64
--> Processing Dependency: cri-tools >= 1.13.0 for package: kubeadm-1.18.3-0.x86_64
---> Package kubectl.x86_64 0:1.18.3-0 will be installed
---> Package kubelet.x86_64 0:1.18.3-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.18.3-0.x86_64
--> Processing Dependency: conntrack for package: kubelet-1.18.3-0.x86_64
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-7.el7 will be installed
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-11.el7 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================================================
 Package                                          Arch                             Version                                   Repository                            Size
========================================================================================================================================================================
Installing:
 kubeadm                                          x86_64                           1.18.3-0                                  kubernetes                           8.8 M
 kubectl                                          x86_64                           1.18.3-0                                  kubernetes                           9.5 M
 kubelet                                          x86_64                           1.18.3-0                                  kubernetes                            21 M
Installing for dependencies:
 conntrack-tools                                  x86_64                           1.4.4-7.el7                               base                                 187 k
 cri-tools                                        x86_64                           1.13.0-0                                  kubernetes                           5.1 M
 kubernetes-cni                                   x86_64                           0.7.5-0                                   kubernetes                            10 M
 libnetfilter_cthelper                            x86_64                           1.0.0-11.el7                              base                                  18 k
 libnetfilter_cttimeout                           x86_64                           1.0.0-7.el7                               base                                  18 k
 libnetfilter_queue                               x86_64                           1.0.2-2.el7_2                             base                                  23 k
 socat                                            x86_64                           1.7.3.2-2.el7                             base                                 290 k

Transaction Summary
========================================================================================================================================================================
Install  3 Packages (+7 Dependent packages)

Total download size: 55 M
Installed size: 246 M
Downloading packages:
(1/10): conntrack-tools-1.4.4-7.el7.x86_64.rpm                                                                                                   | 187 kB  00:00:00
warning: /var/cache/yum/x86_64/7/kubernetes/packages/14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY
Public key for 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm is not installed
(2/10): 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm                                           | 5.1 MB  00:00:01
(3/10): cd5d6980c3e1b15de222db08729eff40f7031b7fa56c71ae3e28e420ba9678cd-kubectl-1.18.3-0.x86_64.rpm                                             | 9.5 MB  00:00:01
(4/10): a23839a743e789babb0ce912fa440f6e6ceb15bc5db42dd91aa0838c994b3452-kubeadm-1.18.3-0.x86_64.rpm                                             | 8.8 MB  00:00:03
(5/10): libnetfilter_cthelper-1.0.0-11.el7.x86_64.rpm                                                                                            |  18 kB  00:00:00
(6/10): libnetfilter_cttimeout-1.0.0-7.el7.x86_64.rpm                                                                                            |  18 kB  00:00:00
(7/10): socat-1.7.3.2-2.el7.x86_64.rpm                                                                                                           | 290 kB  00:00:00
(8/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm                                                                                              |  23 kB  00:00:00
(9/10): d1a0216cfab2fb28e82be531327ebde9a554bb6d33e3c8313acc9bc728ba59d1-kubelet-1.18.3-0.x86_64.rpm                                             |  21 MB  00:00:03
(10/10): 548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc905c9bf8af3b34-kubernetes-cni-0.7.5-0.x86_64.rpm                                      |  10 MB  00:00:04
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                   6.7 MB/s |  55 MB  00:00:08
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key "
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Importing GPG key 0x3E1BA8D5:
 Userid     : "Google Cloud Packages RPM Signing Key "
 Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5
 From       : https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : libnetfilter_cttimeout-1.0.0-7.el7.x86_64                                                                                                           1/10
  Installing : socat-1.7.3.2-2.el7.x86_64                                                                                                                          2/10
  Installing : cri-tools-1.13.0-0.x86_64                                                                                                                           3/10
  Installing : kubectl-1.18.3-0.x86_64                                                                                                                             4/10
  Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                             5/10
  Installing : libnetfilter_cthelper-1.0.0-11.el7.x86_64                                                                                                           6/10
  Installing : conntrack-tools-1.4.4-7.el7.x86_64                                                                                                                  7/10
  Installing : kubernetes-cni-0.7.5-0.x86_64                                                                                                                       8/10
  Installing : kubelet-1.18.3-0.x86_64                                                                                                                             9/10
  Installing : kubeadm-1.18.3-0.x86_64                                                                                                                            10/10
  Verifying  : kubelet-1.18.3-0.x86_64                                                                                                                             1/10
  Verifying  : libnetfilter_cthelper-1.0.0-11.el7.x86_64                                                                                                           2/10
  Verifying  : conntrack-tools-1.4.4-7.el7.x86_64                                                                                                                  3/10
  Verifying  : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                             4/10
  Verifying  : kubeadm-1.18.3-0.x86_64                                                                                                                             5/10
  Verifying  : kubectl-1.18.3-0.x86_64                                                                                                                             6/10
  Verifying  : cri-tools-1.13.0-0.x86_64                                                                                                                           7/10
  Verifying  : kubernetes-cni-0.7.5-0.x86_64                                                                                                                       8/10
  Verifying  : socat-1.7.3.2-2.el7.x86_64                                                                                                                          9/10
  Verifying  : libnetfilter_cttimeout-1.0.0-7.el7.x86_64                                                                                                          10/10

Installed:
  kubeadm.x86_64 0:1.18.3-0                              kubectl.x86_64 0:1.18.3-0                              kubelet.x86_64 0:1.18.3-0

Dependency Installed:
  conntrack-tools.x86_64 0:1.4.4-7.el7         cri-tools.x86_64 0:1.13.0-0                kubernetes-cni.x86_64 0:0.7.5-0  libnetfilter_cthelper.x86_64 0:1.0.0-11.el7
  libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7  libnetfilter_queue.x86_64 0:1.0.2-2.el7_2  socat.x86_64 0:1.7.3.2-2.el7

Complete!
[root@master-node ~]#

6. Start and enable kubectl and docker service
----------------------------------------------
[root@master-node ~]# systemctl restart docker && systemctl enable docker

[root@master-node ~]# systemctl restart kubelet && systemctl enable kubelet
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.

7. Initialize Kubernetes Master with ‘kubeadm init’ && Run the beneath command to  initialize and setup kubernetes master.
--------------------------------------------------------------------------------------------------------------------------

[root@master-node ~]# swapoff -a
[root@master-node ~]#

[root@centos7 ~]# kubeadm init --apiserver-advertise-address=192.168.2.130 --pod-network-cidr=10.244.0.0/16
W0603 19:28:46.895852    3584 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
[init] Using Kubernetes version: v1.18.3
[preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[certs] Using certificateDir folder "/etc/kubernetes/pki"
[certs] Generating "ca" certificate and key
[certs] Generating "apiserver" certificate and key
[certs] apiserver serving cert is signed for DNS names [centos7 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.2.130]
[certs] Generating "apiserver-kubelet-client" certificate and key
[certs] Generating "front-proxy-ca" certificate and key
[certs] Generating "front-proxy-client" certificate and key
[certs] Generating "etcd/ca" certificate and key
[certs] Generating "etcd/server" certificate and key
[certs] etcd/server serving cert is signed for DNS names [centos7 localhost] and IPs [192.168.2.130 127.0.0.1 ::1]
[certs] Generating "etcd/peer" certificate and key
[certs] etcd/peer serving cert is signed for DNS names [centos7 localhost] and IPs [192.168.2.130 127.0.0.1 ::1]
[certs] Generating "etcd/healthcheck-client" certificate and key
[certs] Generating "apiserver-etcd-client" certificate and key
[certs] Generating "sa" key and public key
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
W0603 19:30:06.779785    3584 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[control-plane] Creating static Pod manifest for "kube-scheduler"
W0603 19:30:06.781008    3584 manifests.go:225] the default kube-apiserver authorization-mode is "Node,RBAC"; using "Node,RBAC"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 17.502516 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.18" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node centos7 as control-plane by adding the label "node-role.kubernetes.io/master=''"
[mark-control-plane] Marking the node centos7 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
[bootstrap-token] Using token: a8a9vc.dk8krmn838is5gy6
[bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to get nodes
[bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
[bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
[bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
[bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
[kubelet-finalize] Updating "/etc/kubernetes/kubelet.conf" to point to a rotatable kubelet client certificate and key
[addons] Applied essential addon: CoreDNS
[addons] Applied essential addon: kube-proxy

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 192.168.2.130:6443 --token a8a9vc.dk8krmn838is5gy6 \
    --discovery-token-ca-cert-hash sha256:10c8310cd5dd8b6847bc8fa0404974051ea268bbb68b00020690c8003139975a
[root@centos7 ~]#


[root@master-node ~]# mkdir -p $HOME/.kube
[root@master-node ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@master-node ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

[root@master-node ~]# kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
podsecuritypolicy.policy/psp.flannel.unprivileged created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds-amd64 created
daemonset.apps/kube-flannel-ds-arm64 created
daemonset.apps/kube-flannel-ds-arm created
daemonset.apps/kube-flannel-ds-ppc64le created
daemonset.apps/kube-flannel-ds-s390x created
[root@master-node ~]#

[root@centos7 ~]# kubectl get nodes
NAME          STATUS     ROLES    AGE     VERSION
master-node   NotReady   master   3m38s   v1.18.3
[root@centos7 ~]#

[root@centos7 ~]# kubectl get nodes
NAME          STATUS   ROLES    AGE     VERSION
master-node   Ready    master   3m42s   v1.18.3
[root@centos7 ~]#

[root@centos7 ~]# kubectl get pods --all-namespaces
NAMESPACE     NAME                                  READY   STATUS    RESTARTS   AGE
kube-system   coredns-66bff467f8-d6cbx              1/1     Running   0          3m36s
kube-system   coredns-66bff467f8-x76j7              1/1     Running   0          3m36s
kube-system   etcd-master-node                      1/1     Running   0          3m44s
kube-system   kube-apiserver-master-node            1/1     Running   0          3m44s
kube-system   kube-controller-manager-master-node   1/1     Running   0          3m44s
kube-system   kube-flannel-ds-amd64-bmxjc           1/1     Running   0          25s
kube-system   kube-proxy-8ptqp                      1/1     Running   0          3m35s
kube-system   kube-scheduler-master-node            1/1     Running   0          3m44s
[root@centos7 ~]#

=========================================>
Worker Node-1
--------------

[root@centos7-w1 ~]# setenforce 0
[root@centos7-w1 ~]# sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
[root@centos7-w1 ~]# modprobe br_netfilter
[root@centos7-w1 ~]# echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

[root@centos7-w1 ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.2.131 centos7-w1.localdomain centos7-w1
192.168.2.130 centos7.localdomain centos7
[root@centos7-w1 ~]#

[root@centos7-w1 ~]# firewall-cmd --state
not running

[root@centos7-w1 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.atl.genesisadaptive.com
 * extras: centos.host-engine.com
 * updates: mirror.steadfastnet.com
Resolving Dependencies
--> Running transaction check
---> Package device-mapper-persistent-data.x86_64 0:0.8.5-1.el7 will be updated
---> Package device-mapper-persistent-data.x86_64 0:0.8.5-2.el7 will be an update
---> Package lvm2.x86_64 7:2.02.185-2.el7 will be updated
--> Processing Dependency: lvm2 = 7:2.02.185-2.el7 for package: 7:lvm2-cluster-2.02.185-2.el7.x86_64
---> Package lvm2.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Processing Dependency: lvm2-libs = 7:2.02.186-7.el7_8.2 for package: 7:lvm2-2.02.186-7.el7_8.2.x86_64
---> Package yum-utils.noarch 0:1.1.31-52.el7 will be updated
---> Package yum-utils.noarch 0:1.1.31-54.el7_8 will be an update
--> Running transaction check
---> Package lvm2-cluster.x86_64 7:2.02.185-2.el7 will be updated
---> Package lvm2-cluster.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Processing Dependency: device-mapper = 7:1.02.164 for package: 7:lvm2-cluster-2.02.186-7.el7_8.2.x86_64
---> Package lvm2-libs.x86_64 7:2.02.185-2.el7 will be updated
--> Processing Dependency: lvm2-libs = 7:2.02.185-2.el7 for package: 7:lvm2-python-libs-2.02.185-2.el7.x86_64
---> Package lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Processing Dependency: device-mapper-event = 7:1.02.164-7.el7_8.2 for package: 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64
--> Running transaction check
---> Package device-mapper.x86_64 7:1.02.158-2.el7 will be updated
--> Processing Dependency: device-mapper = 7:1.02.158-2.el7 for package: 7:device-mapper-libs-1.02.158-2.el7.x86_64
---> Package device-mapper.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-event.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-event.x86_64 7:1.02.164-7.el7_8.2 will be an update
--> Processing Dependency: device-mapper-event-libs = 7:1.02.164-7.el7_8.2 for package: 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64
---> Package lvm2-python-libs.x86_64 7:2.02.185-2.el7 will be updated
---> Package lvm2-python-libs.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Running transaction check
---> Package device-mapper-event-libs.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-libs.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================================================
 Package                                              Arch                          Version                                        Repository                      Size
========================================================================================================================================================================
Updating:
 device-mapper-persistent-data                        x86_64                        0.8.5-2.el7                                    base                           422 k
 lvm2                                                 x86_64                        7:2.02.186-7.el7_8.2                           updates                        1.3 M
 yum-utils                                            noarch                        1.1.31-54.el7_8                                updates                        122 k
Updating for dependencies:
 device-mapper                                        x86_64                        7:1.02.164-7.el7_8.2                           updates                        295 k
 device-mapper-event                                  x86_64                        7:1.02.164-7.el7_8.2                           updates                        191 k
 device-mapper-event-libs                             x86_64                        7:1.02.164-7.el7_8.2                           updates                        190 k
 device-mapper-libs                                   x86_64                        7:1.02.164-7.el7_8.2                           updates                        324 k
 lvm2-cluster                                         x86_64                        7:2.02.186-7.el7_8.2                           updates                        750 k
 lvm2-libs                                            x86_64                        7:2.02.186-7.el7_8.2                           updates                        1.1 M
 lvm2-python-libs                                     x86_64                        7:2.02.186-7.el7_8.2                           updates                        188 k

Transaction Summary
========================================================================================================================================================================
Upgrade  3 Packages (+7 Dependent packages)

Total size: 4.8 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : 7:device-mapper-libs-1.02.164-7.el7_8.2.x86_64                                                                                                      1/20
  Updating   : 7:device-mapper-1.02.164-7.el7_8.2.x86_64                                                                                                           2/20
  Updating   : 7:device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64                                                                                                3/20
  Updating   : 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64                                                                                                     4/20
  Updating   : 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64                                                                                                               5/20
  Updating   : device-mapper-persistent-data-0.8.5-2.el7.x86_64                                                                                                    6/20
  Updating   : 7:lvm2-2.02.186-7.el7_8.2.x86_64                                                                                                                    7/20
  Updating   : 7:lvm2-cluster-2.02.186-7.el7_8.2.x86_64                                                                                                            8/20
  Updating   : 7:lvm2-python-libs-2.02.186-7.el7_8.2.x86_64                                                                                                        9/20
  Updating   : yum-utils-1.1.31-54.el7_8.noarch                                                                                                                   10/20
  Cleanup    : 7:lvm2-cluster-2.02.185-2.el7.x86_64                                                                                                               11/20
  Cleanup    : 7:lvm2-2.02.185-2.el7.x86_64                                                                                                                       12/20
  Cleanup    : 7:lvm2-python-libs-2.02.185-2.el7.x86_64                                                                                                           13/20
  Cleanup    : yum-utils-1.1.31-52.el7.noarch                                                                                                                     14/20
  Cleanup    : 7:lvm2-libs-2.02.185-2.el7.x86_64                                                                                                                  15/20
  Cleanup    : 7:device-mapper-event-1.02.158-2.el7.x86_64                                                                                                        16/20
  Cleanup    : 7:device-mapper-event-libs-1.02.158-2.el7.x86_64                                                                                                   17/20
  Cleanup    : 7:device-mapper-libs-1.02.158-2.el7.x86_64                                                                                                         18/20
  Cleanup    : 7:device-mapper-1.02.158-2.el7.x86_64                                                                                                              19/20
  Cleanup    : device-mapper-persistent-data-0.8.5-1.el7.x86_64                                                                                                   20/20
  Verifying  : 7:device-mapper-1.02.164-7.el7_8.2.x86_64                                                                                                           1/20
  Verifying  : device-mapper-persistent-data-0.8.5-2.el7.x86_64                                                                                                    2/20
  Verifying  : 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64                                                                                                               3/20
  Verifying  : 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64                                                                                                     4/20
  Verifying  : 7:lvm2-python-libs-2.02.186-7.el7_8.2.x86_64                                                                                                        5/20
  Verifying  : 7:lvm2-2.02.186-7.el7_8.2.x86_64                                                                                                                    6/20
  Verifying  : 7:device-mapper-libs-1.02.164-7.el7_8.2.x86_64                                                                                                      7/20
  Verifying  : 7:lvm2-cluster-2.02.186-7.el7_8.2.x86_64                                                                                                            8/20
  Verifying  : yum-utils-1.1.31-54.el7_8.noarch                                                                                                                    9/20
  Verifying  : 7:device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64                                                                                               10/20
  Verifying  : 7:device-mapper-event-1.02.158-2.el7.x86_64                                                                                                        11/20
  Verifying  : device-mapper-persistent-data-0.8.5-1.el7.x86_64                                                                                                   12/20
  Verifying  : yum-utils-1.1.31-52.el7.noarch                                                                                                                     13/20
  Verifying  : 7:lvm2-python-libs-2.02.185-2.el7.x86_64                                                                                                           14/20
  Verifying  : 7:device-mapper-event-libs-1.02.158-2.el7.x86_64                                                                                                   15/20
  Verifying  : 7:lvm2-cluster-2.02.185-2.el7.x86_64                                                                                                               16/20
  Verifying  : 7:device-mapper-1.02.158-2.el7.x86_64                                                                                                              17/20
  Verifying  : 7:device-mapper-libs-1.02.158-2.el7.x86_64                                                                                                         18/20
  Verifying  : 7:lvm2-libs-2.02.185-2.el7.x86_64                                                                                                                  19/20
  Verifying  : 7:lvm2-2.02.185-2.el7.x86_64                                                                                                                       20/20

Updated:
  device-mapper-persistent-data.x86_64 0:0.8.5-2.el7                 lvm2.x86_64 7:2.02.186-7.el7_8.2                 yum-utils.noarch 0:1.1.31-54.el7_8

Dependency Updated:
  device-mapper.x86_64 7:1.02.164-7.el7_8.2            device-mapper-event.x86_64 7:1.02.164-7.el7_8.2       device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2
  device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2       lvm2-cluster.x86_64 7:2.02.186-7.el7_8.2              lvm2-libs.x86_64 7:2.02.186-7.el7_8.2
  lvm2-python-libs.x86_64 7:2.02.186-7.el7_8.2

Complete!
[root@centos7-w1 ~]#
[root@centos7-w1 ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Loaded plugins: fastestmirror, langpacks
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo

[root@centos7-w1 ~]# yum install -y docker-ce
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.atl.genesisadaptive.com
 * extras: centos.host-engine.com
 * updates: mirror.steadfastnet.com
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 3:19.03.8-3.el7 will be updated
---> Package docker-ce.x86_64 3:19.03.10-3.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================================================
 Package                              Arch                              Version                                       Repository                                   Size
========================================================================================================================================================================
Updating:
 docker-ce                            x86_64                            3:19.03.10-3.el7                              docker-ce-stable                             24 M

Transaction Summary
========================================================================================================================================================================
Upgrade  1 Package

Total download size: 24 M
Downloading packages:
No Presto metadata available for docker-ce-stable
docker-ce-19.03.10-3.el7.x86_64.rpm                                                                                                              |  24 MB  00:00:04
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : 3:docker-ce-19.03.10-3.el7.x86_64                                                                                                                    1/2
  Cleanup    : 3:docker-ce-19.03.8-3.el7.x86_64                                                                                                                     2/2
  Verifying  : 3:docker-ce-19.03.10-3.el7.x86_64                                                                                                                    1/2
  Verifying  : 3:docker-ce-19.03.8-3.el7.x86_64                                                                                                                     2/2

Updated:
  docker-ce.x86_64 3:19.03.10-3.el7

Complete!
[root@centos7-w1 ~]#

[root@centos7-w1 ~]# cat < /etc/yum.repos.d/kubernetes.repo
> [kubernetes]
> name=Kubernetes
> baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
> enabled=1
> gpgcheck=1
> repo_gpgcheck=1
> gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
>         https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
> EOF
[root@centos7-w1 ~]#

[root@centos7-w1 ~]# yum  install kubeadm docker -y --skip-broken
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.atl.genesisadaptive.com
 * extras: centos.host-engine.com
 * updates: mirrors.tummy.com
kubernetes/signature                                                                                                                             |  454 B  00:00:00
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key "
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
kubernetes/signature                                                                                                                             | 1.4 kB  00:00:00 !!!
kubernetes/primary                                                                                                                               |  69 kB  00:00:00
kubernetes                                                                                                                                                      505/505
Resolving Dependencies
--> Running transaction check
---> Package docker.x86_64 2:1.13.1-161.git64e9980.el7_8 will be installed
--> Processing Dependency: docker-common = 2:1.13.1-161.git64e9980.el7_8 for package: 2:docker-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: docker-client = 2:1.13.1-161.git64e9980.el7_8 for package: 2:docker-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: subscription-manager-rhsm-certificates for package: 2:docker-1.13.1-161.git64e9980.el7_8.x86_64
---> Package kubeadm.x86_64 0:1.18.3-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.7.5 for package: kubeadm-1.18.3-0.x86_64
--> Processing Dependency: kubelet >= 1.13.0 for package: kubeadm-1.18.3-0.x86_64
--> Processing Dependency: kubectl >= 1.13.0 for package: kubeadm-1.18.3-0.x86_64
--> Processing Dependency: cri-tools >= 1.13.0 for package: kubeadm-1.18.3-0.x86_64
--> Running transaction check
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package docker-client.x86_64 2:1.13.1-161.git64e9980.el7_8 will be installed
---> Package docker-common.x86_64 2:1.13.1-161.git64e9980.el7_8 will be installed
--> Processing Dependency: skopeo-containers >= 1:0.1.26-2 for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: oci-umount >= 2:2.3.3-3 for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: oci-systemd-hook >= 1:0.1.4-9 for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: oci-register-machine >= 1:0-5.13 for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: container-storage-setup >= 0.9.0-1 for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: atomic-registries for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
---> Package kubectl.x86_64 0:1.18.3-0 will be installed
---> Package kubelet.x86_64 0:1.18.3-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.18.3-0.x86_64
--> Processing Dependency: conntrack for package: kubelet-1.18.3-0.x86_64
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package subscription-manager-rhsm-certificates.x86_64 0:1.24.26-3.el7.centos will be installed
--> Running transaction check
---> Package atomic-registries.x86_64 1:1.22.1-33.gitb507039.el7_8 will be installed
--> Processing Dependency: python-pytoml for package: 1:atomic-registries-1.22.1-33.gitb507039.el7_8.x86_64
---> Package conntrack-tools.x86_64 0:1.4.4-7.el7 will be installed
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
---> Package container-storage-setup.noarch 0:0.11.0-2.git5eaf76c.el7 will be installed
---> Package containers-common.x86_64 1:0.1.40-7.el7_8 will be installed
--> Processing Dependency: subscription-manager for package: 1:containers-common-0.1.40-7.el7_8.x86_64
--> Processing Dependency: slirp4netns for package: 1:containers-common-0.1.40-7.el7_8.x86_64
--> Processing Dependency: fuse-overlayfs for package: 1:containers-common-0.1.40-7.el7_8.x86_64
---> Package oci-register-machine.x86_64 1:0-6.git2b44233.el7 will be installed
---> Package oci-systemd-hook.x86_64 1:0.2.0-1.git05e6923.el7_6 will be installed
---> Package oci-umount.x86_64 2:2.5-3.el7 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package fuse-overlayfs.x86_64 0:0.7.2-6.el7_8 will be installed
--> Processing Dependency: libfuse3.so.3(FUSE_3.2)(64bit) for package: fuse-overlayfs-0.7.2-6.el7_8.x86_64
--> Processing Dependency: libfuse3.so.3(FUSE_3.0)(64bit) for package: fuse-overlayfs-0.7.2-6.el7_8.x86_64
--> Processing Dependency: libfuse3.so.3()(64bit) for package: fuse-overlayfs-0.7.2-6.el7_8.x86_64
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-11.el7 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
---> Package python-pytoml.noarch 0:0.1.14-1.git7dea353.el7 will be installed
---> Package slirp4netns.x86_64 0:0.4.3-4.el7_8 will be installed
---> Package subscription-manager.x86_64 0:1.24.26-3.el7.centos will be installed
--> Processing Dependency: subscription-manager-rhsm = 1.24.26 for package: subscription-manager-1.24.26-3.el7.centos.x86_64
--> Processing Dependency: python-dmidecode >= 3.12.2-2 for package: subscription-manager-1.24.26-3.el7.centos.x86_64
--> Processing Dependency: python-syspurpose for package: subscription-manager-1.24.26-3.el7.centos.x86_64
--> Running transaction check
---> Package fuse3-libs.x86_64 0:3.6.1-4.el7 will be installed
---> Package python-dmidecode.x86_64 0:3.12.2-4.el7 will be installed
---> Package python-syspurpose.x86_64 0:1.24.26-3.el7.centos will be installed
---> Package subscription-manager-rhsm.x86_64 0:1.24.26-3.el7.centos will be installed
--> Processing Conflict: 3:docker-ce-19.03.10-3.el7.x86_64 conflicts docker
--> Processing Conflict: 3:docker-ce-19.03.10-3.el7.x86_64 conflicts docker-io
--> Processing Conflict: 1:docker-ce-cli-19.03.8-3.el7.x86_64 conflicts docker
--> Restarting Dependency Resolution with new changes.
--> Running transaction check
---> Package docker-ce-cli.x86_64 1:19.03.8-3.el7 will be updated
---> Package docker-ce-cli.x86_64 1:19.03.10-3.el7 will be an update
--> Processing Conflict: 3:docker-ce-19.03.10-3.el7.x86_64 conflicts docker
--> Processing Conflict: 3:docker-ce-19.03.10-3.el7.x86_64 conflicts docker-io
--> Processing Conflict: 1:docker-ce-cli-19.03.10-3.el7.x86_64 conflicts docker
--> Processing Conflict: 1:docker-ce-cli-19.03.10-3.el7.x86_64 conflicts docker-io
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-7.el7 will be installed
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubeadm.x86_64 0:1.18.3-0 will be installed
---> Package kubectl.x86_64 0:1.18.3-0 will be installed
---> Package kubelet.x86_64 0:1.18.3-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-11.el7 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-7.el7 will be installed
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubeadm.x86_64 0:1.18.3-0 will be installed
---> Package kubectl.x86_64 0:1.18.3-0 will be installed
---> Package kubelet.x86_64 0:1.18.3-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-11.el7 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Finished Dependency Resolution

Packages skipped because of dependency problems:
    1:atomic-registries-1.22.1-33.gitb507039.el7_8.x86_64 from extras
    container-storage-setup-0.11.0-2.git5eaf76c.el7.noarch from extras
    1:containers-common-0.1.40-7.el7_8.x86_64 from extras
    2:docker-1.13.1-161.git64e9980.el7_8.x86_64 from extras
    1:docker-ce-cli-19.03.10-3.el7.x86_64 from docker-ce-stable
    2:docker-client-1.13.1-161.git64e9980.el7_8.x86_64 from extras
    2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64 from extras
    fuse-overlayfs-0.7.2-6.el7_8.x86_64 from extras
    fuse3-libs-3.6.1-4.el7.x86_64 from extras
    1:oci-register-machine-0-6.git2b44233.el7.x86_64 from extras
    1:oci-systemd-hook-0.2.0-1.git05e6923.el7_6.x86_64 from extras
    2:oci-umount-2.5-3.el7.x86_64 from extras
    python-dmidecode-3.12.2-4.el7.x86_64 from base
    python-pytoml-0.1.14-1.git7dea353.el7.noarch from extras
    python-syspurpose-1.24.26-3.el7.centos.x86_64 from updates
    slirp4netns-0.4.3-4.el7_8.x86_64 from extras
    subscription-manager-1.24.26-3.el7.centos.x86_64 from updates
    subscription-manager-rhsm-1.24.26-3.el7.centos.x86_64 from updates
    subscription-manager-rhsm-certificates-1.24.26-3.el7.centos.x86_64 from updates

Dependencies Resolved

========================================================================================================================================================================
 Package                                                 Arch                    Version                                        Repository                         Size
========================================================================================================================================================================
Installing:
 kubeadm                                                 x86_64                  1.18.3-0                                       kubernetes                        8.8 M
Installing for dependencies:
 conntrack-tools                                         x86_64                  1.4.4-7.el7                                    base                              187 k
 cri-tools                                               x86_64                  1.13.0-0                                       kubernetes                        5.1 M
 kubectl                                                 x86_64                  1.18.3-0                                       kubernetes                        9.5 M
 kubelet                                                 x86_64                  1.18.3-0                                       kubernetes                         21 M
 kubernetes-cni                                          x86_64                  0.7.5-0                                        kubernetes                         10 M
 libnetfilter_cthelper                                   x86_64                  1.0.0-11.el7                                   base                               18 k
 libnetfilter_cttimeout                                  x86_64                  1.0.0-7.el7                                    base                               18 k
 libnetfilter_queue                                      x86_64                  1.0.2-2.el7_2                                  base                               23 k
 socat                                                   x86_64                  1.7.3.2-2.el7                                  base                              290 k
Skipped (dependency problems):
 atomic-registries                                       x86_64                  1:1.22.1-33.gitb507039.el7_8                   extras                             36 k
 container-storage-setup                                 noarch                  0.11.0-2.git5eaf76c.el7                        extras                             35 k
 containers-common                                       x86_64                  1:0.1.40-7.el7_8                               extras                             42 k
 docker                                                  x86_64                  2:1.13.1-161.git64e9980.el7_8                  extras                             18 M
 docker-ce-cli                                           x86_64                  1:19.03.10-3.el7                               docker-ce-stable                   38 M
 docker-client                                           x86_64                  2:1.13.1-161.git64e9980.el7_8                  extras                            3.9 M
 docker-common                                           x86_64                  2:1.13.1-161.git64e9980.el7_8                  extras                             99 k
 fuse-overlayfs                                          x86_64                  0.7.2-6.el7_8                                  extras                             54 k
 fuse3-libs                                              x86_64                  3.6.1-4.el7                                    extras                             82 k
 oci-register-machine                                    x86_64                  1:0-6.git2b44233.el7                           extras                            1.1 M
 oci-systemd-hook                                        x86_64                  1:0.2.0-1.git05e6923.el7_6                     extras                             34 k
 oci-umount                                              x86_64                  2:2.5-3.el7                                    extras                             33 k
 python-dmidecode                                        x86_64                  3.12.2-4.el7                                   base                               83 k
 python-pytoml                                           noarch                  0.1.14-1.git7dea353.el7                        extras                             18 k
 python-syspurpose                                       x86_64                  1.24.26-3.el7.centos                           updates                           269 k
 slirp4netns                                             x86_64                  0.4.3-4.el7_8                                  extras                             81 k
 subscription-manager                                    x86_64                  1.24.26-3.el7.centos                           updates                           1.1 M
 subscription-manager-rhsm                               x86_64                  1.24.26-3.el7.centos                           updates                           327 k
 subscription-manager-rhsm-certificates                  x86_64                  1.24.26-3.el7.centos                           updates                           232 k

Transaction Summary
========================================================================================================================================================================
Install                         1 Package  (+9 Dependent packages)
Skipped (dependency problems)  19 Packages

Total download size: 55 M
Installed size: 246 M
Downloading packages:
(1/10): conntrack-tools-1.4.4-7.el7.x86_64.rpm                                                                                                   | 187 kB  00:00:00
warning: /var/cache/yum/x86_64/7/kubernetes/packages/14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY
Public key for 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm is not installed
(2/10): 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm                                           | 5.1 MB  00:00:01
(3/10): cd5d6980c3e1b15de222db08729eff40f7031b7fa56c71ae3e28e420ba9678cd-kubectl-1.18.3-0.x86_64.rpm                                             | 9.5 MB  00:00:02
(4/10): a23839a743e789babb0ce912fa440f6e6ceb15bc5db42dd91aa0838c994b3452-kubeadm-1.18.3-0.x86_64.rpm                                             | 8.8 MB  00:00:06
(5/10): libnetfilter_cthelper-1.0.0-11.el7.x86_64.rpm                                                                                            |  18 kB  00:00:00
(6/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm                                                                                              |  23 kB  00:00:00
(7/10): libnetfilter_cttimeout-1.0.0-7.el7.x86_64.rpm                                                                                            |  18 kB  00:00:00
(8/10): socat-1.7.3.2-2.el7.x86_64.rpm                                                                                                           | 290 kB  00:00:00
(9/10): d1a0216cfab2fb28e82be531327ebde9a554bb6d33e3c8313acc9bc728ba59d1-kubelet-1.18.3-0.x86_64.rpm                                             |  21 MB  00:00:05
(10/10): 548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc905c9bf8af3b34-kubernetes-cni-0.7.5-0.x86_64.rpm                                      |  10 MB  00:00:06
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                   4.3 MB/s |  55 MB  00:00:12
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key "
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Importing GPG key 0x3E1BA8D5:
 Userid     : "Google Cloud Packages RPM Signing Key "
 Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5
 From       : https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : socat-1.7.3.2-2.el7.x86_64                                                                                                                          1/10
  Installing : cri-tools-1.13.0-0.x86_64                                                                                                                           2/10
  Installing : libnetfilter_cthelper-1.0.0-11.el7.x86_64                                                                                                           3/10
  Installing : kubectl-1.18.3-0.x86_64                                                                                                                             4/10
  Installing : libnetfilter_cttimeout-1.0.0-7.el7.x86_64                                                                                                           5/10
  Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                             6/10
  Installing : conntrack-tools-1.4.4-7.el7.x86_64                                                                                                                  7/10
  Installing : kubernetes-cni-0.7.5-0.x86_64                                                                                                                       8/10
  Installing : kubelet-1.18.3-0.x86_64                                                                                                                             9/10
  Installing : kubeadm-1.18.3-0.x86_64                                                                                                                            10/10
  Verifying  : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                             1/10
  Verifying  : conntrack-tools-1.4.4-7.el7.x86_64                                                                                                                  2/10
  Verifying  : libnetfilter_cttimeout-1.0.0-7.el7.x86_64                                                                                                           3/10
  Verifying  : kubeadm-1.18.3-0.x86_64                                                                                                                             4/10
  Verifying  : kubectl-1.18.3-0.x86_64                                                                                                                             5/10
  Verifying  : libnetfilter_cthelper-1.0.0-11.el7.x86_64                                                                                                           6/10
  Verifying  : kubelet-1.18.3-0.x86_64                                                                                                                             7/10
  Verifying  : cri-tools-1.13.0-0.x86_64                                                                                                                           8/10
  Verifying  : kubernetes-cni-0.7.5-0.x86_64                                                                                                                       9/10
  Verifying  : socat-1.7.3.2-2.el7.x86_64                                                                                                                         10/10

Installed:
  kubeadm.x86_64 0:1.18.3-0

Dependency Installed:
  conntrack-tools.x86_64 0:1.4.4-7.el7                    cri-tools.x86_64 0:1.13.0-0                           kubectl.x86_64 0:1.18.3-0
  kubelet.x86_64 0:1.18.3-0                               kubernetes-cni.x86_64 0:0.7.5-0                       libnetfilter_cthelper.x86_64 0:1.0.0-11.el7
  libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7             libnetfilter_queue.x86_64 0:1.0.2-2.el7_2             socat.x86_64 0:1.7.3.2-2.el7

Skipped (dependency problems):
  atomic-registries.x86_64 1:1.22.1-33.gitb507039.el7_8                                    container-storage-setup.noarch 0:0.11.0-2.git5eaf76c.el7
  containers-common.x86_64 1:0.1.40-7.el7_8                                                docker.x86_64 2:1.13.1-161.git64e9980.el7_8
  docker-ce-cli.x86_64 1:19.03.10-3.el7                                                    docker-client.x86_64 2:1.13.1-161.git64e9980.el7_8
  docker-common.x86_64 2:1.13.1-161.git64e9980.el7_8                                       fuse-overlayfs.x86_64 0:0.7.2-6.el7_8
  fuse3-libs.x86_64 0:3.6.1-4.el7                                                          oci-register-machine.x86_64 1:0-6.git2b44233.el7
  oci-systemd-hook.x86_64 1:0.2.0-1.git05e6923.el7_6                                       oci-umount.x86_64 2:2.5-3.el7
  python-dmidecode.x86_64 0:3.12.2-4.el7                                                   python-pytoml.noarch 0:0.1.14-1.git7dea353.el7
  python-syspurpose.x86_64 0:1.24.26-3.el7.centos                                          slirp4netns.x86_64 0:0.4.3-4.el7_8
  subscription-manager.x86_64 0:1.24.26-3.el7.centos                                       subscription-manager-rhsm.x86_64 0:1.24.26-3.el7.centos
  subscription-manager-rhsm-certificates.x86_64 0:1.24.26-3.el7.centos

Complete!
[root@centos7-w1 ~]#

[root@centos7-w1 ~]# swapoff -a

[root@centos7-w1 ~]# systemctl restart docker && systemctl enable docker

[root@centos7-w1 ~]# systemctl restart kubelet && systemctl enable kubelet
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
[root@centos7-w1 ~]#

[root@centos7-w1 ~]# kubeadm join 192.168.2.130:6443 --token a8a9vc.dk8krmn838is5gy6 \
    --discovery-token-ca-cert-hash sha256:10c8310cd5dd8b6847bc8fa0404974051ea268bbb68b00020690c8003139975a
[preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.
[root@centos7-w1 ~]#

Testing from Master Node
========================
[root@centos7 ~]# kubectl get nodes
NAME          STATUS     ROLES    AGE   VERSION
centos7-w1    NotReady      40s   v1.18.3
master-node   Ready      master   18m   v1.18.3
[root@centos7 ~]#

[root@centos7 ~]# kubectl get nodes
NAME          STATUS   ROLES    AGE    VERSION
centos7-w1    Ready       100s   v1.18.3
master-node   Ready    master   19m    v1.18.3
[root@centos7 ~]#

========================================================================>
Worker Node-2
---------------

[root@centos7-w2 ~]# setenforce 0
[root@centos7-w2 ~]# sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux
[root@centos7-w2 ~]# modprobe br_netfilter
[root@centos7-w2 ~]# echo '1' > /proc/sys/net/bridge/bridge-nf-call-iptables

[root@centos7-w2 ~]# firewall-cmd --state
not running
[root@centos7-w2 ~]#

[root@centos7-w2 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.dal.nexril.net
 * extras: mirror.hostduplex.com
 * updates: mirror.dal.nexril.net
base                                                                                                                                             | 3.6 kB  00:00:00
docker-ce-stable                                                                                                                                 | 3.5 kB  00:00:00
extras                                                                                                                                           | 2.9 kB  00:00:00
updates                                                                                                                                          | 2.9 kB  00:00:00
docker-ce-stable/x86_64/primary_db                                                                                                               |  43 kB  00:00:00
Resolving Dependencies
--> Running transaction check
---> Package device-mapper-persistent-data.x86_64 0:0.8.5-1.el7 will be updated
---> Package device-mapper-persistent-data.x86_64 0:0.8.5-2.el7 will be an update
---> Package lvm2.x86_64 7:2.02.185-2.el7 will be updated
--> Processing Dependency: lvm2 = 7:2.02.185-2.el7 for package: 7:lvm2-cluster-2.02.185-2.el7.x86_64
---> Package lvm2.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Processing Dependency: lvm2-libs = 7:2.02.186-7.el7_8.2 for package: 7:lvm2-2.02.186-7.el7_8.2.x86_64
---> Package yum-utils.noarch 0:1.1.31-52.el7 will be updated
---> Package yum-utils.noarch 0:1.1.31-54.el7_8 will be an update
--> Running transaction check
---> Package lvm2-cluster.x86_64 7:2.02.185-2.el7 will be updated
---> Package lvm2-cluster.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Processing Dependency: device-mapper = 7:1.02.164 for package: 7:lvm2-cluster-2.02.186-7.el7_8.2.x86_64
---> Package lvm2-libs.x86_64 7:2.02.185-2.el7 will be updated
--> Processing Dependency: lvm2-libs = 7:2.02.185-2.el7 for package: 7:lvm2-python-libs-2.02.185-2.el7.x86_64
---> Package lvm2-libs.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Processing Dependency: device-mapper-event = 7:1.02.164-7.el7_8.2 for package: 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64
--> Running transaction check
---> Package device-mapper.x86_64 7:1.02.158-2.el7 will be updated
--> Processing Dependency: device-mapper = 7:1.02.158-2.el7 for package: 7:device-mapper-libs-1.02.158-2.el7.x86_64
---> Package device-mapper.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-event.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-event.x86_64 7:1.02.164-7.el7_8.2 will be an update
--> Processing Dependency: device-mapper-event-libs = 7:1.02.164-7.el7_8.2 for package: 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64
---> Package lvm2-python-libs.x86_64 7:2.02.185-2.el7 will be updated
---> Package lvm2-python-libs.x86_64 7:2.02.186-7.el7_8.2 will be an update
--> Running transaction check
---> Package device-mapper-event-libs.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2 will be an update
---> Package device-mapper-libs.x86_64 7:1.02.158-2.el7 will be updated
---> Package device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================================================
 Package                                              Arch                          Version                                        Repository                      Size
========================================================================================================================================================================
Updating:
 device-mapper-persistent-data                        x86_64                        0.8.5-2.el7                                    base                           422 k
 lvm2                                                 x86_64                        7:2.02.186-7.el7_8.2                           updates                        1.3 M
 yum-utils                                            noarch                        1.1.31-54.el7_8                                updates                        122 k
Updating for dependencies:
 device-mapper                                        x86_64                        7:1.02.164-7.el7_8.2                           updates                        295 k
 device-mapper-event                                  x86_64                        7:1.02.164-7.el7_8.2                           updates                        191 k
 device-mapper-event-libs                             x86_64                        7:1.02.164-7.el7_8.2                           updates                        190 k
 device-mapper-libs                                   x86_64                        7:1.02.164-7.el7_8.2                           updates                        324 k
 lvm2-cluster                                         x86_64                        7:2.02.186-7.el7_8.2                           updates                        750 k
 lvm2-libs                                            x86_64                        7:2.02.186-7.el7_8.2                           updates                        1.1 M
 lvm2-python-libs                                     x86_64                        7:2.02.186-7.el7_8.2                           updates                        188 k

Transaction Summary
========================================================================================================================================================================
Upgrade  3 Packages (+7 Dependent packages)

Total size: 4.8 M
Downloading packages:
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : 7:device-mapper-libs-1.02.164-7.el7_8.2.x86_64                                                                                                      1/20
  Updating   : 7:device-mapper-1.02.164-7.el7_8.2.x86_64                                                                                                           2/20
  Updating   : 7:device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64                                                                                                3/20
  Updating   : 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64                                                                                                     4/20
  Updating   : 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64                                                                                                               5/20
  Updating   : device-mapper-persistent-data-0.8.5-2.el7.x86_64                                                                                                    6/20
  Updating   : 7:lvm2-2.02.186-7.el7_8.2.x86_64                                                                                                                    7/20
  Updating   : 7:lvm2-cluster-2.02.186-7.el7_8.2.x86_64                                                                                                            8/20
  Updating   : 7:lvm2-python-libs-2.02.186-7.el7_8.2.x86_64                                                                                                        9/20
  Updating   : yum-utils-1.1.31-54.el7_8.noarch                                                                                                                   10/20
  Cleanup    : 7:lvm2-cluster-2.02.185-2.el7.x86_64                                                                                                               11/20
  Cleanup    : 7:lvm2-2.02.185-2.el7.x86_64                                                                                                                       12/20
  Cleanup    : 7:lvm2-python-libs-2.02.185-2.el7.x86_64                                                                                                           13/20
  Cleanup    : yum-utils-1.1.31-52.el7.noarch                                                                                                                     14/20
  Cleanup    : 7:lvm2-libs-2.02.185-2.el7.x86_64                                                                                                                  15/20
  Cleanup    : 7:device-mapper-event-1.02.158-2.el7.x86_64                                                                                                        16/20
  Cleanup    : 7:device-mapper-event-libs-1.02.158-2.el7.x86_64                                                                                                   17/20
  Cleanup    : 7:device-mapper-libs-1.02.158-2.el7.x86_64                                                                                                         18/20
  Cleanup    : 7:device-mapper-1.02.158-2.el7.x86_64                                                                                                              19/20
  Cleanup    : device-mapper-persistent-data-0.8.5-1.el7.x86_64                                                                                                   20/20
  Verifying  : 7:device-mapper-1.02.164-7.el7_8.2.x86_64                                                                                                           1/20
  Verifying  : device-mapper-persistent-data-0.8.5-2.el7.x86_64                                                                                                    2/20
  Verifying  : 7:lvm2-libs-2.02.186-7.el7_8.2.x86_64                                                                                                               3/20
  Verifying  : 7:device-mapper-event-1.02.164-7.el7_8.2.x86_64                                                                                                     4/20
  Verifying  : 7:lvm2-python-libs-2.02.186-7.el7_8.2.x86_64                                                                                                        5/20
  Verifying  : 7:lvm2-2.02.186-7.el7_8.2.x86_64                                                                                                                    6/20
  Verifying  : 7:device-mapper-libs-1.02.164-7.el7_8.2.x86_64                                                                                                      7/20
  Verifying  : 7:lvm2-cluster-2.02.186-7.el7_8.2.x86_64                                                                                                            8/20
  Verifying  : yum-utils-1.1.31-54.el7_8.noarch                                                                                                                    9/20
  Verifying  : 7:device-mapper-event-libs-1.02.164-7.el7_8.2.x86_64                                                                                               10/20
  Verifying  : 7:device-mapper-event-1.02.158-2.el7.x86_64                                                                                                        11/20
  Verifying  : device-mapper-persistent-data-0.8.5-1.el7.x86_64                                                                                                   12/20
  Verifying  : yum-utils-1.1.31-52.el7.noarch                                                                                                                     13/20
  Verifying  : 7:lvm2-python-libs-2.02.185-2.el7.x86_64                                                                                                           14/20
  Verifying  : 7:device-mapper-event-libs-1.02.158-2.el7.x86_64                                                                                                   15/20
  Verifying  : 7:lvm2-cluster-2.02.185-2.el7.x86_64                                                                                                               16/20
  Verifying  : 7:device-mapper-1.02.158-2.el7.x86_64                                                                                                              17/20
  Verifying  : 7:device-mapper-libs-1.02.158-2.el7.x86_64                                                                                                         18/20
  Verifying  : 7:lvm2-libs-2.02.185-2.el7.x86_64                                                                                                                  19/20
  Verifying  : 7:lvm2-2.02.185-2.el7.x86_64                                                                                                                       20/20

Updated:
  device-mapper-persistent-data.x86_64 0:0.8.5-2.el7                 lvm2.x86_64 7:2.02.186-7.el7_8.2                 yum-utils.noarch 0:1.1.31-54.el7_8

Dependency Updated:
  device-mapper.x86_64 7:1.02.164-7.el7_8.2            device-mapper-event.x86_64 7:1.02.164-7.el7_8.2       device-mapper-event-libs.x86_64 7:1.02.164-7.el7_8.2
  device-mapper-libs.x86_64 7:1.02.164-7.el7_8.2       lvm2-cluster.x86_64 7:2.02.186-7.el7_8.2              lvm2-libs.x86_64 7:2.02.186-7.el7_8.2
  lvm2-python-libs.x86_64 7:2.02.186-7.el7_8.2

Complete!
[root@centos7-w2 ~]#

[root@centos7-w2 ~]# yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
Loaded plugins: fastestmirror, langpacks
adding repo from: https://download.docker.com/linux/centos/docker-ce.repo
grabbing file https://download.docker.com/linux/centos/docker-ce.repo to /etc/yum.repos.d/docker-ce.repo
repo saved to /etc/yum.repos.d/docker-ce.repo
[root@centos7-w2 ~]#

[root@centos7-w2 ~]# yum install -y docker-ce
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.dal.nexril.net
 * extras: mirror.hostduplex.com
 * updates: mirror.dal.nexril.net
Resolving Dependencies
--> Running transaction check
---> Package docker-ce.x86_64 3:19.03.8-3.el7 will be updated
---> Package docker-ce.x86_64 3:19.03.10-3.el7 will be an update
--> Finished Dependency Resolution

Dependencies Resolved

========================================================================================================================================================================
 Package                              Arch                              Version                                       Repository                                   Size
========================================================================================================================================================================
Updating:
 docker-ce                            x86_64                            3:19.03.10-3.el7                              docker-ce-stable                             24 M

Transaction Summary
========================================================================================================================================================================
Upgrade  1 Package

Total download size: 24 M
Downloading packages:
No Presto metadata available for docker-ce-stable
docker-ce-19.03.10-3.el7.x86_64.rpm                                                                                                              |  24 MB  00:00:05
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Updating   : 3:docker-ce-19.03.10-3.el7.x86_64                                                                                                                    1/2
  Cleanup    : 3:docker-ce-19.03.8-3.el7.x86_64                                                                                                                     2/2
  Verifying  : 3:docker-ce-19.03.10-3.el7.x86_64                                                                                                                    1/2
  Verifying  : 3:docker-ce-19.03.8-3.el7.x86_64                                                                                                                     2/2

Updated:
  docker-ce.x86_64 3:19.03.10-3.el7

Complete!
[root@centos7-w2 ~]# 

cat < /etc/yum.repos.d/kubernetes.repo
> [kubernetes]
> name=Kubernetes
> baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64
> enabled=1
> gpgcheck=1
> repo_gpgcheck=1
> gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg
>         https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
> EOF
[root@centos7-w2 ~]#

[root@centos7-w2 ~]# yum  install kubeadm docker -y --skip-broken
Loaded plugins: fastestmirror, langpacks
Loading mirror speeds from cached hostfile
 * base: mirror.dal.nexril.net
 * extras: mirror.steadfastnet.com
 * updates: mirror.dal.nexril.net
kubernetes/signature                                                                                                                             |  454 B  00:00:00
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key "
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
kubernetes/signature                                                                                                                             | 1.4 kB  00:00:00 !!!
kubernetes/primary                                                                                                                               |  69 kB  00:00:00
kubernetes                                                                                                                                                      505/505
Resolving Dependencies
--> Running transaction check
---> Package docker.x86_64 2:1.13.1-161.git64e9980.el7_8 will be installed
--> Processing Dependency: docker-common = 2:1.13.1-161.git64e9980.el7_8 for package: 2:docker-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: docker-client = 2:1.13.1-161.git64e9980.el7_8 for package: 2:docker-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: subscription-manager-rhsm-certificates for package: 2:docker-1.13.1-161.git64e9980.el7_8.x86_64
---> Package kubeadm.x86_64 0:1.18.3-0 will be installed
--> Processing Dependency: kubernetes-cni >= 0.7.5 for package: kubeadm-1.18.3-0.x86_64
--> Processing Dependency: kubelet >= 1.13.0 for package: kubeadm-1.18.3-0.x86_64
--> Processing Dependency: kubectl >= 1.13.0 for package: kubeadm-1.18.3-0.x86_64
--> Processing Dependency: cri-tools >= 1.13.0 for package: kubeadm-1.18.3-0.x86_64
--> Running transaction check
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package docker-client.x86_64 2:1.13.1-161.git64e9980.el7_8 will be installed
---> Package docker-common.x86_64 2:1.13.1-161.git64e9980.el7_8 will be installed
--> Processing Dependency: skopeo-containers >= 1:0.1.26-2 for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: oci-umount >= 2:2.3.3-3 for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: oci-systemd-hook >= 1:0.1.4-9 for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: oci-register-machine >= 1:0-5.13 for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: container-storage-setup >= 0.9.0-1 for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
--> Processing Dependency: atomic-registries for package: 2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64
---> Package kubectl.x86_64 0:1.18.3-0 will be installed
---> Package kubelet.x86_64 0:1.18.3-0 will be installed
--> Processing Dependency: socat for package: kubelet-1.18.3-0.x86_64
--> Processing Dependency: conntrack for package: kubelet-1.18.3-0.x86_64
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package subscription-manager-rhsm-certificates.x86_64 0:1.24.26-3.el7.centos will be installed
--> Running transaction check
---> Package atomic-registries.x86_64 1:1.22.1-33.gitb507039.el7_8 will be installed
--> Processing Dependency: python-pytoml for package: 1:atomic-registries-1.22.1-33.gitb507039.el7_8.x86_64
---> Package conntrack-tools.x86_64 0:1.4.4-7.el7 will be installed
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.1)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1(LIBNETFILTER_CTTIMEOUT_1.0)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0(LIBNETFILTER_CTHELPER_1.0)(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_queue.so.1()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cttimeout.so.1()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
--> Processing Dependency: libnetfilter_cthelper.so.0()(64bit) for package: conntrack-tools-1.4.4-7.el7.x86_64
---> Package container-storage-setup.noarch 0:0.11.0-2.git5eaf76c.el7 will be installed
---> Package containers-common.x86_64 1:0.1.40-7.el7_8 will be installed
--> Processing Dependency: subscription-manager for package: 1:containers-common-0.1.40-7.el7_8.x86_64
--> Processing Dependency: slirp4netns for package: 1:containers-common-0.1.40-7.el7_8.x86_64
--> Processing Dependency: fuse-overlayfs for package: 1:containers-common-0.1.40-7.el7_8.x86_64
---> Package oci-register-machine.x86_64 1:0-6.git2b44233.el7 will be installed
---> Package oci-systemd-hook.x86_64 1:0.2.0-1.git05e6923.el7_6 will be installed
---> Package oci-umount.x86_64 2:2.5-3.el7 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package fuse-overlayfs.x86_64 0:0.7.2-6.el7_8 will be installed
--> Processing Dependency: libfuse3.so.3(FUSE_3.2)(64bit) for package: fuse-overlayfs-0.7.2-6.el7_8.x86_64
--> Processing Dependency: libfuse3.so.3(FUSE_3.0)(64bit) for package: fuse-overlayfs-0.7.2-6.el7_8.x86_64
--> Processing Dependency: libfuse3.so.3()(64bit) for package: fuse-overlayfs-0.7.2-6.el7_8.x86_64
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-11.el7 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
---> Package python-pytoml.noarch 0:0.1.14-1.git7dea353.el7 will be installed
---> Package slirp4netns.x86_64 0:0.4.3-4.el7_8 will be installed
---> Package subscription-manager.x86_64 0:1.24.26-3.el7.centos will be installed
--> Processing Dependency: subscription-manager-rhsm = 1.24.26 for package: subscription-manager-1.24.26-3.el7.centos.x86_64
--> Processing Dependency: python-dmidecode >= 3.12.2-2 for package: subscription-manager-1.24.26-3.el7.centos.x86_64
--> Processing Dependency: python-syspurpose for package: subscription-manager-1.24.26-3.el7.centos.x86_64
--> Running transaction check
---> Package fuse3-libs.x86_64 0:3.6.1-4.el7 will be installed
---> Package python-dmidecode.x86_64 0:3.12.2-4.el7 will be installed
---> Package python-syspurpose.x86_64 0:1.24.26-3.el7.centos will be installed
---> Package subscription-manager-rhsm.x86_64 0:1.24.26-3.el7.centos will be installed
--> Processing Conflict: 3:docker-ce-19.03.10-3.el7.x86_64 conflicts docker
--> Processing Conflict: 3:docker-ce-19.03.10-3.el7.x86_64 conflicts docker-io
--> Processing Conflict: 1:docker-ce-cli-19.03.8-3.el7.x86_64 conflicts docker
--> Restarting Dependency Resolution with new changes.
--> Running transaction check
---> Package docker-ce-cli.x86_64 1:19.03.8-3.el7 will be updated
---> Package docker-ce-cli.x86_64 1:19.03.10-3.el7 will be an update
--> Processing Conflict: 3:docker-ce-19.03.10-3.el7.x86_64 conflicts docker
--> Processing Conflict: 3:docker-ce-19.03.10-3.el7.x86_64 conflicts docker-io
--> Processing Conflict: 1:docker-ce-cli-19.03.10-3.el7.x86_64 conflicts docker
--> Processing Conflict: 1:docker-ce-cli-19.03.10-3.el7.x86_64 conflicts docker-io
docker-ce-stable/x86_64/filelists_db                                                                                                             |  20 kB  00:00:00
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-7.el7 will be installed
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubeadm.x86_64 0:1.18.3-0 will be installed
---> Package kubectl.x86_64 0:1.18.3-0 will be installed
---> Package kubelet.x86_64 0:1.18.3-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-11.el7 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Running transaction check
---> Package conntrack-tools.x86_64 0:1.4.4-7.el7 will be installed
---> Package cri-tools.x86_64 0:1.13.0-0 will be installed
---> Package kubeadm.x86_64 0:1.18.3-0 will be installed
---> Package kubectl.x86_64 0:1.18.3-0 will be installed
---> Package kubelet.x86_64 0:1.18.3-0 will be installed
---> Package kubernetes-cni.x86_64 0:0.7.5-0 will be installed
---> Package libnetfilter_cthelper.x86_64 0:1.0.0-11.el7 will be installed
---> Package libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7 will be installed
---> Package libnetfilter_queue.x86_64 0:1.0.2-2.el7_2 will be installed
---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed
--> Finished Dependency Resolution

Packages skipped because of dependency problems:
    1:atomic-registries-1.22.1-33.gitb507039.el7_8.x86_64 from extras
    container-storage-setup-0.11.0-2.git5eaf76c.el7.noarch from extras
    1:containers-common-0.1.40-7.el7_8.x86_64 from extras
    2:docker-1.13.1-161.git64e9980.el7_8.x86_64 from extras
    1:docker-ce-cli-19.03.10-3.el7.x86_64 from docker-ce-stable
    2:docker-client-1.13.1-161.git64e9980.el7_8.x86_64 from extras
    2:docker-common-1.13.1-161.git64e9980.el7_8.x86_64 from extras
    fuse-overlayfs-0.7.2-6.el7_8.x86_64 from extras
    fuse3-libs-3.6.1-4.el7.x86_64 from extras
    1:oci-register-machine-0-6.git2b44233.el7.x86_64 from extras
    1:oci-systemd-hook-0.2.0-1.git05e6923.el7_6.x86_64 from extras
    2:oci-umount-2.5-3.el7.x86_64 from extras
    python-dmidecode-3.12.2-4.el7.x86_64 from base
    python-pytoml-0.1.14-1.git7dea353.el7.noarch from extras
    python-syspurpose-1.24.26-3.el7.centos.x86_64 from updates
    slirp4netns-0.4.3-4.el7_8.x86_64 from extras
    subscription-manager-1.24.26-3.el7.centos.x86_64 from updates
    subscription-manager-rhsm-1.24.26-3.el7.centos.x86_64 from updates
    subscription-manager-rhsm-certificates-1.24.26-3.el7.centos.x86_64 from updates

Dependencies Resolved

========================================================================================================================================================================
 Package                                                 Arch                    Version                                        Repository                         Size
========================================================================================================================================================================
Installing:
 kubeadm                                                 x86_64                  1.18.3-0                                       kubernetes                        8.8 M
Installing for dependencies:
 conntrack-tools                                         x86_64                  1.4.4-7.el7                                    base                              187 k
 cri-tools                                               x86_64                  1.13.0-0                                       kubernetes                        5.1 M
 kubectl                                                 x86_64                  1.18.3-0                                       kubernetes                        9.5 M
 kubelet                                                 x86_64                  1.18.3-0                                       kubernetes                         21 M
 kubernetes-cni                                          x86_64                  0.7.5-0                                        kubernetes                         10 M
 libnetfilter_cthelper                                   x86_64                  1.0.0-11.el7                                   base                               18 k
 libnetfilter_cttimeout                                  x86_64                  1.0.0-7.el7                                    base                               18 k
 libnetfilter_queue                                      x86_64                  1.0.2-2.el7_2                                  base                               23 k
 socat                                                   x86_64                  1.7.3.2-2.el7                                  base                              290 k
Skipped (dependency problems):
 atomic-registries                                       x86_64                  1:1.22.1-33.gitb507039.el7_8                   extras                             36 k
 container-storage-setup                                 noarch                  0.11.0-2.git5eaf76c.el7                        extras                             35 k
 containers-common                                       x86_64                  1:0.1.40-7.el7_8                               extras                             42 k
 docker                                                  x86_64                  2:1.13.1-161.git64e9980.el7_8                  extras                             18 M
 docker-ce-cli                                           x86_64                  1:19.03.10-3.el7                               docker-ce-stable                   38 M
 docker-client                                           x86_64                  2:1.13.1-161.git64e9980.el7_8                  extras                            3.9 M
 docker-common                                           x86_64                  2:1.13.1-161.git64e9980.el7_8                  extras                             99 k
 fuse-overlayfs                                          x86_64                  0.7.2-6.el7_8                                  extras                             54 k
 fuse3-libs                                              x86_64                  3.6.1-4.el7                                    extras                             82 k
 oci-register-machine                                    x86_64                  1:0-6.git2b44233.el7                           extras                            1.1 M
 oci-systemd-hook                                        x86_64                  1:0.2.0-1.git05e6923.el7_6                     extras                             34 k
 oci-umount                                              x86_64                  2:2.5-3.el7                                    extras                             33 k
 python-dmidecode                                        x86_64                  3.12.2-4.el7                                   base                               83 k
 python-pytoml                                           noarch                  0.1.14-1.git7dea353.el7                        extras                             18 k
 python-syspurpose                                       x86_64                  1.24.26-3.el7.centos                           updates                           269 k
 slirp4netns                                             x86_64                  0.4.3-4.el7_8                                  extras                             81 k
 subscription-manager                                    x86_64                  1.24.26-3.el7.centos                           updates                           1.1 M
 subscription-manager-rhsm                               x86_64                  1.24.26-3.el7.centos                           updates                           327 k
 subscription-manager-rhsm-certificates                  x86_64                  1.24.26-3.el7.centos                           updates                           232 k

Transaction Summary
========================================================================================================================================================================
Install                         1 Package  (+9 Dependent packages)
Skipped (dependency problems)  19 Packages

Total download size: 55 M
Installed size: 246 M
Downloading packages:
(1/10): conntrack-tools-1.4.4-7.el7.x86_64.rpm                                                                                                   | 187 kB  00:00:00
warning: /var/cache/yum/x86_64/7/kubernetes/packages/14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm: Header V4 RSA/SHA512 Signature, key ID 3e1ba8d5: NOKEY
Public key for 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm is not installed
(2/10): 14bfe6e75a9efc8eca3f638eb22c7e2ce759c67f95b43b16fae4ebabde1549f3-cri-tools-1.13.0-0.x86_64.rpm                                           | 5.1 MB  00:00:01
(3/10): a23839a743e789babb0ce912fa440f6e6ceb15bc5db42dd91aa0838c994b3452-kubeadm-1.18.3-0.x86_64.rpm                                             | 8.8 MB  00:00:02
(4/10): cd5d6980c3e1b15de222db08729eff40f7031b7fa56c71ae3e28e420ba9678cd-kubectl-1.18.3-0.x86_64.rpm                                             | 9.5 MB  00:00:02
(5/10): libnetfilter_cthelper-1.0.0-11.el7.x86_64.rpm                                                                                            |  18 kB  00:00:00
(6/10): libnetfilter_queue-1.0.2-2.el7_2.x86_64.rpm                                                                                              |  23 kB  00:00:00
(7/10): socat-1.7.3.2-2.el7.x86_64.rpm                                                                                                           | 290 kB  00:00:00
(8/10): libnetfilter_cttimeout-1.0.0-7.el7.x86_64.rpm                                                                                            |  18 kB  00:00:00
(9/10): d1a0216cfab2fb28e82be531327ebde9a554bb6d33e3c8313acc9bc728ba59d1-kubelet-1.18.3-0.x86_64.rpm                                             |  21 MB  00:00:04
(10/10): 548a0dcd865c16a50980420ddfa5fbccb8b59621179798e6dc905c9bf8af3b34-kubernetes-cni-0.7.5-0.x86_64.rpm                                      |  10 MB  00:00:02
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                   8.2 MB/s |  55 MB  00:00:06
Retrieving key from https://packages.cloud.google.com/yum/doc/yum-key.gpg
Importing GPG key 0xA7317B0F:
 Userid     : "Google Cloud Packages Automatic Signing Key "
 Fingerprint: d0bc 747f d8ca f711 7500 d6fa 3746 c208 a731 7b0f
 From       : https://packages.cloud.google.com/yum/doc/yum-key.gpg
Retrieving key from https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Importing GPG key 0x3E1BA8D5:
 Userid     : "Google Cloud Packages RPM Signing Key "
 Fingerprint: 3749 e1ba 95a8 6ce0 5454 6ed2 f09c 394c 3e1b a8d5
 From       : https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : socat-1.7.3.2-2.el7.x86_64                                                                                                                          1/10
  Installing : cri-tools-1.13.0-0.x86_64                                                                                                                           2/10
  Installing : libnetfilter_cthelper-1.0.0-11.el7.x86_64                                                                                                           3/10
  Installing : kubectl-1.18.3-0.x86_64                                                                                                                             4/10
  Installing : libnetfilter_cttimeout-1.0.0-7.el7.x86_64                                                                                                           5/10
  Installing : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                             6/10
  Installing : conntrack-tools-1.4.4-7.el7.x86_64                                                                                                                  7/10
  Installing : kubernetes-cni-0.7.5-0.x86_64                                                                                                                       8/10
  Installing : kubelet-1.18.3-0.x86_64                                                                                                                             9/10
  Installing : kubeadm-1.18.3-0.x86_64                                                                                                                            10/10
  Verifying  : libnetfilter_queue-1.0.2-2.el7_2.x86_64                                                                                                             1/10
  Verifying  : conntrack-tools-1.4.4-7.el7.x86_64                                                                                                                  2/10
  Verifying  : libnetfilter_cttimeout-1.0.0-7.el7.x86_64                                                                                                           3/10
  Verifying  : kubeadm-1.18.3-0.x86_64                                                                                                                             4/10
  Verifying  : kubectl-1.18.3-0.x86_64                                                                                                                             5/10
  Verifying  : libnetfilter_cthelper-1.0.0-11.el7.x86_64                                                                                                           6/10
  Verifying  : kubelet-1.18.3-0.x86_64                                                                                                                             7/10
  Verifying  : cri-tools-1.13.0-0.x86_64                                                                                                                           8/10
  Verifying  : kubernetes-cni-0.7.5-0.x86_64                                                                                                                       9/10
  Verifying  : socat-1.7.3.2-2.el7.x86_64                                                                                                                         10/10

Installed:
  kubeadm.x86_64 0:1.18.3-0

Dependency Installed:
  conntrack-tools.x86_64 0:1.4.4-7.el7                    cri-tools.x86_64 0:1.13.0-0                           kubectl.x86_64 0:1.18.3-0
  kubelet.x86_64 0:1.18.3-0                               kubernetes-cni.x86_64 0:0.7.5-0                       libnetfilter_cthelper.x86_64 0:1.0.0-11.el7
  libnetfilter_cttimeout.x86_64 0:1.0.0-7.el7             libnetfilter_queue.x86_64 0:1.0.2-2.el7_2             socat.x86_64 0:1.7.3.2-2.el7

Skipped (dependency problems):
  atomic-registries.x86_64 1:1.22.1-33.gitb507039.el7_8                                    container-storage-setup.noarch 0:0.11.0-2.git5eaf76c.el7
  containers-common.x86_64 1:0.1.40-7.el7_8                                                docker.x86_64 2:1.13.1-161.git64e9980.el7_8
  docker-ce-cli.x86_64 1:19.03.10-3.el7                                                    docker-client.x86_64 2:1.13.1-161.git64e9980.el7_8
  docker-common.x86_64 2:1.13.1-161.git64e9980.el7_8                                       fuse-overlayfs.x86_64 0:0.7.2-6.el7_8
  fuse3-libs.x86_64 0:3.6.1-4.el7                                                          oci-register-machine.x86_64 1:0-6.git2b44233.el7
  oci-systemd-hook.x86_64 1:0.2.0-1.git05e6923.el7_6                                       oci-umount.x86_64 2:2.5-3.el7
  python-dmidecode.x86_64 0:3.12.2-4.el7                                                   python-pytoml.noarch 0:0.1.14-1.git7dea353.el7
  python-syspurpose.x86_64 0:1.24.26-3.el7.centos                                          slirp4netns.x86_64 0:0.4.3-4.el7_8
  subscription-manager.x86_64 0:1.24.26-3.el7.centos                                       subscription-manager-rhsm.x86_64 0:1.24.26-3.el7.centos
  subscription-manager-rhsm-certificates.x86_64 0:1.24.26-3.el7.centos

Complete!
[root@centos7-w2 ~]#

[root@centos7-w2 ~]# swapoff -a

[root@centos7-w2 ~]# kubeadm join 192.168.2.130:6443 --token a8a9vc.dk8krmn838is5gy6 \
    --discovery-token-ca-cert-hash sha256:10c8310cd5dd8b6847bc8fa0404974051ea268bbb68b00020690c8003139975a

W0531 19:40:35.637237    5557 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
[preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
[kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.18" ConfigMap in the kube-system namespace
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...

This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

[root@centos7-w2 ~]# systemctl restart docker && systemctl enable docker

============================================================================================>
[root@centos7 ~]# kubectl get nodes
NAME          STATUS   ROLES    AGE   VERSION
centos7-w1    Ready       22m   v1.18.3
centos7-w2    Ready       67s   v1.18.3
master-node   Ready    master   40m   v1.18.3
[root@centos7 ~]#

[root@centos7 ~]# kubectl get nodes
NAME          STATUS   ROLES    AGE     VERSION
centos7-w1    Ready       28m     v1.18.3
centos7-w2    Ready       6m42s   v1.18.3
master-node   Ready    master   45m     v1.18.3
[root@centos7 ~]#
============================================================================================>
Testing The Master Node With NGINX

[root@centos7 ~]# kubectl create deployment nginx --image=nginx
deployment.apps/nginx created
[root@centos7 ~]#

[root@centos7 ~]# kubectl describe deployment nginx
Name:                   nginx
Namespace:              default
CreationTimestamp:      Sun, 31 May 2020 20:03:57 -0500
Labels:                 app=nginx
Annotations:            deployment.kubernetes.io/revision: 1
Selector:               app=nginx
Replicas:               1 desired | 1 updated | 1 total | 1 available | 0 unavailable
StrategyType:           RollingUpdate
MinReadySeconds:        0
RollingUpdateStrategy:  25% max unavailable, 25% max surge
Pod Template:
  Labels:  app=nginx
  Containers:
   nginx:
    Image:        nginx
    Port:         
    Host Port:    
    Environment:  
    Mounts:       
  Volumes:        
Conditions:
  Type           Status  Reason
  ----           ------  ------
  Available      True    MinimumReplicasAvailable
  Progressing    True    NewReplicaSetAvailable
OldReplicaSets:  
NewReplicaSet:   nginx-f89759699 (1/1 replicas created)
Events:
  Type    Reason             Age   From                   Message
  ----    ------             ----  ----                   -------
  Normal  ScalingReplicaSet  8s    deployment-controller  Scaled up replica set nginx-f89759699 to 1
[root@centos7 ~]#

[root@centos7 ~]# kubectl create service nodeport nginx --tcp=80:80
service/nginx created
[root@centos7 ~]#

[root@centos7 ~]# kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-f89759699-jgcp4   1/1     Running   0          65s
[root@centos7 ~]#

[root@centos7 ~]# kubectl get nodes
NAME          STATUS   ROLES    AGE   VERSION
centos7-w1    Ready       45m   v1.18.3
centos7-w2    Ready       24m   v1.18.3
master-node   Ready    master   63m   v1.18.3
[root@centos7 ~]#

[root@centos7 ~]# kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.96.0.1                443/TCP        64m
nginx        NodePort    10.103.135.131           80:32701/TCP   40s
[root@centos7 ~]#

[root@centos7 ~]# kubectl edit deployment nginx
------------------------------------------------>
# Please edit the object below. Lines beginning with a '#' will be ignored,
# and an empty file will abort the edit. If an error occurs while saving this file will be
# reopened with the relevant failures.
#
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: "2020-06-04T00:37:35Z"
  generation: 2
  labels:
    app: nginx
  managedFields:
  - apiVersion: apps/v1
    fieldsType: FieldsV1
    fieldsV1:
      f:metadata:
        f:labels:
          .: {}
          f:app: {}
      f:spec:
        f:progressDeadlineSeconds: {}
        f:replicas: {}
        f:revisionHistoryLimit: {}
        f:selector:
          f:matchLabels:
            .: {}
            f:app: {}
        f:strategy:
          f:rollingUpdate:
            .: {}
            f:maxSurge: {}
            f:maxUnavailable: {}
          f:type: {}
        f:template:
          f:metadata:
            f:labels:
              .: {}
              f:app: {}
          f:spec:
            f:containers:
              k:{"name":"nginx"}:
                .: {}
                f:image: {}
                f:imagePullPolicy: {}
spec:
  progressDeadlineSeconds: 600
  replicas: 2 - - - - - --------------------------------->Change Here
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: nginx
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        imagePullPolicy: Always
        name: nginx
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 30
status:
  availableReplicas: 2
  conditions:
  - lastTransitionTime: "2020-06-04T00:37:35Z"
    lastUpdateTime: "2020-06-04T00:37:50Z"
    message: ReplicaSet "nginx-f89759699" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  - lastTransitionTime: "2020-06-04T00:45:31Z"
    lastUpdateTime: "2020-06-04T00:45:31Z"
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
------------------------------------------------>
[root@centos7 ~]# kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
centos7      Ready    master   15m   v1.18.3
centos7-w1   Ready       12m   v1.18.3
centos7-w2   Ready       12m   v1.18.3
[root@centos7 ~]#

[root@centos7 ~]# kubectl get pods
NAME                    READY   STATUS    RESTARTS   AGE
nginx-f89759699-2st5d   1/1     Running   0          56s
nginx-f89759699-zlhhd   1/1     Running   0          8m39s
[root@centos7 ~]#

Testing the Setup:
-------------------

[root@centos7 ~]# curl centos7-w1:30668



Welcome to nginx!



Welcome to nginx!

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.

Thank you for using nginx.

[root@centos7 ~]# [root@centos7 ~]# curl centos7-w2:30668 Welcome to nginx!

Welcome to nginx!

If you see this page, the nginx web server is successfully installed and working. Further configuration is required.

For online documentation and support please refer to nginx.org.
Commercial support is available at nginx.com.

Thank you for using nginx.

[root@centos7 ~]# [root@centos7 ~]# kubectl get pods -o wide NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES nginx-f89759699-2st5d 1/1 Running 0 52m 10.244.1.2 centos7-w1 nginx-f89759699-zlhhd 1/1 Running 0 59m 10.244.2.2 centos7-w2 [root@centos7 ~]#