Friday, September 20, 2013

Oracle Trace File Analyzer (TFA) - Diagnostic Tool

Usage:

Oracle Trace File Analyzer (TFA) is a diagnostic collection utility for Oracle Clusterware/Grid Infrastructure and RAC Systems.

Installation of TFA:

1. Log into the system as the ROOT user
2. Select the TFA location and this location must exist on all cluster nodes
3. Before installting TFA, you must install JRE 1.5 in the same location on all cluster nodes
4. Stage the "installTFALite.sh" on node-1 and it will install automatically on all cluster nodes.

On rac1 instance:

login as: root
root@192.168.56.21's password:

[root@rac1 opt]# cksum installTFALite.sh
715469312 7633428 installTFALite.sh

[root@rac1 opt]# md5sum installTFALite.sh
40101fda2c9df084f4d0451b8ef207a4  installTFALite.sh

[root@rac1 opt]# chmod 775 installTFALite.sh

[root@rac1 opt]# ls -lrth
-rwxrwxr-x. 1 root root 7.3M Sep 20 06:41 installTFALite.sh

[root@rac1 opt]# ./installTFALite.sh -tfabase /u01/app/oracle -javahome /u01/app/11.2.0/grid/jdk

Starting TFA installation

Using JAVA_HOME : /u01/app/11.2.0/grid/jdk

Running Auto Setup for TFA as user root...

Would you like to do a [L]ocal only or [C]lusterwide installation ? [L|l|C|c] [C] : C

The following installation requires temporary use of SSH.
If SSH is not configured already then we will remove SSH
when complete.
Do you wish to Continue ? [Y|y|N|n] [Y] Y

Installing TFA now...

Discovering Nodes and Oracle resources
Checking whether CRS is up and running
Getting list of nodes in cluster . . . . .

List of nodes in cluster
1. rac1
2. rac2

Checking ssh user equivalency settings on all nodes in cluster

Node rac2 is not configured for ssh user equivalency and  the script uses ssh to install TFA on remote nodes.

Without this facility the script cannot install TFA on the remote nodes.

Do you want to configure SSH for user root on rac2 [y/n][y]y
PING rac2.mlg.oracle.com (192.168.56.22) 56(84) bytes of data.
64 bytes from rac2.mlg.oracle.com (192.168.56.22): icmp_seq=1 ttl=64 time=0.349 ms

--- rac2.mlg.oracle.com ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.349/0.349/0.349/0.000 ms

Generating public/private dsa key pair.
Your identification has been saved in /root/.ssh/id_dsa.
Your public key has been saved in /root/.ssh/id_dsa.pub.

The key fingerprint is:
70:82:17:f7:34:75:f1:29:63:80:3e:52:78:db:78:bb root@rac1.mlg.oracle.com

Warning: Permanently added 'rac2,192.168.56.22' (RSA) to the list of known hosts.
root@rac2's password:
root@rac2's password:
SSH setup is complete.

Fri Sep 20 07:07:47 CEST 2013
------------------------------------------------------------------------
SSH verification complete.
Searching for running databases . . . . .
List of running databases registered in OCR
1. flavia
. .
Checking Status of Oracle Software Stack - Clusterware, ASM, RDBMS
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

TFA Will be Installed on the Following Nodes:
++++++++++++++++++++++++++++++++++++++++++++

Install Nodes
=============
rac1
rac2

Do you wish to make changes to the Node List ? [Y/y/N/n] [N] N

TFA will scan the following Directories
++++++++++++++++++++++++++++++++++++++++++++

.------------------------------------------------------------------------------.
|                                     rac1                                                                            |
+-------------------------------------------------------------------+----------+
| Trace Directory                                                                               | Resource |
+-------------------------------------------------------------------+----------+
| /u01/app/11.2.0/grid/cfgtoollogs                                                      | INSTALL  |
| /u01/app/11.2.0/grid/crs/log                                      | CRS      |
| /u01/app/11.2.0/grid/css/log                                      | CRS      |
| /u01/app/11.2.0/grid/cv/log                                       | CRS      |
| /u01/app/11.2.0/grid/evm/admin/log                                | CRS      |
| /u01/app/11.2.0/grid/evm/admin/logger                             | CRS      |
| /u01/app/11.2.0/grid/evm/log                                      | CRS      |
| /u01/app/11.2.0/grid/install                                      | INSTALL  |
| /u01/app/11.2.0/grid/log/                                         | CRS      |
| /u01/app/11.2.0/grid/network/log                                  | CRS      |
| /u01/app/11.2.0/grid/oc4j/j2ee/home/log                           | CRSOC4J  |
| /u01/app/11.2.0/grid/opmn/logs                                    | CRS      |
| /u01/app/11.2.0/grid/racg/log                                     | CRS      |
| /u01/app/11.2.0/grid/rdbms/log                                    | ASM      |
| /u01/app/11.2.0/grid/scheduler/log                                | CRS      |
| /u01/app/11.2.0/grid/srvm/log                                     | CRS      |
| /u01/app/oraInventory/ContentsXML                                 | INSTALL  |
| /u01/app/oraInventory/logs                                        | INSTALL  |
| /u01/app/oracle/cfgtoollogs                                       | CFGTOOLS |
| /u01/app/oracle/diag/asm/+asm/+ASM1/trace                         | ASM      |
| /u01/app/oracle/diag/clients/user_oracle/host_3590113974_80/trace | RDBMS    |
| /u01/app/oracle/diag/rdbms/flavia/flavia1/trace                   | RDBMS    |
| /u01/app/oracle/diag/tnslsnr                                      | TNS      |
| /u01/app/oracle/diag/tnslsnr/rac1/listener/trace                  | TNS      |
| /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs               | INSTALL  |
| /u01/app/oracle/product/11.2.0/dbhome_1/install                   | INSTALL  |
'-------------------------------------------------------------------+----------'

.------------------------------------------------------------------------------.
|                                     rac2                                                                            |
+-------------------------------------------------------------------+----------+
| Trace Directory                                                                              | Resource |
+-------------------------------------------------------------------+----------+
| /u01/app/11.2.0/grid/cfgtoollogs                                  | INSTALL  |
| /u01/app/11.2.0/grid/crs/log                                      | CRS      |
| /u01/app/11.2.0/grid/css/log                                      | CRS      |
| /u01/app/11.2.0/grid/cv/log                                       | CRS      |
| /u01/app/11.2.0/grid/evm/admin/log                                | CRS      |
| /u01/app/11.2.0/grid/evm/admin/logger                             | CRS      |
| /u01/app/11.2.0/grid/evm/log                                      | CRS      |
| /u01/app/11.2.0/grid/install                                      | INSTALL  |
| /u01/app/11.2.0/grid/log/                                         | CRS      |
| /u01/app/11.2.0/grid/network/log                                  | CRS      |
| /u01/app/11.2.0/grid/oc4j/j2ee/home/log                           | CRSOC4J  |
| /u01/app/11.2.0/grid/opmn/logs                                    | CRS      |
| /u01/app/11.2.0/grid/racg/log                                     | CRS      |
| /u01/app/11.2.0/grid/rdbms/log                                    | ASM      |
| /u01/app/11.2.0/grid/scheduler/log                                | CRS      |
| /u01/app/11.2.0/grid/srvm/log                                     | CRS      |
| /u01/app/oraInventory/ContentsXML                                 | INSTALL  |
| /u01/app/oraInventory/logs                                        | INSTALL  |
| /u01/app/oracle/cfgtoollogs                                       | CFGTOOLS |
| /u01/app/oracle/diag/asm/+asm/+ASM2/trace                         | ASM      |
| /u01/app/oracle/diag/clients/user_oracle/host_3945948993_80/trace | RDBMS    |
| /u01/app/oracle/diag/rdbms/flavia/flavia2/trace                   | RDBMS    |
| /u01/app/oracle/diag/tnslsnr                                      | TNS      |
| /u01/app/oracle/diag/tnslsnr/rac2/listener/trace                  | TNS      |
| /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs               | INSTALL  |
| /u01/app/oracle/product/11.2.0/dbhome_1/install                   | INSTALL  |
'-------------------------------------------------------------------+----------'

Installing TFA on rac1

HOST: rac1     
TFA_HOME: /u01/app/oracle/tfa/rac1/tfa_home

Installing TFA on rac2

HOST: rac2     
TFA_HOME: /u01/app/oracle/tfa/rac2/tfa_home

Host rac2 is part of TFA cluster

.----------------------------------------------.
| Host | Status of TFA | PID  | Port         | Version |
+------+---------------+------+------+---------+
| rac1 | RUNNING       | 6056 | 5000 | 2.5.1.5 |
| rac2 | RUNNING       | 5318 | 5000 | 2.5.1.5 |
'------+---------------+------+------+---------'

Summary of TFA Installation:
.---------------------------------------------------------.
|                           rac1                                                       |
+---------------------+-----------------------------------+
| Parameter           | Value                                                   |
+---------------------+-----------------------------------+
| Install location         | /u01/app/oracle/tfa/rac1/tfa_home |
| Repository location | /u01/app/oracle/tfa/repository    |
| Repository usage     | 0 MB out of 6659 MB               |
'---------------------+-----------------------------------'

.---------------------------------------------------------.
|                           rac2                                                      |
+---------------------+-----------------------------------+
| Parameter                                 | Value                             |
+---------------------+-----------------------------------+
| Install location            | /u01/app/oracle/tfa/rac2/tfa_home |
| Repository location       | /u01/app/oracle/tfa/repository    |
| Repository usage         | 0 MB out of 6659 MB               |
'---------------------+-----------------------------------'

Removed ssh equivalency setup on rac2

TFA is successfully installed..

Usage : /u01/app/oracle/tfa/bin/tfactl <command> [options]
<command> =
         print        Print requested details
         purge        Delete collections from TFA repository
         directory    Add or Remove or Modify directory in TFA
         host         Add or Remove host in TFA
         set          Turn ON/OFF or Modify various TFA features
         diagcollect  Collect logs from across nodes in cluster

For help with a command: /u01/app/oracle/tfa/bin/tfactl <command> -help

To start TFA:

[root@rac1 opt]# /etc/init.d/init.tfa start

To stop TFA:

[root@rac1 opt]# /etc/init.d/init.tfa stop

To restart TFA:

[root@rac1 opt]# /etc/init.d/init.tfa restart

To Check PID of TFA:

[root@rac1 opt]# ps -ef | grep tfa

Manual Collection of Diagnostics on rac1:

[root@rac1 bin]# ./tfactl diagcollect -all

Collecting data for the last 4 hours for this component...
Running an inventory clusterwide ...
Run inventory completed locally ...
Collection name tfa_Fri_Sep_20_07_21_28_CEST_2013.zip
Sending diagcollect request to host : rac2
Getting list of files satisfying time range [Fri Sep 20 03:25:24 CEST 2013, Fri Sep 20 07:25:24 CEST 2013]
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_ping_3571.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_lms0_3581.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/crfmond/crfmond.log
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/mdnsd/mdnsd.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/mdnsd/mdnsd.log with original file size : 647kB
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_rbal_2745.trc
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_ora_2610.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/client/ocrconfig_7918.log
rac1: Zipping File: /u01/app/11.2.0/grid/opmn/logs/ons.log.rac1
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_dia0_2723.trc
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_lmd0_2727.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_lmon_3577.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_lgwr_3593.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/gipcd/gipcd.log
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_mmon_3605.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/agent/ohasd/oraagent_oracle/oraagent_oracle.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/agent/ohasd/oraagent_oracle/oraagent_oracle.log with original file size : 9.8MB
rac1: Zipping File: /u01/app/11.2.0/grid/log/diag/tnslsnr/rac1/listener_scan3/trace/listener_scan3.log
Trimming file : /u01/app/11.2.0/grid/log/diag/tnslsnr/rac1/listener_scan3/trace/listener_scan3.log with original file size : 2.7MB
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/alert_flavia1.log
Trimming file : /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/alert_flavia1.log with original file size : 511kB
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_gcr0_3150.trc
rac1: Zipping File: /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch2013-09-20_07-20-13AM.log
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log
Trimming file : /u01/app/oracle/diag/asm/+asm/+ASM1/trace/alert_+ASM1.log with original file size : 561kB
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/crflogd/crflogd.log
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/cssd/ocssd.log with original file size : 16MB
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_lms0_2729.trc
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_lmhb_2733.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/agent/ohasd/oracssdagent_root/oracssdagent_root.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/agent/ohasd/oracssdagent_root/oracssdagent_root.log with original file size : 1MB
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_ora_2760.trc
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_lmon_2725.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/diag/tnslsnr/rac1/listener_scan2/trace/listener_scan2.log
Trimming file : /u01/app/11.2.0/grid/log/diag/tnslsnr/rac1/listener_scan2/trace/listener_scan2.log with original file size : 2.8MB
rac1: Zipping File: /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/lsinv/lsinventory2013-09-20_07-20-13AM.txt
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_dia0_3575.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_rms0_3585.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/evmd/evmd.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/evmd/evmd.log with original file size : 4.4MB
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_diag_3567.trc
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_diag_2719.trc
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_lck0_2753.trc
rac1: Zipping File: /u01/app/11.2.0/grid/evm/log/rac1_evmlogger.info
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/crsd/crsd.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/crsd/crsd.log with original file size : 7.6MB
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/gpnpd/gpnpdOUT.log
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/alertrac1.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/alertrac1.log with original file size : 336kB
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/mdnsd/mdnsdOUT.log
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_gcr0_3750.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/evmd/evmdOUT.log
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_ora_3446.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/client/ocrdump_8014.log
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/client/ocrcheck_7921.log
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_ckpt_2741.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/agent/crsd/scriptagent_oracle/scriptagent_oracleOUT.log
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_qmnc_3748.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_ora_3648.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/gipcd/gipcdOUT.log
rac1: Zipping File: /u01/app/11.2.0/grid/evm/log/rac1_evmdaemon.log
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_asmb_3603.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_mman_3589.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/client/ocrdump_8019.log
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/agent/ohasd/oracssdmonitor_root/oracssdmonitor_root.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/agent/ohasd/oracssdmonitor_root/oracssdmonitor_root.log with original file size : 855kB
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_dbrm_3569.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_vktm_3561.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/agent/crsd/oraagent_oracle/oraagent_oracle.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/agent/crsd/oraagent_oracle/oraagent_oracle.log with original file size : 5.2MB
rac1: Zipping File: /u01/app/11.2.0/grid/rdbms/log/+asm1_ora_2610.trc
rac1: Zipping File: /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch_history.txt
rac1: Zipping File: /u01/app/11.2.0/grid/cfgtoollogs/opatch/lsinv/lsinventory2013-09-20_07-19-59AM.txt
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_ckpt_3595.trc
rac1: Zipping File: /var/log/messages
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/client/olsnodes.log
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/agent/crsd/orarootagent_root/orarootagent_root.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/agent/crsd/orarootagent_root/orarootagent_root.log with original file size : 5.6MB
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/ohasd/ohasd.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/ohasd/ohasd.log with original file size : 1.6MB
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/gipcd/gipcd.l01
Trimming file : /u01/app/11.2.0/grid/log/rac1/gipcd/gipcd.l01 with original file size : 10MB
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/agent/crsd/scriptagent_oracle/scriptagent_oracle.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/agent/crsd/scriptagent_oracle/scriptagent_oracle.log with original file size : 5MB
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/crsd/crsdOUT.log
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_ping_2721.trc
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_gmon_2747.trc
rac1: Zipping File: /u01/app/oracle/diag/tnslsnr/rac1/listener/trace/listener.log
Trimming file : /u01/app/oracle/diag/tnslsnr/rac1/listener/trace/listener.log with original file size : 5MB
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/ctssd/octssd.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/ctssd/octssd.log with original file size : 10MB
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/client/crswrapexece.log
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/cssd/cssdOUT.log
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_dbw0_3591.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_mark_3615.trc
rac1: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM1/trace/+ASM1_vktm_2713.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/agent/ohasd/orarootagent_root/orarootagent_root.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/agent/ohasd/orarootagent_root/orarootagent_root.log with original file size : 5MB
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/client/crsctl_root.log
rac1: Zipping File: /u01/app/oracle/product/11.2.0/dbhome_1/cfgtoollogs/opatch/opatch_history.txt
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_lck0_3624.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/gpnpd/gpnpd.log
Trimming file : /u01/app/11.2.0/grid/log/rac1/gpnpd/gpnpd.log with original file size : 378kB
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_rcbg_3746.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_lmhb_3587.trc
rac1: Zipping File: /u01/app/11.2.0/grid/log/rac1/ctssd/ctssdOUT.log
rac1: Zipping File: /u01/app/11.2.0/grid/cfgtoollogs/opatch/opatch2013-09-20_07-19-59AM.log
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_smon_3597.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_rsmn_3626.trc
rac1: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia1/trace/flavia1_lmd0_3579.trc
Collecting extra files...
Total Number of Files checked : 3275
Total Size of all Files Checked : 820MB
Number of files containing required range : 90
Total Size of Files containing required range : 102MB
Number of files trimmed : 21
Total Size of data prior to zip : 17MB
Saved 93MB by trimming files
Zip file size : 954kB
Total time taken : 174s
Completed collection of zip files.

Logs are collected to:
/u01/app/oracle/tfa/repository/collection_Fri_Sep_20_07_21_28_CEST_2013_node_all/rac1.tfa_Fri_Sep_20_07_21_28_CEST_2013.zip
/u01/app/oracle/tfa/repository/collection_Fri_Sep_20_07_21_28_CEST_2013_node_all/rac2.tfa_Fri_Sep_20_07_21_28_CEST_2013.zip

To print TFA Configuration:

[root@rac1 bin]# ./tfactl print config
.---------------------------------------------------.
| Configuration Parameter                 | Value   |
+-----------------------------------------+---------+
| TFA version                             | 2.5.1.5 |
| Automatic diagnostic collection         | OFF     |
| Trimming of files during diagcollection | ON      |
| Repository current size (MB) in rac1    | 2       |
| Repository maximum size (MB) in rac1    | 6659    |
| Trace level                             | 1       |
'-----------------------------------------+---------'


Check on rac2:

[root@rac2 grid]# ps -ef | grep tfa
root      5262     1  0 07:09 ?        00:00:00 /bin/sh /etc/init.d/init.tfa run
root      5318     1  7 07:09 ?        00:00:09 /u01/app/11.2.0/grid/jdk/jre/bin                                                                                       
Manual Collection of Diagnostics on rac2:

[root@rac2 grid]# cd /u01/app/oracle/tfa/bin/

[root@rac2 bin]# ./tfactl diagcollect -all

Collecting data for the last 4 hours for this component...
Waiting for inventory to complete ...
Collection name tfa_Fri_Sep_20_07_16_02_CEST_2013.zip
Sending diagcollect request to host : rac1
Getting list of files satisfying time range [Fri Sep 20 03:19:08 CEST 2013, Fri Sep 20 07:19:08 CEST 2013]
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_mmon_3672.trc
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_ckpt_3662.trc
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/alert_flavia2.log
Trimming file : /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/alert_flavia2.log with original file size : 446kB
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_lmd0_2735.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/diag/tnslsnr/rac2/listener_scan1/trace/listener_scan1.log
Trimming file : /u01/app/11.2.0/grid/log/diag/tnslsnr/rac2/listener_scan1/trace/listener_scan1.log with original file size : 2.6MB
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_diag_2727.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/client/olsnodes.log
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_ora_3699.trc
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_ora_2769.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/agent/crsd/orarootagent_root/orarootagent_root.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/agent/crsd/orarootagent_root/orarootagent_root.log with original file size : 5.7MB
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_mark_3682.trc
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_lmhb_3654.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/cssd/ocssd.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/cssd/ocssd.log with original file size : 18MB
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_ora_2620.trc
rac2: Zipping File: /u01/app/oracle/diag/tnslsnr/rac2/listener/trace/listener.log
Trimming file : /u01/app/oracle/diag/tnslsnr/rac2/listener/trace/listener.log with original file size : 5.5MB
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_lms0_3648.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/crsd/crsdOUT.log
rac2: Zipping File: /u01/app/11.2.0/grid/rdbms/log/+asm2_ora_2620.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/client/crsctl_root.log
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/agent/crsd/oraagent_oracle/oraagent_oracle.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/agent/crsd/oraagent_oracle/oraagent_oracle.log with original file size : 4.2MB
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_dia0_2731.trc
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_rcbg_3814.trc
rac2: Zipping File: /u01/app/11.2.0/grid/evm/log/rac2_evmlogger.info
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_diag_3634.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/evmd/evmd.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/evmd/evmd.log with original file size : 3.9MB
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_asmb_3670.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/agent/ohasd/orarootagent_root/orarootagent_root.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/agent/ohasd/orarootagent_root/orarootagent_root.log with original file size : 4MB
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_ckpt_2749.trc
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/alert_+ASM2.log
Trimming file : /u01/app/oracle/diag/asm/+asm/+ASM2/trace/alert_+ASM2.log with original file size : 602kB
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/ctssd/ctssdOUT.log
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/crfmond/crfmond.log
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/client/crswrapexece.log
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_mman_3656.trc
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_ping_2729.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/crflogd/crflogd.log
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/gipcd/gipcd.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/gipcd/gipcd.log with original file size : 8MB
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_lmon_3644.trc
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_lck0_2763.trc
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_dbw0_3658.trc
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_lmd0_3646.trc
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_dia0_3642.trc
rac2: Zipping File: /var/log/messages
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_dbrm_3636.trc
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_rbal_2753.trc
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_gcr0_3807.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/agent/ohasd/oracssdagent_root/oracssdagent_root.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/agent/ohasd/oracssdagent_root/oracssdagent_root.log with original file size : 1.2MB
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_ping_3638.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/mdnsd/mdnsdOUT.log
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_vktm_2721.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/client/crsctl_oracle.log
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/alertrac2.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/alertrac2.log with original file size : 268kB
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/cssd/cssdOUT.log
rac2: Zipping File: /u01/app/11.2.0/grid/evm/log/rac2_evmdaemon.log
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_lck0_3688.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/agent/ohasd/oraagent_oracle/oraagent_oracle.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/agent/ohasd/oraagent_oracle/oraagent_oracle.log with original file size : 8.7MB
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/ohasd/ohasd.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/ohasd/ohasd.log with original file size : 10MB
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/agent/crsd/scriptagent_oracle/scriptagent_oracle.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/agent/crsd/scriptagent_oracle/scriptagent_oracle.log with original file size : 2.5MB
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/agent/crsd/scriptagent_oracle/scriptagent_oracleOUT.log
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_gcr0_2761.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/agent/ohasd/oracssdmonitor_root/oracssdmonitor_root.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/agent/ohasd/oracssdmonitor_root/oracssdmonitor_root.log with original file size : 1MB
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_ora_3500.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/gpnpd/gpnpd.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/gpnpd/gpnpd.log with original file size : 416kB
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_gcr0_3363.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/gpnpd/gpnpdOUT.log
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_rsmn_3691.trc
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_gmon_2755.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/ctssd/octssd.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/ctssd/octssd.log with original file size : 8.3MB
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_lmon_2733.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/mdnsd/mdnsd.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/mdnsd/mdnsd.log with original file size : 598kB
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_lms0_2737.trc
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_qmnc_3816.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/gipcd/gipcdOUT.log
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/crsd/crsd.log
Trimming file : /u01/app/11.2.0/grid/log/rac2/crsd/crsd.log with original file size : 7.1MB
rac2: Zipping File: /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_lmhb_2741.trc
Trimming file : /u01/app/oracle/diag/asm/+asm/+ASM2/trace/+ASM2_lmhb_2741.trc with original file size : 617kB
rac2: Zipping File: /u01/app/oracle/diag/rdbms/flavia/flavia2/trace/flavia2_vktm_3628.trc
rac2: Zipping File: /u01/app/11.2.0/grid/log/rac2/evmd/evmdOUT.log
rac2: Zipping File: /u01/app/11.2.0/grid/opmn/logs/ons.log.rac2
Collecting extra files...
Total Number of Files checked : 3218
Total Size of all Files Checked : 712MB
Number of files containing required range : 77
Total Size of Files containing required range : 99MB
Number of files trimmed : 21
Total Size of data prior to zip : 16MB
Saved 91MB by trimming files
Zip file size : 990kB
Total time taken : 88s
Completed collection of zip files.

Logs are collected to:
/u01/app/oracle/tfa/repository/collection_Fri_Sep_20_07_16_02_CEST_2013_node_all/rac2.tfa_Fri_Sep_20_07_16_02_CEST_2013.zip
/u01/app/oracle/tfa/repository/collection_Fri_Sep_20_07_16_02_CEST_2013_node_all/rac1.tfa_Fri_Sep_20_07_16_02_CEST_2013.zip

To print TFA Configuration:

[root@rac2 bin]# ./tfactl print config
.---------------------------------------------------.
| Configuration Parameter                 | Value   |
+-----------------------------------------+---------+
| TFA version                             | 2.5.1.5 |
| Automatic diagnostic collection         | OFF     |
| Trimming of files during diagcollection | ON      |
| Repository current size (MB) in rac2    | 1       |
| Repository maximum size (MB) in rac2    | 6890    |
| Trace level                             | 1       |
'-----------------------------------------+---------'

To Purge TFA Data:

[root@rac1 bin]# ./tfactl purge -h

To Modify TFA Parameters:

[root@rac1 bin]# ./tfactl set -h

To  Add/Remove/Change directories of TFA:

[root@rac1 bin]# ./tfactl directory -h

TFA Commands at a Glance:

[root@rac1 bin]# ./tfactl -h



1 comment:

  1. how to add a node to tfa, am getting the below error when trying to run it from the 3rd node

    User oracle does not have keys to run TFA. Please check with TFA Admin(root)

    ReplyDelete