If you get a lot of these in alert log:
Time drift detected. Please check VKTM trace file for more details.
And <instance_name>_vktm_*.trc files are growing really fast in trace folder. You have probably hit
Bug 9843304 .
NOTE! You can get this lines in alert log sometimes if your servers clock are drifting (may happen for example in vmware). And that is normal. But if you get this all the time (several days) and a lot then it is probably because of this bug.
To fix this bug install patch 9843304 (You can get it from the MOS (My Oracle Support)).
And after that you can disable those debugging info that are added in logs.
Run following from the sqlplus (and restart database after that):
alter system set event="10795 trace name context forever, level 2" scope=spfile;
Read more from MOS document:
VKTM Trace Files Generated With KSTMCHKDRIFT "BACKWARD DRIFT ENDED AT" [ID 1185093.1]
Wednesday, October 31, 2012
Monday, October 29, 2012
Oracle 11.2.0.1 ORA-15061: ASM operation not supported [41]
You might get following error with 11gR1 or 11gR2 when you are trying to resize datafile:
Failed to commit: ORA-01237: cannot extend datafile 37 ORA-01110: data file 37: '+TESTDB_DATA0/testdb/datafile/testdata_01.dbf' ORA-17505: ksfdrsz:1 Failed to resize file to size 219136 blocks ORA-15061: ASM operation not supported [41]
a) Usually this is because you have separe homes for Oracle RDBMS and Grid Infra (clusterware (ASM uses this)). And you have different PSU upgrades installed in these Homes. You can check this with opatch. Run this in both homes and check that they got same PSU's installed:
opatch lsinventory -oh RDBMS_HOME
opatch lsinventory -oh GRID_HOME
More info about this can be seen in MOS document:
ORA-15061 reported while doing a file operation with 11.1 or 11.2 ASM after PSU applied in database home [ID 1070880.1]
b) But if there are same PSU installed in both Homes then problem is probably with relinking.
Patch (for example PSU) is not correctly relinked or there is OS update installed and it's have effected Grid Infra Home files so that they need to be relinked.
If you have Oracle RAC do following for all nodes one by one:
1) Stop first databases for the instance you are relinking then stop crs:
# GRID_HOME/bin/crsctl stop crs
2) Run following as root user:
# cd GRID_HOME/crs/install
# perl rootcrs.pl -unlock
3) Run following as the Oracle Grid Infrastructure for a Cluster owner:
$> export ORACLE_HOME=GRID_HOME
$> GRID_HOME/bin/relink
4) And run following as root user:
# cd GRID_HOME/rdbms/install/
# ./rootadd_rdbms.sh
# cd GRID_HOME/crs/install
# perl rootcrs.pl -patch
Then check that crs is up and running:# GRID_HOME/bin/crsctl check crs
After crs is started start databases and check that you can make resize operations again.
More info about this can be seen in MOS document:
ORA-15061: ASM Operation Not Supported [41] After Apply PSU #1 & PSU #2 [ID 1126113.1]
Failed to commit: ORA-01237: cannot extend datafile 37 ORA-01110: data file 37: '+TESTDB_DATA0/testdb/datafile/testdata_01.dbf' ORA-17505: ksfdrsz:1 Failed to resize file to size 219136 blocks ORA-15061: ASM operation not supported [41]
a) Usually this is because you have separe homes for Oracle RDBMS and Grid Infra (clusterware (ASM uses this)). And you have different PSU upgrades installed in these Homes. You can check this with opatch. Run this in both homes and check that they got same PSU's installed:
opatch lsinventory -oh RDBMS_HOME
opatch lsinventory -oh GRID_HOME
More info about this can be seen in MOS document:
ORA-15061 reported while doing a file operation with 11.1 or 11.2 ASM after PSU applied in database home [ID 1070880.1]
b) But if there are same PSU installed in both Homes then problem is probably with relinking.
Patch (for example PSU) is not correctly relinked or there is OS update installed and it's have effected Grid Infra Home files so that they need to be relinked.
If you have Oracle RAC do following for all nodes one by one:
1) Stop first databases for the instance you are relinking then stop crs:
# GRID_HOME/bin/crsctl stop crs
2) Run following as root user:
# cd GRID_HOME/crs/install
# perl rootcrs.pl -unlock
3) Run following as the Oracle Grid Infrastructure for a Cluster owner:
$> export ORACLE_HOME=GRID_HOME
$> GRID_HOME/bin/relink
4) And run following as root user:
# cd GRID_HOME/rdbms/install/
# ./rootadd_rdbms.sh
# cd GRID_HOME/crs/install
# perl rootcrs.pl -patch
Then check that crs is up and running:# GRID_HOME/bin/crsctl check crs
After crs is started start databases and check that you can make resize operations again.
More info about this can be seen in MOS document:
ORA-15061: ASM Operation Not Supported [41] After Apply PSU #1 & PSU #2 [ID 1126113.1]
Oracle 11.2.0.3 SCAN and VIP IPs are not reachable from other subnets.
There is Linux bug in Oracle 11.2.0.3 concerning SCAN and VIP IPs. When you reboot node from RAC or there is failover where these IP's are moving between nodes then the ip address is
not pingable from a different subnet. This seems to be only Linux problem. And everything is working normally in same cluster and same subnet servers. Problem is that ARP table (on router or on firewall) is getting wrong MAC addresses for these IPs.
When this problem is on and you try to connect this database from server which is in other subnet you get:
ERROR:
ORA-12170: TNS:Connect timeout occurred
As a workaround you can run following:
After vip failover, run command
/sbin/arping -U -c 3 -I <public NIC for vip> <vip ip address>
to update the ARP table of router.
or you can also clean problematic IPs from ARP table on your router/firewall
There is bug report and fix for this problem in MOS (My Oracle Support):
You can find the fix from MOS when you Search from the Patches & Updates with patch number: 13440962
When you are installing this patch you might get errors from file/directory permissions. I needed to give read/write permission for oracle user in following directories (and their files):
$GRID_HOME/lib
$GRID_HOME/jlib
$GRID_HOME/crs
$GRID_HOME/bin
And remember to check that under bin directory following right are set (this needs to be exactly like this. If these are not correct database won't start at all.):
chmod 6751 oracle
chmod 4750 jssu
and I also needed to create this directory (for oracle user):
$GRID_HOME/.patch_storage
This patch like every other patches should be tested first in test environment.
not pingable from a different subnet. This seems to be only Linux problem. And everything is working normally in same cluster and same subnet servers. Problem is that ARP table (on router or on firewall) is getting wrong MAC addresses for these IPs.
When this problem is on and you try to connect this database from server which is in other subnet you get:
ERROR:
ORA-12170: TNS:Connect timeout occurred
As a workaround you can run following:
After vip failover, run command
/sbin/arping -U -c 3 -I <public NIC for vip> <vip ip address>
to update the ARP table of router.
or you can also clean problematic IPs from ARP table on your router/firewall
There is bug report and fix for this problem in MOS (My Oracle Support):
Bug 13440962 - Different subnet failed to connect to vip after restart vip
[ID 13440962.8]
You can find the fix from MOS when you Search from the Patches & Updates with patch number: 13440962
When you are installing this patch you might get errors from file/directory permissions. I needed to give read/write permission for oracle user in following directories (and their files):
$GRID_HOME/lib
$GRID_HOME/jlib
$GRID_HOME/crs
$GRID_HOME/bin
And remember to check that under bin directory following right are set (this needs to be exactly like this. If these are not correct database won't start at all.):
chmod 6751 oracle
chmod 4750 jssu
and I also needed to create this directory (for oracle user):
$GRID_HOME/.patch_storage
This patch like every other patches should be tested first in test environment.
Sunday, October 14, 2012
Oracle 11g GridInfra ocr backup files cleaning/rotating problem
Sometimes Oracle automatic ocr backup cleaning is not working and because of this your $ORA_CRS_HOME/cdata/<cluster_name> directory is filled by .ocr backup files.
Cause of this problem is regular backup files ownership. Check following files:
---
-rw-r--r-- 1 oracle root 6557696 Jun 6 18:07 week.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 13 18:07 week_.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 16 18:07 day.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 17 18:07 day_.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 18 06:07 backup02.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 18 10:07 backup01.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 18 14:07 backup00.ocr
---
These files ownership should be root:root
After you change ownership the new backup files are rotating correctly.
And then you can safely delete older files (which are starting with generic numbers). But don't delete those regular backup files (week*, day*, backup*) they are used for rotating ocr backups.
There is also MOS document about this problem:
Cause of this problem is regular backup files ownership. Check following files:
---
-rw-r--r-- 1 oracle root 6557696 Jun 6 18:07 week.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 13 18:07 week_.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 16 18:07 day.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 17 18:07 day_.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 18 06:07 backup02.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 18 10:07 backup01.ocr
-rw-r--r-- 1 oracle root 6569984 Jun 18 14:07 backup00.ocr
---
These files ownership should be root:root
After you change ownership the new backup files are rotating correctly.
And then you can safely delete older files (which are starting with generic numbers). But don't delete those regular backup files (week*, day*, backup*) they are used for rotating ocr backups.
There is also MOS document about this problem:
OCR Backup Filling Up $ORA_CRS_HOME/CDATA/<CLUSTER_NAME> Directory [ID 741271.1]
Sunday, October 7, 2012
Oracle database 12c is coming
Several sites have said that new Oracle database 12c (c is for cloud) will be launch this years december or start of the next year. This info is based of Larry Ellison's conference call about financial results for the first quarter of fiscal 2013.
More info from these sites:
http://www.channelregister.co.uk/2012/09/20/oracle_openworld_preview_q1_f2012_numbers/
http://www.informationweek.com/software/information-management/oracle-12c-database-open-worlds-centerpi/240008073
This new database have cloud support (Oracle released it's own cloud in OracleWorld 2012). And it is supporting virtualization better than before. For example you can use one database for several customers/services data securely. So that data is isolated and private for each customer/service. It is interesting to see how this is affecting to licenses.
But we still have to wait for final release date and release notes.
More info from these sites:
http://www.channelregister.co.uk/2012/09/20/oracle_openworld_preview_q1_f2012_numbers/
http://www.informationweek.com/software/information-management/oracle-12c-database-open-worlds-centerpi/240008073
This new database have cloud support (Oracle released it's own cloud in OracleWorld 2012). And it is supporting virtualization better than before. For example you can use one database for several customers/services data securely. So that data is isolated and private for each customer/service. It is interesting to see how this is affecting to licenses.
But we still have to wait for final release date and release notes.
Subscribe to:
Posts (Atom)