503 Service Unavailable : Oracle Grid Control

This happened recently when I discovered that Grid Control is not working and the error displayed was:

503 service unavailable servlet error service is not initialized correctly.

I bounced all services from services.msc, our server is on windows .

However the error was still there. I searched on metalink and found note ID 418159.1

On reading the note and diagnosing i found a file emoms.properties was nullified, this file is used by oms when services start.

As per the document this happens when disk space gets full and server is restarted.

So i followed the document and created a new file as no backup was available.

After creating file, Services started nicely...:-)

High ENQ:SQ contention in database

This Happened recently when all of a sudden Database stopped responding.

Only option left was to bounce the DB. Since it is a RAC setup, we shutdown one of the instance first and fortunately after starting things came to normal.

Later on when i analyzed the AWR of Database, first instance i found ENQ:SQ contention in top events.

This was new for me, then i checked TOP SQL section of AWR.

Here all top queries were accessing a particular Sequence , when i checked this sequence it was newly created and the cache size was very less 10.

So as such i increased cache size to 100 initially, and till now things are working fine.

I think it was because of this particular sequence , and this should not occur again.


Will try to dig out more..about it...

ORA-00600 error while querying a table

This happened recently in one of the setups. When user was querying a table for some data he was getting ORA-00600 [4000] ...We got this issue for resolution. Well this was the first time i came across an issue like this related to table corruption, so we to diagnose furthur we did a count(*) from table first.

1. Select count(*) from table;

ORA-00600: internal error code, arguments:[4000]...

Now we did and export of the table and it went fine

2. In this step we did VALIDATE STRUCTURE with CASCADE option.

SQL> Analyze table table name validate structure cascade;

ORA-00600: internal error code, arguments:[4000]...

 This again failed with same error

3. Now just to check whether its an index corruption or table corruption we again ran VALIDATE STRUCTURE but without cascade and it ran fine without any errors.

SQL> Analyze table table name validate structure ;

Table Analyzed.

4. From this we concluded that table was intact but corruption is with indexes, Further diagnosis revealed that  there was one index that too primary key

5. an online rebuild of the index was done

SQL > alter index index name rebuild online parallel 10;

This solved the problem and the error was gone