ORA-27154: post/wait create failed(CRS-4535 and related to ASM instance) on oracle RAC 11g

ORA-27154: post/wait create failed(CRS-4535 and related to ASM instance)

Workaround
==========

 # crs_stat -t  
 CRS-0184: Cannot communicate with the CRS daemon.  
 # crsctl check crs  
 CRS-4638: Oracle High Availability Services is online  
 CRS-4535: Cannot communicate with Cluster Ready Services  
 CRS-4529: Cluster Synchronization Services is online  
 CRS-4533: Event Manager is online 

 after seeing the grid log and CRSD log, 

 i found that +ASM2 instance is not starting up after reboot. 

 so i tried it manually but not started.  

  $ export ORACLE_SID=+ASM2  
 $ amdcmd  
 ASMCMD> ls  
 ASMCMD-08102: no connection to ASM; command requires ASM to run"  

  Whenever it is tried to start ASM instance manually on the second node 

  below error is shown.  
 

amdcmd> startup;  
 ORA-27154: post/wait create failed  
 ORA-27300: OS system dependent operation:semget failed with status: 28  
 ORA-27301: OS failure message: No space left on device  
 ORA-27302: failure occurred at: sskgpsemsper  

Solution:-

Problem is lower settings of semaphores(/etc/sysctl.conf) value so solution is to increase the semaphores value.

like below......

# vi /etc/sysctl.conf
kernel.sem = 250 32000 100 288

On Red Hat Linux system, in order the affect the setting of the value immediately use,

# /sbin/sysctl -p