rac1节点不能启动CRS

今天重启RAC主机后发现RAC1节点的CRS服务不能开启,RAC2正常 [grid@rac1 ~]$ crsctl check crs CRS-4638: Oracle High Availability Services is online CRS-4535: Cannot communicate with Cluster Ready Services CRS-4529: Cluster Synchronization Services is online CRS-4533: Event Manager is online [grid@rac1 ~]$ crs_stat -t CRS-0184: Cannot communicate with the CRS daemon. 检查OCR时发现AMS的共享磁盘有问题 [grid@rac1 ~]$ ocrcheck PROT-602: Failed to retrieve data from the cluster registry PROC-26: Error while accessing the physical storage ASM error [SLOS: cat=8, pn=kgfolclcpi1, dep=204, loc=kgfokge AMDU-00204: Disk N0001 is in currently mounted diskgroup DATA AMDU-00201: Disk N0001: 'ORCL:ASM_DISK1'
[grid@rac1 ~]$ crsctl query css votedisk ##  STATE    File Universal Id                File Name Disk group --  -----    -----------------                --------- --------- 1. ONLINE   8ce3f683f9394fe3bf721506fbc3814c (ORCL:ASM_DISK1) [DATA] Located 1 voting disk(s).

[grid@rac1 ~]$ cat /etc/oracle/ocr.loc ocrconfig_loc=+DATA local_only=FALSE
检查ASM的共享磁盘权限,有读写权限 [root@rac1 disks]# ll total 0 brw-rw---- 1 grid oinstall 8, 17 Aug  7 09:34 ASM_DISK1 brw-rw---- 1 grid oinstall 8, 33 Aug  7 09:34 ASM_DISK2 brw-rw---- 1 grid oinstall 8, 49 Aug  7 09:34 ASM_DISK3
[grid@rac1 disks]$ kfed read ASM_DISK1 kfbh.endian:                          1 ; 0x000: 0x01 kfbh.hard:                          130 ; 0x001: 0x82 kfbh.type:                            1 ; 0x002: KFBTYP_DISKHEAD kfbh.datfmt:                          1 ; 0x003: 0x01 kfbh.block.blk:                       0 ; 0x004: T=0 NUMB=0x0 kfbh.block.obj:              2147483648 ; 0x008: TYPE=0x8 NUMB=0x0 kfbh.check:                  1625325518 ; 0x00c: 0x60e07fce kfbh.fcn.base:                        0 ; 0x010: 0x00000000 kfbh.fcn.wrap:                        0 ; 0x014: 0x00000000 kfbh.spare1:                          0 ; 0x018: 0x00000000 kfbh.spare2:                          0 ; 0x01c: 0x00000000 kfdhdb.driver.provstr:ORCLDISKASM_DISK1 ; 0x000: length=17 kfdhdb.driver.reserved[0]:   1598903105 ; 0x008: 0x5f4d5341 kfdhdb.driver.reserved[1]:   1263749444 ; 0x00c: 0x4b534944 kfdhdb.driver.reserved[2]:           49 ; 0x010: 0x00000031 kfdhdb.driver.reserved[3]:            0 ; 0x014: 0x00000000 kfdhdb.driver.reserved[4]:            0 ; 0x018: 0x00000000 kfdhdb.driver.reserved[5]:            0 ; 0x01c: 0x00000000 kfdhdb.compat:                186646528 ; 0x020: 0x0b200000 kfdhdb.dsknum:                        0 ; 0x024: 0x0000 kfdhdb.grptyp:                        1 ; 0x026: KFDGTP_EXTERNAL kfdhdb.hdrsts:                        3 ; 0x027: KFDHDR_MEMBER kfdhdb.dskname:               ASM_DISK1 ; 0x028: length=9 kfdhdb.grpname:                    DATA ; 0x048: length=4 kfdhdb.fgname:                ASM_DISK1 ; 0x068: length=9 kfdhdb.capname:                         ; 0x088: length=0 kfdhdb.crestmp.hi:             32989035 ; 0x0a8: HOUR=0xb DAYS=0x1b MNTH=0x7 YEAR=0x7dd kfdhdb.crestmp.lo:           1710834688 ; 0x0ac: USEC=0x0 MSEC=0x251 SECS=0x1f MINS=0x19 kfdhdb.mntstmp.hi:             32989417 ; 0x0b0: HOUR=0x9 DAYS=0x7 MNTH=0x8 YEAR=0x7dd kfdhdb.mntstmp.lo:           2690942976 ; 0x0b4: USEC=0x0 MSEC=0x122 SECS=0x6 MINS=0x28 kfdhdb.secsize:                     512 ; 0x0b8: 0x0200 kfdhdb.blksize:                    4096 ; 0x0ba: 0x1000 kfdhdb.ausize:                  1048576 ; 0x0bc: 0x00100000 kfdhdb.mfact:                    113792 ; 0x0c0: 0x0001bc80 kfdhdb.dsksize:                    4094 ; 0x0c4: 0x00000ffe kfdhdb.pmcnt:                         2 ; 0x0c8: 0x00000002 kfdhdb.fstlocn:                       1 ; 0x0cc: 0x00000001 kfdhdb.altlocn:                       2 ; 0x0d0: 0x00000002 kfdhdb.f1b1locn:                      2 ; 0x0d4: 0x00000002 kfdhdb.redomirrors[0]:                0 ; 0x0d8: 0x0000 kfdhdb.redomirrors[1]:                0 ; 0x0da: 0x0000 kfdhdb.redomirrors[2]:                0 ; 0x0dc: 0x0000 kfdhdb.redomirrors[3]:                0 ; 0x0de: 0x0000 kfdhdb.dbcompat:              168820736 ; 0x0e0: 0x0a100000 kfdhdb.grpstmp.hi:             32989035 ; 0x0e4: HOUR=0xb DAYS=0x1b MNTH=0x7 YEAR=0x7dd kfdhdb.grpstmp.lo:           1710333952 ; 0x0e8: USEC=0x0 MSEC=0x68 SECS=0x1f MINS=0x19 kfdhdb.vfstart:                     128 ; 0x0ec: 0x00000080 kfdhdb.vfend:                       160 ; 0x0f0: 0x000000a0 kfdhdb.spfile:                        0 ; 0x0f4: 0x00000000 kfdhdb.spfflg:                        0 ; 0x0f8: 0x00000000 kfdhdb.ub4spare[0]:                   0 ; 0x0fc: 0x00000000 kfdhdb.ub4spare[1]:                   0 ; 0x100: 0x00000000 kfdhdb.ub4spare[2]:                   0 ; 0x104: 0x00000000 kfdhdb.ub4spare[3]:                   0 ; 0x108: 0x00000000 kfdhdb.ub4spare[4]:                   0 ; 0x10c: 0x00000000 kfdhdb.ub4spare[5]:                   0 ; 0x110: 0x00000000 kfdhdb.ub4spare[6]:                   0 ; 0x114: 0x00000000 kfdhdb.ub4spare[7]:                   0 ; 0x118: 0x00000000 kfdhdb.ub4spare[8]:                   0 ; 0x11c: 0x00000000 kfdhdb.ub4spare[9]:                   0 ; 0x120: 0x00000000 kfdhdb.ub4spare[10]:                  0 ; 0x124: 0x00000000 kfdhdb.ub4spare[11]:                  0 ; 0x128: 0x00000000 kfdhdb.ub4spare[12]:                  0 ; 0x12c: 0x00000000 kfdhdb.ub4spare[13]:                  0 ; 0x130: 0x00000000 kfdhdb.ub4spare[14]:                  0 ; 0x134: 0x00000000 kfdhdb.ub4spare[15]:                  0 ; 0x138: 0x00000000 kfdhdb.ub4spare[16]:                  0 ; 0x13c: 0x00000000 kfdhdb.ub4spare[17]:                  0 ; 0x140: 0x00000000 kfdhdb.ub4spare[18]:                  0 ; 0x144: 0x00000000 kfdhdb.ub4spare[19]:                  0 ; 0x148: 0x00000000 kfdhdb.ub4spare[20]:                  0 ; 0x14c: 0x00000000 kfdhdb.ub4spare[21]:                  0 ; 0x150: 0x00000000 kfdhdb.ub4spare[22]:                  0 ; 0x154: 0x00000000 kfdhdb.ub4spare[23]:                  0 ; 0x158: 0x00000000 kfdhdb.ub4spare[24]:                  0 ; 0x15c: 0x00000000 kfdhdb.ub4spare[25]:                  0 ; 0x160: 0x00000000 kfdhdb.ub4spare[26]:                  0 ; 0x164: 0x00000000 kfdhdb.ub4spare[27]:                  0 ; 0x168: 0x00000000 kfdhdb.ub4spare[28]:                  0 ; 0x16c: 0x00000000 kfdhdb.ub4spare[29]:                  0 ; 0x170: 0x00000000 kfdhdb.ub4spare[30]:                  0 ; 0x174: 0x00000000 kfdhdb.ub4spare[31]:                  0 ; 0x178: 0x00000000 kfdhdb.ub4spare[32]:                  0 ; 0x17c: 0x00000000 kfdhdb.ub4spare[33]:                  0 ; 0x180: 0x00000000 kfdhdb.ub4spare[34]:                  0 ; 0x184: 0x00000000 kfdhdb.ub4spare[35]:                  0 ; 0x188: 0x00000000 kfdhdb.ub4spare[36]:                  0 ; 0x18c: 0x00000000 kfdhdb.ub4spare[37]:                  0 ; 0x190: 0x00000000 kfdhdb.ub4spare[38]:                  0 ; 0x194: 0x00000000 kfdhdb.ub4spare[39]:                  0 ; 0x198: 0x00000000 kfdhdb.ub4spare[40]:                  0 ; 0x19c: 0x00000000 kfdhdb.ub4spare[41]:                  0 ; 0x1a0: 0x00000000 kfdhdb.ub4spare[42]:                  0 ; 0x1a4: 0x00000000 kfdhdb.ub4spare[43]:                  0 ; 0x1a8: 0x00000000 kfdhdb.ub4spare[44]:                  0 ; 0x1ac: 0x00000000 kfdhdb.ub4spare[45]:                  0 ; 0x1b0: 0x00000000 kfdhdb.ub4spare[46]:                  0 ; 0x1b4: 0x00000000 kfdhdb.ub4spare[47]:                  0 ; 0x1b8: 0x00000000 kfdhdb.ub4spare[48]:                  0 ; 0x1bc: 0x00000000 kfdhdb.ub4spare[49]:                  0 ; 0x1c0: 0x00000000 kfdhdb.ub4spare[50]:                  0 ; 0x1c4: 0x00000000 kfdhdb.ub4spare[51]:                  0 ; 0x1c8: 0x00000000 kfdhdb.ub4spare[52]:                  0 ; 0x1cc: 0x00000000 kfdhdb.ub4spare[53]:                  0 ; 0x1d0: 0x00000000 kfdhdb.acdb.aba.seq:                  0 ; 0x1d4: 0x00000000 kfdhdb.acdb.aba.blk:                  0 ; 0x1d8: 0x00000000 kfdhdb.acdb.ents:                     0 ; 0x1dc: 0x0000 kfdhdb.acdb.ub2spare:                 0 ; 0x1de: 0x0000 检查ASM进程,无此进程,启进程时报错 [oracle@rac1 ~]$ srvctl start asm -n rac1 PRCR-1070 : Failed to check if resource ora.asm is registered Cannot communicate with crsd
找了很多资料后怀疑是由于ASM实例没启 [oracle@rac1 ~]$ asmcmd Connected to an idle instance. ASMCMD> help asmcmd [-vV] [-a ] [-p] [command]
The environment variables ORACLE_HOME and ORACLE_SID determine the  instance to which the program connects, and ASMCMD establishes a  bequeath connection to it, in the same manner as a SQLPLUS / AS  SYSASM.  The user must be a member of the OSASM group.
Specifying the -V option prints the asmcmd version number and exits immediately.
Specifying the -v option prints extra information that can help advanced users diagnose problems.
Specify the -a option to choose the type of connection. There are only two possibilities: connecting as SYSASM or as SYSDBA. The default value if this option is unspecified is SYSASM.
Specifying the -p option allows the current directory to be displayed  in the command prompt, like so:
ASMCMD [+DATA/ORCL/CONTROLFILE] >
[command] specifies one of the following commands, along with its parameters.
Type "help [command]" to get help on a specific ASMCMD command.
commands: --------
md_backup, md_restore
lsattr, setattr
cd, cp, du, find, help, ls, lsct, lsdg, lsof, mkalias mkdir, pwd, rm, rmalias
chdg, chkdg, dropdg, iostat, lsdsk, lsod, mkdg, mount offline, online, rebal, remap, umount
dsget, dsset, lsop, shutdown, spbackup, spcopy, spget spmove, spset, startup
chtmpl, lstmpl, mktmpl, rmtmpl
chgrp, chmod, chown, groups, grpmod, lsgrp, lspwusr, lsusr mkgrp, mkusr, orapwusr, passwd, rmgrp, rmusr
volcreate, voldelete, voldisable, volenable, volinfo volresize, volset, volstat ASMCMD> startup   --启ASM实例 ASM instance started
Total System Global Area  284565504 bytes Fixed Size                  1336036 bytes Variable Size             258063644 bytes ASM Cache                  25165824 bytes ASM diskgroups mounted 检查ASM进程: [root@rac1 bin]# ps -ef|grep asm grid      4562     1  0 14:00 ?        00:00:01 asm_pmon_+ASM1 grid      4568     1  0 14:00 ?        00:00:02 asm_vktm_+ASM1 grid      4574     1  0 14:00 ?        00:00:00 asm_gen0_+ASM1 grid      4591     1  0 14:00 ?        00:00:02 asm_diag_+ASM1 grid      4596     1  0 14:00 ?        00:00:01 asm_ping_+ASM1 grid      4603     1  0 14:00 ?        00:00:00 asm_psp0_+ASM1 grid      4608     1  0 14:00 ?        00:00:08 asm_dia0_+ASM1 grid      4613     1  0 14:00 ?        00:00:05 asm_lmon_+ASM1 grid      4619     1  0 14:00 ?        00:00:03 asm_lmd0_+ASM1 grid      4624     1  0 14:00 ?        00:00:05 asm_lms0_+ASM1 grid      4633     1  0 14:00 ?        00:00:00 asm_lmhb_+ASM1 grid      4643     1  0 14:00 ?        00:00:00 asm_mman_+ASM1 grid      4656     1  0 14:00 ?        00:00:00 asm_dbw0_+ASM1 grid      4673     1  0 14:00 ?        00:00:00 asm_lgwr_+ASM1 grid      4678     1  0 14:00 ?        00:00:00 asm_ckpt_+ASM1 grid      4695     1  0 14:00 ?        00:00:00 asm_smon_+ASM1 grid      4700     1  0 14:00 ?        00:00:01 asm_rbal_+ASM1 grid      4707     1  0 14:00 ?        00:00:00 asm_gmon_+ASM1 grid      4712     1  0 14:00 ?        00:00:00 asm_mmon_+ASM1 grid      4716     1  0 14:00 ?        00:00:01 asm_mmnl_+ASM1 grid      4853     1  0 14:01 ?        00:00:00 asm_lck0_+ASM1 grid      4928     1  0 14:01 ?        00:00:00 asm_asmb_+ASM1 grid      4940     1  0 14:01 ?        00:00:00 oracle+ASM1_asmb_+asm1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) oracle    5579     1  0 14:04 ?        00:00:00 ora_asmb_rac1 grid      5611     1  0 14:05 ?        00:00:00 oracle+ASM1_asmb_rac1 (DESCRIPTION=(LOCAL=YES)(ADDRESS=(PROTOCOL=beq))) root      6257 13890  0 14:35 pts/3    00:00:00 grep asm 检查CRS相关进程: [root@rac1 bin]# ./crs_stat -t Name           Type           Target    State     Host         ------------------------------------------------------------ ora.DATA.dg    ora....up.type ONLINE    ONLINE    rac1         ora....ER.lsnr ora....er.type ONLINE    ONLINE    rac1         ora....N1.lsnr ora....er.type ONLINE    ONLINE    rac2         ora.asm        ora.asm.type   ONLINE    ONLINE    rac1         ora.eons       ora.eons.type  ONLINE    ONLINE    rac1         ora.gsd        ora.gsd.type   OFFLINE   OFFLINE                ora....network ora....rk.type ONLINE    ONLINE    rac1         ora.oc4j       ora.oc4j.type  OFFLINE   OFFLINE                ora.ons        ora.ons.type   ONLINE    ONLINE    rac1         ora.rac.db     ora....se.type ONLINE    ONLINE    rac2         ora....SM1.asm application    ONLINE    ONLINE    rac1         ora....C1.lsnr application    ONLINE    ONLINE    rac1         ora.rac1.gsd   application    OFFLINE   OFFLINE                ora.rac1.ons   application    ONLINE    ONLINE    rac1         ora.rac1.vip   ora....t1.type ONLINE    ONLINE    rac1         ora....SM2.asm application    ONLINE    ONLINE    rac2         ora....C2.lsnr application    ONLINE    ONLINE    rac2         ora.rac2.gsd   application    OFFLINE   OFFLINE                ora.rac2.ons   application    ONLINE    ONLINE    rac2         ora.rac2.vip   ora....t1.type ONLINE    ONLINE    rac2         ora.scan1.vip  ora....ip.type ONLINE    ONLINE    rac2 
SQL> select OPEN_MODE from v$database;
OPEN_MODE ---------------------------------------- READ WRITE
问题:在重启时没有启动到ASM实例,导致不能打开ASM的共享磁盘,CRS相关进程无法打开,为什么重启后没有启动ASM实例?

来自 “ ITPUB博客 ” ,链接:http://blog.itpub.net/27015256/viewspace-767957/,如需转载,请注明出处,否则将追究法律责任。

转载于:http://blog.itpub.net/27015256/viewspace-767957/


本文来自互联网用户投稿,文章观点仅代表作者本人,不代表本站立场,不承担相关法律责任。如若转载,请注明出处。 如若内容造成侵权/违法违规/事实不符,请点击【内容举报】进行投诉反馈!

相关文章

立即
投稿

微信公众账号

微信扫一扫加关注

返回
顶部