RMAN Restore fails with “PSDRPC returns significant error 3113”

Posted on by • Posted in Oracle, Recovery, RMAN • Tagged , , , , , , • Leave a comment

When using RMAN to restore a database to a new host, the recover database step fails with:

Crosschecked 43 objects
PSDRPC returns significant error 3113.
RMAN-00571: ===========================================================
RMAN-00569: =============== ERROR MESSAGE STACK FOLLOWS ===============
RMAN-00571: ===========================================================
RMAN-03002: failure of recover command at 03/18/2017 19:31:23
ORA-03113: end-of-file on communication channel


If you connect to the recovered database and check the database role, you find that the controlfiles have been restore for a physical standby:

SQL> select name, open_mode, database_role from v$database;

NAME      OPEN_MODE            DATABASE_ROLE
--------- -------------------- ----------------
NYC11     MOUNTED              PHYSICAL STANDBY

The problem here is that if you use RMAN to restore the controlfiles to a new host, and you are connected to the RMAN catalog, the controlfiles are written as standby controlfiles.

Here how the controlfiles in this example were restored to the new host:

[oracle@rstemc64vm31 ~]$ rman target / catalog rman/rman@rcat

Recovery Manager: Release 11.2.0.4.0 - Production on Sat Mar 18 19:29:57 2017

Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

connected to target database: NYC11 (not mounted)
connected to recovery catalog database

RMAN> restore controlfile from tag CF_20170318_1435;

Starting restore at 18-MAR-17
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=6 device type=DISK
allocated channel: ORA_DISK_2
channel ORA_DISK_2: SID=63 device type=DISK
allocated channel: ORA_DISK_3
channel ORA_DISK_3: SID=132 device type=DISK
allocated channel: ORA_DISK_4
channel ORA_DISK_4: SID=191 device type=DISK

channel ORA_DISK_1: starting datafile backup set restore
channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: reading from backup piece /nfs_mount/dd0205_rman/nyc11/cf_NYC11_9471_1.rmn
channel ORA_DISK_1: piece handle=/nfs_mount/dd0205_rman/nyc11/cf_NYC11_9471_1.rmn tag=CF_20170318_1435
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: restore complete, elapsed time: 00:00:01
output file name=+DATA/nyc11/controlfile/current.271.938979023
output file name=+FRA/nyc11/controlfile/current.338.938979023
Finished restore at 18-MAR-17

In order to avoid this problem, we need to restore the controlfiles while NOT connected to the RMAN catalog database.  This means we cannot use the backup tag, but we can use the name of the backupset file:

[oracle@rstemc64vm31 dataguard]$ rman target / catalog rman/rman@rcat

Recovery Manager: Release 11.2.0.4.0 - Production on Sat Mar 18 19:42:14 2017

Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

connected to target database: NYC11 (DBID=334041916, not open)
connected to recovery catalog database

RMAN> list backup of controlfile completed after 'sysdate-(6/24)';
List of Backup Sets
===================
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
398266 Incr 0 31.02M DISK 00:00:05 18-MAR-17
BP Key: 398276 Status: AVAILABLE Compressed: NO Tag: CF_20170318_1435
Piece Name: /nfs_mount/dd0205_rman/nyc11/cf_NYC11_9471_1.rmn
Control File Included: Ckp SCN: 33220754 Ckp time: 18-MAR-17

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
398294 Full 31.05M DISK 00:00:01 18-MAR-17
BP Key: 398296 Status: AVAILABLE Compressed: NO Tag: TAG20170318T144723
Piece Name: +FRA/nyc11/autobackup/2017_03_18/s_938962043.287.938962043
Control File Included: Ckp SCN: 33221214 Ckp time: 18-MAR-17

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
398314 Full 31.02M DISK 00:00:02 18-MAR-17
BP Key: 398317 Status: AVAILABLE Compressed: NO Tag: TAG20170318T144728
Piece Name: /nfs_mount/dd0205_rman/nyc11/NYC11_controlfile_83rver40_1_1.ctl
Control File Included: Ckp SCN: 33221259 Ckp time: 18-MAR-17

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
398325 Full 31.05M DISK 00:00:00 18-MAR-17
BP Key: 398327 Status: AVAILABLE Compressed: NO Tag: TAG20170318T144731
Piece Name: +FRA/nyc11/autobackup/2017_03_18/s_938962051.351.938962051
Control File Included: Ckp SCN: 33221273 Ckp time: 18-MAR-17

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
398462 Full 31.05M DISK 00:00:01 18-MAR-17
BP Key: 398478 Status: AVAILABLE Compressed: NO Tag: TAG20170318T144818
Piece Name: +FRA/nyc11/autobackup/2017_03_18/s_938962098.318.938962099
Control File Included: Ckp SCN: 33221554 Ckp time: 18-MAR-17

BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ---------------
398517 Full 31.05M DISK 00:00:00 18-MAR-17
BP Key: 398519 Status: AVAILABLE Compressed: NO Tag: TAG20170318T144827
Piece Name: +FRA/nyc11/autobackup/2017_03_18/s_938962107.395.938962107
Control File Included: Ckp SCN: 33221632 Ckp time: 18-MAR-17

The controlfile backed up with tag CF_20170318_1435 is in the backupset file /nfs_mount/dd0205_rman/nyc11/cf_NYC11_9471_1.rmn.

So exit RMAN, and start it again without the connection to the catalog:

[oracle@rstemc64vm31 dataguard]$ rman target /

Recovery Manager: Release 11.2.0.4.0 - Production on Sat Mar 18 19:50:43 2017

Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.

connected to target database: NYC11 (not mounted)

RMAN> restore controlfile from '/nfs_mount/dd0205_rman/nyc11/cf_NYC11_9471_1.rmn';

Starting restore at 18-MAR-17
using target database control file instead of recovery catalog
allocated channel: ORA_DISK_1
channel ORA_DISK_1: SID=69 device type=DISK

channel ORA_DISK_1: restoring control file
channel ORA_DISK_1: restore complete, elapsed time: 00:00:02
output file name=+DATA/nyc11/controlfile/current.271.938979023
output file name=+FRA/nyc11/controlfile/current.338.938979023
Finished restore at 18-MAR-17

RMAN> alter database mount;

database mounted
released channel: ORA_DISK_1

Connect to the database using SQL*Plus and verify that the database has mounted the correct controlfiles and that the database role is shown as PRIMARY.

SQL> show parameter control_files

NAME                                 TYPE        VALUE
------------------------------------ ----------- ------------------------------
control_files                        string      +DATA/nyc11/controlfile/curren
                                                 t.271.938979023, +FRA/nyc11/co
                                                 ntrolfile/current.338.93897902
                                                 3

SQL> select name, open_mode, database_role from v$database;

NAME      OPEN_MODE            DATABASE_ROLE
--------- -------------------- ----------------
NYC11     MOUNTED              PRIMARY

Once the correct type of control file has been restored, you can launch RMAN again and this time connect to the catalog to restore and recover the database to the new host.

Leave a comment