OK, I am trying to restore an Oracle database from a TOTALLY complicated setup and really need some help.
Overview: My production Oracle database server is actually a pair of Solaris servers running Symantec HA with one node active and one node in standby. My backups are scheduled through NetBackup which calls an Oracle RMAN script to backup the databases. Full database backups are generated every night and archive log backups are run at noon and 6:00 PM. Backups go to a pair of DataDomain devices, one in my primary datacenter and one at my disaster recovery site. At my disaster recovery site, I have a single Solaris server as a database server. Finally, I have an Oracle RMAN catalog that runs on a Windows server. This catalog is shut down twice a day and a Windows backup of just the RMAN directory is made, again to the DataDomain devices.
Now, I am testing my disaster recovery. We have stopped the replication between the local and remote DataDomain devices. I connect to the Unix server at the DR site and am attempting to recover the databases from backup. We have added entries to the hosts files and created a No.Restrictions file so that the server at the DR site has access to the NetBackup backups.
When I run "bplist -C st31bora01 -t 4 -l -R /" at the DR database server, I see several backup files including the catalog backup I want (database files removed from the list for brevity).
/u01/app/oraprod> /usr/openv/netbackup/bin/bplist -C st31bora01 -t 4 -l -R / -rw-rw---- oraprod dbaprod 24379392 Oct 14 12:55 /PASPROD_c-1968089396-20121014-02 -rw-rw---- oraprod dbaprod 24379392 Oct 14 02:01 /PASPROD_c-1968089396-20121014-01 -rw-rw---- oraprod dbaprod 24379392 Oct 14 01:56 /PASPROD_c-1968089396-20121014-00 -rw-rw---- oraprod dbaprod 24379392 Oct 13 12:04 /PASPROD_c-1968089396-20121013-00 -rw-rw---- oraprod dbaprod 22544384 Oct 12 17:05 /PASPROD_c-1968089396-20121012-03 -rw-rw---- oraprod dbaprod 22544384 Oct 12 12:05 /PASPROD_c-1968089396-20121012-02
I then try and use the following commands to recover the controlfile:
rman set dbid = 1968089396 connect catalog rman10@RMANdb connect target backup_dba list backup of controlfile; RUN { ALLOCATE CHANNEL CH1 TYPE 'SBT_TAPE'; SEND 'NB_ORA_CLIENT=st31bbl01,NB_ORA_POLICY=ORA_PASPROD'; restore controlfile; }
Unfortunately, this does not work and I get the following failure:
Starting restore at 17-Oct-12 15:27:36 channel CH1: starting datafile backupset restore channel CH1: restoring control file channel CH1: reading from backup piece PASPROD_c-1968089396-20121014-01 ORA-19870: error reading backup piece PASPROD_c-1968089396-20121014-01 ORA-19507: failed to retrieve sequential file, handle="PASPROD_c-1968089396-20121014-01", parms="" ORA-27029: skgfrtrv: sbtrestore returned error ORA-19511: Error received from media manager layer, error text: Backup file <PASPROD_c-1968089396-20121014-01> not found in NetBackup catalog failover to previous backup
I don't understand why it's not working. The only thing I can see is that when I "list backup of controlfile", I see media ID's that are at the primary site as indicated by "Media: @aaaac".
RMAN> list backup of controlfile;
List of Backup Sets
===================
BS Key Type LV Size Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ ------------------
5327585 Full 23.25M SBT_TAPE 00:01:03 14-Oct-12 02:02:28
BP Key: 5327593 Status: AVAILABLE Compressed: NO Tag: TAG20121014T020125
Handle: PASPROD_c-1968089396-20121014-01 Media: @aaaac
Control File Included: Ckp SCN: 10909224447 Ckp time: 14-Oct-12 02:01:25
But when I go into the NetBackup Administration Console (at the DR site) and select NetBackup Management --> Catalog in the treeview on the left, I see all the Media ID's are "@aaaae".
Any help you can offer would be greatly appreciated.
Ken