In this post I am going to explore refreshing a storage snapshot of an Oracle 19c RAC database created on a Dell PowerMax storage array.
This post is a companion to the video Dell PowerMax – Refresh a storage snapshot of Oracle RAC database. and a follow on from the blog post Create a storage snapshot of an Oracle 19c RAC database using ASM diskgroups with Dell PowerMax.
This blog post assumes some basic knowledge of PowerMax storage concepts. If you need a basic introduction please check out my PowerMax basics video.
These examples will use the Solutions Enabler command line method of managing the PowerMax, unlike the video which uses the Unisphere graphical interface.
In the previous blog post we used Dell PowerMax to create a storage snapshot of an Oracle 19c RAC database. The process included creating the storage snapshot, creating link devices, linking the snapshot to the desired generation, creating the link storage group masking view, and then expsoing the snapshot to the target hosts.
Compared to some simpler and more limited snapshot technologies, the additional steps might seem cumbersome. However they add an enourmous amount of flexibility, and the additional steps they add to the process are mostly limited to the first time use of the snapshot. Subsequent refreshes can reuse most of what has been created making the refresh process trivial.
To refresh a snapshot then, start by taking the target ASM diskgroups off line and unlabel the disks. If the cloned database is mounted or open on the target host, obviously it will need to be shut down.
srvctl stop diskgroup -g SNAPDATA -n dsib0251,dsib0252
srvctl stop diskgroup -g SNAPREDO -n dsib0251,dsib0252
srvctl disable diskgroup -g SNAPDATA -n dsib0251,dsib0252
srvctl disable diskgroup -g SNAPREDO -n dsib0251,dsib0252
asmcmd afd_unlabel /dev/emcpowerr -f
asmcmd afd_unlabel /dev/emcpowerq -f
asmcmd afd_unlabel /dev/emcpowerw -f
asmcmd afd_unlabel /dev/emcpowerx -f
asmcmd afd_unlabel /dev/emcpowert -f
asmcmd afd_unlabel /dev/emcpowerp -f
asmcmd afd_unlabel /dev/emcpoweru -f
Now remove and drop the SNAPDATA and SNAPREDO diskgroups:
srvctl remove diskgroup -g SNAPDATA -f
srvctl remove diskgroup -g SNAPREDO -f
asmcmd dropdg -r -f SNAPDATA
asmcmd dropdg -r -f SNAPREDO
With Solutions Enabler we can interrogate the PowerMax to see what snapshots are currently linked:
[root@dsib0252 ~]# symsnapvx list -linked
Symmetrix ID : 000120200304 (Microcode Version: 6079)
--------------------------------------------------------------------------------
Sym Link Flgs
Dev Snapshot Name Gen Dev FCMDS Snapshot Timestamp
----- -------------------------------- ---- ----- ----- ------------------------
0014B gctsnap_swingdata_sg 1 00142 .D.X. Wed May 4 13:49:34 2022
0014C gctsnap_swingdata_sg 1 00143 .D.X. Wed May 4 13:49:34 2022
0014D gctsnap_swingdata_sg 1 00144 .D.X. Wed May 4 13:49:34 2022
0014E gctsnap_swingdata_sg 1 00145 .D.X. Wed May 4 13:49:34 2022
00153 gctsnap_swingredo_sg 1 00146 .D.X. Wed May 4 13:49:45 2022
00154 gctsnap_swingredo_sg 1 00147 .D.X. Wed May 4 13:49:45 2022
00155 gctsnap_swingredo_sg 1 00148 .D.X. Wed May 4 13:49:45 2022
In the above example we can see that gctsnap_swingdata_sg and gctsnap_swingredo_sg are linked. We can also interrogate to see what is linked to a given link storage group. Here I am asking what is linked to the SG swingdata_sg_lnk
[root@dsib0252 ~]# symsnapvx -lnsg swingdata_sg_lnk list -linked -by_tgt
Storage Group (SG) Name : swingdata_sg_lnk
SG's Symmetrix ID : 000120200304 (Microcode Version: 6079)
--------------------------------------------------------------------------------
Link Sym Flgs
Dev Dev Snapshot Name Gen FCMDS Snapshot Timestamp
----- ----- -------------------------------- ---- ----- ------------------------
00142 0014B gctsnap_swingdata_sg 1 .D.X. Tue Jul 12 15:01:16 2022
00143 0014C gctsnap_swingdata_sg 1 .D.X. Tue Jul 12 15:01:16 2022
00144 0014D gctsnap_swingdata_sg 1 .D.X. Tue Jul 12 15:01:16 2022
00145 0014E gctsnap_swingdata_sg 1 .D.X. Tue Jul 12 15:01:16 2022
Flgs:
(F)ailed : F = Force Failed, X = General Failure, . = No Failure
: S = SRP Failure, R = RDP Failure
(C)opy : I = CopyInProg, C = Copied, D = Copied/Destaged, . = NoCopy Link
(M)odified : X = Modified Target Data, . = Not Modified
(D)efined : X = All Tracks Defined, . = Define in progress
(S)napshot : X = Has snapshot waiting for define to complete
: . = No snapshot waiting for define to complete
So swingdata_sg_lnk is linked to snapshot gctsnap_swingdata_sg generation 1.
We can look to see what other generations might be available. We can see that our linked snapshot above is based on Sym devices 0014B through 0014E, so let’s see what other snapshots might exist for these volumes:
[root@dsib0251 gct]# symsnapvx list -devs 0014B:0014E -detail
Symmetrix ID : 000120200304 (Microcode Version: 6079)
----------------------------------------------------------------------------------------------------------------------------------------
Snapshot Total
Sym Flags Dev Size Deltas Non-Shared
Dev Snapshot Name Gen FLRG TSEB Snapshot Timestamp (Tracks) (Tracks) (Tracks) Expiration Date
----- -------------------------------- ---- --------- ------------------------ ---------- ---------- ---------- ------------------------
0014B gctsnap_swingdata_sg 0 .... .... Thu May 12 11:50:07 2022 8388615 256 128 NA
gctsnap_swingdata_sg 1 .X.. .... Wed May 4 13:49:33 2022 8388615 8576 1152 NA
0014C gctsnap_swingdata_sg 0 .... .... Thu May 12 11:50:07 2022 8388615 256 128 NA
gctsnap_swingdata_sg 1 .X.. .... Wed May 4 13:49:33 2022 8388615 8704 1152 NA
0014D gctsnap_swingdata_sg 0 .... .... Thu May 12 11:50:07 2022 8388615 256 128 NA
gctsnap_swingdata_sg 1 .X.. .... Wed May 4 13:49:33 2022 8388615 8576 1152 NA
0014E gctsnap_swingdata_sg 0 .... .... Thu May 12 11:50:07 2022 8388615 256 256 NA
gctsnap_swingdata_sg 1 .X.. .... Wed May 4 13:49:33 2022 8388615 8576 1152 NA
---------- ----------
35456 5248
In the above output we see one snapshot gctsnap_swingdata_sg for the specified volumes but two generations each. We can also see that it is generation 1 that is linked.
So to unlink the link devices from the snapshot we will use the symsnapvx command:
[root@dsib0251 ~]# symsnapvx unlink -snapshot_name gctsnap_swingdata_sg -symforce -devs 0014B:0014E -lndevs 00142:00145 -generation 1
Unlink operation execution is in progress for the device range(s). Please wait...
Polling for Unlink................................................Started.
Polling for Unlink................................................Done.
Unlink operation successfully executed for the device range(s)
[root@dsib0251 ~]# symsnapvx unlink -snapshot_name gctsnap_swingredo_sg -symforce -devs 00153:00155 -lndevs 00146:00148 -generation 1
Unlink operation execution is in progress for the device range(s). Please wait...
Polling for Unlink................................................Started.
Polling for Unlink................................................Done.
Unlink operation successfully executed for the device range(s)
Note that in the above example we have unlinked the link devices for both the gctsnap_swingdata_sg and the gctsnap_swingredo_sg snapshots.
It is a feature of PowerMax snapshots that if we now relink the link devices back to the same snapshots and generation, the link devices will revert to the state they were in when the snapshot was first made. Any changes made to the devices while they were mounted on the target server will be lost. Including ASM diskgroup and disk renames. If you want to preserve changes made while the link devices were mounted, then make another snapshot of the link devices before unlinking the snapshot.
[root@dsib0251 ~]# symsnapvx -sg swingdata_sg -lnsg swingdata_sg_lnk -snapshot_name gctsnap_swingdata_sg link -generation 1
Link operation execution is in progress for the storage group swingdata_sg. Please wait...
Polling for Link..................................................Started.
Polling for Link..................................................Done.
Link operation successfully executed for the storage group swingdata_sg
[root@dsib0251 ~]# symsnapvx -sg swingredo_sg -lnsg swingredo_sg_lnk -snapshot_name gctsnap_swingredo_sg link -generation 1
Link operation execution is in progress for the storage group swingredo_sg. Please wait...
Polling for Link..................................................Started.
Polling for Link..................................................Done.
Link operation successfully executed for the storage group swingdata_sgRecall the ASM diskgroups on the target server are offline, and the ASM diskgroups were dropped. No devices were added and none were taken away. The link devices here were never dropped or removed from the masking view. The WWN IDs of the devices did not change. There is no need to rescan the SCSI bus on the target host and PowerPath has nothing to do.
But if you use ASM to scan for new disks, you will see that the ASM disks have reverted to their original disk names, labels and ASM diskgroup names. This is why we deleted labels and dropped diskgroups before unlinking the snapshot. Now we can execute the rename steps again, and ASM should not complain that the new names already exist.
[oracle@dsib0252 ~]$ asmcmd afd_scan
[oracle@dsib0252 ~]$ asmcmd afd_refresh
[oracle@dsib0252 ~]$ asmcmd afd_lsdsk
--------------------------------------------------------------------------------
Label Filtering Path
================================================================================
SWINGDATA00 ENABLED /dev/emcpowerr
SWINGDATA01 ENABLED /dev/emcpowerq
SWINGDATA02 ENABLED /dev/emcpowerw
SWINGDATA03 ENABLED /dev/emcpowerx
SWINGREDO00 ENABLED /dev/emcpowert
SWINGREDO01 ENABLED /dev/emcpowerp
SWINGREDO02 ENABLED /dev/emcpoweru
We could now if we wished repeat the ASM disk/diskgroup rename and relabeling operation from the previous blog post.
The one point to watch for here is if you add new ASM disks to your source diskgroups. Any new disks added must also be added to the corresponding PowerMax SG, and any new snapshot will require the addition of matching link devices for when the revised and lager snapshot is to be mounted to a target server.
Acknowledgements.
Sincere thanks to Yaron Dar at Dell for his generous help and allowing me time on his PowerMax lab system.
the gruffdba 