You are here

Removing a ghost disk From a VG - the PV Key

Removing a „ghost disk“ From a VG - the PV Key

What is a ghost disk?

You may come into a situation where you have to remove a PV from a VG that is failed or not
even physically connected but still recorded in the lvmtab. Such a PV is called a “ghost disk”
or “phantom disk”. You can get a ghost disk if the disk failed and the VG was activated again,
maybe the system was rebooted.

If you cannot use vgcfgrestore to write the original LVM header back to the new disk because
a valid LVM configuration backup file (/etc/lvmconf/vgXY.conf[.old]) is missing or
corrupted you have to remove that PV from the VG (vgreduce), create a new LVM header
(pvcreate) and add it to the VG again (vgextend).

NOTE: In such situations the vgcfgrestore command may fail to restore the LVM header, complaining about a
‘Mismatch between the backup file and the running kernel’. If you are 100% sure that your backup is
valid you may override this check using the –R option.

In order to remove a PV from a VG you have to free it first, i.e. remove all logical extents
from it. If the LVs on such a disk is not mirrored data is lost anyway. If it is mirrored you
need to reduce the mirror, remove the PV run pvcreate and add it again.

A “ghost disk” can be identified if vgdisplay reports more current PVs than active ones.
Additionally LVM commands may complain about the missing PVs:

# vgdisplay vg01
vgdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t11d0":
Chapter 16 LVM
March 2002 Chapter 16 / Page 34
The specified path does not correspond to physical volume attached to
this volume group
vgdisplay: Couldn't query the list of physical volumes.
--- Volume groups ---
VG Name /dev/vg01
VG Write Access read/write
VG Status available
Max LV 255
Cur LV 3
Open LV 3
Max PV 16
Cur PV 2 number of PVs recorded in the lvmtab
Act PV 1 number of PVs recorded in the kernel
Max PE per PV 1016
VGDA 2
PE Size (Mbytes) 4
Total PE 511
Alloc PE 38
Free PE 473
Total PVG 0

The PV c0t11d0 is still recorded in lvmtab so it’s a ghost disk:
# strings /etc/lvmtab
/dev/vg01
/dev/dsk/c0t0d2
/dev/dsk/c1t2d2
/dev/dsk/c0t11d0

Running vgreduce with the -f option would remove all PVs that are “free”, i.e there is no LV
having extents on that PV. Otherwise - if the PV is not free vgreduce -f reports an extent map
to identify the associated LVs:

# vgreduce -f vg01
skip alternate link /dev/dsk/c1t2d2
vgreduce: Couldn't query physical volume "/dev/dsk/c0t11d0":
The specified path does not correspond to physical volume attached to
this volume group
Not all extents are free. i.e. Out of 508 PEs, only 500 are free.

You must free all PEs using lvreduce/lvremove before the PV can be
removed.
Example: lvreduce -A n -m 0 /dev/vg01/lvol1.
lvremove -A n /dev/vg01/lvol1.
Here's the map of used PEs
--- Logical extents ---
LE LV PE Status 1
0000 lvol1 0000 ???
0001 lvol1 0001 ???
0002 lvol1 0002 ???
0003 lvol1 0003 ???
0004 lvol1 0004 ???
0005 lvol1 0005 ???
0006 lvol1 0006 ???
0007 lvol1 0007 ???

In this case there is lvol1 having extents on c0t11d0. You have to remove these extents from
the PV. If the LV is mirrored use lvreduce to remove the mirrored extents, if the LV is not
mirrored, data is lost anyway and you have to use lvremove to delete the LV:

Check the LV state:
# lvdisplay -v /dev/vg01/lvol1
lvdisplay: Warning: couldn't query physical volume "/dev/dsk/c0t11d0":
The specified path does not correspond to physical volume attached to
this volume group
lvdisplay: Couldn't query the list of physical volumes.
--- Logical volumes ---
LV Name /dev/vg01/lvol1
VG Name /dev/vg01
LV Permission read/write
LV Status available/stale
Mirror copies 1
Consistency Recovery MWC
Schedule parallel
LV Size (Mbytes) 32
Current LE 8
Allocated PE 16
Stripes 0
Stripe Size (Kbytes) 0
Bad block on
Allocation strict
IO Timeout (Seconds) default
--- Distribution of logical volume ---
PV Name LE on PV PE on PV
/dev/dsk/c0t0d2 8 8
--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 ??? 00000 stale /dev/dsk/c0t0d2 00000 current
00001 ??? 00001 stale /dev/dsk/c0t0d2 00001 current
00002 ??? 00002 stale /dev/dsk/c0t0d2 00002 current
00003 ??? 00003 stale /dev/dsk/c0t0d2 00003 current
00004 ??? 00004 stale /dev/dsk/c0t0d2 00004 current
00005 ??? 00005 stale /dev/dsk/c0t0d2 00005 current
00006 ??? 00006 stale /dev/dsk/c0t0d2 00006 current
00007 ??? 00007 stale /dev/dsk/c0t0d2 00007 current

You can see, that the LV is mirrored.
Since the disk is not available anymore the PV is not accessable by it’s decivefile
/dev/dsk/c0t11d0 as usual. Alternatively you can access it by it’s PV key.

The PV key

The PV key of a disk indicates it’s order in the VG. The first PV has the key 0, the second has
the key 1, etc. This does not necessarily have to be the order of appearance in lvmtab altough
it is usually like that, at least when a VG is initially created.

The PV key can be used to address a PV that is not attached to the VG, e.g. because it is no
longer available through it’s device file due to a HW problem. If a LV has extents on that PV
the key can be obtained using the -k option of lvdisplay:

# lvdisplay –v –k /dev/vg01/lvol1
--- Logical extents ---
LE PV1 PE1 Status 1 PV2 PE2 Status 2
00000 0 00000 stale 1 00000 current
00001 0 00001 stale 1 00001 current
00002 0 00002 stale 1 00002 current
00003 0 00003 stale 1 00003 current
00004 0 00004 stale 1 00004 current
00005 0 00005 stale 1 00005 current
00006 0 00006 stale 1 00006 current
00007 0 00007 stale 1 00007 current

Compared to the output above the ??? have been replaced with the PV key (= 0).

NOTE: You can use the xd(1) command to display the PV key because it is stored at a fixed position in the
LVM header, exactly 8222 bytes from the beginning of the disk:
# xd –j8222 -N2 /dev/rdsk/c1t6d0

NOTE: Sometimes you see messages like PV[X] is POWERFAILED in syslog. In this case X is the PV key.
Now reduce the mirror with the obtained key as argument:
# lvreduce –k –m 0 /dev/vg01/lvol1 0

After that the PV can be removed from the VG:
# vgreduce -f vg01
skip alternate link /dev/dsk/c1t2d2
vgreduce: Couldn't query physical volume "/dev/dsk/c0t11d0":
The specified path does not correspond to physical volume attached to
this volume group
PV with key 0 sucessfully deleted from vg vg01
Repair done, please do the following steps.....:
1. save /etc/lvmtab to another file
2. remove /etc/lvmtab
3. use vgscan(1m) -v to re-create /etc/lvmtab
4. NOW use vgcfgbackup(1m) to save the LVM setup
Now do exactly what the output above indicates in order to remove the PV from the lvmtab:
# mv /etc/lvmtab /etc/lvmtab.org
# vgscan –v

Scan of Physical Volumes Complete.
*** LVMTAB has been created successfully.
*** If PV links are configured in the system.
*** Do the following to resync information on disk.
*** #1. vgchange -a y
*** #2. lvlnboot -R
Check it:
# strings /etc/lvmtab
/dev/vg01
/dev/dsk/c0t0d2
/dev/dsk/c1t2d2
Reactivate the VG and backup the LVM config:

# vgchange -a y vg01
# vgcfgbackup vg01
Now you can run pvcreate on the PV, add it to the VG again and recreate the mirror:
# pvcreate [-f] /dev/rdsk/c0t11d0
# vgextend vg01 /dev/dsk/c0t11d0
# lvextend –m 1 /dev/vg01/lvol1 /dev/dsk/c0t11d0
If the LV was not mirrored, recreate the LV (lvcreate), create a FS on it (newfs(1M)) and
recover the data from the backup.

Unix Systems: 

Add new comment

Filtered HTML

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <blockquote> <code> <ul> <ol> <li> <dl> <dt> <dd>
  • Lines and paragraphs break automatically.

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
By submitting this form, you accept the Mollom privacy policy.

Fatal error: Class CToolsCssCache contains 1 abstract method and must therefore be declared abstract or implement the remaining methods (DrupalCacheInterface::__construct) in /homepages/37/d228974590/htdocs/sites/all/modules/ctools/includes/css-cache.inc on line 52