A few days ago, we had an incident with one client that he lost the connection with iscsi LUN where have the VM HDD.
The StorageRepository appear with de red blade and isn’t possible «reattach» process. When you do the action, we have the next message:
The attempt to query the iSCSi daemon failed
We don’t found error reported on Support Forums. In the logs of system i can see:
XenServer Log
ERROR XenAdmin.Actions.AsyncAction [3600] – The attempt to query the iSCSi daemon failed
ERROR XenAdmin.Actions.AsyncAction [3600] – en XenAdmin.Actions.SrRepairAction.Run() en XenAdmin.Actions.AsyncAction.RunWorkerThread(Object o)
SMLOG
Raising exception [86, Failed to query the iscsi daemon [opterr=Failed to logout from target]]
FAILED: (errno 5) stdout: », stderr: ‘ Volume group «VG_XenStorage-cc734a73-5a5f-5fd1-24cf- 308258bc1fd2» not found’
We tested in XenServer that you can validate with openiscsi perfectly but isn’t posible connect to LUN. Is strange because all other LUN in the NAS is not affected.
We destroy de PBD connector and recreate again, with de the same problem.
The customer talk with NAS Manufacturer and is necessary update de Firmware. It’s posible a NAS error, but if this the problem, how i recover the VMs on this LUN.
In this post i don’t explain how to troublesshot de iscsi connection. I explain how to recover the LUN across the Live Linux Distribution. Any LiveCD is valid.
You only need to run Linux SO. The first step is install the necessary packets for do the recuperation. You only need the OpeniScsi packets. LVM2 is optional if you like to try mount the filesystem.
We talk with the client and ask that is necessary to create a LUN with the same size that the corrupted lun. With this, i create a VM with XenServer and run the liveCD.
When the Linux Live is running, we need to open a terminal and install the necessary packets.
# sudo apt-get install open-iscsi
When the install is finished is time to connect with Lun and recover the information. For do this…
Start iscsi daemon
# /etc/init.d/open-iscsi start
Discover targets
#iscsiadm -m discovery -t sendtargets -p [IP_Target]
IP_TARGET:3260,1 iqn.2002-10.com.target:RESOURCE1
IP_TARGET:3260,1 iqn.2002-10.com.target:RESOURCE2
IP_TARGET:3260,1 iqn.2002-10.com.target:RESOURCE3
establish the credentials
# iscsiadmin -m node -t IP_TARGET:3260,1 iqn.2002-10.com.target:RESOURCE1 -P [IP_Target] -o update -n node.session.auth.username -v [USER]
# iscsiadmin -m node -t IP_TARGET:3260,1 iqn.2002-10.com.target:RESOURCE1 -P [IP_Target] -o update -n node.session.auth.password-v [PASSWD]
and now connect to target
# iscsiadm -m node -t IP_TARGET:3260,1 iqn.2002-10.com.target:RESOURCE1 -p [IP_Target] -l
with fdisk -l, you can see the device used. In my case we use /dev/sdb.
Realice the same process for the New LUN created. Is mapped with the next device available. In my case: /dev/sdc
In this point is the moment for recover the info. And how i do?
It’s easy, we do a copy block to block from sdb device to sdc device. For do this, we type the next command:
# dd if=/dev/sdb of=/dev/sdc bs=512 conv=noerror,sync
The process is slow and take his time. You can see if the command is running with TOP command or PS command. When the command is finished is time to create again the SR on XenCenter. The new SR with the old SR copy is ready to use.
For more info about this, i recommended read the man pages:
http://linux.die.net/man/8/iscsiadm
————–
¿Do you need help with your Citrix Enviroments?
Visit us at www.ctxdom.com or contract us at www.101-consulting.com
Leave a Reply