NexentaStor - iSCSI example

NexentaStor community edition has some limitations on the iSCSI configurations (GUI or CLI interface)

Most of them can be addressed without switching to alternative solution or to the Enterprise edition, especially if the storage is already in use and downtime better be avoided.

Here is an example on the setup of iSCSI volume shared between 2 hosts.
It can be used for clustered file system or as a volume for Oracle database (ready for a RAC implementation).
Obviously, you can apply similar rules for different scenarios and network topologies.

Storage network topology:

In my setup both Host1 and Host2 are running Solaris 10.
Two fully independent switches were used to provide redundancy and additional throughput.


Workflow:
Configure iSCSI initiator on both hosts:

• Host1
• Initiator node name: iqn.1986-03.com.sun:XXX-f29d
• Host2
• Initiator node name: iqn.1986-03.com.sun:XXX-e792

On the storage server nexenta-ce ( in GUI or CLI )

• Create target point group ( if it is still not created)
• TPG1 (10.1.1.1:3260, 10.1.2.1:3260)
• Create volume
• zVol ( data/shaedvolume10 )
• create 2 targets
• iqn.1986-03.com.sun:XXX-fef7
• iqn.1986-03.com.sun:XXX-07ee

Nexenta-ce(root shell):

Keep in mind that it is not recommended by Nexenta

nmc@nexenta-ce:/$ option expert_mode =1 -s
nmc@nexenta-ce:/$ !bash
root@nexenta-ce# 

Targets must be offline when you are creating target group:

root@nexenta-ce# stmfadm create-tg tg10
root@nexenta-ce# stmfadm offline-target iqn.1986-03.com.sun:XXX-07ee
root@nexenta-ce# stmfadm offline-target iqn.1986-03.com.sun:XXX-fef7
root@nexenta-ce# stmfadm add-tg-member -g tg10 iqn.1986-03.com.sun:XXX-07ee
root@nexenta-ce# stmfadm add-tg-member -g tg10  iqn.1986-03.com.sun:XXX-fef7
root@nexenta-ce# stmfadm online-target iqn.1986-03.com.sun:XXX-07ee
root@nexenta-ce# stmfadm online-target iqn.1986-03.com.sun:XXX-fef7

Let's create a host group which includes both host1 and host2:

root@nexenta-ce# stmfadm create-hg host1    
root@nexenta-ce# stmfadm create-hg host2
root@nexenta-ce# stmfadm add-hg-member -g host1 iqn.1986-03.com.sun:XXX-f29d      
root@nexenta-ce# stmfadm add-hg-member -g host2 iqn.1986-03.com.sun:XXX-e792

root@nexenta-ce# sbdadm create-lu /dev/zvol/rdsk/data/tg10
root@nexenta-ce# sbdadm list-lu
              GUID           DATA SIZE           SOURCE
--------------------------- -------------------  ----------------
600XXXXXXX0001              214748364800          /dev/zvol/rdsk/data/tg10

root@nexenta-ce# stmfadm add-view -t tg10 -h host1 600XXXXXXX0001
root@nexenta-ce# stmfadm add-view -t tg10 -h host2 600XXXXXXX0001

Both host1 and host2 (root shell)
In current setup I'm using only static discovery.

root@host1# iscsiadm modify discovery -s enable
root@host1# iscsiadm add static-config iqn.1986-03.com.sun:XXX-fef7,10.1.1.1
root@host1# iscsiadm add static-config iqn.1986-03.com.sun:XXX-07ee,10.1.2.1
root@host1# iscsiadm list target
   Target: iqn.1986-03.com.sun:XXX-07ee
    ....
   Target: iqn.1986-03.com.sun:XXX-fef7
    ....
root@host1# devfsadm -Cv
root@host1# echo | format

Result - the created volume is ready for operations on both hosts plus high availability on the network layer.