**List disks by ID**
<code bash>
ls -alh /dev/disk/by-id
</code>
Help : make correspondances with disks by-path :
<code bash>
ls -alh /dev/disk/by-path/
</code>
An other view :
<code bash>
lsblk -o NAME,SIZE,FSTYPE,TYPE,MOUNTPOINT
</code>

**Create zpool of RAIDZ1 type**, named "zp1" 3 disk, ashift=12 (4096 bytes pool sector size)

Notes : The ashift needs to be set at least to the sector-size of the underlying disks (2 to the power of ashift is the sector-size), or any disk which might be put in the pool (for example the replacement of a defective disk). 

<code bash>
zpool create -f zp1 raidz1 -o ashift=12 /dev/disk/by-id/scsi-36a4badb038591c00233791390c02c443 /dev/disk/by-id/scsi-36a4badb038591c00233791480cde8092 /dev/disk/by-id/scsi-36a4badb038591c00233791540d9dabe0
</code>

**Get some info about it :**
<code bash>
zpool list
zpool status -Pv zp1
zpool get all

zpool iostat -vq 1
</code>

RQ : by default, it is mounted over /zp1
<code bash>
zfs get all
# You see the mount point here !
</code>

**Use fdisk in order to partition SSD disk and create :**
- 1 partition, size = memory size / 2
- 1 partition with the lefting space
<code bash>
zpool add zp1 log /dev/disk/by-id/scsi-36a4badb038591c002337912c0b32c066-part1
zpool add zp1 cache /dev/disk/by-id/scsi-36a4badb038591c002337912c0b32c066-part2
zpool status -Pv zp1
</code>

**REVERT :**
<code bash>
zpool remove zp1 /dev/disk/by-id/scsi-36a4badb038591c002337912c0b32c066-part2
zpool remove zp1 /dev/disk/by-id/scsi-36a4badb038591c002337912c0b32c066-part1
zpool destroy zp1
</code>

**TESTS**

<code bash>
apt-get install fio

echo 3 > /proc/sys/vm/drop_caches
fio --description="Emulation of Intel IOmeter File Server Access Pattern" \
    --name=iometer --bssplit=512/10:1k/5:2k/5:4k/60:8k/2:16k/4:32k/4:64k/10 \
    --rw=randrw --rwmixread=80 --direct=0 --size=4g --iodepth=64 \
    --zero_buffers --invalidate=1 --ioengine=sync --size=10G \
    --numjobs=50 --runtime=30 --time_based --group_reporting --gtod_reduce=1 \
    --filename=./fio_test.delme
</code>

# 3 SAS 10k disks, 80GB, SLOG 20GB, CACHE 120GB
# before each test, run : echo 3 > /proc/sys/vm/drop_caches
<code bash>
   READ: io=272184KB, aggrb=8469KB/s, minb=8469KB/s, maxb=8469KB/s, mint=32138msec, maxt=32138msec
  WRITE: io=64730KB, aggrb=2014KB/s, minb=2014KB/s, maxb=2014KB/s, mint=32138msec, maxt=32138msec
</code>

# remove CACHE
<code bash>
zpool remove zp1 /dev/disk/by-id/scsi-36a4badb038591c002337912c0b32c066-part2

   READ: io=157142KB, aggrb=4919KB/s, minb=4919KB/s, maxb=4919KB/s, mint=31941msec, maxt=31941msec
  WRITE: io=39336KB, aggrb=1231KB/s, minb=1231KB/s, maxb=1231KB/s, mint=31941msec, maxt=31941msec
</code>

# remove SLOG
<code bash>
zpool remove zp1 /dev/disk/by-id/scsi-36a4badb038591c002337912c0b32c066-part1

   READ: io=210334KB, aggrb=6184KB/s, minb=6184KB/s, maxb=6184KB/s, mint=34012msec, maxt=34012msec
  WRITE: io=50655KB, aggrb=1489KB/s, minb=1489KB/s, maxb=1489KB/s, mint=34012msec, maxt=34012msec
</code>


**Compare to MD (soft raid) RAID5 over same 3 disks :**
Run status group 0 (all jobs):
   READ: io=139365KB, aggrb=3974KB/s, minb=3974KB/s, maxb=3974KB/s, mint=35063msec, maxt=35063msec
  WRITE: io=34421KB, aggrb=981KB/s, minb=981KB/s, maxb=981KB/s, mint=35063msec, maxt=35063msec