What you must never do is utilize a SSD slice for the ZIL.
I've been looking for a valid reference for this statement for a while. I've had a couple people tell me this and simply insist with no valid reasoning.
I understand that ZFS can't send a cache flush to a slice, but if I'm using a SSD with a supercap (say Intel 320) I have never heard any valid argument or reference. I have a LOT to gain from not allowing ZFS to 'use' the whole disk; My 300GB SSD can write something in the neighborhood of 600-800TB if I hand the whole thing to ZFS before I run out the media wear indicator. The same disk vastly under-provisioned (giving ZFS a 15GB slice) my media wear indicator will last about 4.2PB (yes, 4200TB). Under my VMware all-sync NFS load this drive might last me 6 months if I give the whole thing to ZFS where I should get years slicing it. On top of this, the drive performs approximately twice as fast vastly under-provisioned.
Can you give me a valid reasoning I shouldn't do this... or maybe some references?
I disagree. A failed solid state drive typically means 100% data loss, no recovery possible. A failed spinner can often be 'revived', albeit quite expensive if you let it go too long or have too major of a failure - there is quite literal 'physical' data to be retrieved.
There must be more to life than having everything. -- Maurice Sendak