{"id":1322,"date":"2014-01-16T11:57:20","date_gmt":"2014-01-16T10:57:20","guid":{"rendered":"http:\/\/evemaps.dotlan.net\/blog\/?p=1322"},"modified":"2014-01-18T23:46:24","modified_gmt":"2014-01-18T22:46:24","slug":"hardrives-update","status":"publish","type":"post","link":"https:\/\/evemaps.dotlan.net\/blog\/2014\/01\/16\/hardrives-update\/","title":{"rendered":"Harddrives Update (2)"},"content":{"rendered":"<p><img loading=\"lazy\" class=\"alignright size-full wp-image-1326\" alt=\"zfs-vs-hw\" src=\"http:\/\/e.dotlan.net\/blog\/wp-content\/uploads\/2014\/01\/zfs-vs-hw.png\" width=\"250\" height=\"170\" \/>Wow the donation drive is running pretty good. I really appreciate every single donation, so I want share you with my current thoughts with you. Right now I&#8217;m about 50\/50 between dedicated hardware raid vs zfs. In the beginning I was leaning more towards the HW Raid solution cause I know them from work, but the more I dig deeper into ZFS docs or blog posts the more I&#8217;ve to think my setup.<\/p>\n<p>It&#8217;s still some time to go and I don&#8217;t have to hurry up (since there&#8217;s no world going down tomorrow). When spending that much money I&#8217;ll and should take my time to carefully check my options and write them down to have an easier comparison between the 2 final solutions.<!--more--><\/p>\n<p><strong>What do I wanted to do anyway<\/strong><\/p>\n<ul>\n<li>add more RAM (finally fill all banks)<\/li>\n<li>add (Enterpise) SSD (raid1) for MySQL<\/li>\n<li>add a pair of small SSDs or FlashDisks for the Hypervisor (seperate hypervisor and storage which isn&#8217;t the case right now).<\/li>\n<\/ul>\n<p><strong>How will my final setup could look like:<\/strong><\/p>\n<ul>\n<li>2x small flashdisk (pcie) or ssd (sata)\u00a0for Hypervisor (I&#8217;ll keep Debian + Libvirt + KVM which I&#8217;m very happy with)<\/li>\n<li>4x1TB SAS with Raid6 or Raidz2<\/li>\n<li>LVM volumes or ZFS volumes (zvol) for the virtual guests (evemaps,\u00a0www, mail, vpn, development, test, etc)<\/li>\n<li>I&#8217;m very likely adding 2 enterprise SSDs in an raid1 array to store high IO stuff like database to keep it away from he spinning drives.<\/li>\n<\/ul>\n<p>These are the basics. Now lets see what are the up and downsides of the 2 solutions I&#8217;m looking into right now:<\/p>\n<p><strong>HW Raid:<\/strong><\/p>\n<ul>\n<li>Pro\n<ul>\n<li>Usually Fire and Forget and often use in enterprise environments for single servers that doesn&#8217;t require more then 4 or 8 disks<\/li>\n<li>Overall Performance gain through Controller Cache with Cache Protection (Flash or Battery)<\/li>\n<li>I know megaraid controllers from my work how they perform and how they can be controlled<\/li>\n<\/ul>\n<\/li>\n<li>Contra\n<ul>\n<li>Raid Controllers with Cache and Cache Protection are quite expensive<\/li>\n<li>Hardware Dependency on the Raid Controller<\/li>\n<li>Optional SSD Read\/Write Cache from the controller costs license money<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>ZFS:<\/strong><\/p>\n<ul>\n<li>Pro\n<ul>\n<li>Flexible configuration, tuning and additional features<\/li>\n<li>No hardware dependency<\/li>\n<li>Build in SSD Read Cache (L2ARC) and Write Cache (ZIL for sync writes)<\/li>\n<li>For the price of the HW Raid Controller I could get a SAS HostAdapter + Additional Pair of SSDs for Read\/Write Cache<\/li>\n<li>ZFS on Linux was declared production ready about 10 month ago (which I didn&#8217;t noticed)<\/li>\n<\/ul>\n<\/li>\n<li>Contra\n<ul>\n<li>Higher CPU usage due to Software Raid compared with HW Raid<\/li>\n<li>Higher RAM usage for ARC (RAM read\/write cache)\u00a0compared with HW Raid<\/li>\n<li>ZFS on Linux had always the &#8220;better don&#8217;t use it or use it on solaris&#8221; touch and I don&#8217;t intent to go away from Debian+Libvirt\/KVM as hypervisor<\/li>\n<li>Needs more thoughts, planning and likely tuning to find the best configuration in detail<\/li>\n<li>Even I&#8217;m maintaining ZFS servers (mostly backup storage server) at work, I don&#8217;t have much experience how ZFS is reacting under load.<\/li>\n<\/ul>\n<\/li>\n<li>ZFS + HW Raid\n<ul>\n<li>When doing my research I even found people that suggested using a HW Raid Controller to gain Speed through the WriteBack Cache and passing through every single disk as a single array (raid0 or jbod) for zfs to work with. But this sounds very much like voodoo.<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p><strong>Conclusion:<\/strong><\/p>\n<p>Since both are equally expensive (when using the same drives). In the end it all comes down to some very basic points<\/p>\n<ul>\n<li>Software Raid VS Hardware Raid<\/li>\n<li>Flexibility VS Hardware Dependency<\/li>\n<li>Build in read\/write caches when using dedicated SSDs VS Build in Controller Cache (BBU\/Cache Protection) (optional but expensive ssdcache)<\/li>\n<\/ul>\n<p><strong>UPDATE (18.01.2014)<\/strong><\/p>\n<p>I&#8217;ll go with the ZFS solution. If something goes wrong I can still throw out the HBA and buy a HW Raid Controller, but I don&#8217;t think so.<\/p>\n<p>I know the donation drive isn&#8217;t over yet, but I decided to already order about 2.5k \u20ac worth of hardware stuff (SAS Controller, SAS Drives, SSDs, Memory). I know that&#8217;s exceeding my donation drive was, but I was sick of having limited memory (and I know ZFS likes more memory). What&#8217;s better then RAM? More RAM! \ud83d\ude42<\/p>\n<p>I&#8217;ll share more details when I&#8217;m done.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Wow the donation drive is running pretty good. I really appreciate every single donation, so I want share you with my current thoughts with you. Right now I&#8217;m about 50\/50 between dedicated hardware raid vs zfs. In the beginning I was leaning more towards the HW Raid solution cause I know them from work, but [&hellip;]<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"_links":{"self":[{"href":"https:\/\/evemaps.dotlan.net\/blog\/wp-json\/wp\/v2\/posts\/1322"}],"collection":[{"href":"https:\/\/evemaps.dotlan.net\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/evemaps.dotlan.net\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/evemaps.dotlan.net\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/evemaps.dotlan.net\/blog\/wp-json\/wp\/v2\/comments?post=1322"}],"version-history":[{"count":13,"href":"https:\/\/evemaps.dotlan.net\/blog\/wp-json\/wp\/v2\/posts\/1322\/revisions"}],"predecessor-version":[{"id":1338,"href":"https:\/\/evemaps.dotlan.net\/blog\/wp-json\/wp\/v2\/posts\/1322\/revisions\/1338"}],"wp:attachment":[{"href":"https:\/\/evemaps.dotlan.net\/blog\/wp-json\/wp\/v2\/media?parent=1322"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/evemaps.dotlan.net\/blog\/wp-json\/wp\/v2\/categories?post=1322"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/evemaps.dotlan.net\/blog\/wp-json\/wp\/v2\/tags?post=1322"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}