Home Lab Build – NexentaStor Setup

NOTE: This installation was performed with NexentaStor 3.0.4, later versions may have slight differences in the installation process and the GUI interface.

I’m going to skip on insulting your intelligence by providing screen shots of the installation process for Nexenta, or the configuration of the VM if you go that route. I will start with the assumption that you have NexentaStor (Community Edition) installed on either a physical system or a VM, if you have gone the physical route obviously your network interface names are going to be different than I show. Since I am using VT-d of an actual SAS controller card, the rest should be similar.

  1. Proceed and start the configuration wizard

    wpid-voila_capture351-2011-02-4-16-565.png

  2. Select which detected network interface you wish to be your primary (management) – we get more advanced control after the wizard is complete

    wpid-voila_capture352-2011-02-4-16-565.png

  3. Select your configuration option (static)

    wpid-voila_capture353-2011-02-4-16-565.png

  4. Input your IP Address you wish to use

    wpid-voila_capture354-2011-02-4-16-565.png

  5. Proceed through the network configuration defining your subnet mask, DNS servers and gateway

    wpid-voila_capture355-2011-02-4-16-565.png

  6. Review your configuration settings. If your configuration is correct, select N(o). If you need to make a correction, select Y(es)

    wpid-voila_capture356-2011-02-4-16-565.png

  7. Select if you wish to use HTTP or HTTPS for management access. SSL does add CPU overhead and may be less responsive as the system warns.

    wpid-voila_capture357-2011-02-4-16-565.png

  8. Make note of your configured TCP port and change it if desired (default = 2000), this will be the port the web management GUI listens on.

    wpid-voila_capture358-2011-02-4-16-565.png

  9. Make note of the provided URL and access it in order to continue configuration.

    wpid-voila_capture359-2011-02-4-16-565.png

  10. Open the management GUI in a web browser (Flash enabled) to proceed with the configuration wizard (Wizard 1).

    wpid-voila_capture360-2011-02-4-16-565.png

  11. Populate the fields to meet your configuration goals and proceed to the next step.

    wpid-voila_capture361-2011-02-4-16-565.png

  12. Configure your passwords for the two default management contexts and proceed.




    wpid-voila_capture362-2011-02-4-16-565.png

  13. Define your notification preferences and continue to the next step.




    wpid-voila_capture363-2011-02-4-16-565.png

  14. Review your configuration settings and save your configuration.
  15. We are now into the “Wizard 2” stage, this is where we will configure the actual storage options.
  16. Review your current interface settings, you can edit the existing configuration or add a new one. If you wish to aggregate multiple links into a single logical interface you must add a new interface to get that option. I will leave these as they are and can edit them at another time.
  17. Next we are prompted to configure the iSCSI initiator service, this would be used to access another storage device for resources (e.g. to add NFS to an iSCSI only system such as a Dell/EQ). I am not using any other iSCSI systems so this is irrelevant.

    wpid-voila_capture366-2011-02-4-16-565.png
  18. This next screen shows us the list of detected disk devices, if you had configured iSCSI on the previous screen and had mapped storage to this initiator those resources should also be visible. I currently have 2 1TB Seagate drives attached to the SATA controller I assigned through VT-d.

    wpid-voila_capture367-2011-02-4-16-565.png
  19. In this next section we are asked to create volumes (storage pools). The process is to select the physical resources and assign it to the volume. You can select multiple devices and change the “Redundancy Type” to configure for RAID protection (None=stripe, Mirror, RAIDZ1 = ~RAID5, RAIDZ2 = ~RAID6, and RAIDZ3 = paranoid?) 

I am starting with “none” as I will perform some testing comparing different options in a later post.

    wpid-voila_capture371-2011-02-4-16-565.jpg
  20. In the lower section we configure the properties of the pool, including name, deduplication, and Sync settings (which we will discuss more later). I will leave all settings as default at this time.

    wpid-voila_capture373-2011-02-4-16-565.jpg
  21. Verify your volume was created, if not a red error description will flash temporarily across the upper section of the screen.

    wpid-voila_capture374-2011-02-4-16-565.jpg
  22. In this next portion we can create “folders”, each folder can have its own access type (NFS, CIFS, FTP, RSYNC, etc). I will add a single folder which I will configure for NFS, I am selecting a block size of 4KB to match that of most of my guest OS systems. I also am setting the file system to be case sensitive and to enable unicode.
    wpid-voila_capture375-2011-02-4-16-565.jpg
  23. This is the final step of the guided wizard, we can make any additional changes through the actual management interface. Set the checkboxes to meet your comfort level, I will attempt to compare some of these options in a later post for performance impact.
    wpid-voila_capture376-2011-02-4-16-565.jpg
  24. This completes the basic configuration, the rest will be done through the standard management interface.

NexentaStor Storage Concepts

Within the Nexenta, or perhaps Solaris ZFS storage management, there are :

  • Datasets (ZFS Pools) which are made up of physical disk (or logical from outside array)
  • Shares – logical units presented as file services (CIFS, NFS, RSYNC, FTP, etc)
  • ZVols – logical units presented as block storage (iSCSI)

With that being said, there are really just 2 different processes for allocating storage depending on if it is file based or block based storage.

Again, I hope this helps someone. I will cover configuring storage and accessing it from ESX in a later post.

Advertisements

Home Lab Build – Overview

I realize I’m not alone in this process, it seems many of my fellow VMware enthusiasts are putting together home labs. This will be the first of a few blog posts regarding my home lab, I am working on planning out the requirements and my overall goals. Primary requirements are to be able to operate the majority of the VMware products in order to advance my understanding and satisfy curiosity of the growing portfolio, and hopefully to help me obtain more advanced certifications.

I have always had a lab of sorts on my laptop, though the new corporate issued laptop isn’t quite as beefy as the one from my previous employer. I previously had a MacBook Pro 17” with the i7 processor, the new laptop is a 15” with the i5. In order to make the most out of the laptop hardware both have 8GB of RAM and a dual drive configuration, including SSD. I will go into specifics of my MBP configuration in a separate entry. However I needed something more powerful, 8GB of RAM just doesn’t go very far to supporting multiple ESX hosts, vCenter, database servers and the other infrastructure requirements for hosting a vCloud environment. I had contemplated going for a larger virtual host environment, perhaps a Mac Pro 12-core with 32GB of RAM…until I did the math and compared the results to my “budget”. I decided to go for a lower cost route.

Hosts – So far I have determined the following requirements:

  • Minimum of 2 hosts capable of operating ESXi 4.1
  • Each host must provide at least 2 Gigabits of connectivity
  • Ideally is listed on VMware compatibility list

Network – I have set the following network hardware requirements

  • 8 Gigabit Ethernet ports
  • Switch must support 802.1Q VLAN tagging
  • Should support LACP
  • No proprietary software to manage
  • Support for jumbo frames

Storage – As we all know, shared storage is essential. Yes, we can operate without shared storage but every advanced feature requires shared storage. Since this is a home lab my performance requirements are minimal.

  • Provide iSCSI and NFS storage
  • Provide RAID capabilities to increase performance and resiliency
  • Performance scalability
  • Flexibility

With a little bit of time and creativity I believe I found solutions for each of the requirements. I will detail the hardware selected for each area above.

Hosts:
I have selected to use Dell T110 servers, these servers feature the entry quad core Xeon processors (X3400). I settled on these after looking at several options, including those from HP and home built from bare components. The T110 won out in large part on price, the base price with the Xeon X3430 was $379 but I opted for the upgrade to the X3440 with Hyper-Threading for $90. I couldn’t find any VMware specific benchmarks on either of these processors, however the PassMark score for the X3440 was 5303 vs the X3430 with 3638 which represents a 45% improvement. This is in part due to the Hyper-Threading, which is a debate in of itself regarding hypervisor benefits.

EDITED Feb-16-2010 – NOTE: after purchasing my first 2 hosts Dell decreased the pricing to $329 + $90 for $419 per host, Dell does not provide price guarantee but AMEX does…

The server is listed on the VMware CL and the Xeon 34xx processor includes Fault Tolerance support. Additionally, this server and motherboard support both ECC and non-ECC memory which allows for selecting lower priced non-ECC memory. Due to this I was able to max out the RAM on each host to 16GB for a reasonable price.

The servers are scheduled to arrive early next week.

Network:
I considered Netgear, D-Link, HP, Linksys and Cisco switches in trying to pick which was the best value. I would have loved to have a Catalyst switch due the proven track record, however that price alone would have exceeded what I now spent on my 2 ESX hosts. I settled on the Cisco SLM2008, it offers LACP (for when VMware gets around to it), static link aggregation (802.3ad – 2 group limit), jumbo frames and VLANs. Additionally it has a built in management web interface that works from any browser, not requiring any software to be installed is a bonus in my book. If you have a PoE switch to connect it to (or a power injector) it can run from PoE on port 1, otherwise a power brick is included. While I don’t see any value in jumbo frames for IP storage, being able to support MTU sizes larger than 1500 is critical in using Layer2 tunneling options, such as private vCloud Network Isolation features.

The switch arrived today.

EDITED Feb-16-2010
Storage:
As a storage guy I would have loved to have a NetApp FAS3210 with Flash Cache (a.k.a. PAM, or Performance Acceleration Module) but this would neither fit into my budget nor my wife’s noise tolerance. I have selected to go with a software based solution which I haven’t found many using it judging by the blog posts I’ve read. I have decided to use Nexenta Community Edition for my storage build out, I have advised former customers about this as an option for labs but haven’t actually worked with it myself. In a lab environment it can be self contained, in an enterprise solution it should be combined with an enterprise FC SAN.

While an Iomega, Synology, Drobo, or other storage appliance may be simpler to setup I am certain the option I am going with will smoke the competition at a lower price…we’ll see if I can stay on “budget”. For “budget” comparison sake I am going to work with Amazon pricing for devices that I may have considered:

  • Iomega IX4-200d 4TB (4x1TB)        $593.98
  • Thecus 4-bay N4200                         $779.21 + disk 4 x $64.99 = $1039.17
  • Synology DS411+                                 $639.99 + disk 4 x $64.99 = $899.95
  • Drobo FS (5-bay)                                $695.00 + disk 5 x $64.99 = $1019.95

I already had a server purchased that I am adding this role onto, but I also ended up adding a 3rd host to my configuration as the physical server hosting my storage system is clearly taken out of the ability to be “flexible” on maintenance and configuration changes…so I will show both totals. I already had a few parts that I am going to use, however I will try to add a price for those in to keep a fair tally.

The hardware that is added to the ESX host specific to storage is:

  • SAS Controller: Intel SASUC8I PCIe (OEM’d LSI SAS3801 for a big savings) + breakout cable: $154.99 + $19.99
  • 4-bay external SAS/SATA enclosure and cables        $179.00 + $29.50 + $27.50
  • SATA HDDs: 4 x Seagate 1TB 32MB Cache, 2 x Hitachi 1TB – 4 x $64.99 + 2 x $60
  • SSD for cache: OCZ 90GB SandForce controller drive $129.00
  • Dual-port Intel Gigabit ET NIC        $162.99
  • Dell T110 + 16GB RAM                $419.00 + $129.00

So the rough total here is ~$1630 including my ESX host and ~$1082 without the host, which gives me a full 6 disk storage system that I can expand pretty easily with dual GigE and it serves as my management host for the rest of my environment. Now I realize this is over the price of the other systems, but I believe this will provide more flexibility and far better performance than what those other systems are capable of.

EDITED Feb-23-2010

Update:
I’ve had a few questions about the pricing break down, so I thought I would try to make this a more reasonable comparison. In reality the storage I ended up with is far greater than any of the NAS appliances, I have more drive bays (8) and can actually increase to 16-bays for the price of the disk, enclosure and SAS/SATA controller.

In order to keep this an actual comparison to the appliance based options I thought I should show the pricing that is added to my first host.

  • SAS Controller + Breakout cable                        $154.99 + $19.99
  • 4 x Seagate 1TB 7200.12 Drives                        $259.96
  • 1 x 3.5” to 5.25” drive bay adapter (for ESX DASD) ~$5
  • TOTAL =                                                        $439.94

Those are truly the only additional hardware pieces needed, this would give you a 4-disk storage appliance that shares your first host. You can allocate as much or little vCPU and RAM as you wish, realizing that most of those hardware appliance options only have a low end desktop processor and 512MB of RAM.

Edit: You can find more info in the continuation of my lab build here:
Home Lab – Storage Performance Test (Part 1)