Home Lab Build – NexentaStor Setup

NOTE: This installation was performed with NexentaStor 3.0.4, later versions may have slight differences in the installation process and the GUI interface.

I’m going to skip on insulting your intelligence by providing screen shots of the installation process for Nexenta, or the configuration of the VM if you go that route. I will start with the assumption that you have NexentaStor (Community Edition) installed on either a physical system or a VM, if you have gone the physical route obviously your network interface names are going to be different than I show. Since I am using VT-d of an actual SAS controller card, the rest should be similar.

1. Proceed and start the configuration wizard

   

2. Select which detected network interface you wish to be your primary (management) – we get more advanced control after the wizard is complete

   

3. Select your configuration option (static)

   

4. Input your IP Address you wish to use

   

5. Proceed through the network configuration defining your subnet mask, DNS servers and gateway

   

6. Review your configuration settings. If your configuration is correct, select N(o). If you need to make a correction, select Y(es)

   

7. Select if you wish to use HTTP or HTTPS for management access. SSL does add CPU overhead and may be less responsive as the system warns.

   

8. Make note of your configured TCP port and change it if desired (default = 2000), this will be the port the web management GUI listens on.

   

9. Make note of the provided URL and access it in order to continue configuration.

   

10. Open the management GUI in a web browser (Flash enabled) to proceed with the configuration wizard (Wizard 1).

    

11. Populate the fields to meet your configuration goals and proceed to the next step.

    

12. Configure your passwords for the two default management contexts and proceed.




    

13. Define your notification preferences and continue to the next step.




    

14. Review your configuration settings and save your configuration.
15. We are now into the “Wizard 2” stage, this is where we will configure the actual storage options.
16. Review your current interface settings, you can edit the existing configuration or add a new one. If you wish to aggregate multiple links into a single logical interface you must add a new interface to get that option. I will leave these as they are and can edit them at another time.
17. Next we are prompted to configure the iSCSI initiator service, this would be used to access another storage device for resources (e.g. to add NFS to an iSCSI only system such as a Dell/EQ). I am not using any other iSCSI systems so this is irrelevant.
    
    
18. This next screen shows us the list of detected disk devices, if you had configured iSCSI on the previous screen and had mapped storage to this initiator those resources should also be visible. I currently have 2 1TB Seagate drives attached to the SATA controller I assigned through VT-d.
    
    
19. In this next section we are asked to create volumes (storage pools). The process is to select the physical resources and assign it to the volume. You can select multiple devices and change the “Redundancy Type” to configure for RAID protection (None=stripe, Mirror, RAIDZ1 = ~RAID5, RAIDZ2 = ~RAID6, and RAIDZ3 = paranoid?) 

I am starting with “none” as I will perform some testing comparing different options in a later post.
    
    
20. In the lower section we configure the properties of the pool, including name, deduplication, and Sync settings (which we will discuss more later). I will leave all settings as default at this time.
    
    
21. Verify your volume was created, if not a red error description will flash temporarily across the upper section of the screen.
    
    
22. In this next portion we can create “folders”, each folder can have its own access type (NFS, CIFS, FTP, RSYNC, etc). I will add a single folder which I will configure for NFS, I am selecting a block size of 4KB to match that of most of my guest OS systems. I also am setting the file system to be case sensitive and to enable unicode.
    
23. This is the final step of the guided wizard, we can make any additional changes through the actual management interface. Set the checkboxes to meet your comfort level, I will attempt to compare some of these options in a later post for performance impact.
    
24. This completes the basic configuration, the rest will be done through the standard management interface.

NexentaStor Storage Concepts

Within the Nexenta, or perhaps Solaris ZFS storage management, there are :

• Datasets (ZFS Pools) which are made up of physical disk (or logical from outside array)
• Shares – logical units presented as file services (CIFS, NFS, RSYNC, FTP, etc)
• ZVols – logical units presented as block storage (iSCSI)

With that being said, there are really just 2 different processes for allocating storage depending on if it is file based or block based storage.

Again, I hope this helps someone. I will cover configuring storage and accessing it from ESX in a later post.

Advertisement

Home Lab Build – Overview

I realize I’m not alone in this process, it seems many of my fellow VMware enthusiasts are putting together home labs. This will be the first of a few blog posts regarding my home lab, I am working on planning out the requirements and my overall goals. Primary requirements are to be able to operate the majority of the VMware products in order to advance my understanding and satisfy curiosity of the growing portfolio, and hopefully to help me obtain more advanced certifications.

I have always had a lab of sorts on my laptop, though the new corporate issued laptop isn’t quite as beefy as the one from my previous employer. I previously had a MacBook Pro 17” with the i7 processor, the new laptop is a 15” with the i5. In order to make the most out of the laptop hardware both have 8GB of RAM and a dual drive configuration, including SSD. I will go into specifics of my MBP configuration in a separate entry. However I needed something more powerful, 8GB of RAM just doesn’t go very far to supporting multiple ESX hosts, vCenter, database servers and the other infrastructure requirements for hosting a vCloud environment. I had contemplated going for a larger virtual host environment, perhaps a Mac Pro 12-core with 32GB of RAM…until I did the math and compared the results to my “budget”. I decided to go for a lower cost route.

Hosts – So far I have determined the following requirements:

• Minimum of 2 hosts capable of operating ESXi 4.1
• Each host must provide at least 2 Gigabits of connectivity
• Ideally is listed on VMware compatibility list

Network – I have set the following network hardware requirements

• 8 Gigabit Ethernet ports
• Switch must support 802.1Q VLAN tagging
• Should support LACP
• No proprietary software to manage
• Support for jumbo frames

Storage – As we all know, shared storage is essential. Yes, we can operate without shared storage but every advanced feature requires shared storage. Since this is a home lab my performance requirements are minimal.

• Provide iSCSI and NFS storage
• Provide RAID capabilities to increase performance and resiliency
• Performance scalability
• Flexibility

With a little bit of time and creativity I believe I found solutions for each of the requirements. I will detail the hardware selected for each area above.

Hosts:
I have selected to use Dell T110 servers, these servers feature the entry quad core Xeon processors (X3400). I settled on these after looking at several options, including those from HP and home built from bare components. The T110 won out in large part on price, the base price with the Xeon X3430 was $379 but I opted for the upgrade to the X3440 with Hyper-Threading for $90. I couldn’t find any VMware specific benchmarks on either of these processors, however the PassMark score for the X3440 was 5303 vs the X3430 with 3638 which represents a 45% improvement. This is in part due to the Hyper-Threading, which is a debate in of itself regarding hypervisor benefits.

EDITED Feb-16-2010 – NOTE: after purchasing my first 2 hosts Dell decreased the pricing to $329 + $90 for $419 per host, Dell does not provide price guarantee but AMEX does…

The server is listed on the VMware CL and the Xeon 34xx processor includes Fault Tolerance support. Additionally, this server and motherboard support both ECC and non-ECC memory which allows for selecting lower priced non-ECC memory. Due to this I was able to max out the RAM on each host to 16GB for a reasonable price.

The servers are scheduled to arrive early next week.

Network:
I considered Netgear, D-Link, HP, Linksys and Cisco switches in trying to pick which was the best value. I would have loved to have a Catalyst switch due the proven track record, however that price alone would have exceeded what I now spent on my 2 ESX hosts. I settled on the Cisco SLM2008, it offers LACP (for when VMware gets around to it), static link aggregation (802.3ad – 2 group limit), jumbo frames and VLANs. Additionally it has a built in management web interface that works from any browser, not requiring any software to be installed is a bonus in my book. If you have a PoE switch to connect it to (or a power injector) it can run from PoE on port 1, otherwise a power brick is included. While I don’t see any value in jumbo frames for IP storage, being able to support MTU sizes larger than 1500 is critical in using Layer2 tunneling options, such as private vCloud Network Isolation features.

The switch arrived today.

EDITED Feb-16-2010
Storage:
As a storage guy I would have loved to have a NetApp FAS3210 with Flash Cache (a.k.a. PAM, or Performance Acceleration Module) but this would neither fit into my budget nor my wife’s noise tolerance. I have selected to go with a software based solution which I haven’t found many using it judging by the blog posts I’ve read. I have decided to use Nexenta Community Edition for my storage build out, I have advised former customers about this as an option for labs but haven’t actually worked with it myself. In a lab environment it can be self contained, in an enterprise solution it should be combined with an enterprise FC SAN.

While an Iomega, Synology, Drobo, or other storage appliance may be simpler to setup I am certain the option I am going with will smoke the competition at a lower price…we’ll see if I can stay on “budget”. For “budget” comparison sake I am going to work with Amazon pricing for devices that I may have considered:

• Iomega IX4-200d 4TB (4x1TB)        $593.98
• Thecus 4-bay N4200                         $779.21 + disk 4 x $64.99 = $1039.17
• Synology DS411+                                 $639.99 + disk 4 x $64.99 = $899.95
• Drobo FS (5-bay)                                $695.00 + disk 5 x $64.99 = $1019.95

I already had a server purchased that I am adding this role onto, but I also ended up adding a 3rd host to my configuration as the physical server hosting my storage system is clearly taken out of the ability to be “flexible” on maintenance and configuration changes…so I will show both totals. I already had a few parts that I am going to use, however I will try to add a price for those in to keep a fair tally.

The hardware that is added to the ESX host specific to storage is:

• SAS Controller: Intel SASUC8I PCIe (OEM’d LSI SAS3801 for a big savings) + breakout cable: $154.99 + $19.99
• 4-bay external SAS/SATA enclosure and cables        $179.00 + $29.50 + $27.50
• SATA HDDs: 4 x Seagate 1TB 32MB Cache, 2 x Hitachi 1TB – 4 x $64.99 + 2 x $60
• SSD for cache: OCZ 90GB SandForce controller drive $129.00
• Dual-port Intel Gigabit ET NIC        $162.99
• Dell T110 + 16GB RAM                $419.00 + $129.00

So the rough total here is ~$1630 including my ESX host and ~$1082 without the host, which gives me a full 6 disk storage system that I can expand pretty easily with dual GigE and it serves as my management host for the rest of my environment. Now I realize this is over the price of the other systems, but I believe this will provide more flexibility and far better performance than what those other systems are capable of.

EDITED Feb-23-2010

Update:
I’ve had a few questions about the pricing break down, so I thought I would try to make this a more reasonable comparison. In reality the storage I ended up with is far greater than any of the NAS appliances, I have more drive bays (8) and can actually increase to 16-bays for the price of the disk, enclosure and SAS/SATA controller.

In order to keep this an actual comparison to the appliance based options I thought I should show the pricing that is added to my first host.

• SAS Controller + Breakout cable                        $154.99 + $19.99
• 4 x Seagate 1TB 7200.12 Drives                        $259.96
• 1 x 3.5” to 5.25” drive bay adapter (for ESX DASD) ~$5
• TOTAL =                                                        $439.94

Those are truly the only additional hardware pieces needed, this would give you a 4-disk storage appliance that shares your first host. You can allocate as much or little vCPU and RAM as you wish, realizing that most of those hardware appliance options only have a low end desktop processor and 512MB of RAM.

Edit: You can find more info in the continuation of my lab build here:
Home Lab – Storage Performance Test (Part 1)

Time and tide wait for none

I’ve been in professional services for over 8 years now, most of that has really centered around data storage.  I spent a period of time implementing EMC commercial systems for an EMC contracted services partner, I then spent a few years contracted to do the same for NetApp.  In the mix of all of this I worked for EMC, IBM, HDS, and NetApp resellers with exposure to almost all of the systems on a technical pre-sales basis and post-sales implementation efforts.  Out of my experience I have formed some fairly strong and, I’d like to think, informed opinions of what should be in “enterprise” storage systems.

Now during all of this time consulting on storage systems they were always connected to something.  In the earlier years (as if it was so long ago…) it was generally application servers, and really just physical Windows hosts.  At the time I never even had to make a distinction that it was “physical”, because, really, there was no other option.  Yes, on occasion I worked with “virtual” systems through Sun Solaris, HP-UX and IBM AIX systems..but even these were somewhat rare, and many of them weren’t very virtual at all (virtual hardware didn’t exist).  As time progressed the type of systems connected to the storage evolved, and I had to support them all.  A storage system without any connected servers isn’t very useful, it can make lights blink, burn electricity and generate heat, but their usefulness without servers really ends there.

As projects changed with the evolution of applications and what businesses determined were critical the storage systems proceeded from supporting application servers that primarily included databases (e.g. MS SQL, Oracle, etc) to supporting email systems (e.g. MS Exchange).  It was really interesting that in the beginning most customers considered email to not be “valuable enough” to justify shared storage, but email quickly evolved into being one of the most critical applications in all of our environments right behind telephony.  As it turns out, communication is a critical function and we all prefer email for broadcast.

Of course, this all changed even further as we look at the recent years.  VMware quickly became the primary “server” I was connecting to storage.  This matured from being a couple of servers in an environment would be running ESX to all servers operating ESX, this happened in a far shorter time than it did for the progression from only databases to also email on these storage systems.  There are so many variables in deploying virtualization that my informed opinions of storage systems became more validated (at least in my mind), as flexibility became more important than ever.

All of this is to say that a good “storage consultant” never knows just storage, though I know plenty of one-hat experts that can provision storage all day long but can’t ever plan for the actual requirements of the application on the other end of the communication chain.  I always had to keep pace with understanding the application that was connected, as the storage was always a critical piece of meeting SLAs either for performance, availability or data protection.  Storage architecture without awareness of the application will always fail to meet requirements.  Now that being said, I wouldn’t ever consider myself a DBA or an Exchange administrator, in part because I wouldn’t want either job, but I know enough to architect storage to meet business requirements for those applications.

Of course, that evolved into the same for virtualization…but with a distinct difference.  Virtualization changed how storage is managed, provisioned and how data is protected.  If I was only consulting on the small storage portion of a project my billable utilization (critical measure of success in the professional services environment) would have been pretty small, probably less than 25%…however due to my awareness of the other components and dedication to learning VMware I was easily able to fill the other 50-75% of my time with the virtualization components.

I’ve been really fortunate in the past about keeping ahead of the curve, my first “real” tech job was in the ISP/telecom space.  This evolved from being in a support center for business leased line customers (DS0, DS1, DS3, OCx, etc) to being more involved on the managing and planning the backend network.  As I watched the ISPs fade away and consolidate I saw this as the tea leaves telling me that not as many router jockeys were going to be needed, so I switched into the more traditional IT role…as every company has an IT department.

This all changed when my wife and I moved across the country for her to attend law school, I left a perfectly good job that I hated to move to a new job market where I knew no one.  By luck, I found a job traveling as a storage consultant and that progressed to where we are today.

The next advancement in my career was due to the realization that the IT industry is yet again changing.  It doesn’t take much time reading Gartner reports or other IT business case studies to realize that virtualization is here to stay, and the next logical evolution is Cloud Computing.  I have now moved to the next step along my career path and joined the industry leader in creating Cloud Computing solutions, VMware.

I am more excited today about my job than I have been in a long time, I just hope I can keep pace with the shifting tides and the evolution of such a radical change in the industry.  I join a team of individuals that I have a lot of respect for and look forward to learning from within the VMware vCloud Services group.

Crazy schedule and its not letting up

Well, as everyone can tell…I just started this thing and I already fell off, or so it would seem.  I took a week of time off to go to a friends wedding, which meant no laptop…and I just didn’t have anything relevant to post while trying to avoid thinking about work and technology.  This week has been a 3-day work week for me, and its gone by all too quickly.  I’ve been rushing through the inbox trying to get caught up and keep my head above water on the projects I am assigned to, I’m still breathing…but there were times I had to pull out a soda straw and fight for a breath.

Next week is VMworld in San Francisco, I’m really excited and a bit stressed at the same time.  I have a ton to get done before flying out on Sunday, I have week of yard work and other homeowner chores that have been neglected with consecutive weeks of travel and insane temperatures.

I will try to post some details from VMworld during the week, but I can’t make any promises…but I hope to at least have something exciting to share afterwards at the minimum.  I keep hoping my work schedule will slow down just a bit so I can leverage my lab environment to actually generate content anyone would want to read/watch.  Oh well, back to the struggle of leaving the place better than I found it…it is true that not all consultants seem to have that goal, gets frustrating to clean up after “experts” and even aggravating when the mess was created by someone I know.

Another satisfied UCS customer

I spent much of this week deploying hardware to support a customer’s Cisco Unified Communications (UC).  In this case the customer is deploying a new telecommunications solution to add new capabilities to support the growing business, as well as move away from a legacy system that is becoming costly to maintain.

In order to support their overall strategy of maintaining efficiency in the data center through consolidation and virtualization, the design criteria included using Cisco UCS to facilitate consolidating the servers that support the UC environment.  Previously this deployment would have started with at least 3 physical systems, through virtualization we were able to reduce this down to 2 servers.  While this is only a 3:2 reduction, as this project moves from a proof of concept (POC) to a final production rollout the number of servers would have increased if using traditional hosting methods, however though the use of virtualization the customer can add availability and services without requiring additional hardware.

The overall solution included leveraging their existing NetApp storage system, however we needed to add fiber channel protocol support in order to meet Cisco specifications.  The solution included adding a Cisco UCS blade system comprised of a UCS6120 and a UCS 5108, as well as a Nexus 5000.  We are currently running with 2 B200 blades, each with dual 2.53Ghz 4-core CPUs and 32GB of RAM.  Additional blades will be added to this environment in the near future to support the virtualization of the remaining physical application servers in the data center.

As is typical with the Cisco UCS and NetApp deployments the project has gone pretty smoothly, more complexity is generated by the physical environment which we install these systems than are generally present within the product themselves.  In the end it makes for another happy Cisco UCS, VMware vSphere, and NetApp storage customer.  Hopefully their Cisco UC rollout goes just as smooth, thankfully that is in someone else’s hands.

First post

I guess everyone has to make their “first” blog post, I’m certain that others have set the bar very high…but I strive to set expectations low and to over deliver.  With that being said, I will attempt to just state what I hope this blog becomes.

I work every day in helping companies work towards efficiency, primarily as it relates to and around their corporate data centers.  The fact is that the “Data Center” extends far beyond the physical presence that most think of, it impacts how a company does its business internally and externally.  The “Data Center” enables them to deliver applications, services, widgets, or otherwise…but much of the orchestration that makes any business operates starts in the data center.

Most data centers have a lost of waste.  Wasted space.  Wasted energy. Wasted people.  Wasted time.  Waste is almost the norm for data centers and IT operations, it is usually forced by the complexity of the modern business world.  Complex requirements lead to overly complex solutions.

I work (through my employer) to help customers drive this waste out.  There are endless options to approach making the data center, IT, and business more efficient.  Some would look at this as “greening of the data center”, the green could either be the hard cash that is saved or the reduced environmental impact that can be achieved.