Category Archives: Network

Functional Home Gigabit with Century Link

TL;DR (skip to the part you care about and not my rambling in boredom)

I’ve been using Comcast (Xfinity) for my home Internet service since 2003, prior to that I lived in a house that had multiple T1s (back when megabits of home Internet was very rare).  It is somewhat hard to imagine that in such a short period of time we went from hardwired home Internet being measured in kilobits to almost every mobile device we own being capable of sustaining 10s of megabits while roaming about.

I had been holding onto my Comcast Teleworker discounted ‘business’ Internet after leaving VMware, waiting for Google Fiber to come to town as Portland was supposed to be on the relatively near future roadmap and I was trying to avoid adding more unsightly aerial cabling to the exterior of my 110 year old house.  As neat as modern technology is, it doesn’t really go well with the architectural detail of an old craftsman home.  Since Google Fiber is now dead I decided to proceed with the next best option, Century Link.

I never thought I’d suggest that Century Link (formerly Qwest, formerly US West, aka US Worst) was a “best” option for anything.  I worked for large national ISPs for my early career, and US Worst was always one of the most problematic carriers to deal with.  I still have flashbacks about the escalations and yelling customers, but best was when their tech and manager didn’t realize they were connected to voicemail while planning how they were going to lie to explain way their fault on a prolonged outage impacting several of our customers.

Fast forward to today, I ordered Century Link Gigabit to be delivered to my house.  I had read many nightmare stories about this on Nextdoor but figured I’d go the lower risk route and order it online where I could have a paper trail, I tend to never sign up for a contract sold by a solicitor that knocks on my door.  The order went smoothly online, and amazingly they were able to install in less than a week later.  The tech arrived at the beginning of the instal window and spent much of the day running the fiber around our house to the only possible entry point.

What didn’t go well is that Century Link forces you to either buy or lease a “modem”, which is their name for a really crappy router.  The only thing special this “modem” does is it supports VLAN tagging on the WAN interface.  This router offers WiFi, but it only supports 802.11n at the fastest…you are reading correctly, you are required to buy a router that has a max wireless rate of around 100 megabit in order to buy gigabit service.

I had found a few blog posts online hinting at how to bypass their router by putting into “transparent bridge” mode, but I didn’t see any reason to even power this crappy device.  The tech hadn’t even finished cleaning up outside before I had converted back to using my Asus router, my 4-year old Asus readily blows away this brand new required POS.

How did I do it?  Its not so bad, there are a few blogs that you’d have to go to get all of the hints but they all leave out how to get the full thing working.  I was able to get better service using my own router than using the one provided, especially when you include IPv6 in the comparison.

TL;DR start here

I’m not going to include screen shots of all of the steps, as I would like to believe that anyone tackling this can figure it out from the high level steps (and I am too lazy to turn the CL router back on in order to document it).  In my case the CenturyLink 2100T  ZyXEL C1100Z was what was “sold” to me against my wishes.

I assume you know what cables to plug into where on your router and that you know you would need to move the WAN link that comes from the ONT from the Century Link router to your own, so I won’t include that detail here.  

I have Internet *only*, if you are also subscribing to PrismTV there may be additional settings required.

Collect PPPoE Details

  1. Login to the web interface of your Century Link router
  2. Skip to the advanced configuration section
  3. Find the remote management portion, enable telnet (likely the only time you will ever hear/see me suggest to use telnet) and set a password
  4. Telnet to your router IP (likely 192.168.0.1) and login as admin with your set password
  5. Type:
    sh
  6. Press enter, you are now in a  busybox shell.
  7. Run the command:
    /usr/bin/pidstat -l -C pppd
  8. You will get an output string that includes the runtime values being used too configure PPPoE, the parts you care about will look something like this:
    pppd -u lastfirst@qwest.net -p TXlQYXNzd29yZAo= -f 0 -M 1492 -D 0 -n 1 -L 0 -e 1 -X 120
  9. You just need to capture username and the encoded password, the username is the “lastnamefirstname@qwest.net” string and the password is the string after the -p, “TXlQYXNzd29yZAo=” in my example (be sure to include the entire string, including the equal sign as in my example)
  10. You can perform the next step natively on a Mac or you would need to use Linux, I use a Mac so it is easy.  Open a terminal window (aka shell) and run the following command to decide the password:
    echo TXlQYXNzd29yZAo= | base64 --decode
  11. You should get a decoded password back, like this:
    ~# echo TXlQYXNzd29yZAo= | base64 --decode
    
    MyPassword

Congratulations, you now have the PPP info to configure your personal router.  You can proceed to configuring PPPoE on your router WAN link, the only other thing you need to know is that you must tag the WAN with VLAN 201.  On my router’s 3rd party firmware this is under the settings for IPTV.

Now you just need to configure your router, I will include screen shots to help you on this portion.  Your settings may be called something different than what is shown, but there should be a functional equivalent.  If you do not have the ability to configure VLANs on your router you have two options, installed 3rd party firmware or just accept using the Century Link router in “transparent bridge mode” (as set on the WAN configuration under protocol settings).

Configure Your Router

On my Asus this is what I configured (obviously without quotes):

  1. WAN Connection Type: “PPPoE”
  2. PPPoE & MAN access: “DHCP or Static”
  3. Get MAN IP Automatically: “Enabled”
  4. PPP VPN Client Settings (PPPoE settings):
    1. Username: “lastnamefirstname@qwest.net”
    2. Password:  “MyPassword”
    3. Authentication Algorithm: “Auto”
    4. MTU: “1492”
    5. MRU: “1492”asus-pppoe-settings
  5. Ports Isolation and VLAN Filtering:
    1. Choose IPTV STB Port: “No”
    2. VLAN Tagged Traffic Filter: “Enabled”
    3. VLAN CPU (Internet): VID “201”, PRIO “0”
    4. VLAN CPU (IPTV):  defaults
      asus-vlan-settings

That should get you up and running on the Internet, however I wanted IPv6 support as I use it for some work projects.

Configure IPv6

I tried to guess at this but realized the best plan was to reconnect the Century Link router, go into the advanced settings and enable the IPv6 network features and capture the details for re-use.  I don’t know how generic these values are, some of them could be region specific or they may use any cast addresses allowing them to be universal.  Based on the Century Link support pages I assume these are universal.

Asus IPv6.png

You may need to reconnect your clients so that they get new DHCP info after making these changes, if you use static IPs on your workstations you will need to do your own magic to get them to also work with IPv6.  I use static IPv4 addresses on some devices, but just leave IPv6 configured for DHCP.

After making these changes I am able to score 19/20 on the IPv6 test, only lacking inverse DNS which I can’t do much about.  I did have to also enable “Respond Ping Request from WAN” on the firewall pages, as IPv6 requires more ICMP control messages than IPv4.

IPv6 Test Results.png

If you hit a wall you can drop a comment and I’ll try to fill in any details I missed.  If I end up swapping to a different router (e.g. something running pfSense) I will post an update, but the settings should be the same regardless it is just a matter of translating them to a specific configuration nomenclature.

VMware VCNS/NSX Edge Gateway SSLVPN and El Capitan 10.11.1

It is sad that most of my blog posts are related to an SSLVPN client and not my area of expertise, storage.  The previous SSLVPN client post is by far the most popular post on my blog, so here is to another post that will hopefully help someone.  My previous post on the SSLVPN client can be found here.

I had previously been running the NAclient from Edge version 6.1.3 on OS X 10.11 without issue, however the installation will no longer work due to security controls put into place by Apple.  There is a trend here, the previous problem with getting the NAclient installed onto OS X 10.10 (Yosemite) was also due to failure to meet Apple security guidelines, they didn’t implement code signing (amongst using deprecated methods of configuring/starting services).

If you attempt to install the NAclient on a current version of Apple OS X 10.11.1 you will see a lovely error like this one:  “This package is incompatible with this version of OS X and may fail to install.”
Install_VMware_SSL_VPN-Plus_Client_Installer

 

And if you select to Install Anyway it will just fail with the error: “The installation failed.  The installer encountered an error that cause the installation to fail.  Contact the software manufacturer for assistance.”Install_VMware_SSL_VPN-Plus_Client_Installer

It took some digging to figure out what is going on, and I used one of my favorite tools (Pacifist) to open the actual installer package to see where the installation files are written.

Archive_bom

I also remember reading about Apple’s System Integrity Protection locking down access to secure system file locations, including /usr, /System, and others.

In researching this I found Apple’s developer guidelines for System Integrity Protection, and it clearly states that /usr is off limits.

https___developer_apple_com_library_prerelease_ios_documentation_Security_Conceptual_System_Integrity_Protection_Guide_System_Integrity_Protection_Guide_pdf

Source: Apple System Integrity Protection Guide

Now that the root problem is found, I sought a work around.  Previously we had to disable the check for kext signing, though that didn’t permanently solve the issue because the dependent services wouldn’t work after reboot due to use of deprecated methods.  This time I was able to find that you can manage the System Integrity Protection using the command csrutil:

OS_X_10_11_-_Clean_install

In order to manage System Integrity Protection you must reboot your system into Recovery Mode, you can do this by holding down Command+R on system startup.  You should boot to a OS X Utilities menu.  Click Utilities and select to open Terminal.
OS_X_10_11_-_Clean_install_and_untitled_2

Within terminal you simply need to run ‘csrutil disable’ and then reboot the system, the easiest way to do this is with a single command line of ‘csrutil disable; reboot’.
OS_X_10_11_-_Clean_install_and_untitled_2

 

Once the system is booted up you can  now successfully install the NAclient, test that it works.  At this point you can repeat the process and run ‘csrutil enable; reboot’ to turn System Integrity Protection back on…which I strongly encourage.  It is disappointing that we have to disable security controls to install a security tool, however at least we only have to have it disabled upon install.  Any actual package upgrades would require you to repeat this process, but lets face it…VMware hasn’t released an update to this client since the last time I reported a bug in it 😉

Happy VPN’ing.

—– Follow up 11/24/2015 —-

It is becoming obvious that VMware doesn’t take client support for SSLVPN seriously, I will be investigating options to migrate our environments off of SSLVPN within the Edge Gateway.  It is entirely idiotic that you must deploy an entire new version (that isn’t even released yet) in order to fix a simple client installer bug.  That means you have to wait for the new version of the NSX stack to be qualified and supported by GSS with any of your other software components (e.g. vCloud Director), which isn’t an immediate process.  This leaves customers exposed for a long period of time, it increases operational burden for support teams and in general is a horrible way of delivering client software.

Client software should be released and maintained independently of the entire NSX Manager/Edge Gateway bundle.  Requiring every customer to do a massive upgrade of their entire environment just to fix a bug (that they should have known about months ago and should have been included in a 6.1.x release!) in client software is not acceptable.

— Edit 11/30/2015 —

An additional challenge here is if you disable the work around (have System Integrity Protection enabled), as you should, it will result in the client breaking if you run an installer again (say to add another VPN profile).  If you need to add another VPN destination you really should just edit the config file /opt/sslvpn-plus/naclient/naclient.conf directly with the editor of your choice.

— Update 12/16/2015 —

The work around allows the naclient to function on OS X 10.11.2 as well.  If you have the client installed prior to installing 10.11.2 update it will remain working, the work around will also preserve the ability to install after 10.11.2.

VMware vCloud Network & Security Edge – SSLVPN and Mountain Lion Troubles


October 12 2016 Update – Yosemite & El Capitan:

Wow, its been 3 years since posting this thing and it still gets quite a few hits.  The problem did get worse with Yosemite due to required code signing, however VMware corrected the problem with the naclient that was bundled in NSX 6.1.3.  If you have the naclient installed before upgrading to El Capitan it also works, in my limited testing.  I have heard that trying to install it on El Capitan may encounter issues due to a similar version table as noted below, I have not had a chance to test it on clean install and only tested for Yosemite to El Capitan upgrades.


 

 

The addition of client oriented VPN to the vCNS “Edge” (formerly vShield Edge) is a big win, however anyone that attempts to use the product on the current shipping version of Mac OS X will find that it fails to install.  We are using the SSLVPN heavily for a project and encountered this, I decided to dig into the details.

Within the OSX system logs you will find lots of useless errors, ultimately you want to get to the installer errors themselves.  If you open Console.app and look at the /var/log/install.log (or do so from CLI) you will see this error:

installd[4110]: PackageKit: —– Begin install —–
installd[4110]: PackageKit: request=PKInstallRequest <1 packages, destination=/>
installd[4110]: PackageKit: packages=(
“PKJaguarPackage <file://localhost/Volumes/BigFast/Downloads/naclient.pkg>”
)
installd[4110]: PackageKit: Extracting file://localhost/Volumes/BigFast/Downloads/naclient.pkg (destination=/var/folders/zz/zyxvpxvq6csfxvn_n0000000000000/Cleanup At Startup/PKInstallSandboxManager/1.sandbox/Root, uid=0)
installd[4110]: PackageKit: prevent user idle system sleep
installd[4110]: PackageKit: suspending backupd
installd[4110]: PackageKit: opt/sslvpn-plus/naclient/naclient.app relocated to Applications/naclient.app
installd[4110]: PackageKit: Executing script “./preinstall” in /Volumes/BigFast/Downloads/naclient.pkg/Contents/Resources
install_monitor[4115]: Temporarily excluding: /Applications, /Library, /System, /bin, /private, /sbin, /usr
install_monitor[4115]: Re-included: /Applications, /Library, /System, /bin, /private, /sbin, /usr
installd[4110]: PackageKit: releasing backupd
installd[4110]: PackageKit: allow user idle system sleep
installd[4110]: PackageKit: Install Failed: Error Domain=PKInstallErrorDomain Code=112 “An error occurred while running scripts from the package “naclient.pkg”.” UserInfo=0x7fc30b425a80 {NSFilePath=./preinstall, NSURL=file://localhost/Volumes/BigFast/Downloads/naclient.pkg, PKInstallPackageIdentifier=com.vmware.sslvpn, NSLocalizedDescription=An error occurred while running scripts from the package “naclient.pkg”.} {
NSFilePath = “./preinstall”;
NSLocalizedDescription = “An error occurred while running scripts from the package \U201cnaclient.pkg\U201d.”;
NSURL = “file://localhost/Volumes/BigFast/Downloads/naclient.pkg”;
PKInstallPackageIdentifier = “com.vmware.sslvpn”;
}
Installer[4097]: install:didFailWithError:Error Domain=PKInstallErrorDomain Code=112 “An error occurred while running scripts from the package “naclient.pkg”.” UserInfo=0x7f9c8536ce10 {NSFilePath=./preinstall, NSURL=file://localhost/Volumes/BigFast/Downloads/naclient.pkg, PKInstallPackageIdentifier=com.vmware.sslvpn, NSLocalizedDescription=An error occurred while running scripts from the package “naclient.pkg”.}
Installer[4097]: Install failed: The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.
Installer[4097]: IFDInstallController 83028370 state = 7
Installer[4097]: Displaying ‘Install Failed’ UI.
Installer[4097]: ‘Install Failed’ UI displayed message:’The Installer encountered an error that caused the installation to fail. Contact the software manufacturer for assistance.’.
installd[4110]: installd: Exiting.

This error is really not useful, but by looking within the installer package itself I could see that it is using /tmp/naclient_install.log for the install scripts themselves.  Within this log there is a bit more clue as to why it failed:

/tmp/naclient.pkg/Contents/Resources/preinstall: kernel version mismatch
/Volumes/BigFast/Downloads/naclient.pkg/Contents/Resources/preinstall: kernel version mismatch
/Volumes/BigFast/Downloads/naclient.pkg/Contents/Resources/preinstall: kernel version mismatch

In order to fix this you need to define the Mountain Lion kernel as being valid.  To do this, instead of installing the SSL VPN client from the web interface select to download the zipped file.

Extract the contents of the file and you will have a “naclient.pkg” file.  Like many “files” on OSX, this is actually just a special directory…you can either access the contents via CLI or right-click (or Ctrl-Click) and select to “Show Package Contents”.

If we look at the installation scripts themselves (with the arrows above) we find that the scripts are running a uname command to determine OS version:  uname -r | cut -d. -f1

We can also see that they were nice enough to support all the way back to Panther (released in 2003) but that there is no definition for Mountain Lion.

If we execute this command on Mountain Lion the response is “12”, however “12” is not defined as a valid kernel version.  The reality is that Mountain Lion is close enough for most apps to be considered “Lion”, so we will add this definition just the same as for Lion itself.

We will edit the 4 files that are indicated with the arrows, these are shell scripts and you can edit them with your text editor of choice, all 4 files need to be edited exactly the same just adding a definition for Mountain Lion.

Save your changes to all four files including “postinstall”, “postupgrade”, “preinstall”, and “preupgrade”.

Browse up the directory structure until you see the naclient.pkg and run the installer again.

***** Yosemite Update ******

Any of you that have upgraded to Yosemite may find that you cannot connect to the VPN afterword, it fails to establish a connection with an error somewhat like this:

SSL_VPN-Plus_Client_-_Login

In order to fix this here are the steps I took (PROCEED AT YOUR OWN RISK):

  1. Unistall NAclient:  sudo /opt/sslvpn-plus/naclient/uninstall.sh
  2. Enabled developer mode for Kext insertion:  sudo nvram boot-args=”kext-dev-mode=1″
  3. Rebooted
  4. Installed the NAclient again

I owe thanks to @jakerobinson for this as he actually found the solution.

***** Yosemite Update 01-07-2014 ******

Unfortunately it is not possible to get the naclient to run in any reliable fashion on Yosemite.  I have spent a lot of time on this and ended up using a Mavericks VM in Fusion to get the client to work for the day job.

naclient is dependent up on some kexts to load at system boot, however the method invoked to start these has been deprecated for multiple major releases of OS X and were removed in Yosemite.  The problem extends beyond the lack of signing, it is another example of VMware failing to support OS X even as the company issues Apple systems to a large number of employees and all new systems come with Yosemite pre-installed.

I will try to find time to write up my work around, it uses a VM but allows me to use that VM as a very heavy VPN client but I am able to use my (limited) apps in Yosemite as I normally would.

***** El Capitan 10.11.1+ Update 11-19-2015 ******

Rather than keep adding content to this post, I created  new blog post with the work around for OS X El Capitan and it can be found here.