EdgeRouter 4: routing, VLANs and banging one’s head against the wall

I spent most of my Labour Day trying to accomplish two tasks with an EdgeRouter 4 and the other miscellaneous networking gear in the house: setting up a simple VLAN and getting my backup DSL connection working.

Two WANs and a LAN

With two WAN connections (one DHCP/cable, one PPPoE/DSL), I wanted to have specific local network ranges send traffic out to (and receive forwarded traffic from) a specific WAN connection. Note that this isn’t quite the load balancing feature (which I don’t want), but moreso “IP range A uses cable, IP range B uses DSL”. I went through the gauntlet of EdgeRouter support articles and forum posts without much success:

I haven’t yet solved the problem, but I believe the issue is related to the PPPoE connection not injecting default routes into the main table (hence the need for policy-based routing), plus my second SNAT rule didn’t seem to match traffic. The PPPoE connection has a very volatile dynamic IP address, so source NATing based on address translation rather than masquerade wouldn’t work.

In any event, I’m sure this will be another weekend problem, but it was compounded by…

Continue reading

Windows file share and NTFS permissions

For future reference when I inevitably forget whether it is more appropriate to restrict folders with NTFS permissions (Security tab) or file share permissions (Sharing tab).

https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2008-R2-and-2008/cc754178(v%3dws.10)

“For example, some experienced administrators prefer always to set share permissions to Full Control for Everyone, and to rely entirely on NTFS permissions to restrict access.”

Relevant table of examples:

Folder type Share permissions NTFS permissions
Public folder. A folder that can be accessed by everyone. Grant Change permission to the Users group. Grant Modify permission to the Users group.
Drop folder. A folder where users can drop confidential reports or homework assignments that only the group manager or instructor can read. Grant the Change permission to the Users group.

Grant the Full Control permission to the group manager.

Grant the Write permission for the users’ group that is applied to This Folder only. (This is an option available on the Advanced page.)

If each user needs to have certain permissions to the files that he or she dropped, you can create a permission entry for the Creator Owner well-known security identifier (SID) and apply it to Subfolder and files only. For example, you can grant the Read and Write permission to the Creator Owner SID on the drop folder and apply it to all subfolders and files. This grants the user who dropped or created the file (the Creator Owner) the ability to read and write to the file. The Creator Owner can then access the file through the Run command using \\ServerName\DropFolder\FileName.

Grant the Full Control permission for the group manager.
Application folder. A folder containing applications that can be run over the network. Grant Read permission for the Users group. Grant Read, Read and Execute, and List Folder Content permissions to the Users group.
Home folders. Individual folders for each user. Only the user has access to the folder. Grant the Full Control permission to each user on their respective folder. Grant the Full Control permission to each user for their respective folder.

Exchange 2016 + Outlook on iOS and Android: Message size limits and their configuration

Users with the official Microsoft Outlook client on Android or iOS kept running into ~36MB size limits when attempting to send attachments (given the megapixel sizes of most cell phone photos, this can amount to 3 to 4 pictures attached and the whole message is rejected), and none of the conventional transport/mailbox maximum size settings were the cause. I’m hoping the changes in the following articles are the fix:

The settings I specifically believe are responsible are:

  • maxAllowedContentLength in %ExchangeInstallPath%FrontEnd\HttpProxy\ews\web.config
  • maxAllowedContentLength and maxReceivedMessageSize in %ExchangeInstallPath%ClientAccess\exchweb\ews\web.config
  • maxAllowedContentLength and maxRequestLength in %ExchangeInstallPath%FrontEnd\HttpProxy\owa\web.config
  • maxAllowedContentLength, maxRequestLength and maxReceivedMessageSize in %ExchangeInstallPath%ClientAccess\Owa\web.config

Cruise experiences: NCL Epic, December 2017

Neither our first trip on the Norwegian Epic nor subsequent one on the Breakaway in December 2016 scared us off of cruising, so here are some ramblings about the NCL Epic sailing an Eastern Western Caribbean itinerary in December 2017.

Eastern… no, wait, Western

We never made it here.

Due to the hurricanes (Irma and Maria) that absolutely crushed Eastern Caribbean regions in September 2017, nearly every itinerary featuring these destinations was adjusted, regardless of cruise line. As we were only a few days away from the 90-day deadline where one can cancel for a full refund, I was closely monitoring the situation. There wasn’t any coherent news regarding NCL’s plans, mainly because their Miami headquarters was also in a state of disarray around that time.

Cruises sailing to these places in September were definitely being cancelled, cut short, or adjusted, but my theory was that by mid-December, the various Virgin Islands would be up and running again. Maybe the entire region would be worse for wear, but at least the touristy areas where ships drop off several thousand passengers would be up and running. Frankly, the best thing a tourist can do in one of these situations is to continue to visit, and spend hard-earned (or easily-earned) currency with the locals. So it was with that theory in mind that I decided not to adjust our plans.

Unfortunately a swift recovery wasn’t the case. Once NCL got things somewhat settled, they made rumblings about a possible itinerary adjustment 88 days prior to the cruise date, which was just enough time to incur a 25% penalty to switch. The replacement itinerary was officially announced at exactly 75 days out (coinciding with a 50% cancel/change fee.) Not being a common idiot, I knew that we were likely to end up at Falmouth and Grand Cayman as replacement ports, but really didn’t want another $300 in airfare changes or to have to sort out transportation from Orlando/Port Canaveral to a different port. So, we stuck with the Epic and the revised route.

Oh, what’s that you say? Shouldn’t the cruise line have to do something – I mean, they’ve changed two-thirds of the ports on your vacation – harrumph harrumph? I direct you to NCL’s guest ticket contract that basically says they don’t even have to put you on a ship (6b), and they don’t have to stick to the itinerary (6c). Also in the same section, you release NCL from any loss/damage/injury due to piracy, among other egregious things, so don’t expect compensation for any Captain Phillips experience.

Continue reading

Manage your MR10i or other LSI MegaRAID controller on a ESXi 6.5 host

I’ve been arguing with an Exchange 2016 server lately, due to what I suspect is a dodgy IBM-badged MR10i RAID controller in a x3650 M3. It has been kicking disks that seem entirely fine out of RAID1 volumes, which effectively has the same side effect as losing a disk. I intend to publish a few posts with some of the links and practices I’ve used lately.

Original article: How to install LSI MegaRAID Storage Manager (MSM) on ESXi 5.5

The original, excellent instructions from Mike Smith at Serenity-Networks, despite being for ESXi 5.5, seemed to work with some minor adaptations for ESXi 6.5.0 Update 1 (Build 5969303), with the latest versions of software from Avago (Broadcom).

Enable SSH on ESXi host: From the web UI, in the Navigator column, select Host, then choose Actions > Enable Secure Shell (SSH):

Adjust the “acceptance level” to allow installation of unsigned VIB files: In the Navigator column, select Manage, then select the Security & users tab. Then, click the Edit settings button and choose Community.

Get the LSI downloads (SMIS provider and the latest MegaRAID MSM): I found filtering by OEM did not successfully show results. On the Broadcom website, I selected the following categories for download:

  • Group: Storage Controllers, Adapters and ICs
  • Family: Storage Controllers, Adapters and ICs
  • OEM: (left blank – showed up as ‘OEM’ in the search interface)
  • Product: All
  • Asset Type: Management Software and Tools

There were 679 results; I used Ctrl+F and searched for “SMIS”, which offered a link titled “Latest SMIS Providers” for VMWare 6.0 and 6.5: https://docs.broadcom.com/docs/VMware_MR_SAS_Providers-00.67.V0.04.zip

Then I also used Ctrl+F and searched for “MegaRAID Storage Manager”, which offered MSM for a variety of platforms:

Copy the LSI SMIS provider (the file with .vib extension) to the /tmp directory on ESXi host (scp/WinSCP/your client of choice). I found that my sneaky attempt at copying it to a shared volume at /vmfs/volumes/… was hit and miss; when it was a fibre channel mount, the install worked properly, but if the datastore was on a local disk, it died with an error message.

SSH to the ESXi host with appropriate credentials (I did everything as root) and run the following install command:

esxcli software vib install -v /tmp/vmware-esx-provider-lsiprovider.vib --no-sig-check

I also had to disable the firewall on the ESXi host. Bad practice, but I don’t have a list of the specific ports to open at present.

esxcli network firewall set --enabled false

Reboot the ESXi host when complete. You can and probably should do the usual behaviour of taking it into maintenance mode, but in my case everything shut down and came up cleanly as VMWare Tools was installed on each guest.

It was hit and miss as to whether I had to add the line from /etc/hosts on the ESXi server with the hostname to my Windows box. I found that eventually creating both A and PTR records in Active Directory DNS, combined with turning off the ESXi firewall, were sufficient to get the MSM client on a domain-joined Windows server to connect – not even necessarily a guest VM on the same hypervisor.

I also had to change the MSM client settings in the Configure Host dialog to “Display all of the systems in the network of local server”, and not the “ESXi-CIMOM” option:

Fixing WSUS – error 507: “Update services failed its initialization and stopped”

I had a Windows Server Update Services installation that after a reboot, failed to start the WSUS service with a fairly generic error message. Clients issued an “unable to check for updates” message with an 8-character hex error code, differing depending on the client OS.

To fix it, I followed the directions in this source article on vcloudnine.de: WSUS on Windows 2012 (R2) and KB3159706 – WSUS console fails to connect

  • Run elevated Command Prompt and issue the following command:"C:\Program Files\Update Services\Tools\wsusutil.exe" postinstall /servicing
  • Ensure the “HTTP Activation” feature is installed, using Server Manager > Add Roles and Features > Features > .NET Framework 4.5 Features. In my case, it was already installed.
  • Restart “WSUS Service” from services.msc

First cruise review: Norwegian Epic March 1, 2015 – Western Caribbean

This has been sitting in my drafts folder since mid-2015, so the information in it re: UBP charges, menu contents, etc. are all outdated at this point, but I think it’s still a good representation of how our first cruise went. Suffice it to say, I need to be more timely with these – we’ve since gone on a Bahamas cruise on the NCL Breakaway in December 2016, and have a different Caribbean cruise planned on the Epic during December 2017. So even if there are some negatives here, clearly we’ve gone back so it can’t be that bad.

I’ll try and note in the article where things have changed.

In Which We Decide To Cruise

In 2015, Kayla and I got tired of the snow and wind and ridiculously cold temperatures in Southwestern Ontario, and decided to get away to a warmer climate. We’d been to BlueBay Villas Doradas in the Dominican Republic to do the all-inclusive resort trip with a few friends last year and while we liked it, we wanted to try something new before falling back to the same thing. We’d also discussed various Sunwing-promoted destinations flying directly out of YKF to Mexico, but the available resorts in our price range were either too new to have a decent amount of feedback, or had recently begun “focusing on a new concept.”

Then the idea of taking a cruise came up and we started looking. A few years ago in a hotel room in Prague, I’d seen a documentary that was pretty much an expose of the entire cruise ship industry. I managed to locate it after returning – it’s CNBC’s “Cruise Inc: Big Money on the High Seas” (2009).  It went over a cruise on the Norwegian Pearl, describing how passengers are basically just walking ATMs and that the cruise line is constantly running the numbers on every aspect of shipboard operations. The conclusion was that on the last sea day, the operator broke even for the cruise, in no small part to making up $21K in alcohol sales. Being shaken up and down for cash constantly didn’t really appeal to me.

Continue reading

Fix: WSUS Server Cleanup Wizard hangs/stalls when deleting unused updates

Side note: several years ago Kayla caught me talking in my sleep, muttering something about “you’ve got to check the boxes!” This is the actual dialog and process in question.

Full credit to Jeremy Jameson at MSDN. Posting in case the original disappears.

  • Run Server Cleanup Wizard with only the “Unused updates and update revisions” (option #1) box checked. This took about six hours on the server experiencing the problem:screen-shot-2016-11-22-at-11-32-29-am
  • Once finished, run the wizard again with only the “Unneeded update files” (option #3) box checked.
  • Once that’s finished, run the wizard with all the boxes checked.

RiteBite and Invisalign Review: Conclusion

Well, better late than never, but I’m currently in the process of cleaning up paperwork in the home office, and noted that RiteBite had given me a flyer asking for a Google review several months ago. So here’s a conclusion to the review series, which will be combined with the other content and sliced into bits and pieces for the less-verbose social media pages.

Completing the Program

Since last time I wrote, I went through about half of another series of trays with 7-day rotations. I specifically requested to have my treatment wrapped up about a week before my wedding in August 2016, and Dr. Luis and staff were very accommodating since this third set was effectively “finishing touches”. As part of the removal, I had permanent wiring bonded behind both my top and bottom teeth and was given a set of top and bottom harder, clear plastic retainers to wear overnight. One important point is that for the first two weeks, you’re expected to wear the retainers as close to 24/7 as possible, so you’re not “entirely” done. I obviously made an exception to this for the wedding.

A Few Nitpicks

The retainers are not ideal, to put a point on it. Their larger size (compared to the Invisalign trays) and increased rigidity triggers my gag reflex nearly every morning when taking them out, and I still run into similar problems with drooling on my pillow.

I also specifically requested the top permanent wire, and had to ask several times before getting a “yes” – several staff suggested that it wasn’t strictly necessary or had a higher chance of breaking. I wanted to ensure that with my financial investment, there was a “backup” in place to help the teeth from moving as much. The top wire’s presence is still noticeable when I close my mouth several months later, unlike the bottom wire. Both still have a distinct “pebbled” texture where the wire is adhered to the back of each row of teeth.

Despite asking for Google reviews as part of the “exit interview”, RiteBite seems to have several accounts under their name on Google Plus (1, 2, 3, 4, 5) and no link to the official Google profile from their website, nor any content on these pages. I was also disappointed to find that the Case Graphics section has disappeared from my profile since completing treatment.

Overall Results

The change has been quite impressive. It took slightly over a year and a half, I wasn’t seriously inconvenienced, and now that it has been paid off, I begrudgingly admit that it was probably a better personal choice than replacing the laminate flooring in the house or buying the same amount of networking gear.

Continuing the “Router rumble” with pfSense 2.3.2 and a FW-7540

Following up from my previous round of router testing, I managed to get a spare Lanner FW-7540 with an Intel Atom D525 CPU to test how my current pfSense 2.3.2 setup compared to an EdgeRouter Lite. The results were well below what I was expecting: the pfSense box topped out at 490Mbit in the 1MB test and was very spiky when looking at the netdata graphs.

The results file is also available if you’d like to look directly at the ab output.

d525_pfsense

Filesize Average Mbit/s Total Failed Requests Notes
10K 145.07 87 10K concurrency test only resulted in 49Mbit. No failed requests in 10, 100 and 1000 concurrency tests.
100K 421.71 4896 No failed requests in 10, 100 and 1000 concurrency tests.
1MB 489.96 3341 No failed requests in 10, 100 and 1000 concurrency tests.

This test fairly obviously shows a ceiling. For WAN connections of over 500Mbit, it looks like something beefier than an Atom D525 is necessary to run the NAT as anticipated.

I also ran some more informal WAN to LAN iPerf3 testing on direct connection (MDI-X), the EdgeRouter Lite and the pfSense/7540 combination to get some synthetic numbers:

Connection iPerf Result
Direct 941Mbit with no retries
EdgeRouter Lite 939Mbit with retries
pfSense/7540 829Mbit with no retries

Given how well the EdgeRouter Lite seems to perform for its price, and since it beats out the more general purpose hardware, I suspect I will be swapping out for an ERL or ER-Pro very shortly.