Fix forwarding to Gmail with a Linode Postfix/Dovecot mail server

Recently I decided to rebuild our main Debian Squeeze host as a 64-bit Debian Wheezy (7.0; I believe the template is 7.3 as of the time of this writing) VPS. This box runs web hosting, email, internal IRC, shell access and basically any other services that one of our beloved sudoers would like to try. Both of these hosts live in Linode’s Newark datacenter.

Linode will pro-rate your account if you cancel a server in the middle of a month, so both instances (old and new) are currently running for a minimal net cost. You can also assign private IPs to each host and SCP data or mount NFS between servers without cutting into your bandwidth quota, as well as attain a slight transfer speed improvement. This gives us plenty of time to move finicky services and make sure that the new configuration is working as intended. We’ve been cutting over individual user websites and mail services one by one to lessen the impact.

One of the problems I ran into that did not exist on the old host was email forwarding. We use the Linode Library: Email with Postfix, Dovecot, and MySQL article as a basis for a mail server that supports multiple domains, IMAP mailboxes and aliases that forward to multiple accounts. Most of our traffic is forwarding operations, typically to multiple users at once as sort of a poor-man’s distribution list. The main forwarder is an internal alias for Slightly Sauced group discussions, which sends messages out to everyone’s preferred mail provider.

Initially there didn’t seem to be any problems with the mail setup. I use Exchange Online for my personal email and messages were coming and going properly to me. When I went to reply to an existing thread, I soon got a bounceback message from for only a few email recipients the list. Checking /var/log/mail.log, I found the following lines (truncated for brevity and sanitized to not mention any specific email addresses):


Feb 26 21:58:24 services02 postfix/qmgr[28676]: 2F6FA6AB68: from=, size=31176, nrcpt=9 (queue active)
Feb 26 21:58:24 services02 postfix/smtp[10250]: 2F6FA6AB68: to=, orig_to=, relay=ASPMX.L.GOOGLE.com[2607:f8b0:400d:c04::1b]:25, delay=8.6, delays=8/0.01/0.13/0.46, dsn=2.0.0, status=sent (250 2.0.0 OK 1393451904 g88si838453qgf.126 - gsmtp)
Feb 26 21:58:24 services02 postfix/smtp[10253]: 2F6FA6AB68: to=, orig_to=, relay=mx1.sub4.homie.mail.dreamhost.com[208.97.132.226]:25, delay=8.7, delays=8/0.02/0.24/0.39, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 84D4B76807B)
Feb 26 21:58:24 services02 postfix/smtp[10251]: 2F6FA6AB68: to=, orig_to=, relay=gmail-smtp-in.l.google.com[2607:f8b0:400d:c00::1a]:25, delay=8.7, delays=8/0.01/0.06/0.58, dsn=5.7.1, status=bounced (host gmail-smtp-in.l.google.com[2607:f8b0:400d:c00::1a] said: 550-5.7.1 [2600:3c03::f03c:91ff:fe6e:423f 12] Our system has detected that 550-5.7.1 this message is likely unsolicited mail. To reduce the amount of spam 550-5.7.1 sent to Gmail, this message has been blocked. Please visit 550-5.7.1 http://support.google.com/mail/bin/answer.py?hl=en&answer=188131 for 550 5.7.1 more information. r6si610718qcl.69 - gsmtp (in reply to end of DATA command))
Feb 26 21:58:26 services02 postfix/smtp[10252]: 2F6FA6AB68: to=, orig_to=, relay=example-com.mail.protection.outlook.com[207.46.163.215]:25, delay=11, delays=8/0.02/0.34/2.3, dsn=2.6.0, status=sent (250 2.6.0 <72c119e2692a422cbc733234ced8599a@SN2PR03MB046.namprd03.prod.outlook.com> [InternalId=49989124371267, Hostname=BY2PR03MB041.namprd03.prod.outlook.com] Queued mail for delivery)
Feb 26 21:58:26 services02 postfix/bounce[10256]: 2F6FA6AB68: sender non-delivery notification: D6C4C6AB90

The first thing I noticed is that the users on Dreamhost IMAP, Exchange Online, and Google Apps / Google Hosted accounts did not have the forwarded message rejected. Forwards going to straight @gmail.com addresses were immediately rejected. I reviewed the Google Support document without much luck, and then stumbled across some documentation from Tanguy Ortolo about Google’s IPv6-related email restrictions. While Tanguy’s workaround was a good one (force IPv4 connections to Google mail servers), I tried to solve the problem while maintaining IPv6 connectivity. Linode does let you set reverse DNS (PTR) records for both IPv4 and IPv6 addresses, as long as they forward-resolve correctly.

  • In the Linode control panel or your own DNS management system, establish A and AAAA records for the server that is transmitting email. (eg: mailserver01.example.com.) Wait for the records to be resolvable and check with the dig a $hostname; dig aaaa $hostname commands on an IPv6 enabled system.
  • In the Linode control panel, find the individual VPS and access Remote Access. In the Public IPs section, click Reverse DNS.
  • On the Reverse DNS page, look up the domain name (mailserver01.example.com) with the provided tool. If your A and AAAA records are present and functional, Linode will ask whether you would like to use mailserver01.example.com as the reverse DNS entry for both your IPv4 and IPv6 addresses. Click Yes to both options.

Within 24 hours, forwarded mail flow to Gmail accounts should begin working properly.

I also took this opportunity to update my SPF records as Exchange Online was fairly restrictive about certain types of messages that I sent through this server. I had to update my SPF record to: v=spf1 a mx include:spf.protection.outlook.com include:example.com ~all, where example.com was the domain of our Debian mail server that had its own SPF record.

Fixing stuttery or frozen USB on a Supermicro motherboard

For future reference and edification: on certain Supermicro motherboards running OSes of Windows 7 or newer (including the Server variants like 2008 R2 or 2012), USB keyboard and mouse devices will act ‘stuttery’ or freeze during input. The issue I encountered was specifically missing keystrokes or repeated letter presses, and appeared to be present even before the OS had booted.

A response in the Spiceworks community pointed me to the Supermicro FAQ, which states:

Question
In X9DRW-iF, USB Mouse can’t work under Winodws 7.

Answer
Please change “ISOC” setting from disable to enable in the BIOS menu for problem solving. (This item locates at Advanced / Chipset / NB / QPI Config –> ISOC)

To clarify, reboot the server and get into BIOS by pressing Delete at startup when prompted. “NB” refers to North Bridge.

Home networking overkill with a Lanner FW-7540

I’ve recently run into a few issues with my home networking setup. In pure overkill fashion, I’ve bought some new hardware to deal with it all and hopefully, in the process, learn a bit more about different network configurations.

One of my main problems at this point is related to location. After buying a house last year, I still have yet to make significant progress on the “Ethernet to every room” project. Wireless is great and has drastically improved since the early gear, but even the 802.11ac standard and equipment is no substitution for the reliability and consistent speed of a gigabit wired line. ac routers right now can push 180Mbps throughput at 1 meter, but quickly diminish based on additional distance, other devices and the wireless adapters involved in the whole fiasco.

For the wired setup, I have all of the means to complete the process – or at least think I do until moving to whatever the next phase of the process is. At that point there’s usually much cursing, an order or two to Monoprice, and even a trip to Home Depot. Over the year I’ve relocated my folding table of tech gear to the basement, and there’s already quite a convenient hole in the floor to run some wiring through. As a result, my main tech closet in the basement all runs Ethernet, and I’m less inclined to start sawing drywall and drilling holes to the second floor on a whim.

Another problem I was seeing was poor wireless and routing performance in general. I’ve had the Netgear WNDR3700 in place for about two years now, and it’s run both stock firmware and DD-WRT with various success. I’d highly recommend the router with stock firmware for most home configurations, but DD-WRT seems to occasionally stop sending and receiving traffic on the 5GHz wireless interface.

With a router replacement, there are three main components to be aware of:

  • Router/NAT device, to handle Internet connection traffic and route it to the corresponding internal client
  • Switching equipment – usually built in to the router, but additional capacity is generally needed down the line for more than four systems or avoiding lengthy cables
  • Wireless radio interface – again, usually built into the router

I decided to split this up a bit into its logical components. For the router/NAT device, my friend Matt sold me on a Lanner FW-7540, which is essentially a small-form-factor box with four Intel gigabit Ethernet ports and a dual-core Intel Atom CPU. The machine easily runs software like pfSense, which is a FreeBSD distribution with a Web interface and some configuration utilities on top. It’s incredible software and very powerful.

For switching equipment, I turned off DHCP on the Netgear router and am not using the WAN (Internet) port, turning it into a wireless access point plus four-port gigabit switch. I believe there is an option to reassign the WAN port to a LAN port, but I am not entirely lacking for ports near the cable modem at this point. Other locations in the house utilize 8-port Monoprice gigabit switches and that’s probably what I’d put in if the Netgear died or started acting up.

The last part of the equation is wireless access, and I’m waiting for the Ubiquiti UniFi AP AC to become reasonably commercially available. For now, I’m expecting a UniFi AP Pro to start. Even in a residential neighbourhood, I typically see upwards of a dozen networks in range and would like a more powerful, better-located access point to serve the systems here.

So, what have I learned about this setup?

Serial access to the Lanner console is a bit of a fun time. The device includes an RJ-45 to DB-9 serial adapter, so I had to hunt for which devices around the house had a serial port. You’ll also want to have a basic understanding of how serial terminals work.

Installing pfSense – when picking the kernel, select the option that is not symmetric multiprocessing, or you’ll lose console access on the first boot. Initial configuration for making the device behave like a usual router/switch involves not only setting up “OPT1″ and “OPT2″ interfaces to be bridged to the LAN, but configuring the built-in firewall to allow all traffic between them. I accidentally set the firewall allow rules to only let TCP traffic pass between the network interfaces, and that basically ruined functionality for anything plugged into ports 3 and 4 on the Lanner.

IP range selection is a good thing to plan out completely, especially if you’re a moron and pick the same range that your office uses to assign to VPN clients and a number of internal systems. Stick to low-numbered 192.168.x.y subnets to interfere with the least amount of connectivity, and select the appropriate netmask. I picked 10.0.0.0/8 and was in a world of hurt reconfiguring the network the next time I had to work from home.

Don’t dual DHCP or you’ll end up with what looks like periodic packet loss. Running a continuous ping to the router showed maybe two “Request timed out” results every twenty minutes or so. This interrupted music mounted from another computer as well as the Internet connection. Make sure all other DHCP servers are turned off or locked down appropriately!

(Messages in the pfSense logs for this condition look like repeated instances of the following block)

Apr 15 01:18:02 pfsense kernel: arp: 192.168.1.100 moved from 00:1b:21:b0:7e:bb
to 34:bb:1f:bb:0a:f8 on em1
Apr 15 01:18:15 pfsense kernel: arp: 192.168.1.100 moved from 34:bb:1f:bb:0a:f8
to 00:1b:21:b0:7e:bb on em1

Update 1: Useful sites that helped sort this out were:

And finally, have a UPS on all critical parts of the network path. They’re reasonably inexpensive and it’s nice to be able to still have Internet access during a power outage situation.

Fix issues signing in and updating apps from the Mac App Store

Problem: The Mac App Store on my laptop refused to allow me to update existing applications, download new ones, sign in to my account or view existing downloads. Trying the “Store > Sign In” and “Sign In” link from the Featured page both refused to display the usual login dialog. Attempting to update existing applications showed the usual “spinner” in the top toolbar with no progress.

Dead ends: Suggested on the Apple Discussion forums, there were several items suggesting anti-virus and firewall involvement. None of these were applicable to my situation and I was attempting all of these actions from an unrestricted TekSavvy cable connection.

Solution: This post on the Apple Discussion forums provided the initial help, but was incomplete in its solution. First, close out the App Store, then enable the debug menu by running

defaults write com.apple.appstore ShowDebugMenu -bool true

from the Terminal. Launch the App Store again, and choose Debug > Clear Cookies and Debug > Reset Application. Quit and relaunch the App Store, and you should be able to sign in and download updates successfully.

WordPress phpass generator: resetting or creating a new admin user

Again, in case I forget: If you’d like to reset a WordPress password from the database or create a new administrative user:

  1. Generate a PHPass hash using this mainframe8 tool.
  2. Insert a new row, or update an existing row, in the wp_users table. Use the hash from the tool in the user_pass column.
  3. If you’re adding a new administrator, insert the following values into wp_usermeta and replace user_id (2 in this example) with the newly created account’s ID:
    INSERT INTO wp_usermeta (`umeta_id` , `user_id` , `meta_key` , `meta_value`) VALUES
    (NULL , '2', 'wp_capabilities', 'a:1:{s:13:"administrator";b:1;}'),
    (NULL , '2', 'wp_user_level', '10');
  4. Enjoy a fixed WordPress admin account.

More scams: How you can get ripped off using PayPal

One of my most popular posts still seems to be “Apparently you can get scammed using PayPal and Gmail“, in which I received my first scam attempt from a Gmail address. The comments on this post typically are people who have avoided being scammed out of their goods, but a number of people recently are asking how the process actually works and why something’s a scam.

First of all, if the offer is too good to be true, there’s something wrong. Nobody in their right mind is going to willingly add an extra $100 upfront for shipping. People using Craigslist and Kijiji are universally cheap and will offer pennies on the dollar for your merchandise, or worse yet try to “trade” you. If you see an offer that comes in for more than you’re asking for, or adds an exorbitant amount for inconvenience, you’re getting greedy and stupid.

Second, nothing is ever final with PayPal. There are countless horror stories online, but let’s just be clear for people who think PayPal offers any seller peace of mind:

  • If you sell an item and use PayPal for the transaction, and don’t ship with a tracking number, the buyer can just claim they never received the merchandise. PayPal will always side with the buyer and return the funds from your account to theirs.
  • If you do use a tracking number, the buyer can lodge a complaint and claim that the item is not as described. Supposedly they are required to return the item to you, but PayPal will just return the funds to their account.
  • In the event of any issue with the transaction, the buyer always wins.

Oh, wait, you say. But what if I immediately withdraw the funds from my account once the buyer has paid me? Then PayPal can’t do anything?

Actually, they can. PayPal will put your account into a negative balance and any funds you add or receive will first go towards that negative amount. If you keep a negative balance for over 30 days, they will send you to collections and close your account. This will  eventually appear on your credit report and you will be constantly hounded to repay the negative balance. It doesn’t matter that you were scammed out of your money; you’ll have to dispute the payment and likely take the matter to court to have it resolved.

PayPal is not seller-friendly. The only reason people use it on eBay is that it’s the only allowed payment choice in many circumstances, and the high-volume sellers do a pretty good job of staying in PayPal’s good graces. If you’re selling 1000 items and 10 people defraud you, you still have a 99% success rate and it’s really just the cost of doing business. The same scenario doesn’t apply for a single seller selling a single item, especially if there are other signs of sketchy behaviour before the transaction.

Migrating back to WordPress with regex and ‘tr’

I decided that I didn’t really feel like trying to manually upgrade ChintzyCMS to the latest version, so my personal site is now back running WordPress. I wasn’t interested in migrating comments, so if you’ve said something witty in the past you’ll need to look forward for new material.

There were a few tools that helped in the PostgreSQL to MySQL and import process. I’m still a bit upset that WordPress doesn’t support Postgres natively, but such is life.

  • First, I used phppgadmin to export the posts table as XML, which gives a <column> and <row> style output from the table. I deleted the header and column description  tags from the beginning and end of the file.
  • Using the guide at WordPress Codex – Importing Content, I performed the following search and replace operations based on the Importing from [X]HTML instructions:
    • For each column tag that needed to be replaced, I searched for the regex
      <column name="title">([^<]*)</column>

      and replaced it with the appropriate tag pair, such as

      <title>\1</title>
    • For each column that didn’t have an equivalent match in the WordPress database, I searched for the same regex and replaced it with an empty string.
    • I saved the resulting file out as posts.xml.
    • Using the UNIX tr utility described here, I removed all newline characters in the file:
      tr -d '\n' < posts.xml > posts_no_newline.xml

I then was able to import the posts_no_newline.xml file using the WordPress RSS Importer plugin.

There are still some posts that contain remnants from an ASCII to UTF-8 conversion (the new MySQL database stores content in UTF-8) which I’ll likely fix programmatically; when that happens, I’ll update with how that was performed.

Marketing to douchebags: the Voss water experiment

Want to market a product to the average douchebag? First, pick a brand that gives him exclusivity – something purportedly elite but still found at the grocery store. Products like Axe, or services such as oxygen bars cater to segments of society like the “bro” in a wonderful way. If you can sell air through a tube or the idea of two chicks getting up in your grill over musky porpoise-hork cologne, I’m all for it.

One of the concepts along the same line I’ve never really considered is novelty bottled water. Both Kayla and I agree that the best possible type of water is freezing cold, straight out of a garden hose.

Sure, some people don’t want to drink tap water directly. Some cities chlorinate the crap out of it, and I’ve lived in two places with piping that dispensed more rust than anything drinkable. A Brita filter always seemed to resolve the taste. I’ve never seen the need for anything more than that. Bottled water is a nice option for camping, but my brand of choice is whatever’s on sale that week.

Along these lines, Kayla suggested an experiment this evening. She’d recently heard about Voss water from “some douchebag in a movie.” In complete contradiction with this remark, she then purchased a $2.50 bottle of Voss at Zehrs for some god-forsaken reason. Likely because it had a pretty bottle or something.

The point ended up being: could we tell the expensive water from the tap or Brita-filtered stuff, and did it subjectively taste better?

Voss water bottle

Here were the conditions:

  • We would try three kinds of water – tap, Brita and Voss – and try to guess which was which.
  • All water was served at the same temperature out of the same type of glass.
  • Each glass was labelled on the bottom using masking tape. The glasses were then three-card-Monty’d by the opposite participant while our back was turned.
  • Drink the water, guess the type. When all three guesses were in, turn over the cups and check what was in each.

Six glasses of water

The Results

Pretty anti-climactic, really. Whether it was just luck, both Kayla and I were easily able to guess the correct type of water in all three of our glasses within the first sip or two.

Tasting Notes

Oh, hell, you probably don’t want to read any of these. Voss was kind of flat, Brita was kind of mineral-y, and the tap water was kind of flavourful.

Conclusion

The cats tried all the choices and were more impressed that they could get their paws all the way to the bottom of the glasses, nearly knocking several of them off the table. I had some Growers 1927 Dry Premium Cider and enjoyed that much more than the water, and I’d much rather conduct a cider-tasting experiment next time. Kayla gave up interest in the experiment halfway through and turned to her 3DS and Zelda for entertainment.