Emails restored

Last week I noticed I had stopped getting emails about new comments on my blog posts. (And similarly no emails to commenters about new comments in their thread went out.) I'm not quite sure when this happened, but it was at least as early as October, since I got an email in June but didn't get the next comment email that happened in October. It's not unlikely that the problems started after I upgraded Ubuntu and my ec2 instance type.

Now that it's resolved I'm also not sure about the ordering of what went wrong or why. So I'll just describe all the issues and my fixes, maybe it'll help someone.

My blog runs on Amazon's EC2 and uses their SES to send emails. This requires verifying my domain, and the way they want to verify domains is by you adding a TXT entry in your DNS records. When I noticed my emails were dead, I went to the SES console and discovered my domain verification had somehow been expired. I went to my domain administration page, and sure enough the TXT entry I had since 2012 or so was still there. A key of "_amazonses.thejach.com" and a value of some b64 key. I told Amazon to reverify, and went on a business trip.

I came back, and it still hadn't verified. For whatever reason, my TXT entry no longer was being published! I use enom, by the way. Using the command "nslookup -q=TXT thejach.com" resulted in nothing. (If you run it now, you'll see the Amazon verification token.) Amazon's verification process mentions that some domain registrars don't like you putting underscores in the TXT key, so maybe enom changed their policy I thought and tried the alternate form (a key of "thejach.com"). Still nothing.

Enom has a special token "@" that happens to map to "thejach.com" (or whatever your domain is), and after trial and error I discovered that if I used that as the TXT key, it worked fine. Ok, so Amazon verified my domain again, a test email works, but my blog is still not sending email. What's going on now?

I use a really simple PHP script for sending email. At some point long ago I looked up how to set up PHP to use Amazon's SES under the hood through PHP's "mail()" function, so no PHP code needs to be the wiser. I found what I think was my source before, which was this: https://forums.aws.amazon.com/thread.jspa?threadID=72243&tstart=0

It's still mostly correct, I just edited my php.ini for apache. I totally forgot about the ses perl scripts and auth key sitting in my /opt/bin/ directory. So I set up the path, tried again, failed. :( Maybe my auth key was old and invalid? Nope, still valid. I looked at my php error logs and found the issue: the perl script couldn't load the SES module for some reason, despite SES.pm being in the same directory as the script. Maybe some Perl limitation but I'm not sure how I resolved it before. It's possible I just installed the module globally and the one sitting in /opt/bin/ is a red herring. But this time I just modified my perl script. Above the line with "use SES;" I added "use lib '/opt/bin';". That lets Perl know it can load modules from that folder.

After that, all is well, and email is now restored. What a pain.

---

Posted on 2017-01-06 by Jach

Tags: fodder

Permalink: https://www.thejach.com/view/id/331

Trackback URL: https://www.thejach.com/view/2017/1/emails_restored

Back to the top