Yesterday

Disclaimer: this post has nothing to do with the Beatles song.

TLDR at the bottom!

This blog has been languishing for awhile.  I still get quite a few hits for a post I wrote ages ago for a problem I had getting my Logitech mouse to work with Ubuntu, and I even occasionally get comments from people thanking me because it helped them out. It’s nice to know at least it’s not completely lifeless and some others are still finding value in this blog!

However, I’ve had the itch to start posting again.  I was inspired by this YouTube video and I’ve decided to start sharing random things I learn in the hope of educating others too.  Most of it will probably be programming related.  I’ve been in this game for many, many years, but there’s always a ton more to learn.  Not just learning new tools and frameworks, but learning little things you didn’t even know about things you use every day without taking the time to dive deeper until you need to.  Now, I’m not promising to actually post daily, and a lot of these are probably just going to be quick little things, but I’m definitely going to try to post more regularly.

Anyhow, the title of this post has to do with getting yesterday’s date from the command line!

I have a log parsing script that I’ve been running manually as a two step process.  Step 1 is running a command to download the logs from the previous day from AWS Cloudwatch.  Step 2 is running my actual parsing script for that same date to generate the analysis I’m looking for.  In each case, I need to pass yesterday’s date to the script.

I just wanted to write a simple bash wrapper script to run those commands.  Now it’s really easy to get today’s date:

./some-script.sh `date +"%m/%d/%Y"`

Anybody familiar with bash can probably work that out. First it runs the date command, given the format string so that it returns “07/16/2019”. Then ./some-script.sh gets executed with that as an argument.

But what if I need yesterday’s date? My first thought was that I could extract the date portion from the string, and just subtract one from that. Simple, right? Except, what if the day I’m running the script happens to be first of the month? Or even worse, the first of the year? Ugh! Now it’s getting complicated. There’s got to be an easier way to do this, right?

Yep!

If you have the GNU version of the date command, just do one of the following (H/T to this StackOverflow post)

date +"%m/%d/%Y" -d "yesterday"
or
date +"%m/%d/%Y" -d "1 day ago"

I was very excited to find this! I use Linux at home and for personal stuff, but at work I’m using a Mac, and unfortunately the OSX version of the date command doesn’t recognize this option.

But do not despair! On OSX the syntax is just slightly different

date -v-1d + "%m/%d/%Y"

In both cases, you can do more than just yesterday’s date. You can go multiple days in the past or the future. I’ll leave that as an exercise to the reader to work out.

Hope someone else finds this helpful!

TLDR:
To get yesterday’s date from the command line:
GNU/Linux: date +"%m/%d/%Y" -d "yesterday"
OSX: date -v-1d + "%m/%d/%Y"

SOLVED: Docker Networking On Fedora Linux Fails When I’m Connected To VPN

I do a lot of my development work inside docker containers.  Recently I ran into an issue where, when connected to my company’s VPN network, the docker containers on my local machine running Fedora would lose the ability to connect to external resources.  Eventually I discovered the issue was that the default subnet created for the docker bridge interface on my Linux machine was overlapping with the subnet used by our corporate VPN.  So everything worked fine, as long as I wasn’t connected to the VPN.

My home network is a 10.x.x.x subnet, while my corporate VPN (and my default docker bridger interface) were bothing using 172.x.x.x.  So I opted to use 192.168.1.x for my docker bridge.  Making the change was fairly straightforward.  I needed to create the file /etc/docker/daemon.json.  The documentation describes a lot of options in this file, but all I needed was the following:

{
 "bip": "192.168.1.1/24"
}

Then I restarted docker:

sudo service docker restart

Problem solved!

 

Adding SSL with Let’s Encrypt!

screenshot-from-2016-09-18-16-48-46

Adding SSL to a website sure is easier than I remember.  I’ve been vaguely aware of the fact that free SSL certificates were now available from Let’s Encrypt.  I’ve been spending most of the day geeking out with AWS server stuff so I decided now would be a good time to see what exactly is involved, and I was absolutely stunned at how easy the process is!

I started with an Ubuntu Server running Apache; no SSL  configured at all.  I pointed my browser at https://certbot.eff.org/.  It gives you a couple of big, friendly drop-down menus where you specify the web server software and OS you are using, and it redirects you to a page of step-by-step instructions.

If you are at all familiar with working at the command line, the process could not be much simpler.  Following are the steps I took for Apache on Ubuntu Server, but I assume the process will vary depending on your environment.

On my server, I ran the command wget https://dl.eff.org/certbot-auto to get the software that bootstraps the process.  Once it downloaded, I ran chmod a+x certbot-auto to make the file executable, and then ./certbot-auto to kick it off.

At this point, certbot used apt to download all the package dependencies. Since I had a simple, bare-bones Apache configuration, it gave me the following dialog in a text interface:

No names were found in your configuration files. You should specify ServerNames in your config files in order to allow for accurate installation of your certificate. If you do use the default vhost, you may specify the name manually. Would you like to continue?

Being the slacker that I am, I naturally opted for the path of least resistance, and answered affirmatively.  Then it presented another dialog:

Please enter in your domain name(s) (comma and/or space separated)

Simple enough.  I entered my domain and then:

Please enter email address (used for urgent notices and lost key recovery)

After entering my email address, it provided me with a dialog to agree with the TOS.

Please read the Terms of Service at https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf. You must agree in order to register with the ACME server at https://acme-v01.api.letsencrypt.org/directory

Who am I to argue?  I agreed, and then got:

Please choose whether HTTPS access is required or optional.

Easy – Allow both HTTP and HTTPS access to these sites
Secure – Make all requests redirect to secure HTTPS access

Nice!  It even gives you the option of configuring your server so that non-secure requests are redirected to https.  Yes, please!

After a bit more churning:

Congratulations! You have successfully enabled https://ericasberry.com

You should test your configuration at: https://www.ssllabs.com/ssltest/analyze.html?d=ericasberry.com

That was it!  Didn’t even have to restart apache (I presume it did that for me in the background).  I went ahead and verified the configuration as suggested, and my site is now A-rated!  That’s more than I can say for chase.com, which currently only rates a measly B grade.  Take that, mega bank!

The only catch seems to be that the certificates are only good for 90 days.  But it looks like all you have to do is set up a cron job to run “certbot-auto renew” every 3 months to take care of that.  Since I just set it up, I haven’t tried that step, but I’ll try to remember to update this post when the time comes.

Quick tip for joining lines with a separator in vim

Every so often I need to deal with some exported database id’s that come in the form of a CSV file.  The trouble is, instead of having the id’s one per line, I really need them on a single line, comma-separated so that I can use them in an ‘in’ clause in some kind of query.  I always remember this is easy to do in vim, but I can never remember the syntax.  So here it is, for my (and maybe somebody else’s) future reference:

:%s/\n/,/

: to enter command mode

% to select all lines

Then the substitute command to search and replace all newlines in the selected block with a comma.  Of course you could use the pipe character or whatever other delimiter you need in place of the comma.

Now that I’ve written it down somewhere hopefully I’ll never forget it!

janrain social sharing to twitter: an error occurred

Just a quick note, mostly to myself, but maybe this will help out somebody else doing a google search down the road, because I sure didn’t have much luck finding anything.

Today, for the second time, I got bitten by Janrain’s poor diagnostics when it comes to error handling with their social sharing .

I got bitten by this same scenario probably six months ago, but had forgotten the details, which made it all the more aggravating when I wasted time trying to figure it out again today.  I was just putting the finishing touches on a new feature, and regression testing social sharing to Twitter through Janrain, when suddenly every attempt to share to Twitter began failing, with no information from Janrain other than “An error occurred”.  Yeah, really helpful guys.

I was finally able to track down the issue.  I was just sending a test tweet on a private account, so I didn’t really care about the content, and I was reusing the same content over and over.  Apparently the Twitter API detects that the content is being duplicated at some point and begins rejecting the tweets with an error (don’t have the exact code handy as I’m writing this, but it’s obvious that its being rejected because its a duplicate tweet).  Janrain, instead of reporting this detail, just squelches it and reports “An error occurred”.  I was only able to figure this out by looking at the HTTP requests.

So, if you suddenly run into this mysterious generic error with Twitter and Janrain social sharing, and are attempting to tweet the same content, this could be the cuplrit!  Just change up the content and all your problems will be solved (until the next one).