Flush MacOS DNS Cache

Had to restart a dead remote VM server this morning, and it gets its IP address from DHCP. I don’t have physical access to the machine and I needed to SSH into the VM to manually restart some services. My MacOS system was still caching the old DNS address, and IT told me I needed to flush my DNS cache. I had no idea how to do that on MacOS, but turns out to be pretty simple. In the terminal, just run the following command:

sudo killall -HUP mDNSResponder

Magic!

Copying dot files from the shell

This isn’t rocket science, more of a note to self.

Found myself moving to a new work machine at work. Part of that involves copying dot files that Mac OSX finder can’t see. I really only want to move the ones I absolutely need, because I’m moving from a machine that I’ve had for nearly 5 years, so a lot of cruft has accumulated.

I have to return the old machine soon though. So the first thing I did was use Carbon Copy Cloner to back up the entire image to a USB drive. Now, I don’t want to carry that USB drive around everywhere. Once I get the code repositories, etc copied I don’t want to be tethered to it. But I want to hang on to all those dot files and directories in case I miss something and need them.

So what I’ve done in the past is a bunch of ls -la commands, and pick out the ones I know I need, manually copying them one by one. Found myself about to do the same thing again, even though there are some I know for sure I don’t need, and while I could just delete those and copy everything, I still want to keep them on the backup because I’m paranoid.

Copying dot files from the terminal can be a little tricky though. Try doing ls -a .*. You get everything in the current directory and subdirectories. But a couple of grep commands can make it pretty easy:

for x in `ls -a | egrep '^\.\w' | egrep -v 'atom|Web|DS_Store|cache|vagrant'`; 
   do 
   cp -rpv $x ~/dotfiles; 
done

So let’s break this down:

for x in `somecommand`

(note: it’s a backtick, not a single quote!) iterates over the lines of output of somecommand

In this case, somecommand is:

ls -a | egrep '^\.\w' | egrep -v 'atom|Web|DS_Store|cache|vagrant'

The ls command with the -a option lists all files in the current directory including hidden (dot) files, and at this point I’ve already cd’d to the directory containing the dot files on my backup drive.

We then pipe that to:

egrep '^\.\w'

that filters the output of ls to only include filenames that start with a ‘.’ (the backtick must be used to escape the period, since otherwise it means match any character) followed by a word character. We match on the word character because otherwise the output would include entries for “.” and “..” which we don’t want to include.

Finally, that output is piped to egrep again, and in this case we’re telling it to exclude output that matches the given patterns, separated by the | character. In this case, I know I don’t want the .DS_Store directory, or the .cache directory, and I haven’t used atom, WebStorm or vagrant in years, so I don’t want to bother copying those files.

Inside the do/done section, we use the copy command to copy each file from the list to a directory (previously created) called ‘dotfiles’ underneath my home directory. We use -r for recursive (so we copy dot directories and everything underneath them, not just the top level files), -p to preserve the permissions and -v so that we see output seeing the files as they are copied (otherwise the output from cp is silent).

Hopefully someone else finds this useful; if nothing else I will be referring back to this myself the next time I have to do this!

This Is Unsafe

Against my better judgement, I upgraded to MacOS Catalina. After surviving a panic when the upgrade was seemingly stuck “estimating time remaining” (I left it alone; the process ended up taking about 2 hours total) I fired up Chrome and tried to get to work.

The project I work on uses self-signed certificates for local dev, generated on the fly by an npm module called pem. This has always resulted in big warning page from Chrome about the certificate being invalid, but it’s always been easily bypassable by clicking the Advanced button and clicking the option to proceed anyway.

However, after the upgrade, the error page changed as shown above. Now it says the certificate is revoked (NET::ERR_CERT_REVOKED), and no longer gives me the option to proceed.

First, I was a little confused how the MacOS update caused this. I assume it’s tied to some connection between Chrome and the OS, but curiously, I did not have this same problem with Firefox. I thought it might be something that changed in Chrome itself, but I verified I do not have this issue with the same version of Chrome on Linux. I really didn’t want to spend hours going down this rathole (but, alas, so I did) and just wanted to get back to work, but sometimes that’s not how it goes in the life of a software developer.

So, my first sign of hope was this page I found. Seems there was a change where certificates with an expiration date greater than 825 days are no longer accepted. Looking at the code, I see the certificate being generated by pem is set to be valid for 3650 days. That must be it!

This did cause a change in behavior. Now instead of NET::ERR_CERT_REVOKED, I got the same page but with a different error: NET::ERR_CERT_INVALID. This is the same error I’m used to, but I still am not given the option to proceed.

I could not find anybody running into the exact same issue as me. I pondered trying to find a different NPM module (not at all confident that would help), generating a static certificate file and importing it into Chrome, starting Chrome from the command line with “–ignore-certificate-errors”, and myriad other solutions, all of which seemed less than ideal, when I stumbled across this forum thread from 2017. It wasn’t exactly the same scenario, other than the OP was also trying to bypass the NET::ERR_CERT_INVALID error.

When I saw this response, I honest to God thought it was a sarcastic joke, but out of desperation, I tried it anyway: There’s a secret passphrase built into the error page. Just make sure the page is selected (click anywhere on the background), and type thisisunsafe

I clicked back on my Chrome window, typed in the characters “thisisunsafe” (didn’t even have to hit Enter!), and lo and behold, my local dev site is back in all its glory!

Hope this helps someone else out.

Safari blocking new window opened from clicking button in web application

I work on a web application that includes a feature that integrates with another of my company’s products. You trigger the integration by clicking a button in the first web app, which launches the second web app and passes along a document. The code is a little something like this:

$(document).on('click', '.my-button', function(e) {
  $.ajax({
     method: 'POST',
     url: '/api/call/to/get/the/url/we/need/to/open',
       error: function (status, err) { },
       success: function (data) {
         window.open(data.redirectUrl);
        }
     });
});

I mostly use Chrome on either a Mac or Linux laptop and this has worked fine. But yesterday I got a report that the button was not functioning on Safari. Turns out that the window.open was being blocked by Safari’s popup blocker. Of course, the easiest thing is to just tell the user to turn off their popup blocker. But that’s obviously not ideal.

So I did some digging and found other people had run into similar issues with Safari. But the solution is relatively simple, according to this StackOverflow link. Do the window.open first, with no target, and then set that window’s location.

However, this still didn’t fix the issue for me. Digging a little deeper into the comments on the post, one additional thing I discovered is that the ajax call that’s triggering this also needs to _NOT_ be asynchronous. Well, this is not ideal, but the web app is basically blocked until this completes anyway, so this turns out to be a good enough solution for now. The API call is pretty fast, but I throw up a little temporary modal with a spinner while this is being processed so the window doesn’t just completely freeze up with no feedback.

So the fixed code looks a little like this

$(document).on('click', '.my-button', function(e) {
  $.ajax({
     method: 'POST',
     async: false,
     url: '/api/call/to/get/the/url/we/need/to/open',
       error: function (status, err) { },
       success: function (data) {
         var newWindow = window.open();
         newWindow.location = data.redirectUrl;
        }
     });
});

Hope this helps someone! And of course I welcome any feedback on better ways of handling this.

Yesterday

Disclaimer: this post has nothing to do with the Beatles song.

TLDR at the bottom!

This blog has been languishing for awhile.  I still get quite a few hits for a post I wrote ages ago for a problem I had getting my Logitech mouse to work with Ubuntu, and I even occasionally get comments from people thanking me because it helped them out. It’s nice to know at least it’s not completely lifeless and some others are still finding value in this blog!

However, I’ve had the itch to start posting again.  I was inspired by this YouTube video and I’ve decided to start sharing random things I learn in the hope of educating others too.  Most of it will probably be programming related.  I’ve been in this game for many, many years, but there’s always a ton more to learn.  Not just learning new tools and frameworks, but learning little things you didn’t even know about things you use every day without taking the time to dive deeper until you need to.  Now, I’m not promising to actually post daily, and a lot of these are probably just going to be quick little things, but I’m definitely going to try to post more regularly.

Anyhow, the title of this post has to do with getting yesterday’s date from the command line!

I have a log parsing script that I’ve been running manually as a two step process.  Step 1 is running a command to download the logs from the previous day from AWS Cloudwatch.  Step 2 is running my actual parsing script for that same date to generate the analysis I’m looking for.  In each case, I need to pass yesterday’s date to the script.

I just wanted to write a simple bash wrapper script to run those commands.  Now it’s really easy to get today’s date:

./some-script.sh `date +"%m/%d/%Y"`

Anybody familiar with bash can probably work that out. First it runs the date command, given the format string so that it returns “07/16/2019”. Then ./some-script.sh gets executed with that as an argument.

But what if I need yesterday’s date? My first thought was that I could extract the date portion from the string, and just subtract one from that. Simple, right? Except, what if the day I’m running the script happens to be first of the month? Or even worse, the first of the year? Ugh! Now it’s getting complicated. There’s got to be an easier way to do this, right?

Yep!

If you have the GNU version of the date command, just do one of the following (H/T to this StackOverflow post)

date +"%m/%d/%Y" -d "yesterday"

or

date +"%m/%d/%Y" -d "1 day ago"

I was very excited to find this! I use Linux at home and for personal stuff, but at work I’m using a Mac, and unfortunately the OSX version of the date command doesn’t recognize this option.

But do not despair! On OSX the syntax is just slightly different

date -v-1d + "%m/%d/%Y"

In both cases, you can do more than just yesterday’s date. You can go multiple days in the past or the future. I’ll leave that as an exercise to the reader to work out.

Hope someone else finds this helpful!

TLDR:
To get yesterday’s date from the command line:
GNU/Linux:

date +"%m/%d/%Y" -d "yesterday"

OSX:

date -v-1d + "%m/%d/%Y"