I’m addicted to performance metrics on my server infrastructure. For years in our on-prem environment, we monitored thousands of data points using ganglia, nagios, and zabbix. In our new AWS infrastructure, we thought we’d look for some more sophisticated options.Continue reading AWS Adventures, part 5 – monitoring infrastructure
2017 was the year I finally broke down and bought a 3D printer. I went for the cheapest model I could find that looked like it was halfway decent. I ended up ordering a Zonestar P802QR2 printer kit (a Prusa i3 clone). Here are my impressions.Continue reading Zonestar P802QR2 3D Printer impressions
Every time I set up a new Mac, I always forget how to make control-left and control-right work in remote Linux shells opened via iTerm2. If you don’t care about the built-in Mission Control shortcuts “Move left a space” and “Move right a space”, here’s how you do it. These instructions are for Sierra (10.12).Continue reading Control-arrow CLI navigation in iTerm2
I just upgraded to a new MBP, and here are the packages I immediately installed:Continue reading What’s on my MBP
CloudWatch is a great concept — super-easy to configure and inexpensive. And at first glance, it actually looks pretty nice. But after I spent about 30 minutes with it, I realized it wasn’t easy to use. The units used are especially hard to interpret. This is my best attempt to explain what the network values mean.Continue reading AWS Adventures, Part 4 – CloudWatch network monitoring
For a number of years, we have streamed HLS video via CloudFront, using a Wowza Streaming Engine server to convert our RTMP streams to HLS on the fly. CloudFront provides almost infinite scalability for the HLS stream, since the static chunk files are highly cacheable.
For high availability purposes, we want to use two independent WSE servers in two AWS availability zones. But this has been problematic. The two servers are never 100% in sync with their HLS chunking of the incoming live stream. This can cause the client to get a bad response to a request, thereby dropping the live stream.
After a lot of experimentation, I have come up with a way to assemble a multi-AZ, high availability cluster of WSE servers that can reliably stream HLS video from an incoming RTMP stream.Continue reading AWS Adventures, Part 3 – HA Wowza Live HLS
Here are the bands I’m interested in listening more to as Lolla gets closer. Bands marked with an asterisk are those that I know I already want to see (sadly, I’m sure there will be conflicts!)Continue reading Lollapalooza 2017 Playlists (favorites)
I’ve been lucky enough to go to Lollapalooza for the past 5 years. I really like to do my homework before I go so I know who I want to see. There are always tons of bands I’ve never heard of, and every year, some of them end up being my favorites at the show.
So I build Spotify playlists of every band at Lolla, using recent setlists from setlist.fm whenever it is available. A lot of work goes into this, and I’d like for people to have the chance to use these playlists.Continue reading Lollapalooza 2017 playlists
In our AWS migration, we found it necessary to run an FTP server. Yeah, I know — “FTP? In the 20-teens?”. Look, I get it — nobody wants to run an FTP server in this day and age. But it is still a convenient way for partner companies to transfer data to us via automation. This isn’t highly sensitive data; our main concern is keeping the FTP server isolated from our other services so that any vulnerabilities there don’t propagate to more critical systems.
At any rate, we found it surprisingly challenging to build a highly available FTP service in AWS.Continue reading AWS Adventures, part 2 – high-availability FTP service
After a lot of reading about AWS and the failures that have happened over the years, I’ve come to the conclusion that to be truly resilient against complete AZ failure, you need to have enough capacity running in both AZs to handle the entire load of your application.Continue reading AWS: hedging against AZ failure