Upgrading instance class on Aurora cluster with terraform

We recently decided we want to upgrade one of our Aurora RDS clusters from db.r3.large to db.r4.large. Our entire environment is managed by Terraform. It was not clear from Terraform documentation what would happen if we just changed the instance size and applied the changes. Would Terraform be smart enough to upgrade the writer instance in AZ1, failing over to the reader in AZ2, and then when that was complete, upgrade the newly promoted writer instance, failing back to the new instance in AZ1?

Continue reading Upgrading instance class on Aurora cluster with terraform

AWS Adventures, Part 4 – CloudWatch network monitoring

CloudWatch is a great concept — super-easy to configure and inexpensive. And at first glance, it actually looks pretty nice. But after I spent about 30 minutes with it, I realized it wasn’t easy to use. The units used are especially hard to interpret. This is my best attempt to explain what the network values mean.

Continue reading AWS Adventures, Part 4 – CloudWatch network monitoring

AWS Adventures, Part 3 – HA Wowza Live HLS

For a number of years, we have streamed HLS video via CloudFront, using a Wowza Streaming Engine server to convert our RTMP streams to HLS on the fly. CloudFront provides almost infinite scalability for the HLS stream, since the static chunk files are highly cacheable.

For high availability purposes, we want to use two independent WSE servers in two AWS availability zones. But this has been problematic. The two servers are never 100% in sync with their HLS chunking of the incoming live stream. This can cause the client to get a bad response to a request, thereby dropping the live stream.

After a lot of experimentation, I have come up with a way to assemble a multi-AZ, high availability cluster of WSE servers that can reliably stream HLS video from an incoming RTMP stream.

Continue reading AWS Adventures, Part 3 – HA Wowza Live HLS