Blue-green deployments with Materialize

Introduction

Minimizing downtime during any deployment is a key part of any successful deployment strategy. There are many ways to achieve this, and blue-green deployments are one of them.

Prerequisites

Before getting started, you should have the following:

  • Materialize Cloud AccountThe free trial includes access to two instances, so you should be able to test out a blue/green architecture for free.
  • psql or mzcli installed on your machine
  • DNS Service Provider like Cloudflare, AWS Route 53, Google Cloud DNS, or any other DNS service provider

What is a Blue-Green Deployment?

A blue-green deployment strategy allows you to deploy a new version of your application while keeping the old version running. Once the new version is ready, traffic is switched over to it via an update to load balancer rules or DNS. The old version is kept running in the background until the new version has successfully taken over. Once we are sure that the new version is stable, the old instance can be destroyed.

Plan Your Deployment

The rest of this article is a hands-on walkthrough of a blue-green deployment of Materialize Cloud taking the following steps:

  • Create a DNS record pointing to the blue instance
  • Connect to the blue instance and create all of the necessary Materialize sources and views
  • Create a new instance of Materialize via the Materialize Cloud, this will be the green instance
  • At that point, only the blue instance is running and handling the traffic so next, we will switch the DNS to point to the green instance
  • Once the DNS is updated, the traffic is switched over to the green instance but keep in mind that it might take a while for the DNS to propagate before the green instance starts handling all of the traffic
  • Finally, we will shut down the blue instance completely once we are sure that the green instance is up and running and we’ve tested it thoroughly

Creating Materialize Deployments

Create the blue instance via Materialize Cloud UI:

  • Choose the name of your deployment. We will use the name ‘blue’ for this deployment to make sure that we can easily identify it later
  • Next, choose the size and the region of your deployment
  • Finally, click on the ‘Create’ button

Connecting to Materialize

For this demo, we will use the psql command to connect to the instance.

Adding Materialize Sources

For this demo, we will use the PubNub example from the Materialize Cloud documentation.

CREATE SOURCE market_orders_raw
FROM PUBNUB
SUBSCRIBE KEY 'sub-c-4377ab04-f100-11e3-bffd-02ee2ddab7fe'
CHANNEL 'pubnub-market-orders';
CREATE VIEW market_orders AS
SELECT
((text::jsonb)->>'bid_price')::float AS bid_price,
(text::jsonb)->>'order_quantity' AS order_quantity,
(text::jsonb)->>'symbol' AS symbol,
(text::jsonb)->>'trade_type' AS trade_type,
to_timestamp(((text::jsonb)->'timestamp')::bigint) AS ts,
''
FROM market_orders_raw;
CREATE MATERIALIZED VIEW avg_bid AS
SELECT symbol,
AVG(bid_price) AS avg
FROM market_orders
GROUP BY symbol;
SELECT * FROM avg_bid;
symbol | avg
------------+--------------------
Apple | 199.3392717416626
Google | 299.40371152970334
Elerium | 155.04668809209852
Bespin Gas | 202.0260593073953
Linen Cloth | 254.34273792647863

Create the route (via DNS or LB) to the blue instance

Once you have created the views, you can add a DNS record or a load balancer rule to point to the instance.

materialize.example.com. CNAME  12345mz.materialize.cloud.

Performing a Blue-Green Deployment

With all that setup, we can now perform a blue-green deployment. We will first create a new instance of Materialize via the Materialize Cloud just like we did for the first deployment.

CREATE SOURCE market_orders_raw
FROM PUBNUB
SUBSCRIBE KEY 'sub-c-4377ab04-f100-11e3-bffd-02ee2ddab7fe'
CHANNEL 'pubnub-market-orders';
CREATE VIEW market_orders AS
SELECT
((text::jsonb)->>'bid_price')::float AS bid_price,
(text::jsonb)->>'order_quantity' AS order_quantity,
(text::jsonb)->>'symbol' AS symbol,
(text::jsonb)->>'trade_type' AS trade_type,
to_timestamp(((text::jsonb)->'timestamp')::bigint) AS ts
FROM market_orders_raw;
CREATE MATERIALIZED VIEW avg_bid AS
SELECT symbol,
trade_type,
AVG(bid_price) AS avg
FROM market_orders
GROUP BY symbol, trade_type;

Update the route to point to green

Once you are ready, you can head back to your DNS zone and update the DNS record for the instance to point to the new instance hostname. Or in the case of a load balancer, you can update the route to point to the new instance.

materialize.example.com. CNAME  54321mz.materialize.cloud.  # Change this to the new hostname

Verifying the Deployment

To verify that we are successfully connected to the new instance, try to connect to the new instance via the psql command as before by using the DNS name (e.g. materialize.example.com) and run the following query:

SELECT * FROM avg_bid;

Alternative approaches to the DNS change

The described approach using DNS changes and traffic routing is a good one to use but there are other ways to achieve the same result.

Downside of Blue-Green Deployments

Some of the downsides of a blue-green deployment strategy are:

  • The traffic might be routed to the blue instance for a while before the green instance starts handling the traffic
  • If you have Materialize Sinks you would need to plan how to handle the data that is being sent to those sinks while the two instances are running
  • During the deployment, you would have to have two instances running during the same time which could add some overhead
  • In case you have a lot of data with a lot of changes, you might have to wait for the new Materialize instance to ingest the data which could take a while if there is a lot of backpressure.

Conclusion

This is just a brief overview of the steps that you need to take to deploy Materialize in a blue-green deployment strategy.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Bobby Iliev

I am a professional System Administrator with a demonstrated history of working in the internet industry. I am a Linux lover