Using LogicLoop and Materialize with dbt and Redpanda/Kafka

Introduction

LogicLoop allows you to write rules in SQL and then run them against your data and trigger different actions based on the results. LogicLoop also allows you to create and share dashboards and visualizations easily via their web interface.

Prerequisites

Make sure to sign up for a LogicLoop account first.

Starting the demo project

A quick summary of the steps from the “how to use dbt with Materialize and Redpanda” tutorial that you need to take to get the project up and running are as follows:

# Clone the repository:
git clone https://github.com/bobbyiliev/materialize-tutorials.git
# Access the directory:
cd materialize-tutorials/mz-user-reviews-dbt-demo
# Start by running the Redpanda container:
docker-compose up -d redpanda
# Build the images:
docker-compose build
# Then pull all of the other Docker images:
docker-compose pull
# Finally, start all of the services:
docker-compose up -d
# Install dbt:
pip3 install dbt-core==1.1.0
pip3 install dbt-materialize==1.1.0
user_reviews:
outputs:
dev:
type: materialize
threads: 1
host: localhost
port: 6875
user: materialize
pass: pass
dbname: materialize
schema: analytics
target: dev
dbt debug
dbt run
dbt test

Overview

There are three main things that we will be doing in this tutorial:

  1. Then we will write an SQL rule that will check our vipusersbadreviews materialized view which contains the bad reviews left by VIP users.
  2. Next we will create an action destination so that we can get a notification when the rule is triggered. That way we can stay on top of the bad reviews and make sure that our VIP users are taken care of.

Add Materialize as a source to LogicLoop

Start by logging into LogicLoop and navigating to the “Data Sources” page and clicking on the “New Data Source” button.

  • Host: The hostname of your Materialize instance.
  • Port: The port that your Materialize instance is listening on. Usually 6875.
  • User: The username you use to connect to your Materialize instance.
  • Password: The password you use to connect to your Materialize instance.
  • Database Name: set this to materialize as this is the default one.

Add an action destination

LogicLoop has a list of built-in action destinations that you can use like Slack, Webhooks, Email, and more.

Create a rule

Once the data source is created, you can create a rule that will check the vipusersbadreviews materialized view.

SELECT COUNT(*) FROM analytics.vipusersbadreviews;
SELECT * FROM analytics.vipusersbadreviews LIMIT 10;

Add an action

Once the rule is created, you can add an action to the rule. While on the “Rules” page, click on the “Add Action” button at the bottom of the page.

Conclusion

That’s it! You can now use the materialized views in your LogicLoop account. This is a great way to get real-time data visualizations and notifications for your business.

Useful links

Community

If you have any questions or comments, please join the Materialize Slack Community!

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Bobby Iliev

I am a professional System Administrator with a demonstrated history of working in the internet industry. I am a Linux lover