Metis Blog

Do you rely on your monitoring solutions to let you know when things are wrong? That is so brave of you! On a more serious note, please think twice. Monitoring is not enough! It can’t explain why things happen the way they do (because it doesn’t see the past beyond metrics) and it doesn’t tell you what is going to happen (so it can’t predict the future). This is a serious problem and we need a solution (spoiler alert: we need database guardrails). Let’s read on to understand why.

Imagine a world where you don’t have to worry about your database, no bugs are deployed to production, all the performance checks happen early during the development phase, you don’t need DBAs nor the maintenance team, and still, everything works great. No matter how fantastic that sounds, you can get it today. Read on to learn how.

We want to test early and more often. This is what shift-left refers to. Instead of testing applications just before the release, we want to test our applications early to identify issues when it’s still easy to fix them. However, shifting to the left carries much bigger consequences. Let’s see what.

Achieving database reliability is not easy. We need to take care of many components, like data integrity, recoverability, performance, and security. However, we need to do it no matter who we are and what our business is. Our customers demand that. The reliability of a database directly impacts the efficiency, reputation, and success of an organization, making it an indispensable asset in today's data-driven world.

When running multi-tenant applications spanning hundreds of clusters and databases, it’s not uncommon to build extensive maintenance pipelines, monitoring, and on-call rotations to achieve database reliability. Many things can break, starting with bugs, inefficient queries written by developers, or inefficient configuration and indexing. Do you worry about your databases? You’re doing it wrong. Read on to see how to stop worrying and still have continuous database reliability.

In today’s world, it’s even more important for platform engineers and engineering leaders to push ownership of databases to the left and make developers own their databases. The organization needs to automate as much as possible and minimize communication between teams. While we achieved many improvements in the DevOps area and even included other aspects in the same way like security (with DevSecOps), configurations (with GitOps), or machine learning (with MLOps), we still keep databases out of the loop. Let’s see why that is a problem and why we should make developers own their databases.

As developers, we all strive to keep our systems in shape. We maintain them, we review metrics and logs, and we react to alerts. We do whatever it takes to make sure that our systems do not break, especially databases that are crucial to our applications. Wouldn’t it be great if there was no need to do the maintenance at all? Would you like to just have tools that could take care of your databases and let you forget that they exist altogether? Read on how to do that.

As developers, we all strive to write the best code that performs great when deployed. It is true when we’re writing our application code, and the same applies when it comes to our databases! Wouldn’t it be great if you could feel confident when changing your code? Would you like to deploy with no stress? Would you like to get automated confirmation that your changes to the database or business logic will not take your production database down? Read on to learn how to have all of that.

In this article, we’re going to see how database bugs can negatively affect our business and how we can protect ourselves from dire consequences.

In PostgreSQL, the slow query log is a feature that allows you to log queries that take longer than a specified threshold to execute. This log helps you identify and optimize queries that may be causing performance issues in your database. Let’s see how we can use it.

Database observability is crucial for maintaining optimal performance and reliability in modern software systems. It enables organizations to monitor key metrics such as query execution time, resource utilization, and transaction throughput, facilitating the early detection and resolution of issues like slow queries or resource contention. Let's see how to build it.

Database backup and recovery are fundamental aspects of data management, ensuring the preservation and restoration of vital information within a database system. To implement an efficient database backup strategy, we need to learn the methods, understand the pros and cons, and then learn how to avoid pitfalls.

In this how-to guide, we'll walk you through the whole process of setting up Metis on a Kubernetes cluster using StackGres.

There are many database monitoring solutions available in the market. Some of them are ubiquitous and very popular. However, all of them are simply wrong and put the burden of monitoring on the users. They still use the approach that worked well when we didn’t have multi-tenant applications and hundreds of microservices but is not good enough today. In this article, we’re going to see what these solutions lack and what modern monitoring should look like.

Managing an SDK in production over several years is similar to taming a multi-headed beast. Each aspect of the development process demands unique attention and precision. Balancing the evolving needs of a diverse user base, adapting to the ever-changing technology landscape, and ensuring consistent, high-quality performance across various environments presents a complex array of challenges. This article distills the key lessons and strategies we've honed from years of experience in maintaining and evolving our SDKs in a dynamic and demanding technological ecosystem.

PostgreSQL has become super popular thanks to its flexibility and strength - it's this really powerful open-source database that can handle all sorts of tasks, from small projects to huge enterprise needs. But as data keeps getting bigger and bigger, it's super important to tweak things in PostgreSQL to make it run faster and smoother. Let's see how to do that.

Efficient SQL queries stand as a cornerstone for efficient data retrieval. Optimizing these queries isn't merely a best practice; it's the crux of maintaining high database performance. Slow query execution leads to bottlenecks, hampering system throughput and user experience. In essence, the impact of neglecting query optimization in production is profound—it compromises system stability, escalates response times, and ultimately impedes the smooth operation of critical applications. Let's see the techniques for maximizing the database performance.

We still face problems when teams of DevOps, platform engineers, or DBAs are responsible for a staggering amount of databases in their production environments and developers are unable to fix their databases. We need to learn how to prevent issues from reaching production and how to monitor databases efficiently. Metis brings a new and novel approach that reshapes the way we work with databases. Let’s see how.

You may need to deal with many things as a manager. You manage people and make sure they are happy, develop their skills, and don’t leave the company prematurely. You plan roadmaps and execution to make sure that your team meets all the deadlines. You decide which features to build and plan releases to please your customers the most. All of that can go to waste if your production system fails. And the easiest way for it to fail is by breaking your database. Pretty often it’s caused by developers not testing their solutions well enough or because of choosing speed over quality where we shouldn’t do that. But what if you could make sure that ongoing changes won’t break the database and won’t derail your planning? Let’s read on.

DORA group defined four broad metrics that are now known as DORA Four Keys. These metrics can measure the velocity and stability of your applications, processes, and business maintenance in general. In this article we are going to see how these metrics can be used in your organization and how they can be helpful with your databases.

Database maintenance is hard when the ownership and responsibility are scattered. Teams work inefficiently, time is wasted, and time to issue resolution is much longer. Three crucial elements in ownership can greatly improve the organization and reduce time and cost. These are observability with database-oriented tools, well-defined processes, and the mindset of true owners.

DevOps is not new. Ideas to combine deployment and operations began to appear in the late 80s. It took us nearly 20 years to finally formulate the state of DevOps and begin the move towards DevOps engineers. However, we can’t stop here. Read on to understand how to enhance the service ownership in databases and how to shift left the performance checks.

We can’t let our databases fail. We need to have measures in place to guarantee that the crucial business data is not impacted. One way of doing that is real-time database monitoring which involves continuously observing, analyzing, and managing the performance and health of a database system. Let's see what that is and how to use it.

We went through a big transformation in recent decades, we learned how to deliver the software better, and we devised new practices and patterns to streamline the work between the teams. However, the current software development lifecycle (SDLC) is still far from being perfect. Teams spend much time on the handoffs of the artifacts. They communicate inefficiently, and they lack proper checks early in the pipeline. Platform Engineers can reshape the industry by pushing left guardrails to protect production, however, they need to have proper tools and processes. In this article, we are going to see what Platform Engineers can do to make Developers’ lives better.

Current monitoring solutions focus on capturing the raw data and various signals from multiple layers. This lets us build dashboards and alarms to automatically monitor the performance of our applications. While this approach works, it puts too high a burden on the developers without providing them with tools and processes. Even though this sounds contradictory, current monitoring solutions do not show us what is going on and do not let us analyze the performance properly. Let's see how to fix that.

Testing databases is hard. We lack good methods in our industry and we can't prevent the bad code from reaching production. In this blog post, we explain what we need, how to build that, and how database guardrails can help us protect our databases.

We continue our journey into how Metis can prevent, monitor, and troubleshoot our databases. In the previous part we started playing with GORM, a feature-rich ORM for GO. This time we’re going to see more challenging examples. We are going to see actual insights provided by Metis and how they improve the query performance.

Let’s see how Metis can prevent, monitor, and troubleshoot our databases. In this part, we’re going to play with GORM, a feature-rich ORM for GORM. We are going to see actual insights provided by Metis and how they improve the query performance.

Let’s see how Metis can prevent, monitor, and troubleshoot our databases. In this part, we’re going to play with SQLAlchemy, a feature-rich ORM for Python. We are going to see actual insights provided by Metis and how they improve the query performance.

Developers should eat their dog food and use their products as regular users do. In this blog post, we show how Metis helped debug the Metis platform. We’ll see how we identified issues with the deployment and fixed them before affecting the customers.

In the fast-paced digital landscape of today, where data drives critical business decisions, the performance and reliability of databases are paramount. Whether we're a seasoned database administrator or a developer working with databases, encountering performance issues and bottlenecks is inevitable. Slow-running queries, soaring CPU usage, and memory struggles can disrupt workflows and hinder user experiences. Let's see how to tackle all the issues and put your database on steroids!

Pgvector is a novel approach for storing and processing vectors with a PostgreSQL database. This greatly improves any operation on embeddings like text search or Machine Learning inference. In this post, we explore how pgvector works, how to get started, and how to tune it to get the best possible performance.

In this blog post we are going to use Sequelize to explore the IMDb dataset. We’re going to build a proper Database Monitoring around to see the performance changes and how to tune the queries. We’ll write an application in JavaScript with Express and a PostgreSQL database.

Schema migration refers to the strategic alteration of a database's structure, encompassing modifications to tables, relationships, and data types. This dynamic process is not merely a technical task, but a crucial cornerstone in maintaining the integrity, efficiency, and scalability of a database system. See this post to learn how to overcome all the problems you may face.

Fast and efficient database operations are the key for a seamless and responsive user experience that keeps customers engaged and loyal. In this article we focus on eight strategies to improve database performance. We include both hardware and software, query optimizations, configuration, and data normalization aspects.

Database is the most crucial part of each application. While we can accept some issues with the frontend or with the backend, we simply cannot let our database fail or become slow. In this post we will see the crucial database performance metrics and how to improve database performance. We will focus on monitoring, configuration, and various database engines (postgres or mysql), and we will see why they are important.

Partitions let us split the logically one large table into smaller physical ones. This can improve the queries performance by accessing fewer rows, optimizing random-access reads, and using very targeted indexes. Let’s see how it works in PostgreSQL and how Metis handles that.

We continue our journey into how Metis can prevent, monitor, and troubleshoot our databases. In previous part we started playing with Sequelize, a feature-rich ORM for TypeScript. This time we’re going to see more challenging examples. We are going to see actual insights provided by Metis and how they improve the query performance.

Let’s see how Metis can prevent, monitor, and troubleshoot our databases. In this part we’re going to play with Sequelize, a feature-rich ORM for TypeScript. We are going to see actual insights provided by Metis and how they improve the query performance.

Machine Learning is the thing! Let’s see how we can optimize queries with ChatGPT and other online solutions, and how Metis can help even better.

Let’s see how Metis can prevent, monitor, and troubleshoot our databases. In this part we’re going to prepare a database based on IMDb and start with sample queries to see actual examples. We are going to see actual insights provided by Metis and how they improve the query performance.

Databases are complex beasts. They have options for various areas like memory usage, backups, write-ahead-logs, monitoring, replication, and many more. Installing a database is just the beginning, we need to configure it accordingly later on to match our needs. In this blog post we cover the most important settings you should use.

Last time we saw what could be the reason for high CPU usage. In this blog post we are going to see some troubleshooting scenarios and queries that can help us identify and fix the issue.

High CPU usage in our database may indicate multiple issues. There could be a lot of calculations performed in the background. Our database could be lacking available memory and do a lot of paging. The database could as well spend a lot of time on busy waiting. In this post we are going to see how to deal with such cases.

We want our applications to never fail. To achieve that, we need to guarantee that the data storage is always on. How to do that with SQL databases? How to make sure that the engine never fails? And how to scale it to get better performance?

Choosing a database is hard, however, maintaining it afterwards is even harder. In this post we talk about elements to focus on, tooling, monitoring, and other aspects of keeping the database up and running.

Modern SQL engines need to perform multiple operations concurrently and in parallel. This poses multiple problems around protecting data consistency and shared memory. In this blog post we are going to see what mechanisms are used to protect the memory from corruption, and how isolation levels are implemented under the hood.

Most of us know how to SELECT data, JOIN tables, or use subqueries. However, Data Query Language (DQL) part of the SQL gives us much more. Let’s see how to use Common Table Expressions (CTE) and Window Functions to enhance our queries significantly and make them easier to maintain.

SQL database provides multiple ways of auditing what happened in the database. In this blog post, we’re going to examine various solutions, analyze performance, cognitive load, and other aspects of the solution.

Choosing a database for the next project may not be easy. There are numerous aspects to consider, and the decision will mostly stay with us for a long time. In this post we examine various different solutions, and provide a general guidance on what to think about during the selection process.

Object Relational Mapper is a library that takes care of mapping the domain model into the database entities and the other way round. While the library is smart, it sometimes cannot figure out how to do things efficiently. This may result in poor performance and lesser user experience. In this blog post we’re going to see what issues we may face, and how to avoid them.

One of the good practices for database design is normalization. We decompose data into multiple tables to avoid duplication and make data storage more organized. As an outcome, we need to join tables when querying the data. SQL engine needs to calculate the result of the join operation, and there are multiple join strategies (algorithms) that can be used. In this blogpost we’ll understand typical join algorithms and when they’re used.

Collecting data from multiple sources poses multiple risks. Missing values can be represented differently. Dates may be parsed incorrectly. International strings may get wrong encoding. In this post we are going to see various issues that may occur during the problem, and how to address them along the way.

SQL engine needs to execute the query somehow. But how does it do that? And how can we get some insights into details? In this blog post we’ll see what is an SQL, why it's important, and what tools to use to read it.

SQL databases are often used as black boxes. We just create tables, insert some data, and query it later. We rarely think what data we get as an output of a SELECT statement. We just assume it’s “the data”. However, it is not as simple as it may seem. In this post we are going to see how to control the visibility of changes that are not yet persisted in the database.

CLI tools often have similar interface. They have parameters, default values, some auto-completion. We would like to be able to define such an interfance in an easy way. At Metis, we decided to build a library to support these typical CLI elements, and we decided to release it on an MIT license. In this post we describe what we did and how we approached this problem.

PostgreSQL database queries are a common performance bottleneck for many reasons. There are well known techniques for speeding up the queries, like caching or read replicas. They can save a lot of time and headache, however, it may not be straightforward how to introduce them to the existing codebase. One of the solutions that seems like it can be applied much easier is indexing. In this post we’re going to examine indexes on a JSONB data type. We’ll see what they are, how they are constructed under the hood, and what dimensions to consider to assess the performance improvement.

In this post we’re going to learn how to save ourselves from deploying an invalid code to production. We’ll consider various areas, like data quality, performance, data loss, and we’ll see what a good database observability solution is.