Stefan Kecskes

Never stop learning, because life never stops teaching

Delta Lake Evolution and Migrations

In the world of data management, we often talk about schema migrations. But what exactly is schema migrations, and why it is so crucial to have a proper plan for data management? I will show you naive approach, explore some traditional solutions. We will see Delta Lake in action, but also how we can do schema migrations with our own way.

Data Modeling

Did you know that the efficiency of your application hinges significantly on how you model your data?

Most backend software engineers, including myself for many years, don’t think much about data modeling. We understand the basics of SQL and ORM, know a few CRUD operations, and ensure that data is stored and retrieved correctly and that relationships between tables are maintained. Often, we leave the intricate details of data structures and performance to the DB Admin team, assuming that upscaling servers will solve everything. I used to think data modeling was just a fancy term for database design with only one right way to do it. However, as businesses grew and I collaborated with data scientists and analysts, I realized they were often dissatisfied with data structure and performance. Running their queries on the production database slowed down services and took ages to return results. That’s when I learned about the crucial differences between transactional and analytical data modeling, each with its own tools and techniques.


Using Volumes in Kubernetes

Volumes in Kubernetes are used to store data used by our applications. So that concept is very similar to volumes in docker containers. But there are some subtle differences, as always in life; nothing can be simple. :) There are many types of volumes in Kubernetes, and in this post, we will group them by properties and see the some of the most common types. Like with docker, we will group them to local and remote volumes.


Interviews these days...

Interviews these days…

Interviews are the dreaded part of the job search process. They can be nerve-wracking, stressful, and sometimes downright bizarre. Only software engineers will understand what I am talking about. What if I could show how conversations on interviews feel to non-software engineers? Let’s explore some hypothetical scenarios where interviews are conducted in a typical way for software engineers.

If Job Hunters were interviewed like Software Engineers

Interviewer: Welcome, and thank you for joining us today. Let’s jump right into it, shall we?


Declarative Kubernetes

Declarative Kubernetes

With docker, we used docker run ... for each docker instance. We came at some point to situation where our commands became too long with many parameters or that running the same command again and again was time-consuming and mroe error-prone. That is where stopped using imperative approach and started to use declarative approach with docker-compose yaml files. Same is with imperative approach using kubectl .... as I showed you in the previous blog post about Kubernetes basics. This time lets look into how we can achieve same more advanced stuff but with declarative approach with Kubernetes Resource definition files.