Amazons Data Lakehouse architecture is a sophisticated solution that merges the capabilities of data lakes and data warehouses, designed to meet the demands of modern data engineering. This case study delves into the technical aspects of Amazons Data Lakehouse from a Data Engineers perspective, highlighting its architecture, components, and operational efficiencies.

The Infrastructure Behind Amazon Data Lakehouse – Handling Millions of Transactions per Second

In this blog, I dive into Apache Parquet, a powerful columnar file format optimized for big data environments. After spending time exploring its structure, writing and reading processes, and performance benefits, I’ve discovered how Parquet significantly improves storage efficiency, query performance, and scalability. Its seamless integration with big data tools and ability to handle schema evolution make it a go-to format for data engineers working with large datasets in analytics and machine learning pipelines.

Getting to Know Apache Parquet - The Big Data Game Changer

In the fast-paced realm of tech and business, chaos often seems to be the name of the game. As a software engineer with aspirations of stepping into the shoes of a Product Manager or Program Manager, I have realized the importance of seeing the bigger picture. That where System Thinking comes into play.

A Deep Dive into System Thinking

A quick walkthrough and overview of A-star Algorithm

What is A * Algorithm ?

Sieve filters primes, eliminate composites; ancient Greeks ingenious, fast method, find prime numbers with efficiency and ease.

Sieve of Eratosthenes Algorithm - the ancient algorithm

Curve simplification, points embrace, line hugs; preserve shape, efficient path reduction, geometric elegance.

Ramer–Douglas–Peucker Algorithm

Data nodes chain, flexible structure; dynamic size, efficient insertion/deletion, but linear access; foundation for many data structures

Understanding LinkedList in Python

Floyds hare and tortoise race, cycles collide, pointer dance; loop detection, fast and elegant cycle-finding.

Floyd Cycle Detection Algorithm

A quick walkthrough overview of the project setup and the technology stack we used.

Introduction to Multi Threading in Python

R, statistical prowess, data wizardry; open-source, versatile, embraced by analysts, researchers; for data manipulation, visualization, and analysis.

Ayan's Notes

Nov 14, 2024

Sep 13, 2024

Nov 14, 2023

Jun 20, 2020

May 26, 2020

May 16, 2020

Apr 26, 2020

Mar 15, 2020

Apr 30, 2019

Oct 27, 2018