In her post, Xiaoxu introduces the concept of a communication lifecycle for data pipelines. Proactive feedback to stakeholders is essential for establishing trust and building confidence in data. Xiaoxu demonstrates how Mage’s callbacks can be used to automate tasks based on block status, reducing the communication lifecycle and establishing a healthy relationship between data owners and consumers.
Sri Nikitha walks through a demo of Mage: from installation to a sample pipeline that loads data from a NYC taxi dataset, transforms it, and exports to BigQuery.
Using Mage and Python for orchestration and ingestion, Darshil walks through how to take his dataset from source to visualization using a combination of popular tools, including Mage, GCS, BigQuery, and Looker Studio.
Arul discusses the process of constructing an ETL (extract-transform-load) pipeline in Mage. Starting from Netflix’s top 100 movie dataset, Arul extracts data, transforms it to a useable format, ans loads it into a database for further analysis.
