{"product_id":"9781806106370","title":"Data Engineering with Azure Databricks: Design, build, and optimize scalable data pipelines and analytics solutions with Azure Databricks","description":"\u003cb\u003eMaster end-to-end data engineering on Azure Databricks. From data ingestion and Delta Lake to CI\/CD and real-time streaming, build secure, scalable, and performant data solutions with Spark, Unity Catalog, and ML tools.\u003c\/b\u003e\u003cul\u003e\n\u003cli\u003eBuild scalable data pipelines using Apache Spark and Delta Lake\u003c\/li\u003e\n\u003cli\u003eAutomate workflows and manage data governance with Unity Catalog\u003c\/li\u003e\n\u003cli\u003eLearn real-time processing and structured streaming with practical use cases\u003c\/li\u003e\n\u003cli\u003eImplement CI\/CD, DevOps, and security for production-ready data solutions\u003c\/li\u003e\n\u003cli\u003eExplore Databricks-native ML, AutoML, and Generative AI integration\u003c\/li\u003e\n\u003c\/ul\u003e\"Data Engineering with Azure Databricks\" is your essential guide to building scalable, secure, and high-performing data pipelines using the powerful Databricks platform on Azure. Designed for data engineers, architects, and developers, this book demystifies the complexities of Spark-based workloads, Delta Lake, Unity Catalog, and real-time data processing. Beginning with the foundational role of Azure Databricks in modern data engineering, you’ll explore how to set up robust environments, manage data ingestion with Auto Loader, optimize Spark performance, and orchestrate complex workflows using tools like Azure Data Factory and Airflow. The book offers deep dives into structured streaming, Delta Live Tables, and Delta Lake’s ACID features for data reliability and schema evolution. You’ll also learn how to manage security, compliance, and access controls using Unity Catalog, and gain insights into managing CI\/CD pipelines with Azure DevOps and Terraform. With a special focus on machine learning and generative AI, the final chapters guide you in automating model workflows, leveraging MLflow, and fine-tuning large language models on Databricks. Whether you're building a modern data lakehouse or operationalizing analytics at scale, this book provides the tools and insights you need.\u003cul\u003e\n\u003cli\u003eSet up a full-featured Azure Databricks environment\u003c\/li\u003e\n\u003cli\u003eImplement batch and streaming ingestion using Auto Loader\u003c\/li\u003e\n\u003cli\u003eOptimize Spark jobs with partitioning and caching\u003c\/li\u003e\n\u003cli\u003eBuild real-time pipelines with structured streaming and DLT\u003c\/li\u003e\n\u003cli\u003eManage data governance using Unity Catalog\u003c\/li\u003e\n\u003cli\u003eOrchestrate production workflows with jobs and ADF\u003c\/li\u003e\n\u003cli\u003eApply CI\/CD best practices with Azure DevOps and Git\u003c\/li\u003e\n\u003cli\u003eSecure data with RBAC, encryption, and compliance standards\u003c\/li\u003e\n\u003cli\u003eUse MLflow and Feature Store for ML pipelines\u003c\/li\u003e\n\u003cli\u003eBuild generative AI applications in Databricks\u003c\/li\u003e\n\u003c\/ul\u003e\u003cp\u003eThis book is for data engineers, solution architects, cloud professionals, and software engineers seeking to build robust and scalable data pipelines using Azure Databricks. Whether you're migrating legacy systems, implementing a modern lakehouse architecture, or optimizing data workflows for performance, this guide will help you leverage the full power of Databricks on Azure. A basic understanding of Python, Spark, and cloud infrastructure is recommended.\u003c\/p\u003e","brand":"Packt Publishing","offers":[{"title":"Default Title","offer_id":46665943974129,"sku":"9781806106370","price":49.99,"currency_code":"USD","in_stock":true}],"thumbnail_url":"\/\/cdn.shopify.com\/s\/files\/1\/0674\/5433\/7265\/files\/9781806106370_p0.jpg?v=1777055264","url":"https:\/\/shop.barnesandnoble.com\/products\/9781806106370","provider":"Barnes \u0026 Noble","version":"1.0","type":"link"}