https://azurecharts.com/ Latest Azure updates provided by Azure Charts via Azure Terminal aka.ms/aztty RSS feed Wed, 27 May 2026 18:06:41 Z https://aztty.azurewebsites.net/rss/updates?service=8 https://techcommunity.microsoft.com/blog/azurearchitectureblog/how-to-secure-azure-databricks-without-public-exposure-using-waf--private-endpoi/4517721 https://techcommunity.microsoft.com/blog/azurearchitectureblog/how-to-secure-azure-databricks-without-public-exposure-using-waf--private-endpoi/4517721 Announcement This blog outlines a Zero Trust–aligned architecture for securing Azure Databricks using Application Gateway (WAF) and Private Endpoints within a Hub-Spoke network model. Enables a true Zero Trust ...<br />Update Type: Announcement, Services: Databricks, Application Gateway, Categories: Mon, 11 May 2026 23:37:00 Z 2026-05-11T23:37:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/secure-medallion-architecture-pattern-on-azure-databricks-part-ii/4491044 https://techcommunity.microsoft.com/blog/analyticsonazure/secure-medallion-architecture-pattern-on-azure-databricks-part-ii/4491044 Announcement Disclaimer: The views in this article are my own and do not represent Microsoft or Databricks. This article is part of a series focused on deploying a secure Medallion Architecture. The series fo...<br />Update Type: Announcement, Services: Databricks, Categories: Fri, 08 May 2026 15:13:00 Z 2026-05-08T15:13:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/resilient-by-design-azure-databricks-disaster-recovery-strategy/4516464 https://techcommunity.microsoft.com/blog/analyticsonazure/resilient-by-design-azure-databricks-disaster-recovery-strategy/4516464 Announcement Introduction: From Recovery Plans to Resilience Strategy  As organizations increasingly rely on Azure Databricks for mission-critical analytics and data engineering workloads, the need for robust d...<br />Update Type: Announcement, Services: Databricks, Categories: Wed, 06 May 2026 16:28:00 Z 2026-05-06T16:28:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/from-chaos-to-clarity-your-databricks-workspace-on-a-single-pane-of-glass/4516805 https://techcommunity.microsoft.com/blog/analyticsonazure/from-chaos-to-clarity-your-databricks-workspace-on-a-single-pane-of-glass/4516805 Announcement The question that never stays answered — until now As Azure Databricks workspaces evolve, complexity creeps in unnoticed. Every Azure Databricks conversation with customers eventually lands on the ...<br />Update Type: Announcement, Services: Databricks, Categories: Wed, 06 May 2026 16:25:00 Z 2026-05-06T16:25:00Z https://azure.microsoft.com/updates?id=560590 https://azure.microsoft.com/updates?id=560590 GA Azure Databricks now supports Anthropic Claude Opus 4.7 through Azure Databricks AI Model Serving. Claude Opus 4.7 is Anthropic's most capable hybrid reasoning model, delivering stronger performance on complex extraction and agentic reasoning tasks while<br />Update Type: GA, Services: Databricks, Categories: Features Fri, 17 Apr 2026 21:30:12 Z 2026-04-17T21:30:12Z https://techcommunity.microsoft.com/blog/azurearchitectureblog/databricks-delta-sharing-enabling-cost-efficient-cross-cloud-data-access/4511400 https://techcommunity.microsoft.com/blog/azurearchitectureblog/databricks-delta-sharing-enabling-cost-efficient-cross-cloud-data-access/4511400 Announcement What Is Delta Sharing? Delta sharing is an open protocol for secure data sharing that allows organizations to share live data stored in Delta Lake with external consumers—across cloud providers suc...<br />Update Type: Announcement, Services: Databricks, Categories: Thu, 16 Apr 2026 00:00:00 Z 2026-04-16T00:00:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/azure-managed-redis--azure-databricks-real-time-feature-serving-for-low-latency-/4503116 https://techcommunity.microsoft.com/blog/analyticsonazure/azure-managed-redis--azure-databricks-real-time-feature-serving-for-low-latency-/4503116 Announcement This blog content has been a collective collaboration between the Azure Databricks and Azure Managed Redis Product and Product Marketing teams. Executive summary  Modern decisioning systems, frau...<br />Update Type: Announcement, Services: Databricks, Managed Redis, Categories: Wed, 25 Mar 2026 09:00:00 Z 2026-03-25T09:00:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/azure-managed-redis--azure-databricks-real-time-feature-serving-for-low-latency-/4503116 https://techcommunity.microsoft.com/blog/analyticsonazure/azure-managed-redis--azure-databricks-real-time-feature-serving-for-low-latency-/4503116 Announcement This blog content has been a collective collaboration between the Azure Databricks and Azure Managed Redis Product and Product Marketing teams. Executive summary  Modern decisioning systems, frau...<br />Update Type: Announcement, Services: Databricks, Managed Redis, Categories: Wed, 25 Mar 2026 08:00:00 Z 2026-03-25T08:00:00Z https://azure.microsoft.com/updates?id=558927 https://azure.microsoft.com/updates?id=558927 Preview You can now enable OneLake catalog federation in Azure Databricks to query Microsoft Fabric OneLake data without copying or moving it. By connecting Unity Catalog to a OneLake catalog, Azure Databricks automatically syncs schemas and tables so Fabric Lake<br />Update Type: Preview, Services: Databricks, Microsoft Fabric, Categories: Features Thu, 19 Mar 2026 17:00:53 Z 2026-03-19T17:00:53Z https://azure.microsoft.com/updates?id=558810 https://azure.microsoft.com/updates?id=558810 GA Azure Databricks now offers a Lakeflow Connect Free Tier that gives each workspace 100 free DBUs per workspace per day for ingestion from SaaS applications and databases. This allows you to bring roughly 100 million records of data per workspace per day i<br />Update Type: GA, Services: Databricks, Categories: Features Wed, 18 Mar 2026 17:00:05 Z 2026-03-18T17:00:05Z https://techcommunity.microsoft.com/blog/analyticsonazure/announcing-the-new-home-for-the-azure-databricks-blog/4503570 https://techcommunity.microsoft.com/blog/analyticsonazure/announcing-the-new-home-for-the-azure-databricks-blog/4503570 Announcement We’re excited to share that the Azure Databricks blog has moved to a new address on Microsoft Tech Community Hub! Azure Databricks | Microsoft Community Hub Our new blog home is designed to make ...<br />Update Type: Announcement, Services: Databricks, Categories: Wed, 18 Mar 2026 10:44:00 Z 2026-03-18T10:44:00Z https://azure.microsoft.com/updates?id=558060 https://azure.microsoft.com/updates?id=558060 GA Azure Databricks workspaces have two network configurations – Azure Databricks-managed VNet and VNet Injection. The ability to update the Network Configuration of your workspace is now generally available. This enhancement provides greater flexibility and<br />Update Type: GA, Services: Databricks, Virtual Network, Categories: Features Mon, 02 Mar 2026 19:15:14 Z 2026-03-02T19:15:14Z https://azure.microsoft.com/updates?id=557991 https://azure.microsoft.com/updates?id=557991 GA Today we’re announcing the General Availability of Azure Databricks Lakebase, a managed PostgreSQL environment with next generation separation of storage and compute for OLTP, providing instant availability, instant clones, and scale-to-zero. Lakebase let<br />Update Type: GA, Services: Machine Learning, Databricks, Database for PostgreSQL, Categories: Features Mon, 02 Mar 2026 19:15:14 Z 2026-03-02T19:15:14Z https://blog.fabric.microsoft.com/en-US/blog/zero-copy-access-to-onelake-data-in-azure-databricks-preview/ https://blog.fabric.microsoft.com/en-US/blog/zero-copy-access-to-onelake-data-in-azure-databricks-preview/ Preview Most data estates are not single platform, and that is not a problem. The challenge is what usually comes next: extra copies, extra pipelines, extra refresh schedules, and endless debates about which version is the truth. Today, we are introducing OneLake catalog federation (Beta) in Azure Databricks Lakehouse Federation, which simplifies multi-engine analytics by enabling … Continue reading “Zero-copy access to OneLake data in Azure Databricks (Preview)”<br />Update Type: Preview, Services: Databricks, Categories: Mon, 23 Feb 2026 12:00:00 Z 2026-02-23T12:00:00Z https://azure.microsoft.com/updates?id=557595 https://azure.microsoft.com/updates?id=557595 GA Azure Databricks now supports Anthropic Claude Sonnet 4.6 through Azure Databricks AI Model Serving. Claude Sonnet 4.6 delivers frontier-level performance on complex coding, agentic workflows, and professional knowledge work including reasoning at a lower<br />Update Type: GA, Services: Databricks, Categories: Features Wed, 18 Feb 2026 18:00:55 Z 2026-02-18T18:00:55Z https://techcommunity.microsoft.com/blog/appsonazureblog/mcp-driven-azure-sre-for-databricks/4494630 https://techcommunity.microsoft.com/blog/appsonazureblog/mcp-driven-azure-sre-for-databricks/4494630 Announcement Azure SRE Agent is an AI-powered operations assistant built for incident response and governance. MCP (Model Context Protocol) is the standard interface it uses to connect to external systems and too...<br />Update Type: Announcement, Services: Databricks, SRE Agent, Categories: Thu, 12 Feb 2026 08:53:00 Z 2026-02-12T08:53:00Z https://techcommunity.microsoft.com/blog/azurearchitectureblog/securing-multi-agent-ai-with-user-context-entra-id-obo-for-databricks-genie-/4493308 https://techcommunity.microsoft.com/blog/azurearchitectureblog/securing-multi-agent-ai-with-user-context-entra-id-obo-for-databricks-genie-/4493308 Announcement How we built an enterprise-grade multi-agent system that preserves user identity across AI agents and Databricks Introduction When building AI-powered applications for the enterprise, a common ch...<br />Update Type: Announcement, Services: Databricks, Microsoft Entra ID, Categories: Wed, 11 Feb 2026 22:27:00 Z 2026-02-11T22:27:00Z https://azure.microsoft.com/updates?id=557081 https://azure.microsoft.com/updates?id=557081 GA Introducing Supervisor Agent, now Generally Available in Azure Databricks. Supervisor Agent is part of the Azure Databricks Agent Bricks Platform. Supervisor Agent brings a breakthrough in agentic intelligence—using a dynamic supervisor pattern that under<br />Update Type: GA, Services: Databricks, Categories: Features Tue, 10 Feb 2026 18:00:43 Z 2026-02-10T18:00:43Z https://azure.microsoft.com/updates?id=555981 https://azure.microsoft.com/updates?id=555981 GA Azure Databricks now supports Anthropic Claude Opus 4.6 through Mosaic AI Model serving. Claude Opus 4.6 is Anthropic's most advanced model, delivering state-of-the-art performance on agentic coding, complex reasoning, and knowledge work tasks. The model<br />Update Type: GA, Services: Databricks, Categories: Fri, 06 Feb 2026 19:00:01 Z 2026-02-06T19:00:01Z https://azure.microsoft.com/updates?id=550845 https://azure.microsoft.com/updates?id=550845 GA Serverless workspaces in Azure Databricks are now generally available, offering a fully managed workspace type that comes preconfigured with serverless compute and default storage. They deliver an enterprise-ready SaaS experience with zero infrastructure<br />Update Type: GA, Services: Databricks, Categories: Mon, 02 Feb 2026 18:15:39 Z 2026-02-02T18:15:39Z https://techcommunity.microsoft.com/blog/analyticsonazure/serverless-workspaces-are-generally-available-in-azure-databricks/4491314 https://techcommunity.microsoft.com/blog/analyticsonazure/serverless-workspaces-are-generally-available-in-azure-databricks/4491314 GA Recently, we announced that Serverless Workspaces in public preview. Today, we are excited to share that Serverless Workspaces are generally available in Azure Databricks. Azure Databricks now offers...<br />Update Type: GA, Services: Databricks, Categories: Mon, 02 Feb 2026 09:15:00 Z 2026-02-02T09:15:00Z https://blog.fabric.microsoft.com/en-US/blog/mirroring-azure-databricks-catalogs-from-azure-databricks-workspaces-behind-private-endpoints-generally-available/ https://blog.fabric.microsoft.com/en-US/blog/mirroring-azure-databricks-catalogs-from-azure-databricks-workspaces-behind-private-endpoints-generally-available/ GA Enterprises increasingly deploy Azure Databricks workspaces in locked-down network environments where public network access is disabled, and all access is routed through private endpoints. While this security posture is essential for meeting enterprise and regulatory requirements, it has historically limited integration scenarios for the Mirrored Azure Databricks catalog item in Microsoft Fabric. Earlier, the Mirrored … Continue reading “Mirroring Azure Databricks catalogs from Azure Databricks workspaces behind private endpoints (Generally Available) “<br />Update Type: GA, Services: Databricks, Microsoft Fabric, Categories: Security Mon, 02 Feb 2026 09:00:00 Z 2026-02-02T09:00:00Z https://azure.microsoft.com/updates?id=542455 https://azure.microsoft.com/updates?id=542455 GA Azure Databricks Agent Bricks knowledge assistant empowers you to create, deploy, and manage AI agents directly within the Azure Databricks platform, using unified data and AI capabilities. This feature simplifies agent development by providing prebuilt c<br />Update Type: GA, Services: Databricks, Categories: Wed, 28 Jan 2026 18:45:08 Z 2026-01-28T18:45:08Z https://techcommunity.microsoft.com/blog/analyticsonazure/azure-databricks--fabric-disaster-recovery-the-better-together-story/4481323 https://techcommunity.microsoft.com/blog/analyticsonazure/azure-databricks--fabric-disaster-recovery-the-better-together-story/4481323 Announcement Disaster recovery (DR) is a critical component of any cloud-native data analytics platform, ensuring business continuity even during rare regional outages caused by natural disasters, infrastructure ...<br />Update Type: Announcement, Services: Databricks, Categories: Sat, 27 Dec 2025 01:24:00 Z 2025-12-27T01:24:00Z https://azure.microsoft.com/updates?id=536907 https://azure.microsoft.com/updates?id=536907 Preview You can now receive insights where collaboration happens—in Microsoft Teams—with expanded dashboard subscription capabilities in Azure Databricks. This enhancement delivers timely, actionable data directly into team workflows, reducing the need to check e<br />Update Type: Preview, Services: Databricks, Categories: Thu, 11 Dec 2025 20:30:24 Z 2025-12-11T20:30:24Z https://techcommunity.microsoft.com/blog/analyticsonazure/end-to-end-observability-for-azure-databricks-from-infrastructure-to-internal-ap/4475692 https://techcommunity.microsoft.com/blog/analyticsonazure/end-to-end-observability-for-azure-databricks-from-infrastructure-to-internal-ap/4475692 Announcement Author's: Amudha Palani, Peter Lo PeterLo​ , Rafia Aqil Rafia_Aqil​      Observability in Azure Databricks is the ability to continuously monitor and troubleshoot the health, performance, and...<br />Update Type: Announcement, Services: Databricks, Categories: Sun, 07 Dec 2025 09:46:00 Z 2025-12-07T09:46:00Z https://azure.microsoft.com/updates?id=536721 https://azure.microsoft.com/updates?id=536721 Preview Serverless workspaces in Azure Databricks are now available in Public Preview, offering a fully managed workspace type that comes preconfigured with serverless compute and default storage to deliver an enterprise-ready SaaS experience with minimal infrast<br />Update Type: Preview, Services: Databricks, Categories: Thu, 04 Dec 2025 16:15:01 Z 2025-12-04T16:15:01Z https://techcommunity.microsoft.com/blog/analyticsonazure/serverless-workspaces-are-live-in-azure-databricks/4474712 https://techcommunity.microsoft.com/blog/analyticsonazure/serverless-workspaces-are-live-in-azure-databricks/4474712 Announcement Serverless workspaces are now in Public Preview in Azure Databricks<br />Update Type: Announcement, Services: Databricks, Categories: Tue, 02 Dec 2025 23:08:00 Z 2025-12-02T23:08:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/guide-for-architecting-azure-databricks-design-to-deployment/4473095 https://techcommunity.microsoft.com/blog/analyticsonazure/guide-for-architecting-azure-databricks-design-to-deployment/4473095 Announcement Author's: Chris Walk cwalk​, Dan Johnson danjohn1234​, Eduardo dos Santos eduardomdossantos​, Ted Kim tekim​, Eric Kwashie ekwashie​, Chris Hayes Chris_Haynes​, Tayo Akigbogun takigbogun​ and Rafia A...<br />Update Type: Announcement, Services: Databricks, Categories: Wed, 26 Nov 2025 22:54:00 Z 2025-11-26T22:54:00Z https://azure.microsoft.com/updates?id=527678 https://azure.microsoft.com/updates?id=527678 Preview Azure Databricks Genie is now integrated with Microsoft Foundry and in public preview. You can now connect Genie spaces directly to Microsoft Foundry agents using Model Context Protocol (MCP), giving you seamless access to trusted data for building intell<br />Update Type: Preview, Services: Databricks, Categories: Wed, 19 Nov 2025 17:00:30 Z 2025-11-19T17:00:30Z https://azure.microsoft.com/updates?id=527668 https://azure.microsoft.com/updates?id=527668 Preview Azure Databricks Genie is now available in Microsoft Copilot Studio, giving you direct access to intelligent, enterprise-grade AI answers across your full Azure Databricks data estate. This integration unlocks enterprise-grade, conversational analytics ac<br />Update Type: Preview, Services: Databricks, Categories: Wed, 19 Nov 2025 17:00:30 Z 2025-11-19T17:00:30Z https://techcommunity.microsoft.com/blog/analyticsonazure/azure-databricks-genie-integration-with-copilot-studio-and-microsoft-foundry-is-/4471087 https://techcommunity.microsoft.com/blog/analyticsonazure/azure-databricks-genie-integration-with-copilot-studio-and-microsoft-foundry-is-/4471087 Announcement This blog was co-authored by Toussaint Webb, Databricks   We’re excited to announce the Public Preview availability of AI/BI Genie in Microsoft Copilot Studio and Microsoft Foundry via MCP. This ...<br />Update Type: Announcement, Services: Databricks, Categories: Tue, 18 Nov 2025 18:48:00 Z 2025-11-18T18:48:00Z https://blog.fabric.microsoft.com/en-US/blog/microsoft-and-databricks-advancing-openness-and-interoperability-with-onelake/ https://blog.fabric.microsoft.com/en-US/blog/microsoft-and-databricks-advancing-openness-and-interoperability-with-onelake/ Announcement Read and Write integration announcement between Microsoft OneLake and Azure Databricks<br />Update Type: Announcement, Services: Databricks, Categories: Tue, 18 Nov 2025 08:00:00 Z 2025-11-18T08:00:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/azure-databricks-cost-optimization-a-practical-guide/4470235 https://techcommunity.microsoft.com/blog/analyticsonazure/azure-databricks-cost-optimization-a-practical-guide/4470235 Announcement Co-Authored by Sanjeev Nair​    This guide walks through a proven approach to Databricks cost optimization, structured in three phases: Discovery, Cluster/Data/Code Best Practices, and Team Align...<br />Update Type: Announcement, Services: Databricks, Categories: Sat, 15 Nov 2025 07:21:00 Z 2025-11-15T07:21:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/secure-delta-sharing-between-databricks-workspaces-using-ncc-and-private-endpoin/4462428 https://techcommunity.microsoft.com/blog/analyticsonazure/secure-delta-sharing-between-databricks-workspaces-using-ncc-and-private-endpoin/4462428 Announcement This guide walks you through the steps to share Delta tables between two Databricks workspaces (NorthCentral and SouthCentral) and configure Network Connectivity Configuration (NCC) for a Serverless ...<br />Update Type: Announcement, Services: Databricks, Categories: Sat, 18 Oct 2025 00:44:00 Z 2025-10-18T00:44:00Z https://azure.microsoft.com/updates?id=511743 https://azure.microsoft.com/updates?id=511743 GA SAP Business Data Cloud (BDC) Connect to Azure Databricks is now generally available, enabling secure, bi-directional, zero-copy data sharing between SAP BDC and Azure Databricks using Delta Sharing, so teams can unify SAP and external data for analytics,<br />Update Type: GA, Services: Machine Learning, Databricks, Categories: Features Wed, 15 Oct 2025 17:00:38 Z 2025-10-15T17:00:38Z https://techcommunity.microsoft.com/blog/analyticsonazure/sap-business-data-cloud-connect-with-azure-databricks-is-now-generally-available/4459490 https://techcommunity.microsoft.com/blog/analyticsonazure/sap-business-data-cloud-connect-with-azure-databricks-is-now-generally-available/4459490 GA We are excited to share that SAP Business Data Cloud (SAP BDC) Connect for Azure Databricks is generally available. With this announcement, Azure Databricks customers like you, can connect your SAP B...<br />Update Type: GA, Services: Databricks, Categories: Wed, 15 Oct 2025 16:00:00 Z 2025-10-15T16:00:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/secure-medallion-architecture-pattern-on-azure-databricks-part-i/4459268 https://techcommunity.microsoft.com/blog/analyticsonazure/secure-medallion-architecture-pattern-on-azure-databricks-part-i/4459268 Announcement This article presents a security-first pattern for Azure Databricks: a Medallion Architecture where Bronze, Silver and Gold each run as their Lakeflow Job and cluster, orchestrated by a parent job. R...<br />Update Type: Announcement, Services: Databricks, Categories: Tue, 07 Oct 2025 06:25:00 Z 2025-10-07T06:25:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/securing-azure-databricks-serverless-practical-guide-to-private-link-integration/4457083 https://techcommunity.microsoft.com/blog/analyticsonazure/securing-azure-databricks-serverless-practical-guide-to-private-link-integration/4457083 Announcement The Challenge: Outbound Control in a Serverless World Serverless compute resources run in the serverless compute plane, and is managed by Microsoft, for ease of use.  Databricks serverless provides...<br />Update Type: Announcement, Services: Databricks, Private Link, Categories: Thu, 25 Sep 2025 23:42:00 Z 2025-09-25T23:42:00Z https://azure.microsoft.com/updates?id=503408 https://azure.microsoft.com/updates?id=503408 Preview Databricks One in Azure Databricks delivers a unified, simplified experience that brings together data engineering, analytics, and AI development in a single, governed platform. By integrating collaborative workflows, enterprise-grade governance, and perf<br />Update Type: Preview, Services: Databricks, Categories: Features Wed, 17 Sep 2025 16:30:45 Z 2025-09-17T16:30:45Z https://azure.microsoft.com/updates?id=502623 https://azure.microsoft.com/updates?id=502623 Deprecation We are announcing the retirement of Azure Databricks Standard Tier. Azure subscriptions will no longer support creation of Azure Databricks Standard Tier workspaces after April 1st, 2026. Starting October 1st, 2026, all Azure Databricks Standard tier wo<br />Update Type: Deprecation, Services: Databricks, Categories: Retirements Tue, 16 Sep 2025 00:15:32 Z 2025-09-16T00:15:32Z https://techcommunity.microsoft.com/blog/analyticsonazure/approaches-to-integrating-azure-databricks-with-microsoft-fabric-the-better-toge/4453643 https://techcommunity.microsoft.com/blog/analyticsonazure/approaches-to-integrating-azure-databricks-with-microsoft-fabric-the-better-toge/4453643 Announcement Azure Databricks and Microsoft Fabric can be combined to create a unified and scalable analytics ecosystem. This document outlines eight distinct integration approaches, each accompanied by step-by-s...<br />Update Type: Announcement, Services: Databricks, Microsoft Fabric, Categories: Fri, 12 Sep 2025 21:28:00 Z 2025-09-12T21:28:00Z https://azure.microsoft.com/updates?id=502633 https://azure.microsoft.com/updates?id=502633 GA Now you can streamline identity lifecycle operations using Azure Databricks automatic identity management. With this new, generally available capability, you can automate user provisioning and deprovisioning through native integration with Microsoft Entra<br />Update Type: GA, Services: Databricks, Microsoft Entra ID, Categories: Features, Management Wed, 10 Sep 2025 21:30:10 Z 2025-09-10T21:30:10Z https://techcommunity.microsoft.com/blog/analyticsonazure/general-availability-automatic-identity-management-aim-for-entra-id-on-azure-dat/4452206 https://techcommunity.microsoft.com/blog/analyticsonazure/general-availability-automatic-identity-management-aim-for-entra-id-on-azure-dat/4452206 GA In February, we announced that Automatic Identity Management in public preview and loved to hear your overwhelmingly positive feedback. Prior to public preview, you either had to set up an Entra Ente...<br />Update Type: GA, Services: Databricks, Microsoft Entra ID, Categories: Management Wed, 10 Sep 2025 16:00:00 Z 2025-09-10T16:00:00Z https://azure.microsoft.com/updates?id=500583 https://azure.microsoft.com/updates?id=500583 GA Azure Databricks connector for Power Platform is now generally available. You can easily connect Azure Databricks with Microsoft Power Apps in several new ways. The new connector enables you to: Support real-time data access without the need for data cop<br />Update Type: GA, Services: Databricks, Categories: Features Thu, 14 Aug 2025 16:00:43 Z 2025-08-14T16:00:43Z https://techcommunity.microsoft.com/blog/analyticsonazure/closing-the-loop-interactive-write-back-from-power-bi-to-azure-databricks/4442999 https://techcommunity.microsoft.com/blog/analyticsonazure/closing-the-loop-interactive-write-back-from-power-bi-to-azure-databricks/4442999 Announcement This is a collaborative post from Microsoft and Databricks. We thank Toussaint Webb, Product Manager at Databricks, for his contributions. We're excited to announce that the Azure Databricks conne...<br />Update Type: Announcement, Services: Databricks, Power BI Embedded, Categories: Thu, 14 Aug 2025 09:00:00 Z 2025-08-14T09:00:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/supercharge-data-intelligence-build-teams-app-with-azure-databricks-genie--azure/4442653 https://techcommunity.microsoft.com/blog/analyticsonazure/supercharge-data-intelligence-build-teams-app-with-azure-databricks-genie--azure/4442653 Announcement Introduction Are you looking to unlock the full potential of your data investments in Azure Databricks while seamlessly harnessing the capabilities of Azure AI? At Microsoft BUILD 2025, we made the...<br />Update Type: Announcement, Services: Azure AI Services, Databricks, Categories: Tue, 12 Aug 2025 21:10:00 Z 2025-08-12T21:10:00Z https://azure.microsoft.com/updates?id=498573 https://azure.microsoft.com/updates?id=498573 GA Announcing the general availability of Mirroring for Azure Databricks Unity Catalog in Microsoft Fabric—a secure, high-performance integration that provides seamless access to Azure Databricks tables from Fabric. With Fabric and Azure Databricks, we are<br />Update Type: GA, Services: Databricks, Power BI Embedded, Microsoft Fabric, Categories: Features Tue, 15 Jul 2025 18:15:05 Z 2025-07-15T18:15:05Z https://blog.fabric.microsoft.com/en-US/blog/unified-by-design-mirroring-azure-databricks-unity-catalog-in-microsoft-fabric-now-generally-available/ https://blog.fabric.microsoft.com/en-US/blog/unified-by-design-mirroring-azure-databricks-unity-catalog-in-microsoft-fabric-now-generally-available/ GA We are thrilled to announce the general availability of Mirroring for Azure Databricks Unity Catalog in Microsoft Fabric—a secure, high-performance integration that provides seamless access to Azure Databricks tables from Fabric. With Fabric and Azure Databricks, we are building the future of data platforms on a lakehouse foundation, powered by open data formats, full interoperability, … Continue reading “Unified by design: mirroring Azure Databricks Unity Catalog to Microsoft OneLake in Fabric (Generally Available)”<br />Update Type: GA, Services: Databricks, Microsoft Fabric, Categories: Tue, 15 Jul 2025 08:31:37 Z 2025-07-15T08:31:37Z https://azure.microsoft.com/updates?id=490306 https://azure.microsoft.com/updates?id=490306 Preview Now you can easily connect Azure Databricks with Microsoft Power Apps in several new ways. The new connector enables you to: Support real-time data access without the need for data copying. Create dynamic applications that take advanta<br />Update Type: Preview, Services: Databricks, Categories: Features Wed, 11 Jun 2025 17:15:10 Z 2025-06-11T17:15:10Z https://techcommunity.microsoft.com/blog/analyticsonazure/announcing-the-azure-databricks-connector-in-power-platform/4422647 https://techcommunity.microsoft.com/blog/analyticsonazure/announcing-the-azure-databricks-connector-in-power-platform/4422647 Announcement We are ecstatic to announce the public preview of the Azure Databricks Connector for Power Platform. This native connector is specifically for Power Apps, Power Automation, and Copilot Studio wi...<br />Update Type: Announcement, Services: Databricks, Categories: Wed, 11 Jun 2025 13:00:00 Z 2025-06-11T13:00:00Z https://blog.fabric.microsoft.com/en-US/blog/secure-mirrored-azure-databricks-data-in-fabric-with-onelake-security/ https://blog.fabric.microsoft.com/en-US/blog/secure-mirrored-azure-databricks-data-in-fabric-with-onelake-security/ Announcement We’re excited to announce that OneLake security capabilities have been extended to support mirrored data through Azure Mirrored Databricks Catalog. This enhancement brings the full suite of OneLake’s enterprise-grade security features to these mirrored assets, empowering organizations to manage access using table, column, or row level security across all engines.  What’s New?  With this update, Azure … Continue reading “Secure Mirrored Azure Databricks Data in Fabric with OneLake security “<br />Update Type: Announcement, Services: Databricks, Categories: Features, Security Tue, 03 Jun 2025 09:00:00 Z 2025-06-03T09:00:00Z https://blog.fabric.microsoft.com/en-US/blog/integrating-fabric-with-databricks-using-private-network/ https://blog.fabric.microsoft.com/en-US/blog/integrating-fabric-with-databricks-using-private-network/ Announcement Microsoft Fabric and Azure Databricks are widely used data platforms. This article aims to address the requirement of customers who have large data estates in Azure databricks and want to unlock additional use cases in Microsoft Fabric arising out of different business teams. When integrating the two platforms, a crucial security requirement is to ensure … Continue reading “Integrating Fabric with Databricks using private network”<br />Update Type: Announcement, Services: Databricks, Microsoft Fabric, Categories: Security Thu, 29 May 2025 11:00:00 Z 2025-05-29T11:00:00Z https://azure.microsoft.com/updates?id=490578 https://azure.microsoft.com/updates?id=490578 GA Unity Catalog access to Amazon S3 data on Azure Databricks is now generally available. This release gives you seamless, multicloud data governance by unifying permissions and access controls across S3 and Azure Data Lake Storage.With cross-cloud data gove<br />Update Type: GA, Services: Databricks, Data Lake Storage, Categories: Features Fri, 23 May 2025 02:45:39 Z 2025-05-23T02:45:39Z https://techcommunity.microsoft.com/blog/analyticsonazure/announcing-general-availability-of-cross-cloud-data-governance-with-azure-databr/4413859 https://techcommunity.microsoft.com/blog/analyticsonazure/announcing-general-availability-of-cross-cloud-data-governance-with-azure-databr/4413859 GA We are excited to announce the general availability of accessing AWS S3 data in Azure Databricks Unity Catalog. This release simplifies cross-cloud data governance by allowing teams to configure and ...<br />Update Type: GA, Services: Databricks, Categories: Wed, 21 May 2025 16:00:00 Z 2025-05-21T16:00:00Z https://azure.microsoft.com/updates?id=490296 https://azure.microsoft.com/updates?id=490296 Preview You can now easily create AI agents in Azure AI Foundry with Azure Databricks data. Using the new Azure Databricks Azure AI Foundry connector, you can: Connect Azure AI Foundry agents with Azure Databricks for real-time data processing. Build and dep<br />Update Type: Preview, Services: Azure AI Services, Databricks, Azure AI Foundry, Categories: Features Mon, 19 May 2025 18:45:26 Z 2025-05-19T18:45:26Z https://techcommunity.microsoft.com/blog/azurenetworkingblog/cross-tenant-connectivity-between-databricks-and-storage-account-using-private-l/4406700 https://techcommunity.microsoft.com/blog/azurenetworkingblog/cross-tenant-connectivity-between-databricks-and-storage-account-using-private-l/4406700 Announcement High Level Architecture Cross-Tenant Architecture Overview The setup involves two separate Azure tenants: Tenant A: Hosts the Storage Account. Tenant B: Hosts Azure Databricks. ...<br />Update Type: Announcement, Services: Databricks, Private Link, Categories: Mon, 19 May 2025 17:47:00 Z 2025-05-19T17:47:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/announcing-the-availability-of-azure-databricks-connector-in-azure-ai-foundry/4415263 https://techcommunity.microsoft.com/blog/analyticsonazure/announcing-the-availability-of-azure-databricks-connector-in-azure-ai-foundry/4415263 Announcement At Microsoft, Databricks Data Intelligence Platform is available as a fully managed, native, first party Data and AI solution called Azure Databricks. This makes Azure the optimal cloud for runn...<br />Update Type: Announcement, Services: Azure AI Services, Databricks, Azure AI Foundry, Categories: Mon, 19 May 2025 16:00:00 Z 2025-05-19T16:00:00Z https://techcommunity.microsoft.com/blog/azureinfrastructureblog/designing-reusable-bicep-modules-a-databricks-example/4415185 https://techcommunity.microsoft.com/blog/azureinfrastructureblog/designing-reusable-bicep-modules-a-databricks-example/4415185 Announcement In this blog, I’ll walk ythrough how to design a reusable Bicep module for deploying Azure Databricks, a popular analytics and machine learning platform. We'll focus on creating a parameterized and c...<br />Update Type: Announcement, Services: Machine Learning, Databricks, Categories: Mon, 19 May 2025 14:19:00 Z 2025-05-19T14:19:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/announcing-the-availability-of-azure-databricks-connector-in-azure-ai-foundry/4414889 https://techcommunity.microsoft.com/blog/analyticsonazure/announcing-the-availability-of-azure-databricks-connector-in-azure-ai-foundry/4414889 Announcement At Microsoft, Databricks Data Intelligence Platform is available as a fully managed, native, first party Data and AI solution called Azure Databricks. This makes Azure the optimal cloud for ru...<br />Update Type: Announcement, Services: Azure AI Services, Databricks, Azure AI Foundry, Categories: Mon, 19 May 2025 03:28:00 Z 2025-05-19T03:28:00Z https://techcommunity.microsoft.com/blog/azureinfrastructureblog/deploy-and-manage-azure-databricks-infrastructure-using-terraform-and-azure-devo/4413774 https://techcommunity.microsoft.com/blog/azureinfrastructureblog/deploy-and-manage-azure-databricks-infrastructure-using-terraform-and-azure-devo/4413774 Announcement Overview The codebase is organized to support the following processes: Databricks Infrastructure Provisioning Instance pools Shared clusters Secret scopes (integrated with Azure Key Vaul...<br />Update Type: Announcement, Services: Databricks, Categories: Wed, 14 May 2025 17:16:00 Z 2025-05-14T17:16:00Z https://blog.fabric.microsoft.com/en-US/blog/orchestrate-your-databricks-jobs-with-fabric-data-pipelines/ https://blog.fabric.microsoft.com/en-US/blog/orchestrate-your-databricks-jobs-with-fabric-data-pipelines/ Announcement We’re excited to announce that you can now orchestrate Azure Databricks Jobs from your Microsoft Fabric data pipelines! Databrick Jobs allow you to schedule and orchestrate a task or multiple tasks in a workflow in your Databricks workspace. Since any operation in Databricks can be a task, this means you can now run anything in … Continue reading “Orchestrate your Databricks Jobs with Fabric Data pipelines”<br />Update Type: Announcement, Services: Databricks, Microsoft Fabric, Categories: Wed, 14 May 2025 08:00:00 Z 2025-05-14T08:00:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/llama-4-is-now-available-in-azure-databricks/4401277 https://techcommunity.microsoft.com/blog/analyticsonazure/llama-4-is-now-available-in-azure-databricks/4401277 Announcement We are excited to announce the availability of Meta's Llama 4 in Azure Databricks.  As you know, enterprises all over the world already use Llama models in Azure Databricks to power AI enterpr...<br />Update Type: Announcement, Services: Azure AI Services, Databricks, Azure AI Foundry, Categories: Sun, 06 Apr 2025 00:31:00 Z 2025-04-06T00:31:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/power-bi--azure-databricks-smarter-refreshes-less-hassle/4398388 https://techcommunity.microsoft.com/blog/analyticsonazure/power-bi--azure-databricks-smarter-refreshes-less-hassle/4398388 Announcement We are excited to extend the deep integration between Azure Databricks and Microsoft Power BI with the Public Preview of the Power BI task type in Azure Databricks Workflows. This new capability...<br />Update Type: Announcement, Services: Databricks, Power BI Embedded, Categories: Mon, 31 Mar 2025 12:02:00 Z 2025-03-31T12:02:00Z https://azure.microsoft.com/updates?id=485518 https://azure.microsoft.com/updates?id=485518 GA The limited time discount for Azure Databricks Automated and Interactive Serverless Compute will end on April 30, 2025. Serverless compute provides rapid workload startup, automatic infrastructure scaling, and seamless version upgrades of the Azure Databr<br />Update Type: GA, Services: Databricks, Categories: Features, Services Thu, 27 Mar 2025 23:30:16 Z 2025-03-27T23:30:16Z https://azure.microsoft.com/updates?id=486507 https://azure.microsoft.com/updates?id=486507 GA We are thrilled to announce that as of March 26, 2025, Microsoft Azure customers can access and use Anthropic models within their existing Azure Databricks workspaces. This integration is a collaborative effort between Microsoft and Databricks, aimed at e<br />Update Type: GA, Services: Databricks, Categories: Features Wed, 26 Mar 2025 18:15:48 Z 2025-03-26T18:15:48Z https://techcommunity.microsoft.com/blog/analyticsonazure/anthropic-state-of-the-art-models-available-to-azure-databricks-customers/4397140 https://techcommunity.microsoft.com/blog/analyticsonazure/anthropic-state-of-the-art-models-available-to-azure-databricks-customers/4397140 Announcement Our customers now have greater model choices with the arrival of Anthropic Claude 3.7 Sonnet in Azure Databricks. Databricks is announcing a partnership with Anthropic to integrate their state-o...<br />Update Type: Announcement, Services: Databricks, Categories: Wed, 26 Mar 2025 14:11:00 Z 2025-03-26T14:11:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/anthropic-state-of-the-art-models-available-on-azure-databricks/4397140 https://techcommunity.microsoft.com/blog/analyticsonazure/anthropic-state-of-the-art-models-available-on-azure-databricks/4397140 Announcement Our customers now have greater model choices with the arrival of Anthropic Claude 3.7 Sonnet in Azure Databricks. Databricks is announcing a partnership with Anthropic to integrate their state-o...<br />Update Type: Announcement, Services: Databricks, Categories: Wed, 26 Mar 2025 14:11:00 Z 2025-03-26T14:11:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/6-critical-phases-to-prepare-for-a-successful-azure-databricks-migration/4386984 https://techcommunity.microsoft.com/blog/analyticsonazure/6-critical-phases-to-prepare-for-a-successful-azure-databricks-migration/4386984 Announcement As organizations adopt advanced analytics and AI to drive decision-making, moving data applications to Azure Databricks has become a strategic and significant endeavor. This transition requires caref...<br />Update Type: Announcement, Services: Databricks, Categories: Thu, 20 Mar 2025 21:03:00 Z 2025-03-20T21:03:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/part-1-power-bi-service-connections-to-azure-databricks-with-private-networking/4384391 https://techcommunity.microsoft.com/blog/analyticsonazure/part-1-power-bi-service-connections-to-azure-databricks-with-private-networking/4384391 Announcement This blog was written in conjunction with Leo Furlong, Lead Solutions Architect at Databricks. Enhancing Security and Connectivity: Azure Databricks SQL, Unity Catalog, and Power BI Integration T...<br />Update Type: Announcement, Services: Databricks, Power BI Embedded, Categories: Security Fri, 21 Feb 2025 21:50:00 Z 2025-02-21T21:50:00Z https://techcommunity.microsoft.com/blog/analyticsonazure/workarounds-for-maven-json-smart-2-5-2-release-breaking-azure-databricks-job-dep/4377517 https://techcommunity.microsoft.com/blog/analyticsonazure/workarounds-for-maven-json-smart-2-5-2-release-breaking-azure-databricks-job-dep/4377517 Announcement A recent Maven library release corrupted json-smart’s central metadata, ended up removing all previous versions of Maven library other than 2.5.2. This results in DBR ivy resolution failures if custo...<br />Update Type: Announcement, Services: Databricks, Categories: Wed, 12 Feb 2025 14:47:00 Z 2025-02-12T14:47:00Z https://azure.microsoft.com/updates?id=467466 https://azure.microsoft.com/updates?id=467466 GA Azure Databricks Clean Rooms offers a privacy-safe environment for collaboration on all your data and AI assets, ensuring no direct access to sensitive information. In today’s world, organizations require an open, flexible, and secure way to collaborate o<br />Update Type: GA, Services: Databricks, Categories: Features Tue, 04 Feb 2025 14:00:07 Z 2025-02-04T14:00:07Z https://azure.microsoft.com/updates?id=467948 https://azure.microsoft.com/updates?id=467948 GA Azure Databricks is a data analytics platform optimized for the Microsoft Azure cloud services platform. Azure Databricks offers three environments for developing data intensive applications: Databricks SQL, Databricks Data Science & Engineering, and Data<br />Update Type: GA, Services: Databricks, Categories: Regions & Datacenters Mon, 03 Feb 2025 21:15:30 Z 2025-02-03T21:15:30Z https://azure.microsoft.com/updates?id=467457 https://azure.microsoft.com/updates?id=467457 GA We’re excited to announce the extension of the promotional discounts on serverless compute for Notebooks, Jobs, and Pipelines (Delta Live Tables). You will continue to get 50% off serverless compute for Jobs and Pipelines and 30% off serverless compute fo<br />Update Type: GA, Services: Databricks, Categories: Services, Pricing & Offerings Thu, 12 Dec 2024 20:30:21 Z 2024-12-12T20:30:21Z https://techcommunity.microsoft.com/blog/analyticsonazure/replicating-azure-cosmos-db-into-azure-databricks-using-cdc/4302079 https://techcommunity.microsoft.com/blog/analyticsonazure/replicating-azure-cosmos-db-into-azure-databricks-using-cdc/4302079 Announcement <br />Update Type: Announcement, Services: Databricks, Cosmos DB, Categories: Thu, 21 Nov 2024 15:11:00 Z 2024-11-21T15:11:00Z https://techcommunity.microsoft.com/blog/analyticsonazureblog/boosting-power-bi-performance-with-azure-databricks-through/4278890 https://techcommunity.microsoft.com/blog/analyticsonazureblog/boosting-power-bi-performance-with-azure-databricks-through/4278890 Announcement This post is authored in conjunction with Yatish Anand, Senior Solutions Architect at Databricks, and Andrey Mirskiy, Senior Specialist Solutions Architect at Databricks. Figure 1. Power BI automatic aggregations overview, source. Introduction In today’s fast-paced data landscape, timely insights can make all the difference. Power BI’s Automatic Aggregations feature is a breakthrough, designed to push performance boundaries by delivering low-latency query results on large datasets. It harnesses AI-powered caching to streamline DirectQuery models, combining the best-in-class performance of Power BI on Azure Databricks with low-latency BI for modern reporting needs. Used with DirectQuery mode, Automatic Aggregations do not face data volume limitations and allow you to scale regardless of data size without compromising on BI performance. This innovation makes it easy for Power BI users of all skill levels to tap into advanced performance without worrying about backend strain or complex data modeling. Imagine your reports updating in real-time, even with billions of records in play. With this approach, you can deliver actionable insights faster than ever, freeing up time to focus on your business, not your data infrastructure. In this blog, we will showcase the integration of Power BI Automatic Aggregation with Azure Databricks and how this integration will help improve the performance of your Power BI reports. What Are Automatic Aggregations? Automatic aggregations streamline the process of improving BI query performance by maintaining an in-memory cache of aggregated data. This means that a substantial portion of report queries can be served directly from this in-memory cache instead of relying on the backend data sources. Power BI automatically builds these aggregations using AI based on your query patterns and then intelligently decides which queries can be served from the in-memory cache and which are routed to the data source through DirectQuery, resulti<br />Update Type: Announcement, Services: Databricks, Power BI Embedded, Categories: Mon, 28 Oct 2024 20:22:14 Z 2024-10-28T20:22:14Z https://techcommunity.microsoft.com/blog/azurearchitectureblog/data-intelligence-end-to-end-with-azure-databricks-and-microsoft-fabric/4232621 https://techcommunity.microsoft.com/blog/azurearchitectureblog/data-intelligence-end-to-end-with-azure-databricks-and-microsoft-fabric/4232621 Announcement This architecture provides a scalable, secure foundation for analytics, AI, and real-time insights, integrating the best of Azure Databricks with Microsoft Fabric.<br />Update Type: Announcement, Services: Databricks, Microsoft Fabric, Categories: Wed, 04 Sep 2024 00:00:00 Z 2024-09-04T00:00:00Z https://azure.microsoft.com/en-us/updates/v2/Databricks-Jobs-Serverless-Compute https://azure.microsoft.com/en-us/updates/v2/Databricks-Jobs-Serverless-Compute GA Users can now run their Azure Databricks job without configuring and deploying infrastructure.<br />Update Type: GA, Services: Databricks, Categories: Features, Services, Pricing & Offerings Fri, 05 Jul 2024 00:00:00 +0200 2024-07-05T00:00:00+02:00 https://techcommunity.microsoft.com/blog/analyticsonazureblog/part-2-migrate-azure-analysis-services-to-power-bi-premium-using/4134928 https://techcommunity.microsoft.com/blog/analyticsonazureblog/part-2-migrate-azure-analysis-services-to-power-bi-premium-using/4134928 Announcement This post is authored in conjunction with Leo Furlong, Senior Solutions Architect, at Databricks. Many customers choose to migrate their Azure Analysis Services semantic models to Power BI Premium due to the benefits and enhanced functionality described in the Power BI documentation. As customers migrate their semantic models to Power BI, native connections to Azure Databricks SQL become available due to the built-in Databricks SQL connector in Power BI. Databricks SQL Serverless combined with Power BI semantic models can provide customers with a number of benefits including a separation of compute and storage; instant, elastic compute; data refreshes charged by the second; and enterprise Data Warehouse functionality, query performance, and concurrency at any scale. The remainder of this article will focus on the in-and-outs of how to accomplish this migration. Requirements You must use Power BI Premium Capacities. This means that P, A4+, or F SKUs are required.  The XMLA endpoint for your Power BI workspace must be configured for Read/Write. You will need Tabular Editor 2 or Tabular Editor 3 to migrate your semantic model to Databricks SQL. Tabular Editor is a 3rd party tool featured by Microsoft.  Migrating an existing semantic model to use Databricks SQL requires that your existing data source is a structured data source (PowerQuery based connection). Limitations AAS models migrated to Power BI Premium can’t be edited using web authoring per the documented limits, but as of writing this post, it does appear to allow web authoring on migrated models without issues/errors  AAS models migrated to Power BI can’t be downloaded to Power BI Desktop Parallel connection configuration is only available in model Compatibility Level 1600+ Assumptions Your data model in Databricks SQL is in the same structure as your current Azure Analysis Services model in terms of table/column names and column data types.   You are using the same Power BI s<br />Update Type: Announcement, Services: Databricks, Analysis Services, Power BI Embedded, Categories: Services Wed, 29 May 2024 23:02:30 Z 2024-05-29T23:02:30Z https://techcommunity.microsoft.com/blog/analyticsonazureblog/part-1-migrate-azure-analysis-services-to-power-bi-premium-using/4134892 https://techcommunity.microsoft.com/blog/analyticsonazureblog/part-1-migrate-azure-analysis-services-to-power-bi-premium-using/4134892 Announcement This post is authored in conjunction with Leo Furlong, Senior Solutions Architect, at Databricks. In the world of data analytics and business intelligence, the tools and platforms you use can significantly impact the efficiency and capabilities of your data operations. Two major shifts in the landscape are the migration from Azure Analysis Services to Power BI Premium and the move to Azure Databricks SQL as the underlying data source for Power BI. Let's dive into why these changes are worth considering. Migrating from Azure Analysis Services to Power BI Premium Azure Analysis Services has been a staple in the enterprise semantic layer toolset, providing robust semantic layer capabilities. It's the same engine that powers Power BI under the hood, which means there's a shared lineage and compatibility between the two services. However, Power BI has evolved rapidly and now offers a superset of functionalities that were once exclusive to Azure Analysis Services. Here are some compelling reasons to migrate: Enhanced Functionality: Power BI Premium has grown to encompass all the capabilities of Azure Analysis Services and then some. A feature comparison matrix is provided here. Tool Consolidation: By migrating to Power BI Premium, organizations can consolidate their semantic layer and BI tools into a single platform. This not only simplifies the architecture but also reduces the overhead associated with maintaining multiple systems. Microsoft's Direction: Microsoft itself recommends that customers transition their Azure Analysis Services models to Power BI Premium. This is a strong indicator of the strategic direction Microsoft is taking, with Power BI Premium positioned as the go-to enterprise semantic layer solution. For more details on this migration, you can refer to Microsoft's official documentation here. Moving to Azure Databricks SQL for Power BI Azure Databricks SQL offers a modern and optimized approach to handling present and future data analytics cha<br />Update Type: Announcement, Services: Databricks, Analysis Services, Power BI Embedded, Categories: Services Wed, 29 May 2024 23:02:10 Z 2024-05-29T23:02:10Z https://techcommunity.microsoft.com/blog/analyticsonazureblog/revolutionizing-data-intelligence-azure-databricks-updates/4145079 https://techcommunity.microsoft.com/blog/analyticsonazureblog/revolutionizing-data-intelligence-azure-databricks-updates/4145079 Announcement Data Intelligence Platform in Azure Databricks is revolutionizing the Data and AI landscape. This fully managed service, which is built on Lakehouse architecture supported by Delta Lake, and is integrated with Microsoft Azure cloud capabilities, streamlines data, analytics, and AI initiatives by removing infrastructure concerns. Databricks was recently named a Leader in the 2024 Forrester Wave for Data Lakehouses, earning the highest scores in both strategy and current offerings, and excelling in areas like GenAI/LLM, integration, and security. The close partnership between Databricks and Microsoft enhances this integration, enabling users to focus on their data and AI goals and makes Azure the optimal public cloud for Databricks. The platform’s Data Intelligence Engine, DatabricksIQ, is powered by advanced generative AI models to autonomously optimize performance and infrastructure based on the unique characteristics of your data and workloads. Supporting both structured and unstructured data in the Delta Lake format, Azure Databricks combines the best elements of data lakes and data warehouses into a unified system for data storage, processing, security, and governance, facilitating efficient management of data and AI assets at scale. Enhanced Data Governance with Unity Catalog in Azure Databricks Data and AI governance serves as the intricate yet pivotal foundation for data and AI democratization. Azure Databricks offers a unified solution for data and AI governance with Unity Catalog, natively built into the platform. Unity Catalog uniquely manages data and data assets (such as models) and provides coherent governance across all workloads from analytics to ETL to model serving. It provides centralized access control, auditing, lineage, and AI-powered data documentation and discovery across Azure Databricks workspaces. Leveraging Delta Uniform, Unity Catalog can also provide compatibility for any data format (including Hudi or Iceberg), with a si<br />Update Type: Announcement, Services: Databricks, Categories: Security, Services, Management Wed, 22 May 2024 14:00:00 Z 2024-05-22T14:00:00Z https://techcommunity.microsoft.com/blog/analyticsonazureblog/announcing-mosaic-ai-vector-search-general-availability-in-azure/4144712 https://techcommunity.microsoft.com/blog/analyticsonazureblog/announcing-mosaic-ai-vector-search-general-availability-in-azure/4144712 GA Today, at Microsoft Build, we are thrilled to announce the general availability of Mosaic AI Vector Search in Azure Databricks. Vector Search is a serverless vector database that helps customers build high-quality Generative AI applications using Retrieval Augmented Generation (RAG). With its native integration in Azure Databricks, Vector Search supports automatic data synchronization from source to index, eliminating complex and costly pipeline maintenance. It also leverages the same security and data governance tools organizations have already built for peace of mind.   Challenges with Building Production-quality Generative AI Applications The top challenge with deploying Generative AI is getting these applications to production quality. Traditionally, the primary focus around quality was to find the best Large Language Model (LLM).  However, recent research shows that the LLM is only one of the factors of quality for your GenAI-powered application. To achieve high quality, developers need to build an AI system, where they take into account data preparation, the LLM, ranking and post-processing pipelines, prompt engineering, and more. The most popular type of AI system is Retrieval Augmented Generation or RAG. RAG is an architectural approach that can improve the efficacy of large language models by augmenting the prompt with data stored in a Vector Database.   Vector Search in Azure Databricks enables developers to improve the accuracy of their Retrieval Augmented Generation (RAG) and generative AI applications through similarity search over unstructured documents such as PDFs, Office Documents, Wikis, and more. This enriches the LLM queries with context and domain knowledge, improving accuracy and quality of results.   Vector Search is part of the Azure Databricks Data Intelligence Platform, making it easy for your RAG and Generative AI applications to use the proprietary data stored in your data lakes in a fast and secure manner and deliver<br />Update Type: GA, Services: Databricks, Categories: Security Tue, 21 May 2024 15:00:00 Z 2024-05-21T15:00:00Z https://azure.microsoft.com/en-us/updates/the-billing-meter-for-azure-databricks-materialized-views-and-streaming-tables-will-change-1-may-2024 https://azure.microsoft.com/en-us/updates/the-billing-meter-for-azure-databricks-materialized-views-and-streaming-tables-will-change-1-may-2024 Preview Azure Databricks materialized views and streaming tables Billing change <br />Update Type: Preview, Services: Databricks, Categories: Pricing & Offerings Thu, 16 May 2024 00:00:00 Z 2024-05-16T00:00:00Z https://techcommunity.microsoft.com/blog/analyticsonazureblog/introducing-the-databricks-data-intelligence-platform-on-azure/4050749 https://techcommunity.microsoft.com/blog/analyticsonazureblog/introducing-the-databricks-data-intelligence-platform-on-azure/4050749 Announcement This post is authored in conjunction with Bryan Saftler, Global Head of Marketing for Communications, Media & Entertainment at Databricks and Steve Sobel, Global Head - Communications, Media & Entertainment GTM at Databricks. The communications industry is undergoing one of the most significant periods of growth (and change) in its 100+ year history. The dramatic increase in global traffic, the customer expectation for enhanced services, and the need for more network equipment is driving more complex and costly network management. In addition, data shows that as customers demand higher quality service and better customer experiences (CX), they’re more likely to churn when their needs are not met.  As a result, Communications Service Providers (CSPs) are looking to data and AI use cases to help reduce costs to serve and operate, develop and maintain great customer experiences, identify opportunities for revenue growth, and achieve scale with secure and reliable services. No small feat.  To illustrate this, look no further than the explosion of digital technologies — such as 5G, IoT, edge computing, and AI — which has ushered in a new era of big data. For example, according to Cisco's Visual Networking Index (VNI) Global Mobile Data Traffic Forecast, global mobile data traffic has grown to 77.5 exabytes per month. This growth is driven by the proliferation of mobile devices and the increasing demand for data-intensive applications. In addition, the number of connected IoT devices is skyrocketing. According to Statista, the number of connected IoT devices is expected to reach 31 billion by 2025, generating approximately 79.4 zettabytes of data annually. This data, while large and unwieldy, is critical to building a virtuous loop between networks and customers. By bringing this data together, CSPs can better understand the factors that contribute to a positive or negative customer experience. To make this virtuous loop effective, the key is<br />Update Type: Announcement, Services: Databricks, Categories: Services, Management Tue, 06 Feb 2024 23:54:15 Z 2024-02-06T23:54:15Z https://techcommunity.microsoft.com/blog/azuredatablog/announcing-the-general-availability-of-amd-based-confidential/3980969 https://techcommunity.microsoft.com/blog/azuredatablog/announcing-the-general-availability-of-amd-based-confidential/3980969 GA We are excited to announce the general availability of AMD-based confidential virtual machines (VMs) for cluster nodes on Azure Databricks. Confidential VMs are part of the Azure confidential computing (ACC) portfolio and provide a trusted execution environment (TEE) for Azure Databricks clusters, protecting data while in use in memory. It is important to note that Azure already encrypts data at rest and in transit, and the introduction of confidential VMs provides an additional layer of security for sensitive data in use, helping organizations meet compliance requirements and protect their most valuable data. By using Azure confidential computing on Azure Databricks, you gain the capability to encrypt your data end-to-end. This is valuable not only for confidential workloads but also for any scenario where you need to protect highly sensitive data residing in memory and prevent unauthorized access or tampering. The solution also supports Azure Managed HSM, a hardware security module that allows the customer to manage their own encryption keys for data at-rest, in-use, and in-transit. To use confidential VMs on Azure Databricks, customers need to select one of the confidential VM types when creating a cluster. This type of cluster can then be used for any workload that requires the protection of highly sensitive data in memory.  For compute-optimized needs, DCasv5 confidential VMs are available, and for memory-optimized needs, ECasv5 confidential VMs can be used.  These VMs are currently available in the following regions: East US, West US, North Europe, West Europe, Southeast Asia, Central India, East Asia, Switzerland North, Japan East and Italy North, and coming to additional regions soon. Databricks partnership in Confidential Computing: "We are thrilled to have collaborated with Microsoft to introduce Azure Databricks support for Azure confidential computing,” said David Meyer, Senior Vice President of P<br />Update Type: GA, Services: Databricks, Virtual Machines, Categories: Compliance, Security, Regions & Datacenters Wed, 15 Nov 2023 16:00:01 Z 2023-11-15T16:00:01Z https://azure.microsoft.com/en-us/updates/generally-available-amd-confidential-vm-option-for-azure-databricks https://azure.microsoft.com/en-us/updates/generally-available-amd-confidential-vm-option-for-azure-databricks GA Users can now select AMD-based confidential VMs for their Azure Databricks cluster driver node and cluster worker nodes. <br />Update Type: GA, Services: Databricks, Virtual Machines, Categories: Features, Security Wed, 15 Nov 2023 00:00:00 Z 2023-11-15T00:00:00Z https://techcommunity.microsoft.com/blog/azuredatablog/microsoft-and-databricks-deepen-partnership-for-modern-cloud/3640280 https://techcommunity.microsoft.com/blog/azuredatablog/microsoft-and-databricks-deepen-partnership-for-modern-cloud/3640280 Announcement In May, we introduced the Microsoft Intelligent Data platform which deeply integrates our databases, analytics, BI and data governance products into a unified platform. This platform, which is also integrated with the Microsoft Cloud, enables seamless experience and intuitive collaboration for Developers, DBAs, Data Engineers, Data Scientists, Business Analysts and Data Officers. The platform enables customers to “do more with less”, by helping them add layers of intelligence to their applications, unlock predictive insights and drive relevant action, and govern their entire data estate. Microsoft Intelligent Data Platform Databricks has been a key partner for us in delivering a cloud native platform for Analytics in the Microsoft Intelligent Data Platform. Today, I am pleased to share that we have further strengthened our partnership with Databricks to evolve our Analytics platform to an Open and Governed Data Lakehouse foundation. This foundation will enable our customers to unify their most demanding Business Intelligence, Machine Learning, and Artificial Intelligence investments on a single data foundation for analytics viz. an open and governed data lakehouse. The foundation will also enable customers to responsibly democratize their analytics data products to accelerate digital transformation applications across their organizations.    “Databricks is a key partner for Microsoft, and together, we are delivering a modern, cloud-native data foundation in the Microsoft Intelligent Data Platform for the most demanding analytics and machine learning applications,” - Scott Guthrie, EVP, Cloud + AI, Microsoft.  Microsoft and Databricks have partnered to build this foundation in the Microsoft Intelligent Data Platform by integrating their hallmark capabilities to deliver an integrated solution for our customers.   Core to the foundation is the Open and Governed Data Lakehouse, a unified and cloud native data fabric capab<br />Update Type: Announcement, Services: Machine Learning, Databricks, Categories: Mon, 17 Oct 2022 06:04:42 Z 2022-10-17T06:04:42Z https://techcommunity.microsoft.com/blog/azuredatablog/azure-databricks-artifacts-deployment/2913522 https://techcommunity.microsoft.com/blog/azuredatablog/azure-databricks-artifacts-deployment/2913522 Announcement This article is intended for deploying Jar Files, XML Files, JSON Files, wheel files and Global Init Scripts in Databricks Workspace. Overview: In Databricks Workspace, we have notebooks, clusters, and data stores. These notebooks are run on data bricks clusters and use datastores if they need to refer to any custom configuration in the cluster. Developers need environment specific configurations, mapping files and custom functions using packaging for running the notebooks in Databricks Workspace. Developers need a global Init script which runs on every cluster created in your workspace. Global Init scripts are useful when you want to enforce organization-wide library configurations or security screens. This pipeline can automate the process of deploying these artifacts in Databricks workspace. Purpose of this Pipeline: The purpose this pipeline is to pick up the Databricks artifacts from the Repository and upload to Databricks workspace DBFS location and uploads the global init script using REST API's. The CI pipeline builds the wheel (.whl) file using setup.py and publishes required files (whl file, Global Init scripts, jar files etc.) as a build artifact. The CD pipeline uploads all the artifacts to DBFS location and also uploads the global Init scripts using the REST API's. Pre-Requisites: Developers need to make sure that all the artifacts that need to be uploaded to Databricks Workspace need to be present in the Repository (main branch). The location of the artifacts in the repository should be fixed (Let us consider’/artifacts’ as the location). The CI process will create the build artifact from this folder location. The Databricks PAT Token and Databricks Target Workspace URL should be present in the key vault.     Continuous Integration (CI) pipeline: The CI pipeline builds a wheel (.whl) file using the a setup.py file and also creates a build artifact from all files in the artifacts/ folder such as Configurat<br />Update Type: Announcement, Services: Databricks, Categories: Security Wed, 02 Feb 2022 15:26:07 Z 2022-02-02T15:26:07Z https://azure.microsoft.com/en-us/blog/announcing-azure-databricks-unit-pre-purchase-plan-and-new-regional-availability https://azure.microsoft.com/en-us/blog/announcing-azure-databricks-unit-pre-purchase-plan-and-new-regional-availability Announcement Azure Databricks is a fast, easy, and collaborative Apache Spark based analytics platform that simplifies the process of building big data and artificial intelligence (AI) solutions. Azure Databricks provides data engineers and data scientists an interactive workplace where they can use the languages and frameworks of their choice.<br />Update Type: Announcement, Services: Databricks, Categories: Tue, 06 Aug 2019 00:00:00 -0700 2019-08-06T00:00:00-07:00 https://azure.microsoft.com/en-us/blog/azure-databricks-vnet-injection-devops-version-control-and-delta-availability https://azure.microsoft.com/en-us/blog/azure-databricks-vnet-injection-devops-version-control-and-delta-availability Announcement Azure Databricks provides a fast, easy, and collaborative Apache® Spark™-based analytics platform to accelerate and simplify the process of building big data and AI solutions that drive the business forward, all backed by industry-leading SLAs.<br />Update Type: Announcement, Services: Databricks, Virtual Network, Categories: Wed, 13 Mar 2019 00:00:00 -0700 2019-03-13T00:00:00-07:00 https://azure.microsoft.com/en-us/blog/azure-databricks-new-capabilities-at-lower-cost https://azure.microsoft.com/en-us/blog/azure-databricks-new-capabilities-at-lower-cost Announcement Azure Databricks provides a fast, easy, and collaborative Apache Spark™-based analytics platform to accelerate and simplify the process of building big data and AI solutions backed by industry leading SLAs.<br />Update Type: Announcement, Services: Databricks, Categories: Wed, 06 Mar 2019 00:00:00 -0800 2019-03-06T00:00:00-08:00 https://azure.microsoft.com/en-us/blog/azure-databricks-delta-in-preview-9-regions-added-and-other-exciting-announcements https://azure.microsoft.com/en-us/blog/azure-databricks-delta-in-preview-9-regions-added-and-other-exciting-announcements Announcement Azure Databricks provides a fast, easy, and collaborative Apache Spark-based analytics platform to accelerate and simplify the process of building big data and AI solutions that drive the business forward, all backed by industry leading SLAs.<br />Update Type: Announcement, Services: Databricks, Categories: Mon, 24 Sep 2018 00:00:00 -0700 2018-09-24T00:00:00-07:00 https://azure.microsoft.com/en-us/blog/azure-databricks-industry-leading-analytics-platform-powered-by-apache-spark https://azure.microsoft.com/en-us/blog/azure-databricks-industry-leading-analytics-platform-powered-by-apache-spark Announcement The confluence of cloud, data, and AI is driving unprecedented change. The ability to utilize data and turn it into breakthrough insights is foundational to innovation today...<br />Update Type: Announcement, Services: Databricks, Categories: Thu, 22 Mar 2018 00:00:00 -0700 2018-03-22T00:00:00-07:00