This can be a collaborative put up from Databricks and Microsoft Azure. We thank Rajeev Jain, Senior Product Advertising and marketing Supervisor at Microsoft, for his contributions.
Information + AI Summit 2023: Register now to affix this in-person and digital occasion June 26-29 and be taught from the worldwide knowledge group.
Microsoft is a Platinum Sponsor of Information + AI Summit 2023, the premier occasion for the worldwide knowledge group. Satya Nadella, Chairman and CEO of Microsoft, will likely be talking on the Information + AI Summit Keynote Day 1. Be part of Information + AI Summit to be taught from joint Databricks and Microsoft Azure prospects like Akamai, Waterstons, Windfall Well being Providers, ABN Amro, Bradesco, and Division of Veterans Affairs who’ve efficiently leveraged the Databricks Lakehouse Platform for his or her enterprise, bringing collectively knowledge, AI and analytics on one frequent platform.
At Information + AI Summit, Databricks and Microsoft prospects will take the stage for periods that can assist you see how they achieved enterprise outcomes utilizing the Databricks Lakehouse on Azure. Attendees can have the chance to listen to knowledge leaders from Akamai and Waterstons on Tuesday, June twenty seventh, then be part of Windfall Well being Providers, Bradesco, and ABN Amro on Wednesday, June twenty eighth and Division of Veterans Affairs on Thursday, June twenty ninth. At Information + AI Summit, be taught in regards to the newest improvements and applied sciences and listen to thought-provoking panel discussions together with the flexibility for networking alternatives the place you may join with different knowledge professionals in your trade.
The periods under are a information for everybody taken with Azure Databricks and span a spread of matters — from real-time medical operations monitoring to shopper analytics to IoT knowledge gathering. When you have questions on Azure Databricks or service integrations, join with Azure Databricks Options Architects at Information + AI Summit on the Microsoft sales space on the Expo ground.
Keynote
Satya Nadella Keynote
Wednesday, June 28 @8:15 AM
Visionary leaders will share insights on:
Information, analytics and AI panorama: Uncover what’s driving a lot concentrate on knowledge and why knowledge professionals are zeroing in on new methods to deal with their database challenges. Study why there may be a lot curiosity in LLMs, what is going on throughout the information, analytics and AI panorama and the way forward for the market
Evolution of the lakehouse: Check out the bigger universe that the lakehouse lives within, be taught what’s new and discover the evolution with us
Open supply applied sciences: Hear from the open supply group about what’s new and what’s to return for Apache Spark™, Delta Lake and MLflow and find out how this impacts the lakehouse and the general market at massive
Microsoft buyer breakout periods
Taking Your Cloud Vendor to the Subsequent Degree: Fixing Advanced Challenges with Azure Databricks
Tuesday, June 27 @1:00 PM
Akamai’s content material supply community (CDN) processes about 30% of the web’s day by day visitors, leading to an enormous quantity of knowledge that presents engineering challenges, each internally and with cloud distributors. On this session, we’ll talk about the boundaries confronted whereas constructing an information infrastructure on Azure, Databricks, and Kafka to fulfill strict SLAs, hitting the bounds of a few of our cloud distributors’ providers. We’ll describe the iterative strategy of re-architecting an enormous scale knowledge platform utilizing the aforementioned applied sciences.
We will even delve into how at present, Akamai is ready to rapidly ingest and make out there to prospects terabytes of knowledge, in addition to effectively question Petabytes of knowledge and return outcomes inside 10 seconds for many queries. This dialogue will present useful insights for attendees and organizations searching for to successfully course of and analyze massive quantities of knowledge.
Getting Perception From Something: Gathering Information With IoT Gadgets and Delta Reside
Tuesday, June 27 @4:00 PM
The drive to make data-driven choices shapes the technique of many corporations. Nevertheless, this dream has an issue; simply because you should decide doesn’t imply you could have the information to make it. Usually these choices require knowledge from sources usually unconnected to the web. This may very well be an architect decided to know how the development of her constructing is progressing, or a producer desirous to stay their finest trade 4.0 life. It isn’t unusual for corporations to search out this journey to grow to be “good” is a protracted one; with troublesome moral floor to cowl, or prohibitive prices related to procuring the required tools. Nevertheless, the event of low-cost and accessible IoT gadgets permits for the gathering of knowledge from something to be extra inside attain than ever.
This session will have a look at efforts to construct proof-of-concept IoT techniques. We’ll talk about our (persevering with) journey of utilizing and managing {hardware} and the issues we confronted alongside the best way. The main target would be the software program stack. Azure IoT Hub is used for system administration, with Energy BI supporting the entrance finish. Nevertheless, the centerpiece is Delta Reside Tables – permitting us a way of close to real-time evaluation of our knowledge. We’ll talk about the benefits of utilizing Delta Reside for this, and the way we want to prolong our IoT techniques sooner or later.
Enhancing Hospital Operations with Streaming Information and Actual Time AI/ML
Wednesday, June 28 @11:30 AM
Over the previous two years, Windfall has developed a strong streaming knowledge platform (SDP) leveraging Databricks in Azure. The SDP allows us to ingest and course of real-time knowledge reflecting medical operations throughout our 52 hospitals and roughly 1000 ambulatory clinics. The HL7 messages generated by Epic are parsed utilizing Databricks in our safe cloud surroundings and used to generate an up-to-the minute image of precisely what is going on on the level of care.
We’re already leveraging this info to reduce hospital overcrowding and have been actively integrating AI/ML to precisely forecast future situations (e.g., arrivals, size of keep, acuity, and discharge necessities.) This enables us to each enhance useful resource utilization (e.g., nurse staffing ranges) and to optimize affected person throughput. The result’s each improved affected person care and operational effectivity.
On this session, we’ll share how these outcomes are solely potential with the ability and magnificence afforded by our investments in Azure, Databricks, and more and more Lakehouse. We’ll show Windfall’s blueprint for enabling real-time analytics which will be generalized to different healthcare suppliers.
ABN Story: Migrating to Future Proof Information Platform
Wednesday, June 28 @3:30 PM
ABN AMRO Financial institution is likely one of the prime main banks within the Netherlands. It’s the third largest financial institution within the Netherlands by income and variety of mortgages held inside the Netherlands, and has prime administration assist of the target to grow to be a completely data-driven financial institution. ABN AMRO began its knowledge journey virtually seven years in the past and has constructed an information platform off-premises with Hadoop applied sciences. This knowledge platform has been utilized by greater than 200 knowledge suppliers, 150 knowledge shoppers, and greater than 3000 datasets.
To grow to be a completely digital financial institution and deal with the limitation of the on-premises platform requires a future-proof knowledge platform DIAL (digital integration and entry layer). ABN AMRO determined to construct an Azure cloud-native knowledge platform with the assistance of Microsoft and Databricks. Final 12 months this cloud-native platform was prepared for our knowledge suppliers and knowledge shoppers. Six months in the past we began the journey of migrating all of the content material from the on-premises knowledge platform to the Azure knowledge platform, this was a really large-scale migration and was achieved in six months.
On this session, we’ll concentrate on three issues:
- The migration technique going from on-premises to a cloud-native platform
- Which Databricks options have been used within the knowledge platform
- How the Databricks staff assisted within the total migration
Scaling Up Healthcare Information Improvements: Classes Realized from VA’s Summit Information Platform Journey
Thursday, June 29 @2:30 PM
The Division of Veterans Affairs (VA) is the nation’s largest well being system with 170 VA Medical Facilities and over a 1000 outpatient clinics, serving over 9 million beneficiaries, and using virtually 400,000 workers. Three years in the past, VA Workplace of Expertise (OIT) launched into the event of the Summit Information Platform (SDP) to centralize and modernize knowledge and analytics throughout the VA enterprise. At present, SDP helps over 30 use circumstances from Lengthy COVID care, suicide prevention, most cancers care monitoring, medical deterioration, veteran expertise, and plenty of extra. These use circumstances embody among the most progressive technological efforts at VA with the usage of Giant Language Fashions (LLMs), self-service analytics on the cloud, and real-time streaming analytics.
Information Democratization with Lakehouse: An Open Banking Utility Case
Wednesday, June 28 @11:30 AM
Banco Bradesco represents one of many largest corporations within the monetary sector in Latin America. They’ve greater than 99 million prospects, 79 years of historical past, and a legacy of knowledge distributed in a whole bunch of on-premises techniques. With the unfold of data-driven approaches and the expansion of cloud computing adoption, we would have liked to innovate and adapt to new traits and allow an analytical surroundings with democratized knowledge.
We’ll present how greater than eight enterprise departments have already engaged in utilizing the Lakehouse exploratory surroundings, with greater than 190 use circumstances mapped and a multi-bank monetary supervisor. In contrast to with on-premises, the price of every course of will be remoted and managed in close to real-time, permitting fast responses to value and finances deviations, whereas growing the deployment velocity of recent options 36 occasions in comparison with on-premises.
The information is now used and shared safely and simply between totally different areas and corporations of the group. Additionally, the view of dashboards inside Databricks permits panels to be effectively “prototyped” with actual knowledge, permitting a straightforward interplay of the enterprise space with its actual wants after which making a definitive view with all related factors duly confused.
Microsoft-led breakout periods
Sponsored by: Microsoft | Subsequent-Degree Analytics with Energy BI and Databricks
Wednesday, June 28 @2:30 PM
The widely-adopted mixture of Energy BI and Databricks has been a game-changer in offering a complete resolution for contemporary knowledge analytics. On this session, you will find out how self-service analytics mixed with the Databricks Lakehouse Platform can enable customers to make better-informed choices by unlocking insights hidden in advanced knowledge. We’ll present sensible examples of how organizations have leveraged these applied sciences collectively to drive digital transformation, decrease complete value of possession (TCO), and improve income. By the top of the presentation and demo, you will perceive how Energy BI and Databricks might help drive real-time insights at scale for organizations in any trade.
Communications, Media & Leisure Trade Discussion board
Wednesday, June 28 @3:30 PM
Information is on the core of almost each innovation within the Communications, Media & Leisure (CME) trade. Leaders throughout promoting, broadcast & movie, communications, gaming, and sports activities are harnessing the ability of knowledge and analytics to digitally rework and make smarter choices that reduce threat, speed up innovation and drive sustainable worth creation. As a part of our trade programming at Summit, we’re internet hosting a devoted Discussion board for CME. This 90-minute capstone occasion brings collectively over 250 in-person attendees and extra on-line for displays, networking and fascinating discussions on the traits shaping the usage of knowledge + AI throughout the trade.
Processing Prescriptions at Scale at Walgreens
Thursday, June 29 @1:30 PM
We designed a scalable Spark Streaming job to handle 100s of tens of millions of prescription-related operations per day at an end-to-end SLA of some minutes and a lookup time of 1 second utilizing CosmosDB.
On this session, we’ll share not solely the structure, however the challenges and options to utilizing the Spark Cosmos connector at scale. We’ll talk about usages of the Aggregator API, customized implementations of the CosmosDB connector, and the most important roadblocks we encountered with the options we engineered. As well as, we collaborated intently with Cosmos growth staff at Microsoft and can share the brand new options which resulted. When you ever plan to make use of Spark with Cosmos, you will not need to miss these gotchas!
Accomplice-led breakout periods
Sponsored by: Avanade | Accelerating Adoption of Trendy Analytics and Governance at Scale
Wednesday, June 28 @11:30 AM
To compress transformation, allow reinvention because the technique, and use knowledge as the first supply of aggressive benefit, purchasers that we work with are investing into modernizing the foundational components powering their digital core. These constructing blocks supporting the enterprise reinvention are the enterprise metadata and knowledge administration providers, knowledge administration basis, and knowledge providers and merchandise that allow enterprise items to completely use their knowledge and analytics at scale.
On this session, Avanade knowledge leaders will spotlight how Databricks trendy knowledge stack suits Azure knowledge administration basis ecosystem, how Unity catalog metadata helps automated knowledge operations eventualities, and the way we’re serving to purchasers measure trendy analytics and governance enterprise impression and worth.
Sponsored by: Infosys | Topaz and Our AI First Strategy
Wednesday, June 28 @12:30 PM
A dialogue overlaying the journey of enterprise AI First adoption together with standardized framework and toolset, realized with Azure Databricks as a key framework element, for World Organizations.
How Infosys Topaz, our AI-first basis together with Databricks platform will assist our prospects navigate the AI journey for his or her organizations
Well being Care Service Company’s Cloud Migration and Information Analytics Modernization Journey
Wednesday, June 28 @12:30 PM
HCSC is a number one well being care insurer that collaborated with Deloitte to modernize their knowledge analytics and intelligence capabilities by taking the on-premise knowledge lake and knowledge warehouse platforms and shifting them to the cloud.
On this session, Deloitte and HCSC current easy methods to leverage Databricks and automation emigrate options from an on-premise Apache Hadoop platform to Microsoft Azure Cloud. This modernization offers HCSC better agility with an elastic, scalable and responsive engineered resolution strategy, which is paramount as they attempt to be the accomplice of alternative for advantages and care coordination, bettering their members’ well being and well-being by means of all phases of life.
Sponsored by: Avanade | Enabling Actual-Time Analytics with Structured Streaming and Delta Reside Tables
Wednesday, June 28 @1:00 PM
Be part of the panel to listen to how Avanade helps purchasers allow real-time analytics and deal with the folks and course of issues that accompany expertise, powered by Azure Databricks.
Scaling MLOps for a Demand Forecasting Throughout A number of Markets for a Giant CPG
Wednesday, June 28 @1:30 PM
On this session, we have a look at how one of many world’s largest CPG firm setup a scalable MLOps pipeline for a requirement forecasting use case that predicted demand at 100,000+ DFUs (demand forecasting items) on a weekly foundation throughout greater than 20 markets. This implementation resulted in important value financial savings by way of improved productiveness, decreased cloud utilization and sooner time to worth amongst different advantages. You’ll go away this session with a clearer image on the next:
- Greatest practices in scaling MLOps with Databricks and Azure for a requirement forecasting use case with a multi-market and multi-region roll-out.
- Greatest practices associated to mannequin re-factoring and organising normal CI-CD pipelines for MLOps.
- What are among the pitfalls to keep away from in such eventualities?
Actual-Time Streaming Resolution for a Name Heart Analytics: Enterprise Challenges and Technical Enablement
Wednesday, June 28 @1:30 PM
A big worldwide consumer with a enterprise footprint in North America, Europe and Africa reached out to us with an curiosity in having a real-time streaming resolution designed and carried out for its name heart dealing with incoming and outgoing consumer calls. The consumer had a earlier dangerous expertise with one other vendor, who overpromised and underdelivered on the latency of the streaming resolution. The earlier vendor delivered an over-complex streaming knowledge pipeline ensuing within the knowledge taking on 5 minutes to achieve a visualization layer. The consumer felt that structure was too advanced and concerned many providers built-in collectively.
Our instant challenges concerned gaining the consumer’s belief and proving that our design and implementation high quality would supersede a earlier expertise. To resolve a direct problem of the overly sophisticated pipeline design, we deployed a Databricks Lakehouse structure with Azure Databricks on the heart of the answer. Our reference structure built-in Genesys Cloud -> App Providers -> Occasion Hub -> Databricks <-> Information Lake -> Energy BI.
The streaming resolution proved to be low latency (seconds) throughout the POV stage, which led to subsequent productionalization of the pipeline with deployment of jobs, DLTs pipeline, together with multi-notebook workflow and enterprise and efficiency metrics dashboarding relied on by the decision heart employees for a day-to-day efficiency monitoring and enhancements.
Sponsored by: Capgemini | Driving Pace to Insights with an Prolonged Lakehouse Structure
Thursday, June 29 @1:30 PM
Enterprises have lengthy struggled to allow knowledge as an enterprise asset for the complete vary of knowledge consumption patterns. Whereas we proceed to face obstacles with the normal patterns that embody reporting, ad-hoc analytics, real-time analytics and AI/ML, we additionally now face a wholly new sample that we name autonomous analytics.
In a single instance, inside the context of the Azure Public Cloud, we illustrate how corporations can leverage the open knowledge format of the Databricks Lakehouse, utilizing Delta Lake, mixed with different cloud native and ISV options to arrange and distribute enterprise knowledge in an insights first structure. Learn to leverage this structure to offer an expanded enterprise semantic knowledge layer with richer knowledge in real-time that drives essential insights on the velocity of enterprise.
Sponsored by: Qlik | Extracting the Full Potential of SAP Information for World Automotive Manufacturing
Thursday, June 29 @1:30 PM
Yearly, organizations lose tens of millions of {dollars} on account of tools failure, unscheduled downtime, or unoptimized provide chains as a result of enterprise and operational knowledge isn’t built-in. Throughout this session you’ll hear from specialists at Qlik and Databricks on how world luxurious automotive producers are accelerating the invention and availability of advanced knowledge units like SAP. Learn the way Qlik, Microsoft, and Databricks collectively are delivering an built-in resolution for world luxurious automotive producers that mixes the automated knowledge supply capabilities of Qlik Information Integration with the agility and openness of the Databricks Lakehouse platform and AI on Azure Synpase.
We’ll discover easy methods to leverage the IT and OT knowledge convergence to extract the complete potential of business-critical SAP knowledge, decrease IT prices and ship real-time prescriptive insights, at scale, for extra resilient, predictable, and sustainable supply-chains. Learn the way organizations can monitor and handle stock ranges, predict demand, optimize manufacturing and assist their organizations determine alternatives for enhancements.
Delta Sharing: The Key Information Mesh Enabler
Thursday, June 29 @2:30 PM
Information Mesh is an rising structure sample that challenges the centralized knowledge platform strategy by empowering totally different engineering groups to personal the information merchandise in a selected enterprise area. One of many keys to the success of any Information Mesh initiative is choosing the correct protocol for Information Sharing between totally different enterprise knowledge domains that might doubtlessly be carried out by means of totally different applied sciences and cloud suppliers.
On this session you’ll find out about how the Delta Sharing protocol and the Delta desk format have enabled the traditionally stuck-in-the-past power and development trade to be catapulted to the twenty first century by means of a contemporary Information Mesh implementation based mostly on Azure Databricks.
We additionally invite you to go to the Microsoft sales space on the Expo ground, the place you will get to speak 1:1 with Azure knowledge engineering on easy methods to deal with your hardest analytics challenges with Azure.
Register now to affix this free digital occasion and be part of the information and AI group. Learn the way corporations are efficiently constructing their Lakehouse structure with Azure Databricks to create a unified, open, and scalable knowledge platform. Get began utilizing Databricks with $200 in Azure credit and a free trial.