Be a part of AWS Databricks clients at Knowledge + AI Summit 2023


It is a collaborative put up from Databricks and Amazon Net Companies (AWS). We thank Venkat Viswanathan, Knowledge and Analytics Technique Chief, Companion Options at AWS, for his contributions.

 

Knowledge + AI Summit 2023: Register now to affix this in-person and digital occasion June 26-29 and be taught from the worldwide information neighborhood.

Amazon Net Companies (AWS) is a Platinum Sponsor of Knowledge + AI Summit 2023, the premier occasion for the worldwide information neighborhood. Be a part of this occasion and be taught from joint Databricks and AWS clients like Labcorp, Conde Nast, Grammarly, Vizio, NTT Knowledge, Impetus, Amgen, and YipitData who’ve efficiently leveraged the Databricks Lakehouse Platform for his or her enterprise, bringing collectively information, AI and analytics on one frequent platform.

At Knowledge + AI Summit, Databricks and AWS clients will take the stage for periods that will help you see how they achieved enterprise outcomes utilizing the Databricks on AWS Lakehouse. Attendees can have the chance to listen to information leaders from Labcorp on Tuesday, June twenty seventh, then be part of Grammarly, Vizio, NTT Knowledge, Impetus, and Amgen on Wednesday, June twenty eighth and Conde Nast and YipitData on Thursday, June twenty ninth. At Knowledge + AI Summit, be taught concerning the newest improvements and applied sciences and listen to thought-provoking panel discussions together with the flexibility for networking alternatives the place you may join with different information professionals in your trade.

AWS shall be showcasing the way to make the most of AWS native companies with Databricks at each their AWS sales space and Demo Stations:

In Demo Station 1 – AWS shall be showcasing how clients can leverage AWS native companies together with AWS Glue, Amazon Athena, Amazon Kinesis, Amazon S3, to investigate Delta Lake.

  • Databricks Lakehouse platform with AWS Glue, Amazon Athena, and Amazon S3
  • AWS IoT Hub, Amazon Kinesis Knowledge Streams, Databricks Lakehouse platform, Amazon S3 (presumably extending to Quicksight)
  • SageMaker JumpStart, Databricks created Dolly 2.0 and different open supply LLMs, Amazon OpenSearch
  • SageMaker Knowledge Wrangler and Databricks Lakehouse platform

In Demo Station 2 – AWS will completely reveal Amazon Quicksight integration with Databricks Lakehouse platform

  • Databricks Lakehouse platform, Amazon QuickSight, Amazon QuickSight Q

Please cease by the Demo Stations and the AWS sales space to be taught extra about Databricks on AWS, meet the AWS staff, and ask questions.

The periods beneath are a information for everybody excited by Databricks on AWS and span a variety of subjects — from information observability, to reducing whole price of possession, to demand forecasting and safe information sharing. You probably have questions on Databricks on AWS or service integrations, join with Databricks on AWS Options Architects at Knowledge + AI Summit.

Databricks on AWS buyer breakout periods

Labcorp Knowledge Platform Journey: From Choice to Go-Stay in Six Months

Tuesday, June 27 @3:00 PM

Be a part of this session to be taught concerning the Labcorp information platform transformation from on-premises Hadoop to AWS Databricks Lakehouse. We’ll share greatest practices and classes discovered from cloud-native information platform choice, implementation, and migration from Hadoop (inside six months) with Unity Catalog.

We’ll share steps taken to retire a number of legacy on-premises applied sciences and leverage Databricks native options like Spark streaming, workflows, job swimming pools, cluster insurance policies and Spark JDBC inside Databricks platform. Classes discovered in Implementing Unity Catalog and constructing a safety and governance mannequin that scales throughout purposes. We’ll present demos that stroll you thru batch frameworks, streaming frameworks, information examine instruments used throughout a number of purposes to enhance information high quality and velocity of supply.

Uncover how we have now improved operational effectivity, resiliency and diminished TCO, and the way we scaled constructing workspaces and related cloud infrastructure utilizing Terraform supplier.

Be taught extra

How Comcast Effectv Drives Knowledge Observability with Databricks and Monte Carlo

Tuesday, June 27 @4:00 PM

Comcast Effectv, the two,000-employee promoting wing of Comcast, America’s largest telecommunications firm, offers customized video advert options powered by aggregated viewership information. As a world expertise and media firm connecting hundreds of thousands of shoppers to customized experiences and processing billions of transactions, Comcast Effectv was challenged with dealing with huge a great deal of information, monitoring tons of of information pipelines, and managing well timed coordination throughout information groups.

On this session, we’ll talk about Comcast Effectv’s journey to constructing a extra scalable, dependable lakehouse and driving information observability at scale with Monte Carlo. This has enabled Effectv to have a single pane of glass view of their total information setting to make sure client information belief throughout their total AWS, Databricks, and Looker setting.

Be taught extra

Deep Dive Into Grammarly’s Knowledge Platform

Wednesday, June 28 @11:30 AM

Grammarly helps 30 million individuals and 50,000 groups to speak extra successfully. Utilizing the Databricks Lakehouse Platform, we will quickly ingest, remodel, combination, and question complicated information units from an ecosystem of sources, all ruled by Unity Catalog. This session will overview Grammarly’s information platform and the selections that formed the implementation. We’ll dive deep into some architectural challenges the Grammarly Knowledge Platform staff overcame as we developed a self-service framework for incremental occasion processing.

Our funding within the lakehouse and Unity Catalog has dramatically improved the velocity of our information worth chain: making 5 billion occasions (ingested, aggregated, de-identified, and ruled) out there to stakeholders (information scientists, enterprise analysts, gross sales, advertising and marketing) and downstream companies (characteristic retailer, reporting/dashboards, buyer help, operations) out there inside 15. Consequently, we have now improved our question price efficiency (110% quicker at 10% the fee) in comparison with our legacy system on AWS EMR.

I’ll share structure diagrams, their implications at scale, code samples, and issues solved and to be solved in a technology-focused dialogue about Grammarly’s iterative lakehouse information platform.

Be taught extra

Having Your Cake and Consuming it Too: How Vizio Constructed a Subsequent-Era ACR Knowledge Platform Whereas Decreasing TCO

Wednesday, June 28 @1:30 PM

As the highest producer of good TVs, Vizio makes use of TV information to drive its enterprise and supply clients with greatest digital experiences. Our firm’s mission is to repeatedly enhance the viewing expertise for our clients, which is why we developed our award-winning computerized content material recognition (ACR) platform. After we first constructed our information platform nearly ten years in the past, there was no single platform to run an information as a service enterprise, so we bought artistic and constructed our personal by stitching collectively totally different AWS companies and an information warehouse. As our enterprise wants and information volumes have grown exponentially over time, we made the strategic determination to replatform on Databricks Lakehouse, because it was the one platform that would fulfill all our wants out-of-the-box comparable to BI analytics, real-time streaming, and AI/ML. Now the Lakehouse is our sole supply of fact for all analytics and machine studying tasks. The technical worth of the Databricks Lakehouse platform, comparable to conventional information warehousing low-latency question processing with complicated joins due to Photon to utilizing Apache Spark™ structured streaming; analytics and mannequin serving, shall be lined on this session as we speak about our path to the Lakehouse.

Be taught extra

Why a Main Japanese Monetary Establishment Selected Databricks to Speed up its Knowledge and AI-Pushed Journey

Wednesday, June 28 @2:30 PM

On this session, we’ll introduce a case research of migrating the Japanese largest information evaluation platform to Databricks.

NTT DATA is among the largest system integrators in Japan. Within the Japanese market, many firms are engaged on BI, and we are actually within the section of utilizing AI. Our staff offers options that present information analytics infrastructure to drive the democratization of information and AI for main Japanese firms.

The shopper on this case research is among the largest monetary establishments in Japan. This venture has the next traits:

As a monetary establishment, safety necessities are very strict.

Since it’s used company-wide, together with group firms, it’s essential to help varied use instances.

We began working an information evaluation platform on AWS in 2017. Over the subsequent 5 years, we leveraged AWS-managed companies comparable to Amazon EMR, Amazon Athena, and Amazon SageMaker to modernize our structure. Within the close to future, with a view to promote the use instances of AI in addition to BI extra effectively, we have now begun to think about upgrading to a platform that realizes each BI and AI. This session will cowl:

Challenges in growing AI on a DWH-based information evaluation platform and why an information lakehouse is your best option.

Analyzing the structure of a platform that helps each AI and BI use instances.

On this case research, we’ll introduce the outcomes of a comparative research of a proposal primarily based on Databricks, a proposal primarily based on Snowflake, and a proposal combining Snowflake and Databricks. This session is really helpful for individuals who need to speed up their enterprise by using AI in addition to BI.

Be taught extra

Impetus | Accelerating ADP’s Enterprise Transformation with a Trendy Enterprise Knowledge Platform

Wednesday, June 28 @2:30 PM

Be taught How ADP’s Enterprise Knowledge Platform Is used to drive direct monetization alternatives, differentiate its options, and enhance operations. ADP is repeatedly looking for methods to extend innovation velocity, time-to-market, and enhance the general enterprise effectivity. Making information and instruments out there to groups throughout the enterprise whereas lowering information governance threat is the important thing to creating progress on all fronts. Find out about ADP’s enterprise information platform that created a single supply of fact with centralized instruments, information property, and companies. It allowed groups to innovate and acquire insights by leveraging cross-enterprise information and central machine studying operations.

Discover how ADP accelerated creation of the information platform on Databricks and AWS, obtain quicker enterprise outcomes, and enhance general enterprise operations. The session will even cowl how ADP considerably diminished its information governance threat, elevated the model by amplifying information and insights as a differentiator, elevated information monetization, and leveraged information to drive human capital administration differentiation.

Be taught extra

From Insights to Suggestions: How SkyWatch Predicts Demand for Satellite tv for pc Imagery Utilizing Databricks

Wednesday, June 28 @3:30 PM

SkyWatch is on a mission to democratize earth remark information and make it easy for anybody to make use of.

On this session, you’ll find out about how SkyWatch aggregates demand alerts for the EO market and turns them into monetizable suggestions for satellite tv for pc operators. Skywatch’s Knowledge & Platform Engineer, Aayush will share how the staff constructed a serverless structure that synthesizes buyer requests for satellite tv for pc photographs and identifies geographic places with excessive demand, serving to satellite tv for pc operators maximize income and satisfying a broad vary of EO information hungry shoppers.

This session will cowl:

  • Challenges with Achievement in Earth Remark ecosystem
  • Processing massive scale GeoSpatial Knowledge with Databricks
  • Databricks in-built H3 capabilities
  • Delta Lake to effectively retailer information leveraging optimization methods like Z-Ordering
  • Knowledge LakeHouse Structure with Serverless SQL Endpoints and AWS Step Features
  • Constructing Tasking Suggestions for Satellite tv for pc Operators

Be taught extra

Enabling Knowledge Governance at Enterprise Scale Utilizing Unity Catalog

Wednesday, June 28 @3:30 PM

Amgen has invested in constructing fashionable, cloud-native enterprise information and analytics platforms over the previous few years with a give attention to tech rationalization, information democratization, general consumer expertise, enhance reusability, and cost-effectiveness. Considered one of these platforms is our Enterprise Knowledge Cloth which focuses on pulling in information throughout capabilities and offering capabilities to combine and join the information and govern entry. For some time, we have now been attempting to arrange strong information governance capabilities that are easy, but straightforward to handle via Databricks. There have been just a few instruments out there that solved just a few speedy wants, however none solved the issue holistically. To be used instances like sustaining governance on extremely restricted information domains like Finance and HR, a long-term answer native to Databricks and addressing the beneath limitations was deemed vital:

The way in which these instruments had been arrange, allowed the overriding of some safety insurance policies

  • Instruments weren’t UpToDate with the newest DBR runtime
  • Complexity of implementing fine-grained safety
  • Coverage administration – AWS IAM + In instrument insurance policies

To deal with these challenges, and for large-scale enterprise adoption of our governance functionality, we began engaged on UC integration with our governance processes. With an intention to appreciate the next tech advantages:

  • Unbiased of Databricks runtime
  • Straightforward fine-grained entry management
  • Eradicated administration of IAM roles
  • Dynamic entry management utilizing UC and dynamic views

At the moment, utilizing UC, we have now to implement fine-grained entry management & governance for the restricted information of Amgen. We’re within the technique of devising a sensible migration & change administration technique throughout the enterprise.

Be taught extra

Activate Your Lakehouse with Unity Catalog

Thursday, June 29 @1:30 PM

Constructing a lakehouse is easy at the moment due to many open supply applied sciences and Databricks. Nevertheless, it may be taxing to extract worth from lakehouses as they develop with out strong information operations. Be a part of us to learn the way YipitData makes use of the Unity Catalog to streamline information operations and uncover greatest practices to scale your personal Lakehouse. At YipitData, our 15+ petabyte Lakehouse is a self-service information platform constructed with Databricks and AWS, supporting analytics for an information staff of over 250. We’ll share how leveraging Unity Catalog accelerates our mission to assist monetary establishments and companies leverage different information by:

  • Enabling shoppers to universally entry our information via a spectrum of channels, together with Sigma, Delta Sharing, and a number of clouds
  • Fostering collaboration throughout inside groups utilizing an information mesh paradigm that yields wealthy insights
  • Strengthening the integrity and safety of information property via ACLs, information lineage, audit logs, and additional isolation of AWS sources
  • Decreasing the price of massive tables with out downtime via automated information expiration and ETL optimizations on managed delta tables

By way of our migration to Unity Catalog, we have now gained ways and philosophies to seamlessly movement our information property internally and externally. Knowledge platforms should be value-generating, safe, and cost-effective in at the moment’s world. We’re excited to share how Unity Catalog delivers on this and helps you get probably the most out of your lakehouse.

Be taught extra

Knowledge Globalization at Conde Nast Utilizing Delta Sharing

Thursday, June 29 @1:30 PM

Databricks has been a necessary a part of the Conde Nast structure for the previous couple of years. Previous to constructing our centralized information platform, “evergreen,” we had related challenges as many different organizations; siloed information, duplicated efforts for engineers, and a scarcity of collaboration between information groups. These issues led to distrust in information units and made it tough to scale to fulfill the strategic globalization plan we had for Conde Nast.

Over the previous couple of years we have now been extraordinarily profitable in constructing a centralized information platform on Databricks in AWS, totally embracing the lakehouse imaginative and prescient from end-to-end. Now, our analysts and entrepreneurs can derive the identical insights from one dataset and information scientists can use the identical datasets to be used instances comparable to personalization, subscriber propensity fashions, churn fashions and on-site suggestions for our iconic manufacturers.

On this session, we’ll talk about how we plan to include Unity Catalog and Delta Sharing as the subsequent section of our globalization mission. The evergreen platform has grow to be the worldwide customary for information processing and analytics at Conde. With a purpose to handle the worldwide information and adjust to GDPR necessities, we’d like to ensure information is processed within the applicable area and PII information is dealt with appropriately. On the similar time, we have to have a world view of the information to permit us to make enterprise choices on the international degree. We’ll speak about how delta sharing permits us a easy, safe strategy to share de-identified datasets throughout areas with a view to make these strategic enterprise choices, whereas complying with safety necessities. Moreover, we’ll talk about how Unity Catalog permits us to safe, govern and audit these datasets in a straightforward and scalable method.

Be taught extra

Databricks on AWS breakout periods

AWS | Actual Time Streaming Knowledge Processing and Visualization Utilizing Databricks DLT, Amazon Kinesis, and Amazon QuickSight

Wednesday, June 28 @11:30 AM

Amazon Kinesis Knowledge Analytics is a managed service that may seize streaming information from IoT units. Databricks Lakehouse platform offers ease of processing streaming and batch information utilizing Delta Stay Tables. Amazon Quicksight with highly effective visualization capabilities can offers varied superior visualization capabilities with direct integration with Databricks. Combining these companies, clients can seize, course of, and visualize information from tons of and hundreds of IoT sensors with ease.

Be taught extra

AWS | Constructing Generative AI Resolution Utilizing Open Supply Databricks Dolly 2.0 on Amazon SageMaker

Wednesday, June 28 @2:30 PM

Create a customized chat-based answer to question and summarize your information inside your VPC utilizing Dolly 2.0 and Amazon SageMaker. On this discuss, you’ll find out about Dolly 2.0, Databricks, state-of-the-art, open supply, LLM, out there for business and Amazon SageMaker, AWS’s premiere toolkit for ML builders. You’ll learn to deploy and customise fashions to reference your information utilizing retrieval augmented technology (RAG) and extra wonderful tuning methods…all utilizing open-source parts out there at the moment.

Be taught extra

Processing Delta Lake Tables on AWS Utilizing AWS Glue, Amazon Athena, and Amazon Redshift

Thursday, June 29 @1:30 PM

Delta Lake is an open supply venture that helps implement fashionable information lake architectures generally constructed on cloud storages. With Delta Lake, you may obtain ACID transactions, time journey queries, CDC, and different frequent use instances on the cloud.

There are a whole lot of use instances of Delta tables on AWS. AWS has invested lots on this expertise, and now Delta Lake is on the market with a number of AWS companies, comparable to AWS Glue Spark jobs, Amazon EMR, Amazon Athena, and Amazon Redshift Spectrum. AWS Glue is a serverless, scalable information integration service that makes it simpler to find, put together, transfer, and combine information from a number of sources. With AWS Glue, you may simply ingest information from a number of information sources comparable to on-prem databases, Amazon RDS, DynamoDB, MongoDB into Delta Lake on Amazon S3 even with out experience in coding.

This session will reveal the way to get began with processing Delta Lake tables on Amazon S3 utilizing AWS Glue, and querying from Amazon Athena, and Amazon Redshift. The session additionally covers latest AWS service updates associated to Delta Lake.

Be taught extra

Databricks-led periods

Utilizing DMS and DLT for Change Knowledge Seize

Tuesday, June 27 @2:00 PM

Bringing in Relational Knowledge Retailer (RDS) information into your information lake is a crucial and vital course of to facilitate use instances. By leveraging AWS Database Migration Companies (DMS) and Databricks Delta Stay Tables (DLT) we will simplify change information seize out of your RDS. On this discuss, we shall be breaking down this complicated course of by discussing the basics and greatest practices. There will even be a demo the place we convey this all collectively

Be taught extra

Learnings From the Subject: Migration From Oracle DW and IBM DataStage to Databricks on AWS

Wednesday, June 28 @2:30 PM

Legacy information warehouses are pricey to take care of, unscalable and can’t ship on information science, ML and real-time analytics use instances. Migrating out of your enterprise information warehouse to Databricks permits you to scale as what you are promoting wants develop and speed up innovation by operating all of your information, analytics and AI workloads on a single unified information platform.

Within the first a part of this session we’ll information you thru the well-designed course of and instruments that can enable you to from the evaluation section to the precise implementation of an EDW migration venture. Additionally, we’ll handle methods to transform PL/SQL proprietary code to an open customary python code and reap the benefits of PySpark for ETL workloads and Databricks SQL’s information analytics workload energy.

The second a part of this session shall be primarily based on an EDW migration venture of SNCF (French nationwide railways); one of many main enterprise clients of Databricks in France. Databricks partnered with SNCF emigrate its actual property entity from Oracle DW and IBM DataStage to Databricks on AWS. We’ll stroll you thru the shopper context, urgency to migration, challenges, goal structure, nitty-gritty particulars of implementation, greatest practices, suggestions, and learnings with a view to execute a profitable migration venture in a really accelerated timeframe.

Be taught extra

Embracing the Way forward for Knowledge Engineering: The Serverless, Actual-Time Lakehouse in Motion

Wednesday, June 28 @2:30 PM

As we enterprise into the way forward for information engineering, streaming and serverless applied sciences take middle stage. On this enjoyable, hands-on, in-depth and interactive session you may be taught concerning the essence of future information engineering at the moment.

We’ll sort out the problem of processing streaming occasions repeatedly created by tons of of sensors within the convention room from a serverless net app (convey your cellphone and be part of the demo). The main focus is on the system structure, the concerned merchandise and the answer they supply. Which Databricks product, functionality and settings shall be most helpful for our situation? What does streaming actually imply and why does it make our life simpler? What are the precise advantages of serverless and the way “serverless” is a specific answer?

Leveraging the ability of the Databricks Lakehouse Platform, I’ll reveal the way to create a streaming information pipeline with Delta Stay Tables ingesting information from AWS Kinesis. Additional, I am going to make the most of superior Databricks workflows triggers for environment friendly orchestration and real-time alerts feeding right into a real-time dashboard. And since I do not need you to go away with empty fingers – I’ll use Delta Sharing to share the outcomes of the demo we constructed with each participant within the room. Be a part of me on this hands-on exploration of cutting-edge information engineering methods and witness the long run in motion.

Be taught extra

Seven Issues You Did not Know You Can Do with Databricks Workflows

Wednesday, June 28 @3:30 PM

Databricks workflows has come a great distance because the preliminary days of orchestrating easy notebooks and jar/wheel recordsdata. Now we will orchestrate multi-task jobs and create a sequence of duties with lineage and DAG with both fan-in or fan-out amongst a number of different patterns and even run one other Databricks job straight inside one other job.

Databricks workflows takes its tag: “orchestrate something anyplace” fairly critically and is a very fully-managed, cloud-native orchestrator to orchestrate numerous workloads like Delta Stay Tables, SQL, Notebooks, Jars, Python Wheels, dbt, SQL, Apache Spark™, ML pipelines with glorious monitoring, alerting and observability capabilities as effectively. Principally, it’s a one-stop product for all orchestration wants for an environment friendly lakehouse. And what’s even higher is, it provides full flexibility of operating your jobs in a cloud-agnostic and cloud-independent means and is on the market throughout AWS, Azure and GCP.

On this session, we’ll talk about and deep dive on a number of the very attention-grabbing options and can showcase end-to-end demos of the options which is able to can help you take full benefit of Databricks workflows for orchestrating the lakehouse.

Be taught extra

Register now to affix this free digital occasion and be part of the information and AI neighborhood. Find out how firms are efficiently constructing their Lakehouse structure with Databricks on AWS to create a easy, open and collaborative information platform. Get began utilizing Databricks with a free trial on AWS Market or swing by the AWS sales space to be taught extra a couple of particular promotion. Be taught extra about Databricks on AWS.

Related Articles

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Latest Articles