Databricks Notebooks provides builders a managed authoring expertise the place information and AI groups can effectively collaborate on tasks collectively. The group right here is working laborious as we put together to share thrilling new improvements for Notebooks on the Information + AI Summit later this month, and we hope you’ll be a part of us at our session Develop Like a Professional in Databricks Notebooks, led by Weston Hutchins and Neha Sharma. As a heat up, we wished to take a fast look again at some latest additions in Notebooks.
Run Databricks Notebooks on SQL Warehouses
SQL is the 2nd hottest language in Notebooks behind Python, and to higher assist our customers who love SQL, we’re bringing SQL warehouses to Notebooks. SQL warehouses are the identical sources that energy Databricks SQL, they usually ship higher price-performance for SQL execution in comparison with all-purpose clusters. This characteristic is rolling out now, so hold a watch out!
Whereas connected to a SQL warehouse, solely your pocket book’s SQL cells will execute. Cells utilizing different languages (like Python or Scala) will probably be skipped. Markdown cells will proceed to be rendered. To be taught extra, go to our documentation.
View information from Notebooks, SQL editor, and Information Explorer in the identical expertise
The brand new unified schema browser allows you to view the entire information within the Unity Catalog metastore with out leaving a pocket book or the SQL editor. You possibly can choose “For you” to filter the checklist to the energetic tables in your pocket book.
As you sort your search request into the filter field, the show actively updates to point out solely these objects that comprise that textual content. This can search for objects which can be presently open or have been opened earlier within the present session. Be taught extra right here.
Share Notebooks utilizing Delta Sharing
Now you can use Delta Sharing to share pocket book information with the Databricks-to-Databricks sharing circulation. You get the convenience and safety of Delta Sharing. Sharing notebooks empowers you to collaborate throughout metastores and accounts. This allows individuals who share information to unpack the worth of that information with notebooks.
To be taught extra, right here is how one can add notebooks to a share (for suppliers) and the right way to learn shared notebooks (for recipients).
Debug your Notebooks with the Variable Explorer
The Variable Explorer shows the state of all of the Python variables in your pocket book growth session. The title, sort, and worth are surfaced for all easy variable sorts. The Variable Explorer additionally surfaces further metadata for Spark and Pandas DataFrames. The form and column names can be found at-a-glance, and a full view of the schema is out there on hover.
The Variable Explorer additionally permits you to step by way of and debug Python code by leveraging the assist for pdb in Databricks Notebooks. You possibly can set breakpoints with breakpoint() or pdb.set_trace(). Once you run the cell, the execution will pause on the breakpoint and the Variable Explorer will routinely replace with the state of the pocket book at that breakpoint. See our documentation for extra info.
See you at Summit
At Information + AI Summit 2023, we’ll have deep dive classes into utilizing Notebooks. We will even speak in regards to the latest methods to be extra environment friendly whereas utilizing Databricks. We hope to see you there.
