Driving Self-service and Bettering DataOps with Atlan
The Energetic Metadata Pioneers sequence options Atlan prospects who’ve just lately accomplished an intensive analysis of the Energetic Metadata Administration market. Paying ahead what you’ve discovered to the following information chief is the true spirit of the Atlan group! So that they’re right here to share their hard-earned perspective on an evolving market, what makes up their trendy information stack, revolutionary use instances for metadata, and extra!
Within the first interview of this sequence, we meet Heidi Jones, information evaluation and program administration extraordinaire, who explains the historical past of Docker’s information group, how they evaluated the market, and the way they’ll use Atlan to assist their colleagues drive one of many world’s greatest developer experiences.
This interview has been edited for brevity and readability.
Would you thoughts describing Docker and your information group?
Docker is a platform designed to assist builders construct, share, and run trendy purposes. We deal with the tedious setup, so builders can deal with the code.
Knowledge professionals at Docker help quite a lot of totally different departments. So we’ve got a core information group with engineers and analysts, after which we even have information engineers and analysts that help the key capabilities of Docker, equivalent to Advertising, Gross sales, the totally different merchandise at Docker, Finance, et cetera.
A number of skilled information engineers and analysts who’ve joined Docker, have solely began within the final 9 months or so. So we’ve had fairly a little bit of development on the info group, and are actually at that stage the place we’re making an attempt to put money into good processes. That means, our information group can be certain that everybody at Docker has the info that they should do their jobs, and might in the end assist builders do theirs.
And the way about you? May you inform us a bit about your self, your background, and what drew you to Knowledge & Analytics?
I feel the primary purpose I’ve been drawn to information and analytics is as a result of I identical to having the ability to reply folks’s questions.
I got here into information evaluation via a non-traditional route. I’ve been at Docker for about six months now, however I’ve been within the information house for a couple of decade. It began with Excel and offering insights through spreadsheets, as much as PowerBI utilizing Snowflake, that sort of factor.
So I used to be at all times an information analyst, however then additionally a undertaking supervisor. And so what I do at Docker combines each of these. The information of knowledge and the workflows required to get good information and supply good insights, and in addition the undertaking administration and operations facet of it. All of it permits information professionals to deal with what they do greatest, which is modeling information and offering insights with out being blocked by something that has to do with workflow.
What does your stack appear like? Why did you want an energetic metadata resolution?
We ingest information from quite a lot of sources in a number of alternative ways, relying on the supply. After which our information warehouse degree is Snowflake. Our modeling layer is dbt, that’s the place we do modeling and transformation. After which our most important BI device is Looker, that’s the place we do visualization and evaluation.
We have been only a one-person group not too way back. So all of that information work was on one particular person’s plate, together with documentation and understanding information sources. That’s fairly a bit for one particular person.
A number of that burden has been unfold out throughout a number of folks on the group by now. However we’re making an attempt to maneuver away from, “Oh, let me go ask my favourite information particular person,” towards, “I can go test this device and I do know there’s an authorized information asset.”
And so, due to our stack, we have been drawn to Atlan due to issues just like the Looker Chrome extension plugin, the dbt integration, that sort of factor. As a result of proper off the bat we have been capable of say, “Okay, any descriptions we put in our dbt layer will mechanically be uncovered in Atlan.”
So non-engineering customers who wish to know what the info means can go straight to Atlan and see what’s being achieved within the modeling layer.
Did something stand out to you about Atlan throughout your analysis course of?
Atlan is a really cool device that has a very good suite of options that we have been searching for, however the differentiator actually got here all the way down to the folks at Atlan.
You demonstrated very competent understanding of the issues within the information house and in addition very mature buyer help. We might inform that your help was not simply one thing you have been promising for us, however one thing that you simply have been already actively doing with different prospects.
We knew that it will be an actual partnership and that the client help org was ready to help the wants of a company like ours. And that maturity stood out to us after we made our choice.
However then once more, additionally the options like Playbooks, the integrations that I’ve already talked about with dbt, with Looker, and simply the fixed innovation as effectively that we have been capable of observe even through the analysis processes, which I imagine took us about two months.
There have been a number of improvements and releases that occurred throughout that point interval and we might see the cadence the Atlan was on to repeatedly enhance. All of these have been promoting factors to us.
What do you propose on creating with Atlan? Do you have got an concept of what use instances you’ll construct, and the worth you’ll drive?
Our greatest worth that we’re making an attempt to drive with Atlan is to be sure that professionals at Docker can get the knowledge they want concerning the information that they should do their jobs.
We wish to transfer in the direction of self-serve analytics and permit each information professionals, and people who simply need to have the ability to use information extra freely of their work, to have the ability to achieve this with out having to get into the entire SQL and technical particulars of the info.
They know they will belief the info set, they know they will belief the info that they’re , and so they can go forward and make their choices. Finally, it ought to assist us help our mission of delighting builders, and growing instruments that they take pleasure in utilizing.
We’ll be supporting that with Atlan, and in addition supporting our information engineering and analytics groups. They should have extra supported and standardized workflows, in order that they will deal with modeling, actually digging in and doing what they do greatest with information.
Did we miss something?
That’s a very good query. I feel how we found Atlan was attention-grabbing. I’ve been following Prukalpa, truly, for a few years simply as an information skilled, simply sort of watching Atlan.
And so once I joined Docker, they have been already information catalog instruments, however hadn’t been Atlan but. And I stated, “Properly, how about Atlan? Ought to we take a look at Atlan as effectively?”
So one of many first issues I did at Docker was to start out up that dialog, and the explanation why I did that’s as a result of I had appreciated studying what she stated in these areas. Concerning the causes we want information catalog instruments, and past only a catalog, the way it could possibly be a part of information operations. And that piece of it actually had spoken to me over time.
And we noticed some spectacular instruments. It’s a burgeoning house. There’s some nice instruments on the market. However I’m glad that we additionally checked out Atlan as a result of in the end it had a very good mixture of what we would have liked at Docker.
Picture by Annie Spratt on Unsplash
