Making the Most of Current Knowledge with Atlan
The Lively Metadata Pioneers sequence options Atlan clients who’ve not too long ago accomplished a radical analysis of the Lively Metadata Administration market. Paying ahead what you’ve discovered to the following knowledge chief is the true spirit of the Atlan group! So that they’re right here to share their hard-earned perspective on an evolving market, what makes up their trendy knowledge stack, revolutionary use circumstances for metadata, and extra.
On this installment of the sequence, we meet Robert Bayliss, Technical Supervisor, Knowledge Platform from Racing and Wagering Western Australia, who shares how a contemporary knowledge catalog will probably be key to making sure their knowledge is seen, full, correct, and well timed for the enterprise companions they serve.
This interview has been edited for brevity and readability.
Would you thoughts describing RWWA and the way your knowledge crew helps the group?
Working for Racing and Wagering Western Australia (RWWA) is extremely distinctive, given we’re the controlling authority for thoroughbred, harness, and greyhound racing throughout our State, whereas we even have the duty for off-course TAB wagering. We’re right here to offer a sustainable future to the Western Australia racing trade – an trade which is value $1.3 billion for the Western Australia financial system yearly. Nevertheless, racing is value rather more than its financial injection, its social advantages are immeasurable, particularly in regional cities, fostering group connection from Albany to Kununurra and in all places in between.
Given how huge RWWA is, expertise performs a serious position in our day-to-day operations. Our knowledge platform crew is part of a bigger expertise crew, with our core duties and tasks targeted on making certain the information platform offers correct, well timed, full, and reliable knowledge to the whole RWWA crew.
May you inform us a bit about your self, your background, and what drew you to Knowledge & Analytics?
I started my profession constructing data-driven functions, and actually loved making apps that enabled folks to do issues they couldn’t do earlier than. I moved into database improvement, resolution design, ICT challenge administration, after which, main an functions crew.
Lastly, I’ve moved to RWWA the place I lead the information platform crew and implement our knowledge technique. Finally, all my roles have been targeted on enabling folks and groups to realize issues that they couldn’t do earlier than. I imagine knowledge to be an amazing enabler that anybody can use to enhance the way in which they do issues.
What does your knowledge stack seem like?
With our supply knowledge unfold throughout a various expertise panorama together with Oracle, SQL Server and bespoke APIs, we needed to harness a lot of applied sciences to realize our targets.This consists of AWS DMS for knowledge extraction, S3 for intermediate storage and Snowflake for Enterprise Intelligence and reporting. Each Python-based Lambas and DBT scripts orchestrated via Airflow full the transformation stage in our ELT course of.
Why seek for an Lively Metadata Administration resolution?
We’ve got quite a lot of knowledge that may assist our groups make well timed and important selections, so the necessity to supply an Lively Metadata Administration Resolution was to assist improve the visibility and accessibility of our knowledge to everybody within the group, from those who work carefully with our knowledge engineers to people who assist the animal welfare applications.
These stakeholders want entry to correct knowledge, and the information catalog was the device to do this. The basic query was “How can we floor our knowledge, and the way can we make it correct and well timed so that folks can depend on it?”
Why was Atlan match? Did something stand out throughout your analysis course of?
To be trustworthy, Atlan was the one device that made our knowledge engineering crew excited and additionally resonated extraordinarily nicely with the tip customers. It made sense from a usability perspective. It was straightforward to visualise, it flowed and the simplicity of the expertise meant not too many clicks to get to the data.
We have been actually targeted on the truth that our engineering crew was excited by it, and favored the truth that we will see lineage and expose our knowledge units, present the way it’s all related, and likewise floor all of our knowledge high quality metrics, as nicely.
What do you plan on creating with Atlan? Do you might have an thought of what use circumstances you’ll construct, and the worth you’ll drive?
Knowledge accessibility, in addition to surfacing knowledge observability metrics, are actually essential to us. It’s no use getting access to knowledge in case you are not sure of the completeness, accuracy, or timeliness. Atlan helps us to realize this by surfacing our knowledge high quality guidelines and observability metrics alongside our knowledge property.
Moreover, having a strategy to floor knowledge classifications and key glossary phrases is a large profit to all of our groups, to assist deliver a typical understanding and settlement on terminology.
It’s actually about ensuring that we’re leveraging our property. We’ve bought giant volumes of knowledge nevertheless it must be within the fingers of those that can add worth with it.
Anything to share?
For me, it’s all the time in regards to the why. It’s, “Why are we truly doing something?” The rationale for Atlan was all the time about how we get probably the most out of our knowledge. That was the difficulty, then we labored out what we may use to make that occur.
Photograph by Philippe Oursel on Unsplash
