Driving Self-service, Safe, Reliable Information with Atlan
- Swapfiets, the world’s first bicycle subscription service, sought to launch a governance program to satisfy rising inner demand for knowledge
- Selecting Atlan, Swapfiets launched their governance program with a self-service catalog, providing metrics and supply definitions throughout six enterprise domains
- Having efficiently launched the catalog, Swapfiets’ knowledge group now routinely runs governance committees to strengthen cross-functional collaboration, and is driving a tradition of information savviness via coaching and enablement
Based within the Netherlands in 2014, Swapfiets is on a mission to make cities extra livable, introducing the world to the idea of “Bicycle as a Service” with a month-to-month subscription to a bicycle with restore providers included. Now with over 270,000 subscribers throughout eight international locations, Swapfiets’ achievement of sustainability targets, and its continued progress, is determined by dependable, simply accessible knowledge.
“I’m actually obsessed with knowledge. Information is so various, and it has not solely a technical half, but additionally a governance, or extra summary half, as effectively,” shared Lisa Smits, Supervisor of Swapfiets’ Information & Analytics group. “That’s how I got here to Swapfiets, and that’s how I got here to steer this group.”
Drawing upon a various skilled background spanning jobs from provide chain administration to pricing evaluation, and together with the assistance of her group and colleagues, Lisa has helped shepherd in a brand new period of information savviness and collaboration throughout Swapfiets, constructing and rising a Information Governance observe, and making use of best-of-breed know-how like Snowflake and Atlan.
Information is a crucial a part of measuring the place we at the moment are, and what we have to do to develop additional. We have now initiatives like optimizing the elements we use and being extra sustainable, and that’s the place my group, instantly and not directly, contributes to the entire firm.”
Lisa Smits, Information Analytics Supervisor
As soon as structured as one giant group, Swapfiets has since damaged up its knowledge perform into three specialised elements. First is Information Engineering, liable for creating and sustaining Swapfiets’ knowledge platform and driving knowledge high quality and availability. Subsequent, Information Science creates advanced knowledge merchandise via superior modeling, akin to demand forecasting and buyer lifetime worth. Lastly, the analytics group led by Lisa is liable for Swapfiets’ BI environments and activating knowledge merchandise. Powering this knowledge group evolution is a shortly modernizing knowledge stack, atop Snowflake as their knowledge warehouse.
“We migrated to Snowflake. It’s extra versatile to scale up and down, it takes lots much less time for our engineers to arrange, and permissions administration was a giant factor that was simpler to do. This transitioned us into being a extra mature knowledge group,” Lisa shared.
Swapfiets additionally adopted Fivetran, automating the pipelines as soon as run on guide scripts, and dbt, using their semantic layer as a decentralized metrics retailer. Rounding out this modernization was the adoption of Airflow and Tableau.
“We matured in a single large step and actually specialised,” Lisa defined. “We targeted on having instruments to purchase, versus construct, making a acutely aware choice on paying a bit extra, however saving a lot time in the long run, leading to decrease prices.”
Whereas Swapfiets was maturing considerably throughout technical and organizational domains, tribal data about their knowledge belongings persevered. Initially, the group created a Google Sheet with an inventory of 25 metrics and definitions. However with no easy interface to entry these metrics, or perceive the context round them, the quantity of questions posed to the Information & Analytics group elevated.
“We acquired a variety of questions from individuals, like ‘How is that calculated?’ or ‘We see this quantity on this dashboard, however on this dashboard, it’s fully totally different. How does that work?’,” Lisa defined. “There was no possession, no transparency, and no central place for us to jot down down phrases, or clarify what metrics meant and the way they had been calculated. It took a variety of time to reply these ad-hoc issues.”
Normally, there was a superbly legitimate cause for variations between dashboards and belongings, however absent documentation, belief was eroding, main Lisa and her group towards a brand new concentrate on knowledge governance.
To lose belief is straightforward, however to achieve that belief again is tough. That’s once we began taking this severely. We wished to have a high-quality, reliable BI setting, and that’s why we began wanting into our choices for governance, and into Atlan.”
Lisa Smits, Information Analytics Supervisor
Lisa and her group started by trying to find glossary options as a way to enhance the transparency of their knowledge belongings and cut back the quantity of questions from Swapfiets’ knowledge customers. However as she and her group realized extra in regards to the Energetic Metadata Administration market, their expectations for the worth that such a platform may drive grew.
With a versatile consumer interface, knowledge customers may uncover, perceive, and apply knowledge in a self-service method, and with automated lineage, Swapfiets’ knowledge practitioners may higher perceive how knowledge flowed via their modernizing stack and the way it was being utilized. And with Atlan Insights, a metadata-based question builder, much more of Lisa’s colleagues could be empowered to make the most of knowledge.
Quite than only a glossary of phrases, Lisa and her group envisioned a one-stop store for his or her colleagues who used knowledge each day, and for many who ought to, however hadn’t but.
You’ve gotten one place the place you may retailer the phrases, the place you may have a look at your entire knowledge belongings, the place you will have the descriptions, and the place you may see the homeowners. In case you have a query, you may instantly ship it through Slack. It’s all linked in a single place, and I feel having that full image, not solely governance but additionally self-service, was what actually stood out to us about Atlan.”
Lisa Smits, Information Analytics Supervisor
After selecting Atlan, Lisa and her group started by integrating Redshift (since migrating to Snowflake), dbt, and Tableau, shortly enabling lineage throughout their most important applied sciences. With their knowledge property successfully crawled, and knowledge belongings seen, they started the method of assigning homeowners and making a enterprise glossary.
Utilizing the definitions they’d already agreed upon with their enterprise colleagues and had saved in Google Sheets, Lisa’s group enriched a baseline of a brand new enterprise glossary, then labored with their colleagues in Information Engineering to doc the sources of every asset. Whereas utilizing pre-existing definitions saved beneficial time, the method of enriching knowledge belongings and documenting their supply was nonetheless a cumbersome one. In Atlan Playbooks, rule-based bulk automations for duties like documentation, Swapfiets discovered a approach to considerably speed up enrichment.
Using Playbooks, Swapfiets constructed an automation guaranteeing that every time a brand new asset is launched into their metrics retailer, its pre-existing descriptions from upstream programs are mechanically handed via to Atlan. These descriptions are then verified, quite than written from scratch, considerably accelerating their documentation rollout.
“Content material creation was a variety of work, however whenever you launched Playbooks, it was wonderful for us,” Lisa shared. “We now have 125 metrics, and it improved our time spent on documentation.”
Furthering their use of Playbooks, Lisa’s group shortly moved to safe these newly out there knowledge belongings, mechanically tagging delicate or personally identifiable info, and utilizing personas inside Atlan to make sure delicate knowledge was out there solely to these approved to view it, or contribute to documenting it.
In search of to deepen collaboration with material specialists, and to make knowledge governance a routine a part of the way in which Swapfiets operates, Lisa attended an Atlan Masterclass with WeWork to learn the way finest to construct and keep a enterprise glossary.
Lisa’s group maintains a single enterprise glossary, divided into enterprise domains that align material specialists with companions on the info group. Six such enterprise domains exist inside the glossary, throughout groups like Operations, Human Sources, Product, and Client Direct. And for every of those six enterprise domains, Lisa’s group organized cross-functional governance committees composed of information analysts, knowledge homeowners, and knowledge stewards that meet on a month-to-month foundation to debate their progress.
“In that month-to-month assembly, we concentrate on three governance subjects, that are possession, documentation, and high quality. For documentation and possession, Atlan is absolutely useful. It’s additionally useful for high quality to search out out the place within the knowledge lifecycle high quality points come up,” Lisa shared. “Throughout the assembly, we write out motion factors and decide to duties like verifying extra phrases.”
To trace progress towards these commitments, Lisa’s group makes use of Atlan’s reporting performance as a heads-up show of what number of phrases and knowledge sources have been verified or enriched, and at what tempo. Additional simplifying the method, material specialists use filters in Atlan to show the phrases they’re liable for, giving them an inventory of related phrases and knowledge sources they’re tasked with enriching, or to mark unverified however vital phrases and knowledge sources as draft for future enrichment.
Whereas Swapfiets continues to search out new methods to use Atlan and develop their governance observe, the worth yielded from self-service and their metrics catalog, alone, has been important.
“Colleagues ceaselessly ask ‘What does this imply?’ or ‘How do you outline this metric?’ With an built-in workflow in Slack, we ship them to the Atlan glossary to self-serve their query. It’s been a good time saving for us,” Lisa defined.
For questions that may’t be answered by self-service, the Information & Analytics group advantages from clear, assigned possession for every knowledge asset, making it easy to direct inquiries to material specialists, or to tag them instantly right into a Slack thread.
By standardizing the method for outlining metrics, and automating their preliminary enrichment utilizing Atlan Playbooks, Lisa’s group has saved many hours of effort, thus far. However extra noteworthy to their group is the truth that they’re now assured their course of can be adopted every time a brand new metric is launched.
The entire standardization of all of the metrics saves between 10 and half-hour for each metric. So in case you multiply that by 125, it’s a pleasant quantity, however it’s not like, ‘Oh, we saved 5 weeks this 12 months.’ It’s simply good that we don’t have to consider this anymore. It’s not in our course of anymore whereas we create a metric. It’s already taken care of.”
Lisa Smits, Information Analytics Supervisor
With a catalog of metrics in place, and a sustainable course of to additional enrich it, Lisa and her group started a collection of packages designed to extend knowledge savviness throughout the Swapfiets group and additional cut back the burden of service on their group.
Information Literacy Program
Foundational to how Swapfiets allows self-service is their Information Literacy Program. A part of this program is a compulsory coaching for Atlan & Tableau. This teaches new Swapfiets workers about Atlan and its potentialities, and the best way to make the most of Swapfiets’ present Tableau dashboards. When full, customers are then given entry to Atlan and Tableau. On this program, the Analytics group additionally gives a SQL-Atlan coaching for his or her Information Champions.
For self-service questions, we like educating individuals the fundamentals of querying. And with Atlan, you will have the visible question builder. Even when persons are not tremendous knowledge literate, they will mess around with the question builder and it’s a lot simpler to know than one thing like ‘SELECT * FROM’. If they should filter, it’s truly known as filter as a substitute of the technical ‘WHERE’ assertion. It simply speaks to them much more.”
Lisa Smits, Information Analytics Supervisor
With the mixture of the Information Literacy Program and Atlan Insights, the usage of Tableau is now reserved for long-term knowledge merchandise and visualization, with the majority of customized querying now carried out utilizing Atlan, doubtlessly decreasing licensing prices over Tableau and decreasing the technical boundaries to knowledge evaluation with a visible question instrument.
Information Champions
For Swapfiets workers with a better technical aptitude or ambition, or for knowledge stewards liable for updating their glossary, a extra superior degree of coaching dives deeper into visualization in Tableau, and the best way to discover belongings and translate them into queries in Atlan.
Utilizing fine-grained entry controls in Atlan, Lisa’s group ensures these Information Champions are in a position to conduct self-service evaluation at a deeper degree, with out the danger of accessing and making use of improper or restricted knowledge.
“We don’t have any PII in Tableau anymore, as a result of entry administration is less complicated in Atlan. Now, it’s clear what number of belongings have PII knowledge, who can entry these, when they’re used, and who queries which tables,” Lisa shared.
Extract Champions
The most recent addition to Swapfiets’ democratization efforts are Extract Champions, data-savvy colleagues from every enterprise area that at the moment are empowered to assist Lisa’s group when extract requests for knowledge happen.
In Swapfiets’ knowledge help channel on Slack, customers provoke a workflow to request knowledge that’s unavailable in Tableau, or too superior to self-service. With one click on of an emoticon, Extract Champions from the related knowledge area are tagged, settle for the duty, and start querying the info utilizing Atlan Insights for supply to the requester.
“Once they’re carried out, they simply click on the ‘End’ button on the ticket, and we don’t even have to take a look at the ticket. So it’s a win-win. I’m so passionate about this, it’s been nice,” Lisa shared.
Furthering the worth Extract Champions yield from Atlan, Lisa and her group realized the repetitive nature of some requests, and labored to automate them utilizing Atlan Scheduled Queries.
“Atlan helped us schedule extracts to individuals outdoors of Atlan, which helped an Extract Champion who’s in control of communication with B2B clients,” Lisa defined. “And now he’s scheduled his guide work, and Atlan automated what he would normally share through electronic mail. He doesn’t have to achieve out to us anymore.”
Sustaining the unimaginable momentum of Swapfiets’ Information Governance Program is a concentrate on change administration, fixed communication, and driving cultural change.
Governance Committees proceed to satisfy on a month-to-month foundation, guaranteeing every enterprise area retains their enrichment and possession tasks top-of-mind. Updates and newsletters are routinely despatched to stakeholders, educating them on subjects like high quality, possession, safety, or accessibility, and updating them on essential subjects like how documentation is progressing towards their targets.
However probably the most noteworthy change, past how shortly Swapfiets’ know-how has modernized and the way effectively their Information Governance Program has been stewarded, is a cultural shift. The place Lisa’s group was as soon as liable for answering questions on knowledge and connecting the dots for knowledge customers, that possession is shifting. Stewards and Information Homeowners from throughout their group now really feel empowered to subject questions on their knowledge, and liable for the standard and availability of those belongings.
We have now the info help channel, and we may reply most questions, however we gained’t anymore. We tag the info proprietor or steward and they’re going to look into it. This ends in extra motivation to search for sustainable options and take into consideration Information Governance after they’re taking part in their half contributing to Swapfiets’ mission. I feel it’s created a tradition the place knowledge is one thing that impacts all of us, and never simply the info group. That’s been the largest change we’ve completed, thus far.”
Lisa Smits, Information Analytics Supervisor
Photograph by Ernest Ojeh on Unsplash
