As data tools and technology mature, organizations are realizing the need for their people to transform alongside their technology to achieve true value.
In the first installment of this series, I wrote about how the evolution of data technology is fueling data-driven value for organizations. Specifically, the concept of DataOps – which seeks to add flexibility and scalability to traditionally rigid data pipelines – is an approach at the forefront of Data Platform Modernization. Digital transformation in all its manifestations has altered technology and how organizations utilize it, but it is also transforming the way that teams are forming, collaborating, and delivering. Organizations across all industries feel the pressure to do more with less, and if firms are not transforming their people and processes alongside their technology, true value will not be realized.
Therefore equally, if not more critical, to the success of effective Digital Transformation is the modernization of the teams which leverage data and analytics to drive value and insight. Of course, the skills of various data and analytics teams are becoming more modern (also discussed in the aforementioned article), but in this case, I am referring to the operating model of the various teams throughout the lifecycle or along the spectrum of data-to-insight conversion. Whether the team is squarely in the CIO’s IT organization or if it is embedded within a business unit, approaching the delivery of data and analytics in a more modern way is paying dividends.
What does it mean to say your data team is ‘modern’? At their core, modern data teams operate in the service of growing the partnership between Business and IT. My colleague, Amanda Inman, writes about this in her article tying outcomes to product-centric delivery. If those on the technical side of data delivery seek to delight their customer just like business and product teams, then a synergistic operating model forms that drives better outcomes across the whole value chain.
Characteristics of a Modern Data Team
To summarize the Modern Data Team: it is product oriented, collaborative, and Agile. The only way for a team to balance the competing demands and priorities of stakeholders is to be laser-focused on its products and to infuse agility and collaboration into its ways of working.
Digital transformation in all its manifestations... is also transforming the way that teams are forming, collaborating, and delivering.
Every team, whether it manages databases, develops pipelines, creates dashboards, or trains machine learning models, has customers. Having customers means output – however technical in nature – is seen as a product by someone else. The ideas of product and customer exist nearly interchangeably: if a team thinks about its customer, then it must think about what it produces and how the customer derives value from it. If that same team thinks about its product, then the only product roadmap that makes sense is one where features are implemented to increase the customer’s perception of value.
Consider the scenario where a team’s product is ‘self-service analytics.’ Such an initiative needs a vision, a roadmap, and marketing to internal stakeholders. Similar to new-product development, the team must understand the customer demand and appetite for self-service and what product features are essential versus negotiable. Focusing on the product ensures that no team reports to management that they have ‘done all this work, and nobody uses it.’
Characteristic #2: Collaborative
Orienting around products reflects that all teams working with data participate in a supply chain. This complex ecosystem has many players and detailed processes by which data ‘raw material’ is transformed into finished goods. Unlike the supply chains of the physical world, this supply chain can be more easily integrated through batch-data transfers, streaming queues, and REST APIs. Efficient integration is only the beginning of the collaboration that must happen between the players of this supply chain to be maximally effective.
Collaboration morphs the vertical orientation of various data and analytics teams and realigns it horizontally. No longer can teams blindly prepare their data output according to stale best practices, but instead must keep in mind the nuances and use cases for downstream users. The quintessential example of this is curating data for both Business Intelligence (BI) and Data Science teams. BI (i.e., report and dashboard builders) requires very clean, often aggregated data to facilitate rapid analytical queries. On the other hand, Data Scientists extract a lot of value not only from what is in the data but what is not reflected in it. Access to granular data which can be engineered into statistically significant covariates is nearly the opposite of what their BI counterparts might require. Accordingly, the data engineering team should not blindly apply business rules which exclude data critical to one downstream team in favor of another.
Characteristic #3: Agile
If there was a word that could nearly completely summarize the essence of the Modern Data Team, it would be ‘Agile.’ While it is probably the most abused and over-used word so far in the 21st century, it captures not only the paradoxical nature of modernization but also the standard by which modernization is measured. And by ‘Agile’ I am not talking about any specific framework envisioned by a group of brainiacs in a ski lodge in the past, but instead a commitment to continuously improve and reflect on what a data team produces and how it produces it.
It is perhaps easier to think of training a machine learning model in an Agile fashion over, let’s say, managing a cluster of database servers (in the age of the Cloud, should you really be doing this anyway?). But infusing Agile principles like responsiveness and customer focus can result in a healthier, outcome-focused team regardless of its product.
Guiding Principles and Best Practices for the Modern Data Team
To summarize and specify a few guiding principles (which are contextualized into your organization to become a ‘best practice’) for the Modern Data Team:
- Adapt a hybrid Agile framework that works for your team and your customers: Often a Kanban-style backlog of operational/support tasks can work better than Scrum, while a Machine Learning team might find a lot of value in organizing their work into more defined scrum-style sprints. While there are rules of thumb for how long a Sprint should last, data-centric teams are often adopting three-week Sprints to give them adequate time to design, deliver, and validate incremental functionality.
- Define the role and responsibilities of your Product Owner and yes, every data team has a Product Owner: Teams along the data supply chain need a Product Owner to influence the product strategy and to observe what the team is accomplishing. Eliciting feedback from the Product Owner is the most direct way to ensure that continuous improvement is something you take to heart. Data-related Product Owners do not typically start as dedicated resources; they are stewards, analysts, and line of business leaders who have a vested interest in the output of the team.
- Treat MVPs as “Minimum Valuable Products”: Instead of approaching your products as what could be ‘minimally viable,’ think of how thin, vertical slices of functionality can be delivered to still add value to your customer. While it may not have all the bells and whistles, this take on an MVP iteratively demonstrates to your customer that you understand their needs and you can deploy a solution to meet them.
Often people and processes are overlooked when Digital Transformation takes hold at an organization. Inflection points for success during modernization initiatives aren’t based on how well you configure a new data tool but instead are defined by how well your teams modernize their ways of delivering data and analytics to their stakeholders. While data needs to be consistent, complete, and trusted, that does not presuppose approaching delivery in a way that is product oriented, collaborative, and Agile. Approaching the delivery of data and analytics in a way that puts customer delight at the forefront will greatly improve quality and the likelihood of transformative success.