Jon Loyens
By now, all companies are fundamentally data driven. This is true regardless of whether they operate in the tech space. Therefore, it makes sense to examine the role data management plays in bolstering — and, for that matter, hampering — productivity and collaboration within organizations.
While the term “data management” inevitably conjures up mental images of vast server farms, the basic tenets predate the computer age. From censuses and elections to the dawn of banking, individuals and organizations have long grappled with the acquisition and analysis of data.
One oft-quoted example is Florence Nightingale, a British nurse who, during the Crimean war, recorded and visualized patient records to highlight the dismal conditions in frontline hospitals. Over a century later, Nightingale is regarded not just as a humanitarian, but also as one of the world’s first data scientists.
As technology began to play a greater role, and the size of data sets began to swell, data management ultimately became codified in a number of formal roles, with names like “database analyst” and “chief data officer.” New challenges followed that formalization, particularly from the regulatory side of things, as legislators introduced tough new data protection rules — most notably the EU’s GDPR legislation.
This inevitably led many organizations to perceive data management as being akin to data governance, where responsibilities are centered around establishing controls and audit procedures, and things are viewed from a defensive lens.
That defensiveness is admittedly justified, particularly given the potential financial and reputational damages caused by data mismanagement and leakage. Nonetheless, there’s an element of myopia here, and being excessively cautious can prevent organizations from realizing the benefits of data-driven collaboration, particularly when it comes to software and product development.
Taking the offense
Data defensiveness manifests itself in bureaucracy. You start creating roles like “data steward” and “data custodian” to handle internal requests. A “governance council” sits above them, whose members issue diktats and establish operating procedures — while not actually working in the trenches. Before long, blockages emerge.
Blockages are never good for business. The first sign of trouble comes in the form of “data breadlines.” Employees seeking crucial data find themselves having to make their case to whoever is responsible. Time gets wasted.
By itself, this is catastrophic. But the cultural impact is much worse. People are natural problem-solvers. That’s doubly true for software engineers. So, they start figuring out how to circumvent established procedures, hoarding data in their own “silos.” Collaboration falters. Inconsistencies creep in as teams inevitably find themselves working from different versions of the same data set.
This top-down approach resembles the waterfall methodologies of yesteryear, which have since ceased to serve their purpose in an industry that requires — nay, demands — speed.
Individuals over processes
The first value of the Manifesto for Agile Software Development (often referred to simply as the “Agile Manifesto”) states: “Individuals and interactions over processes and tools.” Those seven words can be interpreted to mean that developers should be given the autonomy needed to complete the task at hand, establishing their own processes as necessary. That notion is anathema to the aforementioned hierarchical approach to data governance.
Flexibility is key to balancing the needs of the team, as well as the data protection and governance needs of business. Ultimately, this starts with collaboration between those tasked with protecting data and those who use it in their day-to-day work. Culture needs to change from a hierarchical model to a flatter, more lateral one.
By understanding the needs of all stakeholders, organizations can start to figure out how to remove blockages. Ultimately, this requires businesses to shift from a “defensive” to an “offensive” mindset, with gatekeepers replaced by simple, straightforward rules that encourage iteration, understanding and collaboration, and can scale as needed.
Reaping the rewards
It’s no coincidence that some of the world’s most successful companies have embraced an agile data-driven culture.
Take Netflix. The firm’s core philosophy is “people over process,” which is itself a reference to the Agile Manifesto. One of the values it seeks in potential employees is a willingness to use data to inform intuition. It deliberately allows employees freedom to make decisions and take responsibility for them.
This overarching emphasis on data is no surprise considering the business Netflix is in. It routinely spends eight-figure sums on an original TV series or movie and must be able to decisively determine whether a series should be greenlit, renewed or canceled. Being able to communicate insights to decision-makers based on real-world conditions is therefore essential.
Then there’s Airbnb, which was forced to decentralize its data management and science approaches as a result of a rapid international expansion far beyond its original San Francisco heartland.
The house-sharing startup opted to embed data scientists within individual teams, removing blockages and improving access to insights. The agile approach emphasizes iteration, and data has informed each incremental step in Airbnb’s growth.
Both companies adopted this data-driven ethos either from the beginning or at the start of their swift northward ascent. But you don’t need to be a venture-backed behemoth to apply the principles of agile data management.
At its core, the main prerequisite is a willingness to consider workflows, moving from a rigid philosophy based around immutable rules to one that considers the needs of the individual and is prepared to change with circumstances. It’s not a tech problem — it’s a thinking problem.
Comment