Data Silos in Research
How silos increase administrative burdens
Data management challenges are common in research. Individual research projects demand greater sophistication and more collaboration (be that across disciplines or borders), and this creates a data landscape of growing complexity. Such challenges are echoed on a larger scale across the broader university body: questions like ‘What information is stored?’ and ‘Who should have access to that information?’ and ‘How are different stakeholders going to use this data?’ are common.
This blog discusses the challenge of data silos, their impacts and their relationship to data governance. Silos are collections of information, which, stored in isolation, are more challenging to share (and therefore use) across systems or departments. The effect of a data silo is that, often, data must be moved around manually.
The key challenge of data silos — for any organisation — is balancing the need for security with the need for utility.
The data projects in which ResearchMaster engages frequently reflect uncertainties of administrative practices within universities. Research administration processes commonly rely on spreadsheets, emails, and other manual activities to support tasks that are both repetitive and high-volume. Data is manually entered into software that is only present in one school or department, for which a licence fee is inevitably charged, and then when it must be received and processed by another department, this whole process occurs again: spreadsheets, emails, manual data entry.
These manual interventions often do seem benign at a glance. Individually, they tend to have a low risk of error and take a relatively short amount of time to execute. But when you execute a task that has, for example, a 0.5% risk of manual error and which takes ten minutes of every working day in a year, you ensure at least one error and waste just over an entire working week in administrative labour over the course of each year. When there are several such tasks, the burden of them simply multiplies and multiplies.
Even for academics who are not employed in a strictly administrative role, the burden of administration has become disproportionate over the past years. Although the traditional breakdown of an academic’s workload is 40% teaching, 40% research and 20% administrative tasks, a 2019 case study indicated that they might be spending as much as 35% of their time on administrative tasks instead, a circumstance inimical to executing on their research responsibilities.
This is in part an effect of siloed systems, such as between departments. It is also, in part, the result of software solutions which are taken up piecemeal and which are simply not equal to the needs of their users.
High-volume, repetitive processes are excellent candidates for automation, whether that be basic workflow automation or more sophisticated AI automation. When correctly implemented, automation eliminates the need for personnel to intervene in data processes and reduces their gross administrative burden. But such automations are stymied by siloed data. When systems within an organisation cannot “speak” to each other, it begets double- or even triple-handling and unnecessary manual activity.
Research management draws on multiple functions within the university, from human resources to finance to candidate management, and has to carefully balance the needs of a variety of disparate stakeholders. Consequently, such administrative burden is usually felt, not just in the research office, but across the whole university’s infrastructure.
Dissolving silos starts with data governance
Universities can improve their administrative function and future-proof their data to comply quickly with changing regulations.
Regulatory pressures in the higher education sector are fast-changing and hard to keep up with. It’s a regrettable truth that as the pace of technological advancement hastens, so too does the pace at which it must be regulated — which means the compliance landscape only grows less and less stable.
Between cyber- and national security obligations, changing research quality assessment requirements, and an environment that encourages more cross-discipline and international collaboration, the shape of the challenge emerges: much data, from a variety of sources, must be collected, updated, stored and maintained. It has to be managed at the appropriate level of privacy and security without compromising access for staff undertaking genuinely necessary tasks.
Regulatory compliance is a common reason for a data transformation project. But if data isn’t maintained and stored appropriately to begin with, these projects can become enormously burdensome. A robust data governance framework means having complete visibility of, and control over, the institution’s data at any given moment.
What does excellent data governance look like?
Data is a foundational requirement for all digital capabilities in research, including research management. From the simplest to the most complex, technology systems all rely on correct, valid, and organised data flowing in and out to provide value to the institution.
Data governance supports critical functions of the university, and owing to the evolving landscape of quality assessment, regulation, and administrative load, there has never been a more appropriate time to review whether the data government frameworks in place are fit for purpose and capable of supporting the future of research.
Excellent data governance means you have a strong, organisation-wide understanding of what data is collected, and that you know how it’s supposed to be stored and maintained, how it’s used, who has access to it, and what purposes it’s there for. It means having a complete idea of data activities across your whole organisation.
Any organisation with a robust data governance framework should be able to answer questions like:
- What kind of data does the whole organisation hold? And about what and whom?
- Where is that information stored? In what formats?
- For what purpose is that data used?
- What are your current obligations with regard to the stewardship of that data, and how do you meet those obligations?
The principles of data governance are simple in theory, but in practice they become challenging due to the complexity of organisational structures and systems, and the volumes of information involved.
However, the key components of any data governance framework do have some things in common.
A checklist for data governance
The following checklist addresses those commonalities:
- Data strategy
Develop a data strategy that recognises the long-term goals, resources and requirements necessary to manage your data assets. Most universities have a data strategy. Like many large organisations, however, it is not always followed or even considered when engaging with data management.
- Policies and procedures
The organisation should have clear, formal policies with defined responsibilities and procedures, enforced with appropriate roles and user permissions when interacting with technology systems. These are structures that need to be maintained to support staff practicing good data stewardship.
Policies need to be easily distributed and followed in practice, as well as in theory, and must therefore strike a balance between the value of the policy and the burden of executing it.
- Oversight and measurable metrics
With a view to supporting your organisation’s strategic goals, organisations should maintain regularly reviewed, clearly defined metrics to make sure your systems and your data within them are serving your needs.
- Data quality and integrity
Poor quality data embeds flaws in analyses and processes. Make sure the organisation is operating with maximised data quality: clean, verified and valid data that supports both critical functions and regulatory obligations for the university.
- Issues management
The defined roles and responsibilities in your policies will determine who should be responsible for decisions when problems do arise. Model the most common data problems you might reasonably encounter and equip your staff to identify and refer or address them.
