Rethinking Data Selection and Appraisal in a Data-Driven World



In an era defined by data abundance, the challenge is no longer simply how to store information, but how to decide what deserves to be preserved. The selection and appraisal of data have become central to effective data management, shaping not only what is retained but also what knowledge remains accessible for future use. These processes are far from straightforward they are deeply influenced by institutional priorities, resource limitations, and evolving user needs .

Traditionally, appraisal has been framed as a technical or archival function, primarily concerned with identifying records of long term value. Eastwood (2004) describes appraisal as a professional judgment grounded in evidential and accountability requirements. While this perspective is valuable, it tends to position appraisal as a static decision making process. In contrast, Hodge and Frangakis (2004) argue that appraisal must align with broader data management policies and anticipated patterns of reuse. Building on these perspectives, I argue that appraisal should be understood as a dynamic and iterative process that evolves alongside changing data environments and user expectations.

One of the central tensions in appraisal lies in balancing preservation with practicality. Data are generated in large volumes, often in diverse formats, making it neither feasible nor desirable to retain everything indefinitely . Borgman (2015) supports this view, emphasizing that data curation is constrained by financial, technical, and human resources. However, reducing appraisal decisions to cost efficiency risks overlooking the potential long term value of data. What may appear insignificant today could hold substantial importance in future contexts. Therefore, appraisal frameworks must move beyond immediate utility and consider the broader, and sometimes unpredictable, value of data over time.

Equally important is the role of context in ensuring that preserved data remain meaningful and reusable. Evidence suggests that data without adequate metadata and documentation quickly lose their interpretability . This is reinforced by Tenopir et al. (2020), who demonstrate that data reuse is strongly influenced by the quality of documentation and accessibility. Despite this, data management practices often remain underprioritized. I contend that this reflects a structural weakness in current approaches, where appraisal is treated as an afterthought rather than as an integral component of the data lifecycle.

Another challenge lies in the diversity of data practices across different domains. Variations in data types, standards, and institutional policies make it difficult to establish universal appraisal criteria . While frameworks such as OAIS provide useful guidance, their application must remain flexible to accommodate contextual differences. In my view, effective appraisal should be guided by shared principles such as value, usability, and sustainability while allowing for adaptation to specific environments.

Ultimately, the selection and appraisal of data are not merely technical exercises but strategic decisions that shape what knowledge is preserved and made accessible. By engaging critically with existing approaches, it becomes evident that appraisal must be reimagined as a collaborative and forward looking practice. It is not just about deciding what to keep, but about determining what will matter in the future.

https://youtu.be/BEf6gDNPPs0?si=5qMPL7KRDmb42bUA

References

Borgman, C. L. (2015). Big data, little data, no data: Scholarship in the networked world. MIT Press.
Eastwood, T. (2004). Appraising digital records for long-term preservation. Data Science Journal, 3, 214–220.
Hodge, G., & Frangakis, E. (2004). Digital preservation and permanent access to scientific information. CENDI.
Tenopir, C., et al. (2020). Data sharing, management, use, and reuse: Practices and perceptions of scientists worldwide. PLOS ONE, 15(3).

Comments

Post a Comment

Popular posts from this blog

Storing Data

Image
Every day, institutions generate vast amounts of data, yet the challenge of storing it effectively remains one of the most underestimated issues in the digital age . Far from being a neutral or purely technical activity, data storage has become a strategic function central to records and archives management . Contemporary scholarship increasingly demonstrates that storage systems must support not only preservation but also accessibility, sustainability, and long-term usability (Katuu & Ngoepe, 2021). What this means in practice is that storing data is no longer about keeping information safe it is about ensuring its continued value. One of the most notable shifts in this space is the growing adoption of free and open source software (FOSS) such as DSpace and Archivematica . These platforms are widely recognised for their flexibility and cost effectiveness in managing digital repositories (Katuu, 2020). However, focusing solely on affordability risks oversimplifying their strategic...
Read more

Data collection and Repositories

Image
Data is the lifeblood of modern knowledge systems, yet its value is only realised when it is systematically collected, organised, and stored in environments that support long term access and reuse. Data collection and repositories sit at the very heart of data curation, forming the infrastructure upon which research, policy, and institutional memory depend. Data collection refers to the deliberate process of gathering raw information from various sources  surveys, sensors, experiments, administrative records, or observations in a structured manner (Corti et al., 2019). However, collecting data without a curation strategy is insufficient. Borgman (2015) argues that data does not speak for itself; it requires context, documentation, and governance to become meaningful and reusable across different communities. This insight shifts the responsibility of data professionals beyond mere collection toward stewardship an active, ongoing engagement with the data lifecycle . https://youtube.c...
Read more

Data curation preservation issues (threats to digital materials)

Image
What would happen if the digital records documenting climate change, public health trends, government decisions, or scientific discoveries suddenly became inaccessible? While many of us assume that digital information lasts forever , the reality is quite different. Digital materials are surprisingly fragile. Without deliberate efforts to curate and preserve them, valuable information can become unreadable, corrupted, or lost altogether. In an age where knowledge is increasingly created and stored digitally, the preservation of digital materials has become one of the most pressing challenges facing information professionals. Data curation refers to the active management and preservation of digital assets to ensure that they remain accessible, understandable, and reusable over time (Jeffery, 2020). However, preserving digital materials involves much more than simply storing files on servers or cloud platforms. It requires protecting the context, metadata, software, and technological en...
Read more