NCEAS Open Science Synthesis for the Delta Science Program - 5 Data Management Lens for Publishing and Accesing Data

Learning Objectives

Understand the importance of data management for successfully preserving data
Learn about metadata guidelines and best practices for reproducibility
Become familiar with environmental data repositories for accessing and publishing data

5.1 The Big Idea

The ultimate goal of this lesson is to provide an overview of a reproducible open science framework for your research, when either you are accessing published data (data user) to – for example use it for synthesis or you want to publish your own data (data author). To achieve this, we are going to talk about the following topics.

The Data Life Cycle
The importance of data management
Metadata best practices
Data preservation

We will discuss how these topics relate to each other and why they are the building block for you to use others’ data and for others to access, interpret, and use your data in the future.

5.2 The Data Life Cycle

The Data Life Cycle gives you an overview of meaningful steps in a research project. This step-by-step breakdown facilitates successful management and preservation of data throughout a project. Some research activities might use only part of the life cycle. For example, a meta-analysis might focus on the Discover, Integrate, and Analyze steps, while a project focused on primary data collection and analysis might bypass the Discover and Integrate steps.

Data Life Cycle graphic with each stage following the next to create a circle. — Source: DataOne

The first step to working with data is identifying where my project is starting in the Data Life Cycle. Using the data life cycle stages, create your own cycle that best fits your project needs.

A way to use the Data Life Cycle in practice is to:

Think about the end goal, outcomes, and products of your project
Think and decide steps in the Data Life Cycle you need to include in your project
Review best practices for that step in the cycle and start outlining action items in each of those steps.

DataOne’s Data Management Skillbuilding Hub offers several best practices on how to effectively work with your data throughout all stages of the data life cycle.

No matter how your data life cycle looks like, Plan should be at the top of the cycle. It is advisable to initiate your data management planning at the beginning of your research process before any data has been collected or discovered. The following section will discuss more in-depth data management and how to plan accordingly

5.3 Managing your data

Successfully managing your data throughout a research project helps ensures its preservation for future use.

5.3.1 Why manage your data?

From a Researcher Perspective

Keep yourself organized – be able to find your files (data inputs, analytic scripts, outputs at various stages of the analytic process, etc.)
Track your science processes for reproducibility – be able to match up your outputs with exact inputs and transformations that produced them
Better control versions of data – easily identify versions that can be periodically purged
Quality control your data more efficiently
To avoid data loss (e.g. making backups)
Format your data for re-use (by yourself or others)
Be prepared: Document your data for your own recollection, accountability, and re-use (by yourself or others)
Gain credibility and recognition for your science efforts through data sharing!

Advancement of Science

Data is a valuable asset – it is expensive and time consuming to collect
Maximize the effective use and value of data and information assets
Continually improve the quality including: data accuracy, integrity, integration, timeliness of data capture and presentation, relevance, and usefulness
Ensure appropriate use of data and information
Facilitate data sharing
Ensure sustainability and accessibility in long term for re-use in science

5.3.2 Tools to Manage your Data

A Data Management Plan (DMP) is a document that describes how you will use your data during a research project, as well as what you will do with your data long after the project ends. DMPs are living documents and should be updated as research plans change to ensure new data management practices are captured (Environmental Data Initiative).

A well-thought-out plan means you are more likely to:

stay organized
work efficiently
truly share data
engage your team
meet funder requirements as DMPs are becoming common in the submission process for proposals

A DMP is both a straightforward blueprint for how you manage your data, and provides guidelines for your and your team on policies, access, roles, and more. While it is important to plan, it is equally important to recognize that no plan is perfect as change is inevitable. To make your DMP as robust as possible, treat it as a “living document” that you periodically review with your team and adjust as the needs of the project change.

5.3.3 How to Plan

Plan early: research shows that over time, information is lost and this is inevitable so it’s important to think about long-term plans for your research at the beginning before you’re deep in your project. And ultimately, you’ll save more time.
Plan in collaboration: high engagement of your team and other important contributors is not only a benefit to your project, but it also makes your DMP more resilient. When you include diverse expertise and perspectives to the planning stages, you’re more likely to overcome obstacles in the future.
Utilize existing resources: don’t reinvent the wheel! There are many great DMP resources out there. Consider the article Ten Simple Rules for Creating a Good Data Management Plan (Michener 2015), which has succinct guidelines on what to include in a DMP. Or use an online tool like DMPTool, which provides official DMP templates from funders like NSF, example answers, and allows for collaboration.
Make revising part of the process: Don’t let your DMP collect dust after your initially write it. Make revising the DMP part of your research project and use it as a guide to ensure you’re keeping on track.
Include tidy and ethical lens: It is important to start thinking through these lenses during the planning process of your DMP, it will make it easier to include and maintain tidy and ethical principles throughout the entire project. We will discuss in depth about tidy data, FAIR principles and data ethics though the CARE principles later this week.

More details on what to include in a Data Management Plan in Additional Resources

5.4 Metadata Best Practices

Within the data life cycle you can be collecting data (creating new data) or integrating data that has all ready been collected. Either way, metadata plays plays a major role to successfully spin around the cycle because it enables data reuse long after the original collection.

Imagine that you’re writing your metadata for a typical researcher (who might even be you!) 30+ years from now - what will they need to understand what’s inside your data files?

The goal is to have enough information for the researcher to understand the data, interpret the data, and then reuse the data in another study.

5.4.1 Overall Guidelines

Another way to think about metadata is to answer the following questions with the documentation:

What was measured?
Who measured it?
When was it measured?
Where was it measured?
How was it measured?
How is the data structured?
Why was the data collected?
Who should get credit for this data (researcher AND funding agency)?
How can this data be reused (licensing)?

5.4.2 Bibliographic Guidelines

The details that will help your data be cited correctly are:

Global identifier like a digital object identifier (DOI)
Descriptive title that includes information about the topic, the geographic location, the dates, and if applicable, the scale of the data
Descriptive abstract that serves as a brief overview off the specific contents and purpose of the data package
Funding information like the award number and the sponsor
People and organizations like the creator of the dataset (i.e. who should be cited), the person to contact about the dataset (if different than the creator), and the contributors to the dataset

5.4.3 Discovery Guidelines

The details that will help your data be discovered correctly are:

Geospatial coverage of the data, including the field and laboratory sampling locations, place names and precise coordinates
Temporal coverage of the data, including when the measurements were made and what time period (ie the calendar time or the geologic time) the measurements apply to
Taxonomic coverage of the data, including what species were measured and what taxonomy standards and procedures were followed
Any other contextual information as needed

5.4.4 Interpretation Guidelines

The details that will help your data be interpreted correctly are:

Collection methods for both field and laboratory data the full experimental and project design as well as how the data in the dataset fits into the overall project
Processing methods for both field and laboratory samples
All sample quality control procedures
Provenance information to support your analysis and modelling methods
Information about the hardware and software used to process your data, including the make, model, and version
Computing quality control procedures like testing or code review

5.4.5 Data Structure and Contents

Everything needs a description: the data model, the data objects (like tables, images, matrices, spatial layers, etc), and the variables all need to be described so that there is no room for misinterpretation.
Variable information includes the definition of a variable, a standardized unit of measurement, definitions of any coded values (i.e. 0 = not collected), and any missing values (i.e. 999 = NA).

Not only is this information helpful to you and any other researcher in the future using your data, but it is also helpful to search engines. The semantics of your dataset are crucial to ensure your data is both discoverable by others and interoperable (that is, reusable).

For example, if you were to search for the character string “carbon dioxide flux” in a data repository, not all relevant results will be shown due to varying vocabulary conventions (i.e., “CO2 flux” instead of “carbon dioxide flux”) across disciplines — only datasets containing the exact words “carbon dioxide flux” are returned. With correct semantic annotation of the variables, your dataset that includes information about carbon dioxide flux but that calls it CO2 flux WOULD be included in that search.

5.4.6 Rights and Attribution

Correctly assigning a way for your datasets to be cited and reused is the last piece of a complete metadata document. This section sets the scientific rights and expectations for the future on your data, like:

Citation format to be used when giving credit for the data
Attribution expectations for the dataset
Reuse rights, which describe who may use the data and for what purpose
Redistribution rights, which describe who may copy and redistribute the metadata and the data
Legal terms and conditions like how the data are licensed for reuse.

5.4.7 Metadata Standards

So, how does a computer organize all this information? There are a number of metadata standards that make your metadata machine readable and therefore easier for data curators to publish your data.

Note this is not an exhaustive list.

5.4.8 Data Identifiers

Many journals require a DOI (a digital object identifier) be assigned to the published data before the paper can be accepted for publication. The reason for that is so that the data can easily be found and easily linked to.

Some data repositories assign a DOI for each dataset you publish on their repository. But, if you need to update the datasets, check the policy of the data repository. Some repositories assign a new DOI after you update the dataset. If this is the case, researchers should cite the exact version of the dataset that they used in their analysis, even if there is a newer version of the dataset available.

5.4.9 Data Citation

Researchers should get in the habit of citing the data that they use (even if it’s their own data!) in each publication that uses that data.

5.6 Summary

The Data Life Cycle help us see the big picture of our data project.
Once we identify the necessary steps it is helpful to think through each one and plan accordingly.
It is extremely helpful to develop a data management plan is to stay organized.
Document everything. Having rich metadata is a key factor to enable data reuse. Describe your data and files and use an appropriate metadata standard.
Publish your data in a stable long live repository and assign a unique identifier.

5.7 Data Users Example

Data Life Cycle

Plan
Discover: Finding and understanding data
Integrate: Accessing Data
Analyze
Describe: Metadata, Citing data (provenance)
Preserve: Publishing a derived data package

5.8 Exercise: Evaluate a Data Package on the EDI Repository

Explore data packages published on EDI assess the quality of their metadata. Imagine you are planning on using this data for a synthesis project.

Setup

Break into groups and use the following data packages:

You and your group will evaluate a data package for its: (1) metadata quality, and (2) data documentation quality for reusability.

Exercise: Evaluate a data package

Open our Data Package Assessment Rubric (Note: Evaluate only the Metadata Documentation and Quality section), make copy and:
1. Investigate the metadata in the provided data
  1. Does the metadata meet the standards we talked about? How so?
  2. If not, how would you improve the metadata based on the standards we talked about?
2. Investigate the overall data documentation in the data package
  1. Is the documentation sufficient enough for reusing the data? Why or why not?
  2. If not, how would you improve the data documentation? What’s missing?
Elect someone to share back to the group the following:
1. How easy or challenging was it to find the metadata and other data documentation you were evaluating? Why or why not?
2. What documentation stood out to you? What did you like or not like about it?
3. Do you feel like you understand the research project enough to use the data yourself (aka reproducibility?

If you and your group finish early, check out more datasets in the bonus question.

5.9 Bonus: Investigate metadata and data documentation in other Data Repositories

Not all environmental data repositories document and publish datasets and data packages in the same way. Nor do they have the same submission requirements. It’s helpful to become familiar with metadata and data documentation jargon so it’s easier to identify the information you’re looking for. It’s also helpful for when you’re nearing the end of your project and are getting ready to publish your datasets.

Evaluate the following data packages at these data repositories:

How different are these data repositories from the EDI Data Portal? Would you consider publishing you data at one or multiple of these repositories?

5.10 Additional Resouces

5.10.1 What to include in a DMP

DMP Section	Guiding Questions
Funder Requirements	Does the funder have a template or specific DMP guidelines? Do you thoroughly understand all the requirements? Or do you need to reach out for clarification? Is there a page-limit to what you can submit in your proposal? Would it beneficial to have an appendix or a longer version of your DMP for internal use elsewhere (and not for submission)?
Study Design	What analytical methods do you plan to use? What experiments, if any, are needed to answer your research question? What are the end products you plan to produce? What ethical considerations do you have about your project?
Data Collection	What type of data do you plan to collect (text, audio files, images, models, spreadsheets)? Where do you plan to source your data? Is it observational, already existing, or does it need to be collected? Do you need to obtain a license to access the data? Do you need an IRB review? How much data do you plan to collect or use? What format is the data in? Is it open source or is it proprietary?
Data Organization	How will you manage your data? Will you be using open source or proprietary software programs? Do you need a database to manage your data? Are there existing databases you can utilize or do you need to build one? What software tools do you plan to use to manage and organize your data?
Quality Assurance and Quality Control	How will you ensure that your data is of quality? How will you maintain data integrity throughout your analysis? What tests will you run on your raw data and processed data? Will you be utilizing outside partners to implement testing or QA/QC measures?
Data Policies	What licenses do you plan to use for your data? Are there open source licenses that meet your funders requirements? What are the policies for sharing, retaining, and licensing the data? Whose responsibility is that? Are there any legal or ethical restrictions on your data? Do you have sensitive data that cannot be shared? Is a metadata documentation appropriate?
Data documentation & Metadata	What information is required for you and others to accurately interpret, reuse, and access your data? Will you be using a metadata standard? What information is needed for you to write comprehensive metadata? Where and how will you maintain this documentation? Is it possible for you to have the documentation open source?
Data Sharing	How will the data be shared after the project ends? Is this an accessible location? When will the data and project be available? Immediately after the project ends or a time period after? Will you be publishing the project and the data to a journal? What data products do you plan to share?
Roles and Responsibilities	Who is in charge of collecting the data? Managing it? Storing it? Archiving it? Running quality control? Overall project management? There are lots of roles to consider here. What kind of expertise is needed for these roles? What happens if a role needs to change? How do you plan to handle this kind of change?
Long-term Storage & Data Preservation	Where do you plan to archive your data? How long will the data be accessible? How will the data be accessed for future use? How will you be storing the data during your project? Is this different than where you will store it after the project ends? Does your institution or funder have long-term storage options for you to use?
Budget	Do you need to purchase any proprietary software? Do you need to purchase any hardware? Do you need to pay for any services? Will you need to hire employees? Consultants? Do you anticipate that you will need to pay for any professional development or training either for yourself or your team?

Michener, William K. 2015. “Ten Simple Rules for Creating a Good Data Management Plan.” PLOS Computational Biology 11 (10): 1–9. https://doi.org/10.1371/journal.pcbi.1004525.