ACCLIP Data Protocols

Storage Location

All data will be uploaded via the web to an archive maintained by NCAR (for GV data) or LaRC (for WB-57 data). Links to each archive will be provided at each separate site.

File Format

Data file naming and content will follow ICARTT or HDF5 standards.

https://www-air.larc.nasa.gov/missions/etc/IcarttDataFormat.htm

https://portal.hdfgroup.org/display/HDF5/HDF5

ICARTT is preferred for time series data. HDF5 is preferred for multi-dimensional data (e.g., remote sensing curtains). ICARTT files from each platform should be screened with FScan prior to archiving. A copy of FScan can also be downloaded at

https://www-air.larc.nasa.gov/missions/etc/FScanBrowser.zip.

Variable Names

Each variable has two names. The first is the PI-generated “short name.” There is no fixed convention for this, but a suggested format is variableName_instrumentName_units (e.g. CH2O_ISAF_pptv). It is highly recommended that the standard “CoreName” (see below) is used as the variable name.

The second name is a “standard name,” which is used for long-term archival and allows machine readability for searching across multiple datasets. This usually consists of a “coreName” along with other identifiers for specific measurement attributes. Naming conventions are described here:

https://www-air.larc.nasa.gov/missions/etc/AtmosphericCompositionVariableStandardNames_V11.pdf

Uncertainties

All uncertainties should be described in metadata. It also helps to describe how errors should be propagated (e.g., precision improves with averaging, but accuracy does not). Unfortunately, this metadata is typically not included in merge files. Thus, if reasonable, uncertainties should also be reported as a separate variable in the data file.

Time Alignment

All fast (1 Hz) data should be time-aligned to the DLH (WB57) or VCSEL (GV) water vapor measurement.

Merges

Merging will be handled by NCAR personnel. In addition, a custom merge tool is available on the LARC archive website.Standard merges include 1-second, 10-second, TOGA-, and WAS-interval. Field data merges will be built within 1 month of mission’s end. Partial field merges during the mission will be made intermittently as needed. Final data merges will be built within 1 month of the final data deadline.

Storage and Distribution Schedule

Field data is due 24 hours after the end of a flight. Data quality should be sufficient for in-field data analysis.

Final data is due 6 months after end-of-mission. This data should be publication-quality.

Public data will be released 6 months after end-of-mission.

Data Use, Presentation, and Co-Authorship

NASA’s data policy, described here (https://science.nasa.gov/earth-science/earth-science-data/data-information-policy), states:

“NASA commits to the full and open sharing of Earth science data obtained from NASA Earth observing satellites, sub-orbital platforms and field campaigns with all users as soon as such data become available. There will be no period of exclusive access to NASA Earth science data. Following a post-launch checkout period, all data will be made available to the user community.”

The NSF data policy similarly promotes open and full sharing of data. From https://www.nsf.gov/pubs/policydocs/pappg20_1/pappg_11.jsp#XID4:

a. Investigators are expected to promptly prepare and submit for publication, with authorship that accurately reflects the contributions of those involved, all significant findings from work conducted under NSF grants. Grantees are expected to permit and encourage such publication by those actually performing that work, unless a grantee intends to publish or disseminate such findings itself.

b. Investigators are expected to share with other researchers, at no more than incremental cost and within a reasonable time, the primary data, samples, physical collections and other supporting materials created or gathered in the course of work under NSF grants. Grantees are expected to encourage and facilitate such sharing….

ACCLIP will follow these policies for appropriate archiving, use, and dissemination of ACCLIP science products. Measurements, data, and model products shall be available to all members of the ACCLIP science team. Publication, presentation, and dissemination of the ACCLIP measurements and model products is a high priority for team members and collaborators.

Some general guidelines for this policy are:

1. Data files: metadata cannot contain stipulations that restrict use, but they can and should contain statements such as: “users are strongly encouraged to contact with the PI or DM for proper data use.”
2. Analysis: If someone else’s data is important to your analysis, let them know you are using it in the early stages of analysis.
3. Presentations: Presentations at meetings and conferences will use only final, publicly available datasets. Presentations at Science Team meetings, or in meetings with collaborators, may use the field data archive.
4. Publications: Publications will only use final, publicly available datasets. They will cite data DOIs where appropriate. Co-authorship will be offered to any groups whose data or model products have been used in the development of the publication.