Developing and Managing GCOS, GOOS and GTOS Data Sets

The integrated observing systems of the Global Observing Systems (GTOS, GOOS, GCOS) consist of a variety of end-to-end data collection and management programs that serve national and international users, as well as the Global Observing Systems. These data are routinely used for both research and operational applications.

The Global Observing Systems (GTOS, GOOS, GCOS) users tend to be concerned more and more with operational applications for the detection and prediction of climate and other change. There are some differences in how research and operational datasets are managed. Some of the differences are described here. Primarily operational applications need datasets that meet a defined standard for sampling in space and time, quality, and analysis methods.

Opportunistic Data Sets and Global Observing Systems Needs

A large number of oceanographic, meteorological, and terrestrial environmental data sets and data products are available from scientific and operational organizations around the world. Many of them have overlapping content, are versions of the same dataset, are combinations of several of the same data sets, have been quality controlled and analyzed by different and unspecified techniques, and have varying and possibly incomplete metadata attached to them. A scientist conducting a general search on the Internet for a certain type of data will face the problem of a large number of hits many of which will not have sufficient supporting metadata for him to make an informed decision on whether he can apply to the data to his problem. In these circumstances the scientist would have to spend significant time in asking questions of the holders of the datasets and its collectors, and would probably have to acquire the dataset and do processing and further checking before he could be satisfied it was appropriate for use. The next scientist needing that type of data would have to repeat this work. Even after all of this many of the datasets will have limitations that will make them less than satisfactory for studying global change.

An operational scenario cannot depend on such opportunistic datasets. Although the user would gradually develop a list of datasets with relevant data, they would not be tailored to needs and would require filtering, further processing, more QC, etc.

General Internet searches using available search engines do have a role to play. They will be used by Global Observing Systems (GCOS, GOOS and GTOS) developers as a step in the process of identifying, studying, and specifying existing or historical datasets relevant to the Global Observing Systems for registry in GOSIC. However these searches and analyses will not have to be done by every scientist looking for Global Observing Systems data. This, of course, demands that the Global Observing Systems scientific and technical advisory groups and panels be conversant with the needs of their users by having good contacts and close working relationships with them. In fact a cross section of the user communities should serve on the advisory groups and panels.

Development of Global Observing Systems Datasets for Global Observing Systems Applications

The scientific and technical advisory groups or panels of the Global Observing Systems design specific observational systems that produce the scientific and operational datasets and data products needed for research and operations. This includes agreeing on the methods and precision of collection (including recommended best practices), the supporting metadata to be gathered and circulated with the data, the analyses that are needed, and the datasets and products that should be produced.

The scientific and technical groups also examine historical data sets and identify those that meet Global Observing Systems requirements in terms of relevance of variables, completeness of metadata, standards of collection, spatial and temporal coverage, etc.  These datasets are then referenced in the data set registry in GOSIC.

The end-to-end data management systems that compose the integrated observing systems for the three observing systems (GTOS, GOOS, GCOS) each have a number of centers that carry out processing tasks, archive new data and historical datasets, and provide services. The data management system supporting the Global Observing Systems is thus highly distributed. Another characteristic is that it deals with quite specific data that have been collected according to an experimental design created for economy and effectiveness, and which have been analyzed and presented according to agreed standards.

The Role of the Global Observing Systems Information Center

The Global Observing Systems observational data systems are being developed to collect and manage the subset of data that meets the needs of GCOS, GOOS, and GTOS. This will include some opportunistic data from other programs, but all data that are collected in the atmosphere, ocean, and on land are not Global Observing Systems data. At any time a list can be made of the datasets, products, and services that are Global Observing Systems. The strategy for Internet searches being developed by the Information Center is designed to identify those datasets and services that qualify to be on such a list. The "list" of datasets and products will be kept as a database which will form the basis of the GOSIC Data Set Registry.