QinetiQ Space || CCSDS

CEOS Archive Management Task Team

WGISS Task Profile Outline (Task ID: 16) - revised 27^th April 2000

Title:	Archive Management
Output:	Guidelines and recommendations for long-term archive data management, preservation and exchange among Agencies. First goals are the generation of a document defining the archive data processing level in unambiguous terms (baseband data) and the definition of an archive baseband data interchange format.
Task leader:	Gian Maria Pinna, ESA/ESRIN Via G. Galilei C.P. 64 00044 Frascati (Rome) Italy Tel: +39 (06) 94180644 Fax: +39 (06) 94180632 E-mail: GianMaria.Pinna@esrin.esa.it
Objectives & benefits:	Establish archive guidelines and foster use of guidelines to ensure data integrity of data archived within CEOS agencies, and foster data integrity of auxiliary data and appropriate unique satellite data archived outside CEOS agencies. Establish a Purge Alert system, including an on-line bulletin board. Advertise the Purge Alert system appropriately. To provide an unambiguous definition and reach a consensus among the Agencies of all the sensible requirements to be satisfied by an EO dataset archive format to ensure the long term preservation of the information. Establish a common and stable framework for the definition of the data that will be the object of the future work to be carried out by the Archive Task Team. To define a standard for the exchange of data among EO long term archives. A wide adoption of a standard interchange format will have a major beneficial impact on all EO programmes that foresee an international cooperation and an exchange of archive data.
Rationale:	CEOS Plenary charged WGD to establish "Data Management Guidelines", including archive integrity, to preserve useful, relevant satellite and auxiliary data. Presently any Agency uses its own definition for the data archived and some ambiguity exists about the processing level applied to them. Also, Agencies use their own format for the EO data archived and the exchange of these datasets among cooperating Agencies is difficult. It is clear that there's no driving force in the industry to adopt a universal and common format for the data archiving systems they develop for the data holders. Role for CEOS ATT would be to propose and define, in collaboration with industry, an effective way to translate the internal archive format in a new InterChange Format (ICF) to be eventually supported by all systems. Agencies would then exchange data by mean of the ICF.
Target end date:	Two years planned lifetime for the ATT. Tasks like Purge Alert System maintenance may go over the ATT currently planned lifetime.
*Further info*
Milestones: (major delivery or review dates)	December 1998: draft Baseband WP March 1999: Review of draft Baseband WP at SGs meeting and final modifications to be agreed April 1999 (TBC): Final Baseband WP to be presented to WGISS for approval March 1999 (TBC): proposal for InterChange Format document September 1999 (TBC): draft InterChange Format document March 2000 (TBC): Final InterChange Format document to be presented to WGISS for approval
Summary description of activity:	The definition in non ambiguous terms of the "level of processing" required by the EO data for long term archiving is felt as being critical for any future job the ATT will carry out. Presently a restricted working group is working on a draft white paper for the definition of the Baseband Data concept. Already, a unanimous consensus exists on the fact that the Baseband data will be in some form of computer compatible format, which means that the original serial telemetry stream is frame-synchronised. An almost final version of the WP should be ready by March 1999, for the next ATT meeting in Osaka. In Osaka, the ATT plenary will discuss the document and final adjustment will be done, after which the document will be presented to WGISS for approval and official distribution to member Agencies. Regarding the InterChange Format the ATT would develop and issue a CEOS controlled definition of the Archive Baseband Data InterChange Format, which would be approved by WGISS and endorsed by the member Agencies. Work will start in Osaka.
Links to CEOS agency programme plans, to WGISS primary objectives, and to other WGISS tasks (e.g. dependencies or scheduling):	Almost all Agencies are currently undertaking a huge job to transcribe their data holdings from old technology media like the HDDTs into more modern compact cassettes and into computer compatible format. The definition of an archive Interchange format would have a direct link to Agencies plans for development of EO data archiving systems. CCSDS is developing, under the auspices of ISO, an archive model standard. ATT is interested in following this work.
Participation (agencies involved, names, contacts, and any specific roles):	ESA (Task Leader) Baseband data definition WG: NASA/JPL, NASDA, NSC, ESA Contributing agencies: NASA, NOAA, ESA, NASDA, RESTEC, BNSC, DLR, Eumetsat, CNES, JRC, CSIRO, CCRS, NSC, plus representatives from Industry
Strategy for adoption of results (including specific agency programme plans):	Documents to be endorsed and approved by WGISS, acknowledged by member Agencies and Industry. Advertising within and outside CEOS, e.g. CEOS Newsletter article on Purge Alert. Creation of an ATT Web home page. Co-ordination with other relevant international organisations (e.g. CCSDS)
Outline resource requirements to achieve output (staff time or key needs):	Computer and programming resources for ATT home page. Documents editor: resources as required by document complexity, between 7 man-days and several man-months. All ATT members (Agencies and Industry representatives) to review and provide inputs to documents.