Newsletter 16 - August 2003


                    _____   __________________  ____  ____
                   /  _/ | / /_  __/ ____/ __ \/ __ \/ __ \
                   / //  |/ / / / / __/ / /_/ / / / / /_/ /
                 _/ // /|  / / / / /___/ _, _/ /_/ / ____/
                /___/_/ |_/ /_/ /_____/_/ |_|\____/_/

                    _   __                  __     __  __
                   / | / /__ _      _______/ /__  / /_/ /____  _____
                  /  |/ / _ \ | /| / / ___/ / _ \/ __/ __/ _ \/ ___/
                 / /|  /  __/ |/ |/ (__  ) /  __/ /_/ /_/  __/ /
                /_/ |_/\___/|__/|__/____/_/\___/\__/\__/\___/_/
*************************************************************************
Number 16     ***         INTEROP NEWSLETTER      ***     20 August 2003
*************************************************************************

In This Issue:
--------------
I.    Preparations for Next CEOS meeting in Chiang Mei, Thailand
        See information at:
http://www.gistda.or.th/Gistda/HtmlGistda/Html/HtmlTraining/WGISS/index.htm

II.   How the CEOS IDN Interoperability Listserver (Interop) Works and
       the Status of Current Proposals.
        - L. Olsen and G. Major

III.  Experimenting with the Semantic Web Using the CEOS IDN.
        - by Femke Reitsma, University of Maryland Student in Graduate Summer
          Student Program with the GCMD.

IV.   Update from the IDN's Science User Working Group
       - by Dr. John Porter, UWG Chairperson, University of Virginia

V.    GOSIC Contributes More Than 200 New Data Set Descriptions and Suggests
       Keyword Changes.
        - by Christina J. Lief, University of Delaware, GOSIC

VI.   An Update from the Joint Committee on Antarctic Data Management (JCADM)
       - by Dr. Dean Peterson, Chief Officer of JCADM

VII.  Using PostgreSQL for the IDN
        - Viktor Pusztai, UNEP GRID-Budapest

VIII. Minutes from the CEOS meeting in Toulouse, France

IX.   Tentative Agenda for Upcoming Meeting
       The IDN session is scheduled for the morning of Wednesday, 
September 17th.
        - by L. Olsen and D. Kendig

X.    Issues - Discussion (Refer to Interop)

*************************************************************************

I.   Preparations for Next CEOS meeting in Chiang Mei, Thailand
       - by Lola Olsen and Dave Kendig

Preparations are being made by the International Directory Network (IDN)
participants for the next CEOS meetings in Chiang Mei.  The IDN
session is scheduled for the morning of Wednesday, September 17th.  All
participating node representatives should prepare a status report.
See

http://www.gistda.or.th/Gistda/HtmlGistda/Html/HtmlTraining/WGISS/index.htm

**************************************************************************
II.  How The CEOS IDN Interoperability Listserver (Interop) Works and the
      Status of Submitted Proposals
      - by Lola Olsen and Gene Major

The Committee on Earth Observation Satellites' (CEOS) International
Directory Network (IDN) hosts an Interoperability Forum.  A mailing
list is available to all interested parties for the purpose of
discussing modifications and additions to the CEOS IDN's Directory
Interchange Format (DIF) and the Service Entry Resource Format (SERF)
syntax. The procedures are as follows:

1.  A proposal for modifications or additions to the DIF or SERF
syntax is circulated through the Interop list.   All members of the
list have an opportunity to discuss the proposal for a specified
period of time - usually 3 to 4 weeks.

2.  After the initial discussion period has ended, the proposal can
be resubmitted with additional modifications that have been
suggested.  This process may be repeated until there appears to be
general agreement among the group of participants.

3.  Finally, a formal vote is solicited from the Interop Voting
Committee to assure that a majority of IDN representatives approve
the change.  The Voting Committee is composed of nine members,
representing groups participating in CEOS.

The current Voting Committee positions are held by:
Andrea Buffam    (CCRS/GeoConnections)  Andrea.Buffam at ccrs.nrcan.gc.ca 
Cheryl Solomon   (USGS/BRD/GCMD)        Cheryl_Solomon at gcmd.nasa.gov 
Jolyon Martin    (ESA)                  jolyon.martin at esa.int 
Lee Belbin       (JCADM)                Lee.Belbin at aad.gov.au 
Osamu Ochai      (NASDA)                ochiai.osamu at nasda.go.jp 
Lyne Yohe        (NSIDC)                yohe at nsidc.org 
Victor Pusztai   (UNEP GRID-Budapest)   pusztai at mail5.ktm.hu 
Lorant Czaran    (UNEP)                 czaran at grida.no 
Sherry Harrison  (GHRC)                 sherry.harrison at msfc.nasa.gov 

4.  A majority vote constitutes passage (note Item 5).  The results
of the voting are then conveyed to the entire Interop list.

5.  The prioritization, scheduling, and completion of software
changes, as suggested by the acceptance of any proposal, are dependent
on the workload of participating members.

Status of Submitted Proposals  (See http://idn.ceos.org/User/forum):

Status of "model" proposal:  On November 18, 2002, Anne Sophie
Archambeau proposed the addition of a Numerical_Experiment group to
the DIF.  Comments were received (see Interop).  The addition of the
Numerical_Experiment group does not affect the keyword search;
therefore, this information could be included in the summary and be
available for free-text searches at this time.  Development schedules
do not permit the prioritization of this addition to the database at this
time.  Therefore, comments on the unresolved issues related to this proposal
will be extended until such time that development schedules might
permit the initiation of change.  The proposal will be presented to a
modeling group in London at the end of September for comment.

Status of ISO proposal:  All comments received on this proposal were
internal and positive.  Little controversy surrounds these changes -
as proposal acceptance and implementation would simply bring the
DIF into compliance with the international standard.

New proposal for the addition of "valids" for the "Data_Resolution"
field to include valid values for searching for both spatial and
temporal resolution:  This proposal will be sent through the Interop
List within the next week.  The proposal affects the search 
capability and requires database changes.

**************************************************************************

III.  Experimenting with the Semantic Web Using the CEOS IDN.
        - by Femke Reitsma, UMD Graduate Student in Graduate Summer Student
          Program with the GCMD. (femke at geog.umd.edu )

The proliferation of spatial data providers often makes it difficult
to find the information needed, particularly with increasingly
interdisciplinary approaches to research.  The Semantic Web, in
essence, provides a default standard for interoperability.  The
expression of content that is semantically marked up in a web
ontology language (i.e. RDF, RDFS, or OWL) can be queried by
autonomous agents in the same manner regardless of the database
structure.

The first experimental steps are being implemented towards distributing
the CEOS IDN data via the Semantic Web.  Two key components are required:
the creation of an ontology describing the objects, properties, and 
relationships that can be found in the DIF; and marking up those DIF
files in a semantic language.  Thus far, the keywords have been directly
converted into ontologies; the DIF files have been semantically 
marked up, referring back to the ontologies; and a software 
environment, Sesame (see http://www.aidministrator.nl/index.html), 
has been successfully used
to test queries over these DIF files using the ontology.

The next steps for the GCMD involve: enriching the ontologies with
more content, thereby supporting more comprehensive querying;
creating a method to automatically update the ontology with any
change in the keywords; and uploading this work to an experimental
web site where users can access the DIF files in RDF and query them.

#Note:  We had the pleasure of communicating with Dirk-Willem van 
Gulik about this work.  Dirk is a former CEOS participant who worked 
with Dr. Clive Best
at JRC in Ispra.  He is now working on Semantic Web technology.

************************************************************************

IV.  Science User Working Group Report
        - Dr. John Porter, University of Virginia

The Global Change Master Directory Science Users' Working Group (GCMD UWG)
for the American Coordinating Node of the International Directory Network
(IDN) is planning to meet in January of 2004.  The 2002 report will be
available at https://gcmd.gsfc.nasa.gov/Aboutus/UWG/uwg_2002.html


************************************************************************

V.  GOSIC Contributes More Than 200 New Data Set Descriptions and Suggests
     Keyword Changes
     - by Christina J. Lief, University of Delaware, GOSIC

The Global Observing Systems Information Center (GOSIC) is a data and
information center in support of the three Global Observing Systems
(G3OS), consisting of the Global Climate Observing System (GCOS), the
Global Ocean Observing System (GOOS), and the Global Terrestrial
Observing System (GTOS).  The G3OS are international programs that
provide a systematic and comprehensive global set of observations
that will allow participating nations to:
- Detect climate change at the earliest possible time
- Document natural climate change variability and extreme climate events
- Model, understand, and predict climate variability and change
- Assess the potential impact on ecosystems and socioeconomics
- Develop strategies to diminish potentially harmful effects
- Support sustainable development

The global observing systems use a variety of observation methods
ranging from space platforms to in-situ measurements. They encompass
all components of the climate system including the atmosphere,
biosphere, cryosphere, hydrosphere, and land surface, as well as
socioeconomic relationships.  They use existing operational programs
such as the World Weather Watch and the Global Atmospheric Watch, and
ongoing research programs, such as the Global Climate Precipitation
Project, the Global Energy and Water Cycle Experiment, the Climate
Variability and Predictability Program, and other elements of the
World Climate Research Program.  GOSIC is being developed at the
University of Delaware and is in the fifth year of development.  Part
of this development is in cooperation with the Committee on Earth
Observation Satellites International Directory Network (CEOS IDN) of
NASA's Global Change Master Directory (GCMD). This arrangement
provides benefits for both the G3OS and NASA in that it:
a. Provides standardization of the metadata fields used to describe
the data sets
b. Provides a single search strategy for both metadata directories
c. Makes the records gathered by the two organizations available to
users of both
d. Suggests keywords for more effective queries of GTOS databases

The evolving system is available on line at http://www.gosic.org. The link
to the GOSIC IDN can be found at http://www.gosic.org/search.asp.  GOSIC
provides effective data and information management in support of
global change research. With the great variety of data types taken
from global, regional, and local sources, the necessity of having a
strong and active central source of data information is essential to
meeting G3OS objectives. GOSIC is intended to serve as that
information source.

The role of GOSIC is to:
- Function as a central source of information for all G3OS data and
information by providing particulars on observing requirements,
operational data systems, and access procedures for finding and
obtaining data and products of the G3OS.
- Provide users with the ability to search for and identify the
availability, the data processing status, the location(s), and the
accessibility of relevant data.
- Provide users with metadata sufficient for them to determine if
data meet their requirements in terms of content, coverage, and
quality.
- Provide access to an integrating overview of the management and
development of the G3OS programs including such things as observing
requirements and standards and terms of reference of the panels and
expert teams.

As of July 2003, GOSIC has identified more than 50 program elements, 256
databases, 165 data centers, and consists of four main features:
1. The dataset registry consists of a directory-level catalogue of
historical and operational data sets that have been identified as a
component of one of the G3OS.
2. Data-flow diagrams are used to illustrate the operational
observing system components for the G3OS that have been established
to provide data into the future.
3. Models for the development and distribution of information useful
for data-flow monitoring are being developed as part of the data-flow
diagrams and as part of the GOOS National Commitments section of the
site as a first step in beginning to automate much of the work of the
data coordinators in monitoring and troubleshooting the data-flows to
ensure the systems are working.
4. Documentation of the observing systems is also being implemented
in GOSIC for on-line access including design of the networks,
scientific requirements, data management plans and strategies, users
and their requirements, etc.

Faculty and staff working on GOSIC at the University of Delaware are
Ferris Webster, Ron Wilson, Christina Lief and Doug White. For
further information please consult http://www.gosic.org
**************************************************************************

VI. An Update from the Joint Committee on Atmospheric Data Management
      - by Dr. Dean Peterson, Chief Officer of JCADM, Science Strategy Manager,
        Antarctica, New Zealand.

The Joint Committee on Antarctic Data Management (JCADM) recently
held its seventh annual meeting on data management at the Federal
Office for Scientific, Technical and Cultural Affairs of Belgium in
Brussels.  Over the past seven years there have been many changes in
direction and membership for JCADM.  Currently, JCADM has a total
membership of 26 countries.  Eighteen countries were represented at
the recent meeting, a huge success for JCADM and the Scientific
Committee on Antarctic Research (SCAR) and the Council of Mangers of
National Antarctic Programmes (COMNAP), the two sponsoring committees
for JCADM.

JCADM has a mandate to:
1. Recruit National Antarctic Data Centres (NADCs) within SCAR countries.

Over the past year the JCADM executives have spent considerable time
and effort in recruiting.  JCADM membership has grown from 16
representatives to a total of 26 countries.  This proactive
recruitment created the largest JCADM meeting to date, with 18
countries represented, including 8 new member countries.

2. To empower those NADCs to collect data set descriptions of
Antarctic scientific data for the Antarctic Master Directory (AMD).

The total number of Directory Interchange Format files (DIFs) in the
AMD has increased from 2032 (May 2002) to 2244 (April 2003), a 25%
growth in the AMD.  The growth can be attributed to the excellent
support received from the Global Change Master Directory (GCMD) staff
in creating and deploying tools to assist NADCs in creating DIF
metadata records to the AMD.  GCMD representatives have also
supported the creation of NADC portals to the AMD. These portals,
hosted by the GCMD, provide a national view of their metadata in the
AMD and a national DIF creation facility.

The 7th meeting of JCADM was structured to: educate emerging
countries on populating and using the AMD and general data management
protocol; to plan future strategies to better align to SCAR and
COMNAP needs; and to discuss issues arising over the past year.
Because of the large number of new countries added to JCADM over the
past year the meeting began with a series of presentations on
information about data management and the positioning of JCADM within
the Antarctic Treaty System.

Reports were given by the national delegates and the Chief Officer on
the 2002/03 achievements and from that, a list of tasks specific for
each NADC was created and agreed to.  There was also an expressed
need from all countries that a list of databases associated with the
Antarctic and Southern Ocean would be very useful.  Canada has
volunteered to begin a list, which will be added to and commented on
by all countries involved with JCADM.

David Kendig from the GCMD presented a report on the status and
recent developments of the AMD and the associated toolkit.  The
committee identified the need for a modified Antarctic-specific MD-9
(MD-9 lite) version.  The GCMD also ran a half-day workshop on
metadata issues for the emerging data management countries at the
meeting.  This was a great success and brought a number of
representatives quickly up-to-speed with many of the latest tools
available for data management.  The starter kit available on the
JCADM website (www.jcadm.scar.org) will be updated over the next 12
months, and two national data management workshops in the coming year
will be held in the Ukraine and Estonia.

Following the meeting in Brussels a presentation was given to both
SCAR and COMNAP representatives on JCADM's progress and direction.
The representatives were very pleased with JCADM's recruitment
success and the increase in the total number of DIFs in the AMD.  The
representatives were also committed to continuing the AMD especially
because of its association with the GCMD.


***************************************************************************
VII.  Using PostgreSQL for the IDN
       - by Viktor Pusztai, UNEP GRID-Budapest

Open source databases, like PostgreSQL software, are valuable to the
CEOS IDN in that they allow the nodes to interoperate with commercial
products using an open source solution.  The MD Server and the
Operations Client can be run with PostgreSQL and with the Local
Database Agent (LDA) as well.

The major advantage of Postgres database:
- There is no need for paying licence.
- Very easy installation on both unix and windows operating systems
- PostgreSQL does not need as much supporting memory and hardware
configuration behind as Oracle has.

The experiment using PostgreSQL is not an initiative to drop our
currently running Oracle database instances.  However, for those
nodes that cannot afford to pay for Oracle licences, this can be a
solution to get MD software running.

GRID Budapest implemented a Postgres version of MD8 server, where all
the functions available as it was with Oracle database instance.
Currently, one major step remains - to get Local Database Agent to
communicate between two nodes' database servers and make the
mirroring between the nodes.

The final step is to create or add the possibility in InstallAnywhere
to be able to choose among the types of databases and make the MD
server software configuration and installation according to these
options.

Theoretically, the JDBC driver and connection string need to be
changed and  the installer made to point to the selected database to
create and load the latest GCMD export and configure LDA.

Jakarta-Tomcat and Python can be configured and installed as before.
Finally,  the WWW interface will connect automatically to the
selected database during the configuration.

****************************************************************************
VIII.  Minutes from CEOS meeting in Toulouse, France

You may view the minutes from the last meeting at:
http://idn.ceos.org/IDN/Meetings/2003_05_Toulouse

***************************************************************************

IX.  Tentative Agenda for CEOS IDN Task Team
      Wednesday, September 17th.

      A.  Review of Minutes from Toulouse (5 min)

      B.  Status at the American Coordinating Node (20 min)
         1. Metrics for content and usage
         2. Ongoing Activities
          3. New Portals

      C.  IDN Node Reports (30 min)

          Please include in these reports, sections on:
         1. Metrics
            i. Number of DIFs contributed
           ii. Usage Statistics
         2. Status of Node Software
         3. Update Method/Frequency
         4. IDN Node Pages
         5. Advertising

D.  Proposed MD9 Contributions to the IDN from the Budapest site  (Viktor
     Pusztai)

E.  The CEOS IDN Interoperability Voting Committee and the Status of Proposals
     (40 min)
      1.  Voting Committee
      2.  Status of ISO Modifications
      3.  Status of Spatial and Temporal Resolution Data Sets
      4.  Status of model output proposal
      5.  Other suggestions for modification

F.  Advances with Lucene Search Engine

G.  Analysis of Viable Spatial Search Engine at the Directory Level

H.  The New HTML version of the DocBuilder Tool
      - Update/Demo

I.  MD9
     1. Update on what has been done

     2. Schedule for next 12 months.

***************************************************************************

X. Issues - Discussion

For more information or questions about this Newsletter or the agenda
for the upcoming meeting, please contact Lola Olsen, (Lola.M.Olsen at nasa.gov 
),
IDN Task Team lead.

**********************************************************************