NOAA National Operational Model Archive & Distribution System

First, please make sure the GDS is not in a reload phase. If the server is not throwing 404 (or other) error codes and just hanging when you try to do anything, this is more then likely the case. Give it at least 15 minutes. If it remains unresponsive for more then ~3 hours, you can send a message to the NCDC-NOMADS support team at:

NOMADS.ncdc@noaa.gov

What are the data conventions in NOMADS and is there any convergence as to a "standard"?

COARDS
_cf
GRIB
BUFR
GRIB2
FGDC
HDF
If it does OPeNDAP it's NOMADS
Non-COARDS compliant data sets can be accessed
ascii (using ftp) services available.
_cf appears to be gaining popularity across the climate community and NOMADS will embrace this form as (one of its) "standards".

What are the core Application Programming Interfaces (API's)?

JAVA-OPeNDAP
C++-OPeNDAP
NetCDF
THREDDS
Python and Climate Data Analysis Tools (CDAT)
Eventually include the work of the BADC for Grid based direct GRIB and BUFR interface.

NOMADS Project Questions

What is NOMADS?

NOMADS is an agreement between agencies, who participate, to have common data and observation distribution software, using a format independent and description methodology, and for a unified documentation and organizational framework for data distribution.
NOMADS provides a forum to plan and organize these structures for use by university, federal agencies and organizations as participants (servers of data) or as clients (users of data).
NOMADS is a platform for these agencies to obtain support for the dissemination of their data sets and to make use of these data sets. NOMADS participants will provide their own resources to accomplish this. A resource means the hardware and the programming staff to execute their plans. Clients should invest in the training it takes to use a web page client, like ftp2u, or client software like GrADS, IDL, MatLab, Ferret, Live Access Server, idv, etc.
Eventually, NOMADS will integrate Grid based technologies- the Open Source Python language, to allow easy access to large data sets (time series of multiple files (e.g., 5 years of u/v from the Eta model) in a modular programming language. Through choices of user services- GDS, LAS, OPeNDAP, HTTP, ESG, and CEOS-Grid, these choices will allow for model, observational and other (i.e., satellite data in HDF format, and OpenGIS data structures), data using LLNL developed Climate Data Analysis Tools. Under Python, and the modular Open Source design, other discipline specific routines would become available to the NOMADS user (e.g., high energy plasma physics statistics routines would be "plug and play" for the NOMADS user
NOMADS is a user of the OPeNDAP framework. OPeNDAP is a stand-a-lone server. The DAP protocol (OPeNDAP's core) has no dependencies to the HTTP library or any other communications library. ESG has been working on a new Point-to-Point Transfer (PPT) communication library for OPeNDAP. This lightweight library allows easy communications with OPeNDAP servers so data can be transferred from OPeNDAP servers to GridFTP servers. GridFTP uses Globus Grid technology to transfer large amounts of data securely and quickly.

What are the Goals of NOMADS?

Short Term Goals

Development of a NOMADS Program Plan and Operating Procedures and Project Milestones for FY03. Action:Rutledge. Due (for team review): 1/03
Work to secure resource funding. Action: NCDC, NCEP, GFDL, CDC. Due: on-going
Determine relationship between NOMADS and DOE's Earth System Grid (ESG). Action: Rutledge, Williams.
Finalize NCDC's GDS system and get NOA APort data online. Action: Rutledge. Due 3/03
Complete BUFR station data handling capability for GrADS. Action: Doty (COLA) Due: 12/02
Develop an integrated Web presence for NOMADS: a unified front-end Web Page for NOMADS. Action: Hankin, Williams, Kerr, Rutledge. Due: 3/03
Work with existing efforts (GODAE, GCOS, BADC, et al.,), to ensure for cross disciple inter-operability and plan for international data management meeting (see Section 4)

Long Term Goals

Further develop NOMADS and CEOS-Grid and ESG partnerships
Secure funding resources for all core NOAA participants: NCEP, GFDL, NCDC, PMEL, and CDC.
Integrate Python-based technologies (CDAT, Globus) into NOMADS in parallel with standard back-end services (GrADS, LAS, OPeNDAP).
Develop Steering Committee or integrate NOMADS philosophy into existing NOAA operational structures (CIOS?).

What is the Real-Time NWP NOMADS Server?

The NCEP Real-time NOMADS (RT-NOMADS) prototype project serves real time operational data only. NCDC is the operational archive focal point and project leader for NOMADS and holds data sets older than real time. The NCEP RT-NOMADS server sends NCDC data for archival. The NCDC NOMADS goal is to save model (run history) data but budget realities provide that the archives can save initial conditions and observations sufficient to restart NCEP models to reconstruct model run history. A NOMADS goal at NCDC is to add to this archive, the NCEP operational run history forecasts as soon as sufficient storage is funded. Other NOMADS participants serve their own data sets.The NOMADS participant data sets will be connected by various search engines now under development at UCAR (THREDDS), NCAR, and NASA.The RT-NOMADS project is a prototype with the goal to produce an Operational Specification which if built is planned to be located in the operational data distribution component of NWS.

What will the NOMADS Program Plan do?

The NOMADS Program Plan will outline roles and responsibilities across the participants. It will contain a User's Guide and a template for others to establish and serve their own data sets under the NOMADS umbrella.The Plan will contain the list of data conventions currently in use and how to deal with intercomparison of different data types and appendices of currently available data.

What data forms do we expect to use in the future?

OpenGIS
others tbd

Who are the core NOMADS participants and data suppliers?

NCDC
PMEL
NCEP
GFDL
CDC
NCAR
PCMDI
USGODAE
COLA
NASA/GCMD (metadata and data locations)
International (long term)

Who are the potential data suppliers?

Other data centers (NESDIS, NWS, NOS, NMFS, Unidata, etc.)
CLASS
IOOS
GOOS
GCOS
CEOP
CEOS
Grids: NERC- Data Grid, e-science, IPG, ESG
Other International:
- GODIVA (Keith Haines) Reading
- ESMF (modeling code framework)
- PRISM (NOMADS like activity in Europe)
- ESG (Dean Williams)
- BOM (Australia thru GODAE)
- BADC (Bryan Lawrence)
- CEOS-Grid (Rutledge, Diamond)
- Project CEOP (Japan)
- Climateprediction.net

What are the core NOMADS communications protocols?

OPeNDAP
HTTP
FTP
LDAP
THREDDS
XML
Globus/CEOS-Grid/ESG

What is the relationship between NOMADS and the "grid" especially with DOE's Earth System Grid?

The long-term viability of NOMADS will be directed toward the collaborative nature of the current NOMADS partners, and future expected projects particularly the Earth System Grid, Ceos-Grid, and the British Atmospheric Data Center (BADC). BADC will provide access to the European community suite of data including ECMWF, Hadley and eventually UKMET. Australia and Japan are already participating under USGODAE, and CEOP (not yet a firm CEOP commitment). The vision currently taking shape is that NOMADS will be the umbrella under which ESG, CEOS-Grid, NERC DataGrid (UK), and other efforts will operate. Under an open source Python-based operating environment, NOMADS users will be able to use whatever tool they so desire and the NOMADS-ESG will allow users to intercompare data of differing formats and from differing locations using either thin-client (Web browser), or up to thick-clients (host services using CDAT, and/or Internet2). Once we have this framework, day-to-day maintenance will be secured first via NOAA level initiatives then on to NOAA core mission base funding.