The heart of the SDPS is a relational database management system (RDBMS). The RDBMS provides controlling, scheduling, and cataloging functions. The SDPS currently uses the SQL Server product from Sybase, but any RDBMS can be used.
There are two types of databases within the SDPS: Core Databases and Mission-specific Databases. The core databases consist of the processing, catalog, and admin databases. These are databases that support processing, cataloging, and administration functions within the SDPS.
The processing database's primary function is to provide controlling and scheduling functions for the SDPS. Semaphores are used to control the primary SDPS modules, Scheduler and VDC. Each module monitors a specific field in the control table that governs its operating mode. The Utility GUI provides the user with a means of interfacing with the control table and thus gives the user the ability to control Scheduler and VDC.
For scheduling, a table called todolist is used. This table contains records that describe jobs (tasks) that need to be done for the current day. The task's attributes define the characteristics of the job such as its type, when it should be run, which machine it should run on, and its status. The Scheduler monitors the records in the todolist table, and submits jobs to the operating system at the appropriate time. When a task completes, successfully or unsuccessfully, it records its status in the todolist table. The following graphic illustrates this:
This scheme works well for high-level tasks and simple processing, but for low-level, full-stream processing, the Visual Database Cookbook (VDC) is more suitable. For VDC, the activeproc and streams tables function as the to-do list, but instead of being based on time, they act as a gatekeeper. When a file is queued for VDC processing, a new record is inserted into the activeproc table. Here it waits in line until all of the ancillary data needed for processing have been identified and staged. Once this occurs, the file is admitted into VDC.
VDC uses two abstract concepts: a virtual CPU and a recipe. A virtual CPU is a work space defined on a machine that can be used by VDC. This work space is one or two directories, defined as input and output buffers, that will contain all of the files needed and produced by the individual steps of the processing. The work spaces are defined in the hosts and resources tables and are allocated and freed by VDC automatically. A recipe is a processing scheme that consists of a set of steps, each step performing a separate task. VDC invokes the steps sequentially, just as a chef would follow a cooking recipe. The recipes are defined in the recipe_lookup and recipes tables.
The catalog database serves as the liaison between the processing database and the mission-specific databases. Some of the catalog database's primary functions include remote-file accounting, data request processing, internal file management and migration, and housing production reporting procedures. Each of these functions may involve interacting with one or more of the mission-specific databases, and the maintenance involved with these functions is minimized by consolidating the database objects in the catalog database.
The admin database is used for maintaining records pertaining to the administrative aspects of the SDPS. These include the Mission Events and Major Software Changes log, current known bugs and planned enhancements, and SDPS staff information.
Mission-specific databases are used to isolate all of the tables and procedures for a mission. Each mission will have its own products and data flow. The architecture for a mission-specific database is governed by the products defined for that mission and the logical flow of the data through the system. There are currently two mission-specific database within the SDPS: seawifs and octs. The seawifs database comprises all of the tables and procedures that support the SeaWiFS mission. Likewise, the octs database supports the ADEOS/OCTS mission.