GRANITE 9 and GRANITE 10 Data Table Storage

Fixed Number of Records vs. Auto-Allocated Data Tables

Fixed size data storage tables are stored in battery backed, non-volatile SRAM whereas auto-allocated data tables are buffered in the DDR-SDRAM and then copied to the SSD (Solid State Drive) for final storage. The DDR-SDRAM is high-speed, volatile memory that is well suited for storing high volumes of data at sustained higher rates (e.g., sub second). The combination of buffering in the DDR-SDRAM and copying into the SSD is very powerful for high speed data. However, there is a window of vulnerability with this DDR-SDRAM/SSD combination. With auto-allocated tables, data resides on the volatile DDR-SDRAM until it can be transferred to the non-volatile SSD hard drive. If a power fault or some other failure occurs, all data that is in the DDR-SDRAM prior to being written to the SSD, will be lost. The slower, less data efficient, battery backed SRAM does not suffer from this window of vulnerability; a power loss does not result in compromised data. The ideal use of these different memories is to store slower and/or smaller data sets in the battery backed SRAM (e.g., fixed size tables) and high speed, high volume data on the SSD (e.g., auto-allocated tables). Note that in addition to fixed size tables, SRAM holds communication buffers, CRBasic program operating memory and, if allocated the USR drive. Data tables with a fixed number of records fill whatever memory remains after all other memory requirements are satisfied. A compile error occurs if insufficient memory is available for fixed size data tables.

Available memory on the SSD

Note that even though the SSD has 64 GB (or 128 GB for XD dataloggers) of available memory, individual file size is limited to 2 GB. Hence, the Tablefile() instruction must be used if more than 2 GB of memory is needed for a final storage data table on SSD. Tablefile() allows multiple small files to be written from the same data table, so storage for a single table can exceed 2 GB. See the Tablefile() help for details.

Managing high-speed data

The most efficient method to store high-speed data is to stream the data from local, final-storage data tables to a FTP server. This is achieved with the optional FTP streaming parameters of the FTPClient (Use FTP, or FTPS, or SFTP to Manage Files) instruction.