Improved Performance of Delete Node Queue Management

A very long server shutdown was noticed in a particular environment due to delete node processing. Two areas of server operation were enhanced to minimize this time spent in delete node processing.

Delete Node Queue Entries

Previously, the delete node queue exercised the following pseudo code for each queue entry:

• read queue
• open file
• begin transaction (for TRANPROC files)
• prune tree
• commit transaction
• close file

This basic loop was modified in two ways:

1. An attempt is made to peek at the next queue entry to determine if the next tree pruning will be for the same index file (the host index or one of its members).
2. If the files match, skip the commit/begin operations and skip the close/open operations. More precisely, the number of skipped transaction commits and skipped file closures are actually tracked and an artificially imposed limit is set on the number of skips to avoid very long transactions and keeping the file open in shared mode for an extensive period.

By default, NODEQ_TRNLEN is set to ten (10), and NODEQ_OPNLEN is set to twenty (20). Once shutdown begins, these limits are increased by a factor of ten.

Limited testing shows that shutdown processing of the delete queue runs at least twice as fast with this new behavior.

Note: These changes have little effect if the queue entries are widely varied across different files.

c-tree Server Shutdown Processing

The c-tree Server now supports a configurable limit on the number of delete node queue entries that the c-tree Server processes when it is shutting down. If the option DNODEQ_SHUTDOWN_LIMIT is specified in the c-tree Server configuration file, then when the c-tree Server shuts down, if there are more than the specified number of entries in the delete node queue, the delete node thread writes all the unique queue entries to a disk stream file named DNODEQUE.FCS. A memory-based index file is used to eliminate duplicate queue entries, and only the unique entries are written to disk. If the number of unique entries is less than DNODEQ_SHUTDOWN_LIMIT, those entries are returned to the delete node queue and are processed by the delete node thread before the c-tree Server completes the shut down.

DNODEQ_SHUTDOWN_LIMIT 0 causes the c-tree Server to process all entries in the delete node queue when shutting down.

The c-tree Server always attempts to open and read all entries from the file DNODEQUE.FCS into the delete node queue at startup, regardless of the DNODEQ_SHUTDOWN_LIMIT setting. The c-tree Server deletes the file DNODEQUE.FCS after populating the delete node queue with its contents. An administrator can delete the file DNODEQUE.FCS before starting the c-tree Server to avoid processing these persistent delete node queue entries.