Memory is full – what does the database do?

With today’s storage capacities in the terabyte range, your memory should never be full. Unfortunately, this does happen, albeit very rarely. As of version 5.7.05, the database server handles these cases better.

Database growth

in the database falls below a certain limit or there is not a sufficiently large contiguous memory area available for a BLOB. The limit is set dynamically and is 1% of the database size, but this value is restricted to the range between 512 KB and 16 MB. If the free memory falls below this limit, it is increased by the limit value.

Database     Limit    Expansion
   5 MB      512 KB      512 KB
 200 MB        2 MB        2 MB
   1 GB       10 MB       10 MB
  50 GB       16 MB       16 MB

When expanding a BLOB, the expansion is the sum of the uncompressed size of the BLOB plus the limit value. A database with a size of 1 GB and a new BLOB with a size of 20 MB is therefore extended by 30 MB.

The expansion process also includes the initialization of the additional memory area in the database and therefore causes a series of I/O operations. These no longer take place immediately when free segments are requested, but are carried out in the background by the database update thread. This speeds up mass imports in particular, as an operation no longer has to wait for the successful enlargement. It also avoids segment initialization taking place at the same time as an Update-Event, which previously led to an I/O load peak.

No available memory

If the expansion process fails due to a lack of available mass storage, the warning “Database expansion failed” is logged. In this case, the server repeats the expansion attempt at intervals of one minute, but the warning is not regenerated each time if it fails again. If space can still be created on the data carrier, the memory problem can be eliminated without additional intervention.

If the requested free segment or group of free segments (for BLOB) is outside the current size of .ca1, a test is also carried out to check whether the database has been successfully resized. If this test fails, the database is unlikely to be successfully expanded, meaning that the new segment cannot be saved in the database. To avoid potential data loss, the database operation and the current transaction are aborted, the user is logged out and error “Database free space exhausted” is logged.

The expansion test can only be performed if there is read/write access to the database. If the database is in read-only mode, for example in backup mode, during a diagnosis or hot standby synchronization, the expansion test is not carried out – in this case, the necessary expansion can only take place once the database can be written to again.

Troubleshooting

If the database cannot be expanded (warning “Database expansion failed”), space should be created on the data carrier as quickly as possible. Under no circumstances should the database be closed, the database server stopped or the system shut down before sufficient storage space is available again. In the case of data carriers on which only the database is located, a dummy file can be created as a preventive measure, which is deleted in an emergency and thus creates free storage space.