What's New in InterBase XE7

From InterBase
Jump to: navigation, search

Go Up to Main Page


The following pages describe the new features for InterBase XE7 updates:

Released:May, 2014

Welcome

InterBase XE7 introduces many new features and enhancements to further extend the capabilities of InterBase. A few of the new or enhanced features are highlighted below:

  • Change Views: Allows you to quickly answer the question: "What data has changed since I last viewed it?"
  • Linux kits: The Linux kits, 32-bit and 64-bit are now available in the InterBase XE7 release.
  • Performance Enhancements: These enhancements include improved SMP performance, distinguished dump enhancements, and improved Read Committed (RC) Transaction Processing.
  • ODS Changes: New ODS version 16 was introduced to support Change Views and new Index Managers for better performance. You can also now restore your database backup file to an older supported ODS version; supported older ODS versions include 13 and 15.

System Requirements

For system requirements to install and run InterBase XE7, please see System Requirements/Prerequisites.

New Features

Change Views Feature

The Change Views feature uses InterBase multigenerational architecture to capture changes to data. This feature allows you to quickly answer the question: "What data has changed since I last viewed it?"

Previously it involved triggers, logging, and/or transaction write-ahead log scraping. This was time-consuming for the developer and affected the database performance for a certain transaction load or change volume. Now with Change Views, there is no performance overhead on existing transactions because it maintains a consistent view of changed data observable by other transactions.

The Change Views™ feature uses InterBase multigenerational architecture to capture changes to data. This feature allows you to quickly answer the question: "What data has changed since I last viewed it?" Previously it involved triggers, logging, and/or transaction write-ahead log scraping. This was time-consuming for the developer and affected the database performance for a certain transaction load or change volume. Now with Change Views, there is no performance overhead on existing transactions because it maintains a consistent view of changed data observable by other transactions.

The Change Views mechanism is not dependent on its own underlying data, but is based on data already stored for existing base tables or views derived from base tables. This implicit view mechanism is temporal based and returns data that have changed since the prior transaction in which the implicit view was observed.

Change Views can be subscribed to in order to view data that has changed across database connections. The effect is a long-lived transaction spanning multiple database connections.

  • Specifically, the subscription tracks all row inserts, updates, and deletes to one or more tables at a column-level granularity over a disconnected, extended period of time.
  • The InterBase SQL query language is modified to search on columns where data has changed since the prior observation.
  • These data changes are tracked at a column granularity.

Linux 32-bit and 64-bit

The Linux kits, 32-bit and 64-bit, were introduced in the InterBase XE7 release. New Linux builds (12.0.0.124) are available for InterBase XE7.

License Manager

The License Manager changed from InterBase XE3 to InterBase XE7.

Performance Enhancements

  • Improved SMP Performance: The performance of multiple reads and writes improved for this release.
  • Distinguished Dump: "Incremental Dump" in InterBase XE3 required the database server to read all pages from the database file, but only write the pages that had been modified to the target database dump file. With the implementation of a tracking system in XE7, only those pages that need updating since the last dump would be fetched. This provides instantaneous updates to the target. There can only be 1 "Distinguished Dump" per source database.
The choice of a "distinguished dump" is as follows:
  • The first "dump" on the source database file will be a "distinguished dump"; all further dump targets are "normal dump" targets.
  • Should the "first" dump be made online and thus sever its link to the source database, the next "dump" to be incrementally updated will now become the "disinguished dump".
  • Improved Read Committed (RC) Transaction Processing for large (OIT - OAT) transaction gap: In XE3 and prior versions, RC transaction processing can grind to a halt as the "housekeeping" (OIT - OAT) gap grows larger. This usually does not happen unless the database has disabled automatic internal sweep function or the housekeeping gap is set to a very high value. This gap is usually in the 10's of millions for the performance degradation to be noticed. The OIT usually sticks after an abnormal server termination or a transaction savepoint is discarded because it has grown too large trying to track changes of a running transaction and that transaction ultimately rolls back.
    • With XE7, RC transactions perform much faster in the above scenario. In one of our tests to count 1 million records in a table, in XE3 the test took 14.48 seconds. In XE7 the time to execute the query was 0.97 seconds. The performance of the RC transaction is now on par with Snapshot mode transactions. Just to note, Snapshot transactions did not exhibit any slowdown in earlier versions.
  • Quicker transaction creation: With XE7, you can create more transactions/sec than with prior InterBase versions.

Performance Monitoring Counters

The performance monitoring counters have been scaled up from 32-bit to 64-bit values. It includes all data stored on tables within the database. IBConsole has been updated for this release.

There are some ODS-related differences to be aware of:

  • ODS <= 15 will continue to have 32-bit INTEGER counters as before (for both dialect 1 and dialect 3 databases).
  • ODS >= 16 will have the counters defined as "double precision" data type for dialect 1 databases.
  • ODS >= 16 will have the counters defined as "NUMERIC(18,0)" data type for dialect 3 databases. As you know, by default, any new database is created as ODS 16, dialect 3.

For dialect 3, the Performance Monitoring data counters have been updated to 64-bit Integer type. Since dialect 1 cannot support 64-bit Integer type, a 64-bit Integer type will be internally converted to type "double precision" as it is the same size (8 bytes) and will also accommodate the large values for 64-bit addresses and counter values.

The best way to identify the differences between dialect 1 and dialect 3 is to check field definitions for the result of the SQL query against your database. So if you ran the following query: SELECT RDB$FIELD_NAME FROM RDB$FIELDS WHERE RDB$FIELD_LENGTH=8 AND RDB$FIELD_TYPE=16;, the results would show as follows:

  • FIELD_TYPE 16 indicates NUMERIC(18,0) and will be set in dialect 3.
  • Dialect 1 database will have none of this field type since it will be double.

Backward Compatibility

With the support of 64-bit Integer data counters for Performance Monitoring, backward compatibility is maintained for the new server working with ODS 15 databases where such counters are 32-bit Integers.

To do this, enable gpre with the "-ods <major_ods_number>" command-line option that allows GPRE to limit its parsing of GDML/ESQL programs to relations/tables that are only valid until that major ODS.

 gpre [-ods <number>] ... <foo.e>
 For e.g.:
 gpre -ods 15 tmp.e

Partial Segment Selectivity

InterBase currently maintains a single selectivity value per index (whether single or composite key). The values are stored in descending order and track information on how many duplicates exist in the whole index – but only first level. However, there are times when it would be helpful to have information on lower levels. InterBase XE7 can now track this. Changes have been implemented for a new ODS (version 16) that allows gathering segment-specific selectivity to be stored and retrieved from RDB$INDEX_SEGMENTS.RDB$STATISTICS.

For example: an index exists on columns F1, F2, and F3, and the query provides a conjunct (condition) on only the first two columns F1 and F2. The optimizer can use a value that only reflects the underlying selectivity of the columns F1 and F2 instead of F1+F2+F3 and then make a decision on whether to use the index or not.

You will see certain queries run faster, but will not see any change in data.

64-bit Transaction IDs

InterBase XE3 supported only (signed) 32-bit Transaction IDs. This allowed only 2 Billion transactions to be executed in a database before the transaction ID needed to be reset using a backup/restore procedure. This was limiting for 24/7 databases and for those who had transaction-intensive application suites.

InterBase XE7 scales transaction IDs to 64-bit (actual 48-bit) so databases now can accommodate more transactions without the old limitations. It limits itself to 48-bit, as to be aware of future transaction ID growth.

The transaction limits now work as follows:

  • Databases with ODS <=15 need to be backed up and restored before they hit the 2-billion transaction ID limit.
  • Databases with ODS >=16 can continue to be online beyond 2-billion transactions since they support 64-bit transaction ID. A benefit of this is you can have your databases online to service your applications closer to a 24/7 scenario without the need to back up/restore due to this earlier 32-bit limit.
  • 64-Bit Conversion.
  • This project assumes conversion to 64-bit (more likely 48-bit) transaction IDs with the assumption that they could not be exhausted in a reasonable timeframe.
  • For example, a database with a 4KB page size could run continuously in excess of 10,000 tps for 100 years with a 48-bit transaction ID.

ODS Changes

Allowing a Database Backup to be Restored to a Particular Older ODS Version

A database restore operation automatically creates a database with the latest ODS version (as supported by the engine). However, there are situations where the developer may need to restore an older version:

  • To enable the same database file to be deployed to other compatible-database engine versions.
  • To provide a workaround for hard-to-resolve current ODS bugs.
  • To test performance-sensitive operations between current ODS version and older ODS versions by using a single server/engine to restore multiple copies of the same database (one per ODS) and then running the queries against each one of those database files. For example, to test certain versions of index managers released in newer ODS versions.

You can use this implementation in several different ways:

IBConfig Parameter

DATABASE_ODS_VERSION. Allows a major ODS version number to be specified server-wide. A newly-created database or one restored using GBAK is restored with this ODS version. It now has full support for new Performance Monitoring artifacts for 64-bit integer counters with full backward compatibility with older ODS databases.
The IBConfig file has a new parameter for this action as documented below.
#DATABASE_ODS_VERSION 16
##Platforms: All
##Version: starting in InterBase XE7
##The database server/engine will automatically create/restore
##databases to this major ODS version number, if specified.
##Valid ODS major versions are in the range 13 to 16
##Default major ODS version is the latest version supported by the product

DPB Parameters

Allows the application to selectively create/restore databases using a new DPB parameter isc_dpb_ods_version_major.

GBAK Restore Option

"-ods_version <n>" where <n> is a supported major ODS version. The target database engine will have its own allowed major ODS version list. XE7 will allow ODS versions 13 through 16.
A new GBAK restore option is available for restoring a specific major ODS version. When this option is specified at restore time, it overrides the server-side IBConfig setting DATABASE_ODS_VERSION.
gbak -r employee.gbk emp15.ib -ods_version 15 -user sysdba -pass masterkey
gbak -c employee.gbk emp16.ib -ods_version 16 -user sysdba -pass masterkey
gbak -service localhost:service_mgr -r <path>/employee.gbk <path>/emp13.ib -ods_version 13 -user sysdba -pass masterkey

OpenSSL Upgrade

OpenSSL libraries included in InterBase XE7 are upgraded from version 1.0.0d to version 1.0.1i with Security Vulnerability fixes. OpenSSL is used for Encryption and OTW/SSL features in InterBase.

For more information on using OpenSSL in InterBase XE7, please reference Network Configuration in the Operations Guide, as well as Encrypting your Data in the Data Definition.

For additional information on OpenSSL, please refer to: ssl.

Services API Support for Online Dump and Journal Archive Operations

Online Dump and Incremental Dump

Database Online Dump and Incremental Dump were, before this release, only supported as DPB options. It was not available as a Services API option, which made it difficult for connectivity layers such as IBX to identify this "backup" functionality differently from other "Services API"-aware backup functionality. Services API now supports online dumps and incremental dumps.

The following lists the features for this function:

  • The task isc_action_svc_dump has been added with the purpose to dump a database to a file. This is equivalent to gbak -d.
  • Use the cluster identifier isc_action_svc_dump to request that the Services Manager perform a dump operation. This is a programmatic method invoking the gbak tool as a thread in the ibserver process. You must specify the path of the database primary file, and the path of the dump output file or files. Paths of dump files are relative to the server. Because the Services Manager executes dump tasks on the server host, the Services Manager reads and writes dump files on the server host. The Services Manager also creates files in the context of the server.
  • The following Online Dump and Incremental Dump actions have been added via isc_action_svc_dump:
Action Argument Purpose Argument Length Argument Value
isc_action_svc_dump isc_spb_dmp_create Request to execute an Online Dump operation and create an online dump database from a source database. This is passed as a standalone option value to isc_spb_options. This operation is equivalent to the "gbak -dump" command. 0 0
isc_action_svc_dump isc_spb_dbname Path of the primary file of the database, from the point of view of the server. 2 bytes + string String
isc_action_svc_dump isc_spb_dmp_file Path of a dump output file; you can specify multiple output files. 2 bytes + string String
isc_action_svc_dump isc_spb_dmp_overwrite Overwrite existing dump file; absence connotes incremental update of existing dump file. This operation is equivalent to the "gbak -overwrite_dump" command. This parameter should be provided just by itself in the SPB byte array. This is an optional argument. 0 0

Note: To create Online and Incremental database dumps without requiring a database, and by only using Service API, use the new SPB parameter isc_spb_dmp_create. This is passed as a standalone option value to isc_spb_options and is equivalent.

Journal Archive Management Actions

The Journal Archive Management actions were, in the past, only supported from the following command-line tools:

  • GBAK supports Archive Database, Archive Journals, and Archive Recover actions.
  • GFIX supports Archive Dumps and Archive Sweep actions.

Previously, no Service API support existed that applications could invoke. However, now by using the cluster identifiers isc_action_svc_backup, isc_action_svc_restore, or isc_action_svc_properties, you can request that the Services Manager perform various operations on InterBase Journal Archives.

This is a programmatic method to invoke the gbak tool as a thread in the ibserver process. You must specify the absolute full path of the database primary file, and recovery related database files. Because the Services Manager executes archive management tasks on the server host, the Services Manager reads and writes journal archive location files on the server host. The Services Manager also creates files in the context of the server.

  • The following Services API Journal Archive Management Actions have been added:
Action Argument Purpose Argument Length Argument Value
isc_action_svc_backup isc_spb_bkp_archive_database Request to execute an Archive Database operation and create an archive database copy from a source database. This is passed as a standalone option value to isc_spb_options. This operation is equivalent to the "gbak -archive_database" command. 0 0
isc_action_svc_backup isc_spb_dbname Path of the primary file of the database, from the point of view of the server. This database will be copied by the database engine to the Journal Archive location defined for that database using CREATE JOURNAL ARCHIVE command. 2 bytes + string String
isc_action_svc_backup isc_spb_bkp_archive_journals Request to execute an Archive Journals operation and copy journal files for the source database to the journal archive directory location. This is passed as a standalone option value to isc_spb_options. This operation is equivalent to the "gbak -archive_journals" command. 0 0
isc_action_svc_backup isc_spb_dbname Path of the primary file of the database, from the point of view of the server. The journal files for this database will be copied by the database engine to the Journal Archive location defined for that database using CREATE JOURNAL ARCHIVE command. 2 bytes + string String
isc_action_svc_restore isc_spb_res_archive_recover Request to execute an Archive Recover operation and recover from a database archive in the journal archive directory location. This is passed as a standalone option value to isc_spb_options. This operation is equivalent to the "gbak -archive_recover" command. 0 0
isc_action_svc_restore isc_spb_bkp_file Path of the database in the Journal Archive location, from the point of view of the server. The database file mentioned will be used as the source database to recover from followed by recovering data from associated journals from the same archive location. 2 bytes + string String
isc_action_svc_restore isc_spb_dbname Path of the database file to restore to (or create), from the point of view of the server. The newly-restored database once fully recovered will be set to "read_only" mode and can be brought back online later. 2 bytes + string String
isc_action_svc_restore isc_spb_res_archive_recover_until This is an optional argument for Point-in-time-recovery; this operation is equivalent to "gbak -until <timestamp>" command. Argument value is string indicating Timestamp format. Accepted Timestamp string format detailed in Embedded SQL Guide (EmbedSQL.pdf) chapter "Working with Dates and Times", section "Formatting Dates for Input".

For e.g. "2006-08-21 18:08:15".

2 bytes + string String
isc_action_svc_properties isc_spb_prp_archive_dumps Request to execute an Archive Dumps operation to set a limit of database dump versions in the journal archive directory location. This operation is equivalent to the "gfix -archive_dumps <n>" command. 4 bytes Unsigned 32-bit integer value indicating number of dump files to maintain.
isc_action_svc_properties isc_spb_prp_archive_sweep Request to execute an Archive Sweep operation to sweep (delete) database dump files that have a sequence number less than the provided value in the journal archive directory location; the sweep also deletes associated journal files for those database dump files. This operation is equivalent to the "gfix -archive_sweep <n>" command. 4 bytes Unsigned 32-bit integer value indicating sequence number below which to sweep database dump files.

Sample applications are provided in the <interbase>/examples location for all the SPB parameters listed above. start_dump.c exercises the online/incremental dump operations. ib_archive.c exercises the Journal Archive related operations.

Topics

See Also