B.08.20 changes (relative to rc6) ------------------------------------- This is a B.08.20 release candidate. Unless a critical issue is detected it will be released unchanged. The release candidate fixes a few issues that were found in testing. - Fixed a problem when assigning COM variables that could result in an internal failure. - Fix list utility command line handling - Suppress redundant messages in eloqdb panic - Allow schema "passwords" up to 64 characters - Trigger selective FTS rebuild on parser config changes - Fixed a problem where in a rare case a data block in the volume files might get corrupted when on-line backup mode is stopped (#4108). - Fix alignment problem on eloqdb IA64 Linux build - Fix potential data corruption, lock against concurrent checkpoint (#4128) - Fixed a possible transaction journal corruption if the database server was stopped (or aborted) while in on-line backup mode after using the dblogreset or dbrecover utility (#4126). - Don't increment volume generation on slave server startup after dbrecover (#4129) - Add "other" session stat to cover non-image database calls B.08.20 rc6 changes (relative to rc5) ------------------------------------- The B.08.20 release is functional complete, there are no known critical issues. The RC6 release fixes a few issues that were found in testing. - Fix a problem with FTS mixed aggregated/non-aggregated results - Fixed a problem that displayed only first member variable of a user defined type in the IDE B.08.20 rc5 changes (relative to rc4) ------------------------------------- The B.08.20 release is functional complete, there are no known critical issues. The RC5 release fixes a few issues that were found in testing. - Fixed result type on mixed aggregated/non-aggregated FTS searches. In previous versions, a mixed query was always converted to an aggregated result. - Added Perforation Skip Disable mode support to eqpcl - Fix PCL Page length resulting in an invalid text length - Fix Windows 8 priting problem with xps drivers B.08.20 rc4 changes (relative to rc3) ------------------------------------- The B.08.20 release is functional complete, there are no known critical issues. The RC4 release fixes a few issues that were found in testing. - Changed timestamp format in log files to YYYY-MM-DD HH:MM:SS - The [server] LogTimeFmt config item may be used to specify the logfile timestamp format for the database server. - Removed old example files, they will be available for download separately - Fixed a problem with PCL reset in the eqpcl utility that could result in bad output. - Updated eqpcl barcode library. - LOAD DLL w/o specifying a DLL file name (in addition) searches for files with a .DLL extension. - Fixed a bug with ASSIGN;APPEND failing with error 56 instead of creating the target file. B.08.20 rc3 changes (relative to rc2) ------------------------------------- The B.08.20 release is functional complete, there are no known critical issues. However, QA testing has not concluded yet. The RC3 release fixes a few issues that were found in testing. - fixed a crash in eloqcore when copying string arrays - added information on memory consumption to dump stack output - fixed bad include path in eqpcl utility - fixed eqpcl quoting of output device - fixed eqpcl quoting on Windows B.08.20 rc2 changes (relative to rc1) ------------------------------------- The B.08.20 release is functional complete, there are no known critical issues. However, QA testing has not concluded yet. The RC2 release fixes a few issues that were found in testing. - fixed a race condition in eloqcore when obtaining taskid - fixed a potential deadlock condition when updatig FTS index - improved FTS locking during keyword updates to be more robust - revert image3k dbinfo behavior to separate image and TPI items B.08.20 rc1 changes (relative to beta6) --------------------------------------- The RC1 release indicates the B.08.20 is functional complete, and there are no known critical problems. Highlights - a number of defect fixes - improved ODX/DBI/image3k documentation - improved dbrepack analyze phase - improved FTS search scalability database server --------------- - Improved FTS search scalability image3k ------- - dbinfo TPI item number behavior was modified. A TPI item number is only returned for composite FTS/index items. Otherwise the item number of the data item is returned. This implements established behavior of the dbinfo TPI modes. - dbfind mode 1/21 performs FTS lookup if item is not ambiguous. A dbfind mode 1/21 may be ambiguous if the item is both an FTS field as well as a search item or index. The dbfind was modified to not perform a FTS lookup if the item parameter is ambiguous. - New status -54 indicates a bad FTS argument (syntax) A status -54 indicates a bad FTS argument. This typically indicates improper syntax. - New status -55 indicates improper use of FTS. This typically indicates no FTS Search result is available or the FTS search result does not apply. language -------- - dbfind mode 1 performs FTS lookup if item is not ambiguous. A dbfind mode 1 may be ambiguous if the item is both an FTS field as well as a search item or index. The dbfind was modified to not perform a FTS lookup if the item parameter is ambiguous. - New status -54 indicates a bad FTS argument (syntax) A status -54 indicates a bad FTS argument. This typically indicates improper syntax. - New status -55 indicates improper use of FTS. This typically indicates no FTS Search result is available or the FTS search result does not apply. dbrepack -------- - The dbrepack analyze phase was improved to take advantage of the record free list if the number of deleted records is below a limit. This substantially improves performance of the analyze phase for large data sets with a limited number of deleted records. B.08.20 beta6 changes (relative to beta5) ----------------------------------------- Highlights: - new dbpasswd utility and support for protected password files - dbrepl service on Windows - dbrepack utility enhancements - dbctl utility enhancements The B.08.20 beta6 completes the B.08.20 functionality and adds a number of fixes. database server --------------- - The DBGET modes 2x (21,22,23,25,26) were revised to allow additional flexibility using FTS indexes. The first DBGET mode 2x after a DBFIND on an FTS field will now define the "retrieval" data set, which could be different from the last data set a DBFIND was used on. DBGET supports using database chains to appy FTS search results to a detail set. For example, one could search for customers in a specific region and then list their orders, assuming both sets have a common master set. - The server was improved to properly dispose unreferenced keywords from the dictionary index. - The syntax for numeric ranges in FTS searches was simplified. A numeric range may be used with FTS fields on numeric items or when the NR option is present on a text field. The database will then use a numeric comparison to determine if a value matches the search condition. By default a string comparison is used. Numeric ranges also support specifying decimal numbers. - The dbctl functionality was enhanced. The "dbctl replication monitor" returns the slave server replication status information in a form that is easier to parse. The following information is returned, separated by spaces. The "dbctl volume" command is experimental and allows to change database volume file properties without a server restart. language -------- - The QFIND statement may be used on FTS indexes with the "matches" clause. For example: QFIND "CITY","MATCHES","wup@ or berlin" Where "CITY" is an FTS field. This will locate all records where the city has words starting with "wup" or the word berlin. database utilities ------------------ - The new dbpasswd utility may be used to create database password files and/or change database passwords. This provides a convenient and secure alternative to change database user passwords and stop using clear text passwords. - The database client library and utilities were enhanced to support protected password files. - The stand-alone utilities that can modify the volume files (such as dbfsck, dbcfix or dbrepack) were enhanced to increment the volume generation in write mode. This makes sure the forward recovery and replication detect their use. Directly changing the volume files is not compatible with applying forward logs and may potentially result in difficult to track down corruption problems. - The new dbftswords utility may be used to list fts keywords for a specified FTS field. - The ftsstat utility was renamed to ftsdump and its command line options were changed. NOTE: If you installed a previous Eloquence B.08.20 beta release on HP-UX it may be necessary to manually remove previous versions of the ftsdump utility. It seems that swinstall does not handle this automatically. As root execute the following commands # rm /opt/eloquence/8.2/bin/ftsstat # rm /opt/eloquence/8.2/bin/dbftsstat - The dbrepl utility may be used as a service on Windows platform - The startup banner message was removed from most database utilities to simplify use in scripting. In previous versions the banner message was output to stderr when the utility was started. Affected utilities were changed so the utility name is present in any error messages. - The dbctl utility was enhanced to support interactive mode. If the new -i option is present, any dbctl commands are read from stdin rather than the command line. This allows a single dbctl invocation to be used to execute server commands repeatedly. - The dbinfo utility was enhanced to indicate a data set has encrypted or masked fields. In this case an indicator column is output in addition. - The dbtables utility was enhanced to list FTS fields. - The schema utility was enhanced to create new databases. This makes the use of dbcreate optional. dbcreate may still be used but has no effect if the database is already created. - The dbrepack utility was enhanced to support encrypted databases and FTS indexes. With encrypted databases, the new -M command line option must be specified and the master key must be present. database client library ----------------------- - The database client library was changed to support protected password files ======================================================================= B.08.20 beta5 changes (relative to beta4) ----------------------------------------- Highlights: - improved index rebuild performance - new dbrepack utility This beta includes a number of bug fixes in various product components. Please find below a list of user visible changes. When updating from a previous B.08.20 beta version any FTS indexes MUST be rebuilt as internals have changed to allow for a larger number of keyword references per database. eloqdb ------ - Fixed a problem with FTS bulk rebuild with the 64 bit eloqdb64 on standalone FTS fields. In this case a record reference could hold a corrupted record number. - New NR option for FTS fields. When the NR option is set numeric range searches on text fields are supported. A number extracted from a text field is then added to both the keyword string as well as numeric index in addition. If the option NR is not set, any numeric range searches on text fields will use a text comparision rather than numeric. For example, 01:10 would not include the value 5. - Changed internal format of FTS references to allow for a larger number of keyword references per database. The current version will support at least 1 billion FTS references per database. - The database restructure procedure was enhanced to support rebuilding FTS indexes incrementally. Previous versions recreated all FTS indexes when any FTS index changed. - The database restructure procedure was enhanced to use a more efficient bulk btree rebuild procedure. This makes temporary use of memory to accelerate btree indexing performance noticeably. The max. amount of temporary memory may be configured with the [config] RestructMemLimit config item. - The default memory limit for the btree and FTS index bulk rebuild procedure was changed to 100 MB. The previous default of 20 MB was overly conservative. - Rebuildung a btree index should use less disk space in the forward log. - The dbctl replication status and the HTTP config status page on a slave server has additional information on the last replication activity and the estimated replication lag. This is intended to simplify monitoring the replication status. For example: $ dbctl replication status slave Server is configured as SLAVE Replication is active (activity 158 sec ago, lag 0 sec) Last checkpoint is 45-1.1 (2012-09-05 16:57:32) Forward-logging is enabled Forward-log is '/data/test/S/fw/fw-44-1.log' The activity field ("last activity") indicates the last time when replication data was received by the slave server (difference in seconds from the current time). The lag field ("estimated replication delay") indicates the time difference between the master timestamp in the replication data and the time the data was received on the slave server (difference in seconds). The lag field is only present when replication is active (dbrepl connected to the slave server). The activity field is present when dbrepl was previously connected to the slave server. - The dbrepl utility may now be used with operator capabilities. Previous server versions required dba capabilities for replication. dbrepack -------- An initial version of the new dbrepack utility is now available. dbrepack allows to repack a database to reclaim disk space occupied by deleted records. Deleted records are re-used but the disk space is not freed for use by other data sets or databases. usage: dbrepack [options] database [set ...] options: -help - show usage (this list) -u user - set user name -p pswd - set password -c cfg - server configuration file -b size - Buffer cache size (MB) -v - verbose output -w - enable write mode -P pct - skip set unless at least pct% of disk space reclaimable -S mb - skip set unless at least mb MB disk space reclaimable By default dbrepack runs in read-only mode and analyzes the specified database. The option -v enables additional progress messages. dbrepack accesses the database volume files directly. The database server process may not be active (or must be in online-backup mode when dbrepack is used in read-only mode). $ dbrepack -c eloqdb.cfg -v toydb2 Running in Read-Only mode PRODUCTS : no space reclaimable, 1 records repackable (5.0%) CUSTOMERS : skipped - data set has associated FTS data ORDER-MASTER : no space reclaimable, no records repackable INVOICES : no space reclaimable, no records repackable ORDERS : no space reclaimable, 15 records repackable (100.0%) ORDER-DETAILS : skipped - data set has associated FTS data OTHER-DETAILS : no space reclaimable, no records repackable BOOKING : 7.3 MB reclaimable (3.1%), 14500 records repackable (3.2%) The reclaimable disk space indicates the disk space that could be reclaimed by repacking a data set. The number of repackable records indicates the number of gaps found in the data set. Please note that the current dbrepack version does not yet support encrypted databases and data sets with FTS indexes. dbrepack in write mode changes the database volume files and will likely leave the database in a corrupted state on failure. Please make sure you have a current backup. Runing dbrepack in write mode will invalidate replication and forward recovery. eloqcore -------- - Fix incompatible behavior with DLG LOAD when .dlg file does not exist and WinDLG is used. - Fix FN decompiler bug with array argument. The statement X=FNA(FNB(X.Y(*),1)) was not decompiled correctly. dbutil ------ New NR option for FTS fields. When the NR option is set numeric range searches on text fields are supported. A number extracted from a text field is then added to both the keyword string as well as numeric index in addition. If the option NR is not set, any numeric range searches on text fields will use a text comparision rather than numeric. For example, 01:10 would not include the value 5. libeqdb ------- - Fix dbinfo mode 515 when fts item number passed. ftsstat ------- - The dbftsstat utility was renamed to ftsstat and the command line options were reorganized. Linux platform -------------- - Added workaround to improve systemd coexistence. The Eloquence start/stop script uses extended modes and additional arguments and is not compatible with the systemd wrappers. A workaround was added to skip systemd wrappers. ======================================================================= B.08.20 beta4 changes (relative to beta3) ----------------------------------------- Highlights: - improved eloqsd - improved FTS index rebuild performance - dbrepl improvements - replication integrated in start/stop script - new prdbutil utility to obtain database setup in dbutil syntax eloqsd ------ An improved version of the eloqsd daemon is included as of the B.08.20 release. User visible changes include - User passwords may be stored in encrypted format. - Added PAM support for user authentication: eloqsd now supports authenticating against OS users. - Added network address filter to selectively limit access to eloqsd. - Added support for user groups to limit application use. - Added support to configure applications in eloqsd rather then relying on client setup. This reduces redundant information and improves security. - The new eqpasswd utility is used to maintain encrypted eloqsd passwords. For more information see http://eloquence.marxmeier.com/support/B0820/relnotes/eloqsd.html eloqdb ------ The database server restruct funtionality uses a new FTS bulk parser to create FTS indexes. The new FTS bulk parser uses memory to create the FTS indexes more efficiently. By default the FTS rebuild will use up to 20 MB of memory. A larger memory limit is likely to further improve performance. This may be specified with the new [config] RestructMemLimit config item. In our testing a memory limit of 200 MB resulted in the best performance for a large database. The eloqdb.cfg template configuration file (newconfig/config/eloqdb.cfg) was updated to include this config item. The search strategy for excluded words was enhanced to improve performance for larger exclusion lists. dbrepl ------ The dbrepl utility was enhanced to support a replication config file. The replication config file is specified with the new dbrepl -C option and provides a default for the replication parameters. The settings in the config file my be overridden using the equivalent command line options. A template configuration file (newconfig/config/repl.cfg) is included. The dbrepl utility was enhanced to allow retrying on a slave server connection failure. This functionality is enabled with the RetryCnt and RetryDelay config items. RetryCnt specifies the max. number of retries and RetryDelay the delay in seconds between each attempt. The default RetryCnt is 0 (no retries), the default RetryDelay is 300 seconds (5 minutes). On HP-UX and Linux the new -D option may be used to run dbrepl in background. start/stop script (HP-UX and Linux) ----------------------------------- The start/stop script was enhanced to support replication. This allows to reduce custom scripting to use replication. When configured in the startup config file (/etc/.../eloquence82) the eloq82 startup script may be used to manage replication. A replication instance is configured as below: REPL_CFG[0]=repl.cfg REPL_START[0]=1 REPL_ID[0]="repl" A new startup config file template (newconfig/startup/eloquence.rc) is included and provides additional details. Eloquence language ------------------ A bug when copying member arrays (MAT V.A=B) was fixed. This could result in aborting eloqcore. The REDIM statement was enhanced to support member arrays. dbutil ------ - Fixed a problem in dbutil when creating new databases. - Fixed a problem decoding dbproperty flags when the database was maintained on a remote host with a different architecture. prdbutil -------- The new prdbutil utility may be used to obtain the database configuration. usage: prdbutil [options] [database] options: -help - show usage (this list) -u name - user name (default "public") -p pswd - password -h host - host name or ip address (and service) -s service - service name or port number -l section - list of sections schema, group, priv, access, enc, index, fts dbproperty, member, user, image3k The -h option may be used to specify the database server instance (host and service). The EQ_DBSERVER environment variable may be used as an alternative. The option -l may be used to specify a comma separated list of information categories. When the database name is not specified only the user list is available. For example: prdbutil -l user -h server prdbutil -l image3k,index,fts toydb prdbutil toydb Please note that any user passwords cannot be obtained as they are not stored in the database server.