Eloquence B.08.10 (RC) ====================== Revision: RC2, 2010-06-15 Thank you for your interest in the Eloquence B.08.10 beta test. This Eloquence test release provides a release candidate of the upcoming Eloquence B.08.10 version. This release is expected to be feature complete and only minor changes or bug fixes are expected until the production release. By making the test versions available publicly we hope to encourage wider testing. Please contact support@marxmeier.com to share your feedback or report a problem. Please note: This release is available under the terms of the Eloquence Beta Test Agreement which is specified in the file AGREEMENT. http://www.marxmeier.com/eloquence/download/beta/B0810/AGREEMENT Downloading and installing the software indicates your agreement to the Beta Test terms and conditions. This beta release does not meet the release criteria for quality or performance and is only intended for test usage. If it breaks you get to keep the pieces. Introduction ------------ This beta version currently includes a preliminary version of the Eloquence B.08.10 release. Major Eloquence B.08.10 database goals include: - item masking - data encryption Eloquence B.08.10 may be installed in parallel with any previous Eloquence release. Eloquence B.08.10 is installed in the /opt/eloquence/8.1 directory and the configuration files reside in the /etc/opt/eloquence/8.1 directory. Requirements ------------ To use the B.08.10 beta the following requirements must be met: - On HP-UX, the HP OpenSSL software must be installed. It may be obtained from the HP Software Depot web site. https://h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=OPENSSL11I - For HP-UX 11i v1 it is recommended to install the KRNG kernel support for strong random numbers in addition. It is available from the HP Software Depot web site. https://h20392.www2.hp.com/portal/swdepot/displayProductInfo.do?productNumber=KRNG11I - On Linux based systems, OpenSSL 0.9.7 or 0.9.8 must be installed. - Eloquence B.08.10 requires a license key version B.08.10 or above. - To use Eloquence encryption, an additional license key option is required. Installation ------------ The Eloquence B.08.10 test releases are available for download from the following location: HTTP protocol: http://www.marxmeier.com/download/beta/B0810/ FTP protocol: ftp://ftp.marxmeier.com/eloq/beta/B0810/ To install, please follow the installation instructions in the platform specific INSTALL document. B.08.00 compatibility --------------------- Eloquence B.08.10 is upwards compatible with previous Eloquence versions. However, a database created with B.08.10 should not be used with previous Eloquence releases. To revert to B.08.00 the following procedure is required: * Please refer to the compatibility section of item masking functionality on databases created with B.08.10 when downgrading to a previous Eloquence version. * Please refer to the compatibility section of item encryption functionality on databases created with B.08.10 when downgrading to a previous Eloquence version. Documentation ------------- INSTALL-HPUX HP-UX platform specific installation notes INSTALL-LINUX Linux platform specific installation notes eq810_item_access.txt Describes preliminary implementation of item masking eq810_dbkeyutil.txt Describes the dbkeyutil utility to maintain master keys eq810_dbutil.txt Describes the dbutil changes to support data encryption eq810_dbkeyupdate.txt Describes the dbctl dbkeyupdate utility to support retiring data encryption keys Please refer to the Eloquence B.08.00 release notes and reference documentation for additional information. http://eloquence.marxmeier.com/support/B0800/ Summary of enhancements (relative to the initial B.08.00 release) ----------------------------------------------------------------- * All B.08.00 patches (as applicable) are merged to B.08.10 * Add support for item masking * Add support for data encryption Known issues and limitations ---------------------------- The following limitations are known in the current test version: - Applications using the fwutil library are unable to access encrypted content as it is currently not supported to specify a master key. - Installation of the OpenSSL library is required if the encryption license option is present. It is required even if data is not encrypted. - Attempting to open an encrypted database where encryption keys are unavailable fails with status -812 (database encrypted). Opening an encrypted database in mode 8 (readonly) will succeed but encrypted data where the master key is unavailable will be "blanked" and access using encrypted key, search or index items will not work. - An Eloquence B.08.10 release for Windows is not yet available Recent Changes -------------- User visible changes since the RC1 release include: database server - HTTP: Include supplemental audit information in the session status. - A "dbctl encryption status" results in an error message if encryption is not available. - A "dbctl encryption status" no longer outputs a message that database encryption is supported. This is redundant as the command fails otherwise. - Removed dbctl encryption list keys as this is redundant to "dbctl encryption status". dbutil utility - Changed indication of group capabilities in the group list. "A" indicates DADMIN and "P" indicates DBPRIV capability. - Improved error message when restructuring process in the server fails. - Fixed a problem with B.07.10 and B.08.00 backwards compatibility that caused an internal error due to a missing tspw column. - Added CHANGE SET TYPE syntax to allow changing set type between automatic and manual master. - Added CHANGE PATH syntax to allow changing path definition in a detail set. Currently, only the sort item may be changed. dbexport utility - The new -m command line option may be used to specify the DBOPEN mode. By default, DBOPEN mode 9 is used (read-only/shared). - Added a warning message if data is masked or not available due to a missing master key. - Clarified wording of warning message when encountering a corrupted P/Z item value dbinfo utility - The new -m command line option may be used to specify the DBOPEN mode. By default, DBOPEN mode 9 is used (read-only/shared). dbtables utility - Added a fallback to DBOPEN mode 8 when an encrypted database is not available. - Fixed a spelling error in the dbtables output ("PATHES" -> "PATHS") prschema utility - Added a fallback to DBOPEN mode 8 when an encrypted database is not available. dbcfix utility - Fixed a bug when using an encrypted detail set with an unencrypted master. This could result in data corruption. dbkeyutil utility - Fixed duplicate message when revoking a master key. database client library - Changed library revision to "B.08.10.03" - Corrected error message on db status -31 from "illegal mode" to "bad mode" eloqcore - Fixed a problem executing a MERGE or LINK statement in a COMMAND statement. User visible changes since the B4 release include: database server - The server was enhanced to implicitly grant a DBA user administrative capabilities on a database (DADMIN and DBPRIV privileges). As a consequence, the following behavior is changed: Any user with DBA privileges may - purge or erase any database - dbstore or dbrestore any database - perform a dbdumpcat on any database - change the database structure - change the database access rights With previous server versions only the user that created a database (or was granted administrative capabilities subsequently) was able to perform these tasks. Other users, even with DBA privileges would fail. - A DBOPEN of an encrypted database returns status -812:0 if one or more master keys are unavailable. - The DBUPDATE mode 2 (or DBUPDATE using the CIUPDATE flag for image3k) on a master set was changed to return status 43:0 instead of -804:0 when a duplicate key item value is specified. - Added support for user password modification timestamp. The timestamp is updated when the user account is created or the password is updated. With dbdumpcat it is returned in the tspw column of the server sysuser table. When starting the B.08.10 database server for the first time, the server catalog is upgraded to support this column. When the volume set is used with a previous Eloquence version this additional column is ignored. - Fixed a problem where in rare cases the a status -700:-3 was returned when establishing a connection to the db server. - Fixed a race condition deleting semaphores on shutdown. This is harmless but may result in confusing error messages when shutting down a busy server process. dbutil utility - Support hidden entry of user passwords in the user properties dialog. The password needs to be entered twice and both fields need to match. - Added support for user password change timestamp. This is output in the user properties dialog. - Improved checks if attempting to grant privileges exceeding own or changing more privileged users. This complements the server verification and provides better error messages. - Simplified error message when a catalog operation failed - Changed indication of group capabilities in the group list. "D" indicates DADMIN and "G" indicates DBPRIV capability. - Show set permissions in list when maintaining database security (#3678) - Improved on-line help to describe the dbutil syntax - Fixed a bug that could result in aborting dbutil with a segmentation fault after a failed permission check. dbdumpcat utility - dbdumpcat was enhanced to output timestamps as date. By default (when not using the -n option), a timestamp is output as a human readable date (YYYY-MM-DD format). When the -n option is preset, a timestamp is output as a UNIX date value (seconds since epoch). This currently affects the tspw column of the sysuser catalog table and the tskey column of the syskey catalog table. dbcfix utility - The dbcfix utility was changed to use stdout as the default output for results. Previous versions required to specify a log file. - Display or verbose progress is disabled unless a log file is specified. - Fixed bug when specifying set number on command line (instead of a set name). Changes since B3 database server - The dbctl dbkeyupdate command was added to allow retiring data encryption keys. This command is described in detail in the eq810_dbkeyupdate.txt document. - Fixed a race condition during shutdown of a busy server process that could result in error messages like below: semctl(IPC_RMID): Invalid argument (errno 22) During its final shutdown phase the database server process ensures to delete any semaphores it previously allocated. However, some application threads may still be in the process of closing their connection, possibly resulting in an attempt to delete the same semaphore twice, which would result in an error message. This is a cosmetic problem and has no impact. - Fixed a problem with the dbctl killthread command that could fail to wakeup an idle thread on the Linux platform. The Linux kernel does not interrupt a blocked thread when a socket is closed. database utilities - The fwaudit utility does support supplying a master key to access encrypted content. When the -M command line option is present, the EQ_MKEYID and EQ_MKEYFILE environment variables are used to provide master key(s) to access encrypted data. This requires the master key file(s) to be present and a passphrase for each master key id must be entered upon start. EQ_MKEYID specifies a colon separated list of master key id's. Up to 6 master key id's may be present. EQ_MKEYFILE specifies a colon separated list of master key files. Up to 3 key files may be specified. The default key file is eqdb.key. For example: export EQ_MKEYID=test:test2 export EQ_MKEYFILE=test.key fwaudit ... -M ... This would prompt for the passphrases for the listed master keys and gain access to data protected with the master key. - The dbcfix utility does support supplying a master key to access encrypted content. When the -M command line option is present, the EQ_MKEYID and EQ_MKEYFILE environment variables are used to provide master key(s) to access encrypted data. - The dbbexp utility does support supplying a master key to access encrypted content. When the -M command line option is present, the EQ_MKEYID and EQ_MKEYFILE environment variables are used to provide master key(s) to access encrypted data. Please notice that data exported by the dbbexp utility are not encrypted. image3k library - changed library revision to "B.08.10.03" - A compatibility problem with the DBFIND mode 4 when using wildcard search was fixed (#3860). DBFIND mode 4 allows to provide information on the length of its arguments. However, with a wildcard search this only spcifies the max. argument length. A wildcard character may terminate the argument. Previous Eloquence versions used the argument length to determine the search term, possibly allowing multiple wildcard characters. However, this behaviour differs from TurboIMAGE and might result in an unexpected status 17 returned. - A corner case problem with DBFIND mode 1 and 21 was fixed that did result in an internal failure when passing a bad search argument for an index (#3957). According to the TPI specifications angled brackets may be used to enclose the search argument to allow multiple wildcard characters to be used (<< .. >>). Failing to supply closing brackets could trigger an internal consistency test and cause the application process to abort instead of returning the appropriate status code. distribution - The Eloquence WebDLG module for the Apache 1.3 web server is no longer shipped. Changes since B2 dbkeyutil utility - The dbkeyutil was enhanced to support PBKDF2 as recommended by RFC 2898 (PKCS#5 v2.0). Newly created master keys will be incompatible with previous Eloquence beta versions. However, master keys created by previous beta versions are still supported with this version. A new "cipher" config entry is used to describe the password derivation and encryption algorithm. - The dbkeyutil utility will default to using a 1024 bit RSA key to communicate with the db server process. Previous versions used a 2048 bit RSA session key which turned out to be too slow to be practical for older hardware. The dbkeyutil -b command line option may be used to specify a longer RSA session key. - The dbkeyutil "auth" operation was changed to "submit". - Restore terminal echo if dbkeyutil was interrupted while entering a passphrase. dbutil utility - The dbutil utility was enhanced to support deleting the data encryption keys if a database does not use encryption. The following syntax is supported: DELETE ALL ENCRYPTION KEYS; Deleting encryption keys must be performed in a separate session than removing database encryption. - The dbutil utility was enhanced to produce more helpful error messages for crypto related failure causes. The following cases were enhanced: - creating a data encryption key when encryption is not available - creating a data encryption key when the specified master key is not available to the database server - changing the associated master key for a data encryption key when the new or previous master key is not available to the database server. db server - A DBOPEN will no longer fail if a data encryption key is not available (eg. due to a missing master key) if the database does not have encrypted information. - Fixed a problem with timeout handling of the HTTP status if the TCP connection is hung. - Add message log notice when creating, changing or deleting a data encryption key. dbclient library - Resolve binary incompatibility when using the suprtool fastmode. - Changed library revision to "B.08.10.02" image3k library - Fix DBINFO mode 114 returning negative flag values if the data set is writable. - Changed library revision to "B.08.10.02" installation - HP-UX: Create Eloquence specific symbolic link to HP-UX OpenSSL libcrypto (if present) during configure step of the installation procedure. - Fixed a problem where terminal types and map files were not properly installed. Changes since B1 - The dbkeyutil utility uses a secure communication channel to submit the master key to the server process. Depending on the CPU performance this could result in a short delay when submitting a master key to the server while a temporary session key is generated. - The data base server now encrypts indexes on fields marked as encrypted. - The "operator" user property was added. This may be used to indicate user accounts permitted to perform operational tasks. This allows to be more restrictive with administrative accounts. - The DBINFO mode 114 was added to allow obtaining field status. DBINFO mode 114 is similar to DBINFO mode 104 but returns item status information rather than item numbers. DBINFO mode 114 is available in both the image3k and the native client library. However, it is currently not available in eloqcore. The returned status information is bit encoded (per item) as indicated below: bit 0 - set if field is stored on disk in encrypted format bit 1 - set if some encryption key for the database is not available. If this affects actual record, the field is blanked (if a string item) or zeroed. bit 2 - set if an item mask exists for this item bit 3 - set if if an item mask affects information in this field (eg. information is truncated). Bit 0 and and bit 2 may be used by an application to understand a field has sensitive information, so it should be handled with extra care (eg. not included in application logs). Bit 1 and bit 3 may be used to indicate the field content is not available or only partially returned. Please note that DBINFO 114 is considered experimental at the moment and your feedback is appreciated. - Improved server messages on submitting and revoking master keys - The dbkeyutil utility no longer links directly against libcrypto - The dbkeyutil adds a note to the key file when creating a new master key - The dbutil utility was enhanced to support the operator user property (both interactive and in batch use). - The dbutil utility supports changing all data encryption keys of a database to a new master key - The dbutil utility emits a warning message if encrypted fields are used as search items and are not encrypted in a related set - The dbutil utility emits an error message if encryption is used but no data encryption key was created. - The dbutil utility emits a warning message if data encryption keys present but encryption is not used. - A problem in the fwutil library was fixed that could result in an abort due to an alignment problem with encrypted information - Fixed a problem in the server process that could result in a corrupted database structure when upgrading the database catalog. - Fixed a problem in the dbutil utility that could result in memory corruption. Changes since A3 - Provide full Eloquence distribution Changes since A2 - Fixed problem with dbkeyutil chpass command - Fixed problem on HP-UX accessing the OpenSSL library Changes since A1 - Incorporated the most recent B.08.00 fixes - Added new dbkeyutil utility to maintain master keys - Added support for data encryption - Fixed a problem causing database restructuring to fail on databases created with previous Eloquence versions.