| Bookmark Name | Actions |
|---|
RDBMS Conversion
This section provides the appendices pertaining to the RDBMS conversion.
Appendix A- Example RDBMS conversion
Conversion is completed in three stages for additional demonstration.
Stage 1 - Scan VOC and exit COMO rdbmsConversion_1_1169395751.6407 established 16:09:11 21 JAN 2007 16:09:11 21 JAN 2007 T24 RDBMS conversion version 2.0 16:09:11 Options phase Enter number of agents to use for process <default=2> :2 Enter database <XMLORACLE/XMLDB2/XMLMSSQL> <default=J4> :XMLDB2 Convert file data in ../mbdemo.data <Y/N> <default=Y> :Y Convert dict data in ../mbdemo.dict <Y/N> <default=N> :Y Convert jnl/help/arc files <Y/N> <default=N> :Y Compare new database with original <Y/N> <default=N> :Y Convert stub files to voc table entries <Y/N> <default=N> : Options selected, Continue to scan phase Y/N <default=Y> : 16:09:40 Create work files Create conversion directory .\rdbmsConversion Created for 'dict' file voc entries .\rdbmsConversion\DictFiles Created for 'data' file voc entries .\rdbmsConversion\DataFiles Created for 'other' file voc entries .\rdbmsConversion\OtherFiles Created for invalid file voc entries .\rdbmsConversion\InvalidFiles Created for part file voc entries .\rdbmsConversion\PartFiles Created for duplicate voc entries .\rdbmsConversion\DuplicateFiles Created for preservation voc entries .\rdbmsConversion\eldVOC Created for new voc entries .\rdbmsVOC Created to control conversion processes .\rdbmsConversion\Control Created to log conversion errors .\rdbmsConversion\Errors Created to log conversion data errors .\rdbmsConversion\DataErrors Created to confirm 'dir' type conversion .\rdbmsConversion\ConvertedDir Created to confirm 'dict' type conversion .\rdbmsConversion\ConvertedDict Created to confirm 'data' type conversion .\rdbmsConversion\ConvertedData Created to confirm 'other' type conversion .\rdbmsConversion\ConvertedOther Created to confirm dict path conversions .\rdbmsConversion\DictNames Created to retain original company .\rdbmsConversion\EldCompanyFiles Created to confirm new company .\rdbmsConversion\NewCompanyFiles Created to log comparison completed .\rdbmsConversion\CompareFiles Created to log comparison errors .\rdbmsConversion\CompareErrors Created to log voc conversion errors .\rdbmsConversion\VocErrors Created to record converted voc entries .\rdbmsConversion\ConvertedVoc Created to preserve stubfile .\rdbmsConversion\StubFiles 16:09:47 Scan phase Processing entries in voc file 'E:\T24R6\tinymbdemo\mbdemo.run\VOC' . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1000 entries 00:00:02 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16:12:55 VOC Scan complete... Processed 24673 voc entries (origVOC), 13137 retained (rdbmsVOC), 11304 deprecated Detected 248 invalid (InvalidFiles), 8 duplicates (DuplicateFiles) Scanned 22242 voc files, 12497 dict (DictFiles), 9720 data (DataFiles) Jnl/Help 25 (OtherFiles), 17 (DataOnly), 2810 (DictOnly) PartFiles 0 (PartFiles) Elapsed scan time 00:03:08 Use "LIST .\rdbmsConversion\InvalidFiles F1 F2 F3" for invalid files Use "LIST .\rdbmsConversion\DuplicateFiles F1 F2 F3" for duplicates files Encountered errors in scan, continue with next phase <Y/N> :y Must resolve Duplicated voc files before continuing conversion Use "LIST .\rdbmsConversion\DuplicateFiles F1 F2 F3" to see duplicated files Encountered errors in scan, continue with next phase <Y/N> :n COMO completed 16:16:44 21 JAN 2007 Stage 2 - Skip VOC Rescan and Convert Data, Dict and Other then Compare COMO rdbmsConversion_1_1169396214.8968 established 16:16:54 21 JAN 2007 16:16:54 21 JAN 2007 T24 RDBMS conversion version 2.0 RDBMS conversion option RDBMS_ALLOW_DUPLICATES configured 16:16:54 Options phase Enter number of agents to use for process <default=2> :2 Enter database <XMLORACLE/XMLDB2/XMLMSSQL> <default=J4> :XMLDB2 Convert file data in ../mbdemo.data <Y/N> <default=Y> :Y Convert dict data in ../mbdemo.dict <Y/N> <default=N> :Y Convert jnl/help/arc files <Y/N> <default=N> :Y Compare new database with original <Y/N> <default=N> :N Convert stub files to voc table entries <Y/N> <default=N> :N Rescan original voc <Y/N> <default=N> :N Options selected, Continue to scan phase Y/N <default=Y> :Y 16:17:09 Create work files ….. 16:17:12 Scan phase Skip scan of voc file 'E:\T24R6\tinymbdemo\mbdemo.run\VOC' Scan file .\rdbmsConversion\DataFiles contains 9720 entries Scan file .\rdbmsConversion\DictFiles contains 12497 entries Scan file .\rdbmsConversion\OtherFiles contains 25 entries Continuing =============================================================================== 16:17:12 Data Conversion phase 16:17:13 Starting 2 conversion agents for data file conversion of 9720 files 16:17:23 Conv Data 5 of 9720, 0 errors, total elapsed 00:00:10 16:18:24 Conv Data 73 of 9720, 0 errors, total elapsed 00:01:10 16:19:24 Conv Data 175 of 9720, 0 errors, total elapsed 00:02:10 16:20:24 Conv Data 239 of 9720, 0 errors, total elapsed 00:03:10 ……… 18:01:35 Conv Data 9597 of 9720, 7 errors, total elapsed 01:44:21 18:02:30 Agents 1 completed, 0 appears inactive, 1 still in progress 18:02:35 Conv Data 9713 of 9720, 7 errors, total elapsed 01:45:21 18:02:35 Agents 2 completed, 0 appears inactive, 0 still in progress 18:02:35 Starting 2 conversion agents for dict file conversion of 12497 files 18:02:47 Conv Dict 7 of 12497, 0 errors, total elapsed 00:00:10 ………… 18:33:49 Conv Dict 12497 of 12497, 0 errors, total elapsed 00:31:12 18:33:49 Agents 2 completed, 0 appears inactive, 0 still in progress 18:33:49 Starting 2 conversion agents for other file conversion of 25 files 18:34:02 Conv Other 13 of 25, 0 errors, total elapsed 00:00:10 18:35:02 Conv Other 21 of 25, 0 errors, total elapsed 00:01:10 18:35:12 Agents 2 completed, 0 appears inactive, 0 still in progress Converted 12497 dict files, 9738 data files Converted 5 directories, 290924 records Detected 7 errors, total elapsed 02:17:59 Use "LIST .\rdbmsConversion\Errors F1 F2" for conversion errors Use "LIST .\rdbmsConversion\DataErrors F1 F2" for conversion anomalies Errors/anomalies noted in conversion, continue with next phase <Y/N> :y =============================================================================== 18:40:27 Data Comparison phase 18:40:28 Starting 2 comparison agents for data file compare of 9713 files 18:40:38 Comp Data 60 of 9713, 0 errors, total elapsed 00:00:10 ……… 18:57:40 Comp Data 9506 of 9713, 0 errors, total elapsed 00:17:11 18:57:55 Agents 2 completed, 0 appears inactive, 0 still in progress 18:57:56 Starting 2 comparison agents for dict file compare of 12497 files 18:58:11 Comp Dict 25 of 12497, 0 errors, total elapsed 00:00:10 …….. 19:20:13 Comp Dict 12376 of 12497, 0 errors, total elapsed 00:22:12 19:20:23 Agents 2 completed, 0 appears inactive, 0 still in progress 19:20:24 Starting 2 comparison agents for other file compare of 25 files 19:20:37 Comp Other 2 of 25, 0 errors, total elapsed 00:00:10 19:21:07 Comp Other 2 of 25, 0 errors, total elapsed 00:00:40 19:21:27 Agents 2 completed, 0 appears inactive, 0 still in progress Compared 12497 dict files, 9738 data files, 0 errors, total elapsed 00:41:06 Use "LIST .\rdbmsConversion\CompareErrors F1 F2" for compare errors Errors/anomalies noted in comparison, continue with next phase <Y/N> :y =============================================================================== 23:07:36 Procedures complete Converted 12497 dict, 9738 data files, 5 directories, 290924 records Detected 7 errors, elasped time 02:17:59 Converted total of 22235 files Compared 12497 dict, 9738 data files, 802911 records Detected 0 errors, elapsed time 00:41:06 Compared total of 22235 files Use the script '.\rdbmsConversion\rdbmsMove.bat' to reconfigure area which will preserve originals and replace with converted directories Process can be reversed by '.\rdbmsConversion\rdbmsRevert.bat' script 23:07:36 Process completed, total elasped 02:59:05 COMO completed 23:07:36 21 JAN 2007 Stage 3 - Convert STUBFILE files to VOC table entries and remove STUBFILE files COMO rdbmsConversion_1_1169421232.8709 established 23:13:53 21 JAN 2007 23:13:53 21 JAN 2007 T24 RDBMS conversion version 2.0 RDBMS conversion option RDBMS_ALLOW_DUPLICATES configured 23:13:53 Options phase Enter number of agents to use for process <default=2> :2 Enter database <XMLORACLE/XMLDB2/XMLMSSQL> <default=J4> :XMLDB2 Convert file data in ../mbdemo.data <Y/N> <default=Y> :N Convert dict data in ../mbdemo.dict <Y/N> <default=N> :N Convert jnl/help/arc files <Y/N> <default=N> :N Compare new database with original <Y/N> <default=N> :N Convert stub files to voc table entries <Y/N> <default=N> :Y Rescan original voc <Y/N> <default=N> :N Options selected, Continue to scan phase Y/N <default=Y> :Y 23:14:10 Create work files Create conversion directory .\rdbmsConversion …… 23:14:15 Scan phase Skip scan of voc file 'E:\T24R6\tinymbdemo\mbdemo.run\VOC' Scan file .\rdbmsConversion\DataFiles contains 7 entries Scan file .\rdbmsConversion\DictFiles contains 0 entries Scan file .\rdbmsConversion\OtherFiles contains 0 entries Continuing =============================================================================== 23:14:16 Data Conversion phase No files selected for conversion Continuing =============================================================================== 23:14:17 VOC Conversion phase 23:14:18 Starting 2 voc conversion agents for 9713 converted data files 23:14:28 Voc Data 99 of 9713, 0 errors, total elapsed 00:00:10 23:14:43 Voc Data 399 of 9713, 0 errors, total elapsed 00:00:25 23:14:58 Voc Data 599 of 9713, 0 errors, total elapsed 00:00:40 23:15:13 Voc Data 999 of 9713, 0 errors, total elapsed 00:00:55 23:15:28 Voc Data 1499 of 9713, 0 errors, total elapsed 00:01:10 23:15:43 Voc Data 2399 of 9713, 0 errors, total elapsed 00:01:25 23:15:58 Voc Data 3599 of 9713, 0 errors, total elapsed 00:01:40 23:16:13 Voc Data 6799 of 9713, 0 errors, total elapsed 00:01:55 23:16:18 Agents 2 completed, 0 appears inactive, 0 still in progress 23:16:19 Starting 2 voc conversion agents for 12497 converted dict files 23:16:29 Voc Dict 1599 of 12497, 0 errors, total elapsed 00:00:10 23:16:39 Agents 2 completed, 0 appears inactive, 0 still in progress 23:16:39 Starting 2 voc conversion agents for 25 converted other files 23:16:49 Voc Other 25 of 25, 0 errors, total elapsed 00:00:10 23:16:49 Agents 2 completed, 0 appears inactive, 0 still in progress 23:16:49 Starting 2 voc conversion agents for 7 scanned data files 23:17:00 Voc Data 0 of 7, 0 errors, total elapsed 00:00:10 23:17:00 Agents 2 completed, 0 appears inactive, 0 still in progress Skip scanned 'data' voc conversion, no entries Skip scanned 'other' voc conversion, no entries Skip scanned 'duplicate' data voc conversion, no entries Skip scanned 'duplicate' dict voc conversion, no entries VOC Convert 12497 dict entries, 9738 data entries, 0 errors, total elapsed 00:02:43 Continuing to stub removal phase 23:17:00 Removing StubFiles Processed 2000 stubfiles in 00:00:00 Processed 4000 stubfiles in 00:00:00 Processed 10000 stubfiles in 00:00:02 Processed 14000 stubfiles in 00:00:01 Processed 18000 stubfiles in 00:00:00 Processed 20000 stubfiles in 00:00:00 Processed 22000 stubfiles in 00:00:00 Processed 24000 stubfiles in 00:00:00 Processed 30000 stubfiles in 00:00:01 Processed 35881 stubfiles, removed 13646, total elapsed 00:00:14 Stub files can be regenerated if required using 'rdbmsRestoreStubs' command =============================================================================== 23:17:15 Procedures complete Converted voc entries 12497 dict, 9738 data files Detected 0 errors, elapsed time 00:02:43 Converted total of 22235 voc entries Use the script '.\rdbmsConversion\rdbmsMove.bat' to reconfigure area which will preserve originals and replace with converted directories Process can be reversed by '.\rdbmsConversion\rdbmsRevert.bat' script 23:17:15 Process completed, total elasped 00:02:57 COMO completed 23:17:15 21 JAN 2007
Appendix B- Format of VOC table entry
The following table lists the format of a VOC entry for replacement of STUBFILE file information for FBNK.ACCOUNT.
|
Field Number |
VoC Entry |
Description |
|---|---|---|
|
001 |
TABLE |
VOC TABLE entry identifier |
|
002 |
XMLORACLEInit |
Database Driver Initialisation Function |
|
003 |
D_F_ACCOUNT NOXMLSCHEMA |
Dictionary table argument for database driver |
|
004 |
acFBNK_ACCOUNT |
Data table arguments for database driver |
The format of the VOC TABLE entry also allows either the DICT or DATA definition to be a file path (absolute or relative) in which case the reference will operate similarly to the F pointer type.
The pseudo Inode value to be used for the record lock mechanism is derived from the Hash value of the following.
-
D_F_ACCOUNT Dictionary table, for a record lock taken on the Dictionary
-
acFBNK_ACCOUNT Data table, for a record lock taken on the Data table
The pseudo device value for the Dictionary or Data table will be the Hash value of the XMLORACLEInit Database Driver Initialisation Function. If the JBASE_DATABASE environment variable configured, the device number will be the Hash value of the specified string.
The VOC TABLE entry also allows the paths (absolute or relative) to be specified so that the usage of hash file versions of dictionaries can be continued, if required.
003 ../bnk.dict/F.ACCOUNT]D - Path to dictionary hash file
Appendix C- Database Schema and Table space creation
This section provides the database schema and table space creation details for Oracle RDBMS.
SQL> -- Create T24 User SQL> create user t24 identified by t24 2 default tablespace T24DATA 3 quota unlimited on T24DATA 4 quota unlimited on T24INDEX; User created. SQL> grant connect to t24; SQL> grant resource to t24; SQL> grant query rewrite to t24; SQL> grant create synonym to t24; SQL> grant drop any procedure to t24; SQL> grant create any table to t24; SQL> grant select any table to t24; SQL> grant update any table to t24; SQL> grant insert any table to t24; SQL> grant delete any table to t24; SQL> grant drop any table to t24; SQL> grant create any index to t24; SQL> grant alter any index to t24; SQL> grant drop any index to t24; SQL> grant create any view to t24; SQL> grant drop any view to t24; SQL> grant select_catalog_role to t24; SQL> revoke unlimited tablespace from t24;
SQL> -- Create FBNK User SQL> create user fbnk identified by fbnk 2 default tablespace FBNKDATA 3 quota unlimited on FBNKDATA 4 quota unlimited on FBNKINDEX; SQL> grant connect to fbnk; SQL> grant resource to fbnk; SQL> grant query rewrite to fbnk; SQL> grant create synonym to fbnk; SQL> grant drop any procedure to fbnk; SQL> grant create any table to fbnk; SQL> grant select any table to fbnk; SQL> grant update any table to fbnk; SQL> grant insert any table to fbnk; SQL> grant delete any table to fbnk; SQL> grant drop any table to fbnk; SQL> grant create any index to fbnk; SQL> grant alter any index to fbnk; SQL> grant drop any index to fbnk; SQL> grant create any view to fbnk; SQL> grant drop any view to fbnk; SQL> grant select_catalog_role to fbnk; SQL> revoke unlimited tablespace from fbnk;
SQL> CREATE TABLESPACE T24DATA LOGGING DATAFILE 2 '/home/oracle/T24/R6/T24DATA1.dbf' SIZE 1000M AUTOEXTEND ON NEXT 50M MAXSIZE UNLIMITED 3 EXTENT MANAGEMENT LOCAL 4 SEGMENT SPACE MANAGEMENT AUTO; Tablespace created. SQL> CREATE TABLESPACE T24INDEX LOGGING DATAFILE 2 '/home/oracle/T24/R6/T24INDEX1.dbf' SIZE 300M AUTOEXTEND ON NEXT 50M MAXSIZE UNLIMITED 3 EXTENT MANAGEMENT LOCAL 4 SEGMENT SPACE MANAGEMENT AUTO; Tablespace created.
SQL> CREATE TABLESPACE FBNKDATA LOGGING DATAFILE 2 '/home/oracle/T24/R6/FBNKDATA1.dbf' SIZE 1400M AUTOEXTEND ON NEXT 50M MAXSIZE UNLIMITED 3 EXTENT MANAGEMENT LOCAL 4 SEGMENT SPACE MANAGEMENT AUTO; Tablespace created. SQL> CREATE TABLESPACE FBNKINDEX LOGGING DATAFILE 2 '/home/oracle/T24/R6/FBNKINDEX1.dbf' SIZE 800M AUTOEXTEND ON NEXT 50M MAXSIZE UNLIMITED 3 EXTENT MANAGEMENT LOCAL 4 SEGMENT SPACE MANAGEMENT AUTO; Tablespace created.
Appendix D – Contents of rdbmsConversion subdirectory
Created for 'dict' file voc entries .\rdbmsConversion\DictFiles Created for 'data' file voc entries .\rdbmsConversion\DataFiles Created for 'other' file voc entries .\rdbmsConversion\OtherFiles Created for invalid file voc entries .\rdbmsConversion\InvalidFiles Created for part file voc entries .\rdbmsConversion\PartFiles Created for duplicate voc entries .\rdbmsConversion\DuplicateFiles Created for preservation voc entries .\rdbmsConversion\eldVOC Created to control conversion processes .\rdbmsConversion\Control Created to log conversion errors .\rdbmsConversion\Errors Created to log conversion data errors .\rdbmsConversion\DataErrors Created to confirm 'dir' type conversion .\rdbmsConversion\ConvertedDir Created to confirm 'dict' type conversion .\rdbmsConversion\ConvertedDict Created to confirm 'data' type conversion .\rdbmsConversion\ConvertedData Created to confirm 'other' type conversion .\rdbmsConversion\ConvertedOther Created to confirm dict path conversions .\rdbmsConversion\DictNames Created to retain original company .\rdbmsConversion\EldCompanyFiles Created to confirm new company .\rdbmsConversion\NewCompanyFiles Created to log comparison completed .\rdbmsConversion\CompareFiles Created to log comparison errors .\rdbmsConversion\CompareErrors Created to log voc conversion errors .\rdbmsConversion\VocErrors Created to record converted voc entries .\rdbmsConversion\ConvertedVoc Created to preserve stubfile .\rdbmsConversion\StubFiles
Appendix E-Troubleshooting Techniques
The conversion process will record the problems (if any) with the files processed. The rdbmsConversion directory contains various files, which record the progress and outcome of the conversion. You need to check InvalidFiles, DuplicateFiles, Errors and DataErrors to check for potential errors. Entries will have the ID of the VOC entry name and the associated record will have the details of the failure. Also, verify the <RDBMS>driver.log file for potential reasons for a transfer failure.
The rdbmsConversion/Errors records list the files, which had conversion failure. You need to examine this record and DC Driver log file to determine the reason for failure.
The files listed in rdbmsConversion/DataErrors have been converted to an RDBMS table. However, some illegal characters are found during conversion and have been removed. You can use CT DataErrors <filename> to the location of each character found. Most of these are 09 and 0D characters although other characters also occur. Such characters will normally be removed, but these characters are technically legal in XML. These characters need to be reviewed for their usage in those files and decide whether or not they can be modified to not use them. There is now an override option, which allows these characters.
The following table lists few examples for the errors.
|
FILENAME |
No. Errors |
Example of error |
|---|---|---|
|
F.DE.O.MSG.SWIFT |
480 |
bad record D20071127000336230800.1, char 0D at offset 69 |
|
F.DB.DPK.ENTRIES.CE |
34 |
bad record 14739000286170,, char 09 at offset 116 |
|
F.DB.DPK.ENTRIES.RE |
286 |
bad record 147390003361827.61, char 09 at offset 117 |
The following command is used to re-run conversion for an individual file.
rdbmsConvert -v rdbmsConversion/Datafiles <vocName>
For example, rdbmsConvert -v rdbmsConversion/Datafiles FBNK.ACCOUNT
This works only if the rdbmsConversion has already completed at least the first phase. You need to ensure any record for the file is removed from the Errors file (if present) using cd rdbmsConversion and DELETE Errors FBNK.ACCOUNT. To re-convert the files converted already, you need to copy the file record from ConvertedData (or ConvertedDict and so on) to DataFiles (or DictFiles and so on) using the following commands.
-
cd rdbmsConversion
-
JCOPY from ConvertedData to DataFiles FBNK.ACCOUNT
-
DELETE ConvertedData FBNK.ACCOUNT
Appendix F - Conversions Requiring Codepage Translation
If your source data is not stored in UTF-8 format, then the rdbmsConversion need to translate it prior to copying it to the RDBMS database, for which you need to specify the corresponding codepage encoding of your data. This situation occurs most frequently for users that use the Latin alphabet but make use of the extended ASCII range to encode accented or special characters.
It is mandatory to map the correct codepage to ensure correct translation.
Procedure
-
Start up your Terminal Emulator and login to a Temenos Transact account
-
Start Temenos Transact and ensure a sample of records from a number of applications are displayed correctly, especially locale specific characters
-
Check the translation settings of your terminal emulator. The codepage below is ISO-8859-1
The JUTF8 utility can be used to perform a conversion outside of the rdbmsConversion process to verify the codepage you have is correct.
-
CREATE-FILE DATA TESTFILE 1
-
JED TESTFILE T1
-
Insert text (including special or accented characters specific to your codepage)
-
File the record
-
Enable jBASE UTF-8 support
-
export JBASE_I18N=1
-
export JBASE_CODEPAGE=utf8
-
export JBASE_LOCALE=en_US
-
-
Perform UTF-8 translation using JUTF8 –c < codepage> TESTFILE
-
Change translation settings of Terminal Emulator to display UTF-8
-
Redisplay the text and ensure it displays exactly like the original text
-
JED TESTFILE T1
-
This codepage information can now be used by the rdbmsConversion by setting
-
export RDBMS_FORCE_UTF8_CONVERSION_CODEPAGE=<codepage>
-
export RDBMS_FORCE_UTF8_CONVERSION=1
Some Temenos Transact files contain binary data fields, that is F.SPF.CHECK and F.USER. The Temenos Transact utility CONVERT.SYSTEM.UTF8.JBASE (run prior to rdbmsConversion) encodes these binary fields using codepage latin1. If the rdbmsConversion performs another UTF-8 conversion on these records, the data in these binary fields is rendered unreadable by Temenos Transact.
This file contains only binary data. Codepage conversion should be skipped by adding a VOC item named CODEPAGE_EXEMPT.
Example
$ ED VOC CODEPAGE_EXEMPTFILES New record CODEPAGE_EXEMPTFILES TOP .i 001+F.SPF.CHECK 002+ BOTTOM .fi Record 'CODEPAGE_EXEMPTFILES' written to file 'VOC'
F.USER contains both binary fields (encoded using codepage latin1) and customer data requiring translation to UTF-8 using a different codepage. The conversion must translate these records for the customer data to get displayed correctly. Unfortunately, this results in binary password fields being double encoded and unusable by Temenos Transact.
You can do either of the following.
-
Clear the password fields in F.USER. Users will be prompted to re-enter their passwords when they sign in.
-
Write a small program to copy only the binary fields from the original F.USER to the new version (created by the conversion process).
Add Bookmark
save your best linksView Bookmarks
Visit your best linksIn this topic
Are you sure you want to log-off?