RDBMS Conversion

This section provides the appendices pertaining to the RDBMS conversion.

Appendix A- Example RDBMS conversion

Conversion is completed in three stages for additional demonstration.

Stage 1                                  - Scan VOC and exit
COMO rdbmsConversion_1_1169395751.6407 established 16:09:11  21 JAN 2007
16:09:11 21 JAN 2007 T24 RDBMS conversion version 2.0
16:09:11 Options phase
Enter number of agents to use for process <default=2>                             :2
Enter database <XMLORACLE/XMLDB2/XMLMSSQL> <default=J4>                :XMLDB2
Convert file data in ../mbdemo.data <Y/N> <default=Y>                            :Y
Convert dict data in ../mbdemo.dict <Y/N> <default=N>                            :Y
Convert jnl/help/arc files <Y/N> <default=N>                                           :Y
Compare new database with original <Y/N> <default=N>                            :Y
Convert stub files to voc table entries <Y/N> <default=N>                         :
Options selected, Continue to scan phase Y/N <default=Y>                        :
16:09:40 Create work files
Create conversion directory .\rdbmsConversion
Created for 'dict' file voc entries  .\rdbmsConversion\DictFiles
Created for 'data' file voc entries  .\rdbmsConversion\DataFiles
Created for 'other' file voc entries .\rdbmsConversion\OtherFiles
Created for invalid file voc entries .\rdbmsConversion\InvalidFiles
Created for part file voc entries    .\rdbmsConversion\PartFiles
Created for duplicate voc entries    .\rdbmsConversion\DuplicateFiles
Created for preservation voc entries .\rdbmsConversion\eldVOC
Created for new voc entries          .\rdbmsVOC
Created to control conversion processes    .\rdbmsConversion\Control
Created to log conversion errors           .\rdbmsConversion\Errors
Created to log conversion data errors      .\rdbmsConversion\DataErrors
Created to confirm 'dir' type conversion   .\rdbmsConversion\ConvertedDir
Created to confirm 'dict' type conversion  .\rdbmsConversion\ConvertedDict
Created to confirm 'data' type conversion  .\rdbmsConversion\ConvertedData
Created to confirm 'other' type conversion .\rdbmsConversion\ConvertedOther
Created to confirm dict path conversions   .\rdbmsConversion\DictNames
Created to retain original company         .\rdbmsConversion\EldCompanyFiles
Created to confirm new company             .\rdbmsConversion\NewCompanyFiles
Created to log comparison completed        .\rdbmsConversion\CompareFiles
Created to log comparison errors           .\rdbmsConversion\CompareErrors
Created to log voc conversion errors       .\rdbmsConversion\VocErrors
Created to record converted voc entries    .\rdbmsConversion\ConvertedVoc
Created to preserve stubfile               .\rdbmsConversion\StubFiles
16:09:47 Scan phase
Processing entries in voc file 'E:\T24R6\tinymbdemo\mbdemo.run\VOC'
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .  1000 entries 00:00:02
. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
16:12:55 VOC Scan complete...
Processed 24673 voc entries (origVOC), 13137 retained (rdbmsVOC), 11304 deprecated
Detected  248 invalid (InvalidFiles), 8 duplicates (DuplicateFiles)
Scanned   22242 voc files, 12497 dict (DictFiles), 9720 data (DataFiles)
Jnl/Help  25 (OtherFiles), 17 (DataOnly), 2810 (DictOnly)
PartFiles 0 (PartFiles)
Elapsed scan time 00:03:08 
Use "LIST .\rdbmsConversion\InvalidFiles F1 F2 F3" for invalid files
Use "LIST .\rdbmsConversion\DuplicateFiles F1 F2 F3" for duplicates files
Encountered errors in scan, continue with next phase <Y/N>                    :y
Must resolve Duplicated voc files before continuing conversion
Use "LIST .\rdbmsConversion\DuplicateFiles F1 F2 F3" to see duplicated files
Encountered errors in scan, continue with next phase <Y/N>                    :n
COMO completed 16:16:44  21 JAN 2007
Stage 2                      -           Skip VOC Rescan and Convert Data, Dict and Other then Compare
COMO rdbmsConversion_1_1169396214.8968 established 16:16:54  21 JAN 2007
16:16:54 21 JAN 2007 T24 RDBMS conversion version 2.0
RDBMS conversion option RDBMS_ALLOW_DUPLICATES configured
16:16:54 Options phase
Enter number of agents to use for process <default=2>                             :2
Enter database <XMLORACLE/XMLDB2/XMLMSSQL> <default=J4>                 :XMLDB2
Convert file data in ../mbdemo.data <Y/N> <default=Y>                             :Y
Convert dict data in ../mbdemo.dict <Y/N> <default=N>                             :Y
Convert jnl/help/arc files <Y/N> <default=N>                                            :Y
Compare new database with original <Y/N> <default=N>                             :N
Convert stub files to voc table entries <Y/N> <default=N>                          :N
Rescan original voc <Y/N> <default=N>                                                   :N
Options selected, Continue to scan phase Y/N <default=Y>                        :Y
16:17:09 Create work files
…..
16:17:12 Scan phase
Skip scan of voc file 'E:\T24R6\tinymbdemo\mbdemo.run\VOC'
Scan file .\rdbmsConversion\DataFiles contains 9720 entries
Scan file .\rdbmsConversion\DictFiles contains 12497 entries
Scan file .\rdbmsConversion\OtherFiles contains 25 entries
Continuing
===============================================================================
16:17:12 Data Conversion phase
16:17:13 Starting 2 conversion agents for data file conversion of 9720 files
16:17:23 Conv Data        5 of     9720,        0 errors, total elapsed 00:00:10
16:18:24 Conv Data       73 of     9720,        0 errors, total elapsed 00:01:10
16:19:24 Conv Data      175 of     9720,        0 errors, total elapsed 00:02:10
16:20:24 Conv Data      239 of     9720,        0 errors, total elapsed 00:03:10
………
18:01:35 Conv Data     9597 of     9720,        7 errors, total elapsed 01:44:21
18:02:30 Agents 1 completed, 0 appears inactive, 1 still in progress
18:02:35 Conv Data     9713 of     9720,        7 errors, total elapsed 01:45:21
18:02:35 Agents 2 completed, 0 appears inactive, 0 still in progress
18:02:35 Starting 2 conversion agents for dict file conversion of 12497 files
18:02:47 Conv Dict        7 of    12497,        0 errors, total elapsed 00:00:10
…………
18:33:49 Conv Dict    12497 of    12497,        0 errors, total elapsed 00:31:12
18:33:49 Agents 2 completed, 0 appears inactive, 0 still in progress
18:33:49 Starting 2 conversion agents for other file conversion of 25 files
18:34:02 Conv Other       13 of       25,        0 errors, total elapsed 00:00:10
18:35:02 Conv Other       21 of       25,        0 errors, total elapsed 00:01:10
18:35:12 Agents 2 completed, 0 appears inactive, 0 still in progress
Converted 12497 dict files, 9738 data files
Converted 5 directories, 290924 records
Detected  7 errors, total elapsed 02:17:59
Use "LIST .\rdbmsConversion\Errors F1 F2" for conversion errors
Use "LIST .\rdbmsConversion\DataErrors F1 F2" for conversion anomalies
Errors/anomalies noted in conversion, continue with next phase <Y/N>          :y
===============================================================================
18:40:27 Data Comparison phase
18:40:28 Starting 2 comparison agents for data file compare of 9713 files
18:40:38 Comp Data       60 of     9713,        0 errors, total elapsed 00:00:10
………
18:57:40 Comp Data     9506 of     9713,        0 errors, total elapsed 00:17:11
18:57:55 Agents 2 completed, 0 appears inactive, 0 still in progress
18:57:56 Starting 2 comparison agents for dict file compare of 12497 files
18:58:11 Comp Dict       25 of    12497,        0 errors, total elapsed 00:00:10
……..
19:20:13 Comp Dict    12376 of    12497,        0 errors, total elapsed 00:22:12
19:20:23 Agents 2 completed, 0 appears inactive, 0 still in progress
19:20:24 Starting 2 comparison agents for other file compare of 25 files
19:20:37 Comp Other        2 of       25,        0 errors, total elapsed 00:00:10
19:21:07 Comp Other        2 of       25,        0 errors, total elapsed 00:00:40
19:21:27 Agents 2 completed, 0 appears inactive, 0 still in progress
Compared 12497 dict files, 9738 data files, 0 errors, total elapsed 00:41:06
Use "LIST .\rdbmsConversion\CompareErrors F1 F2" for compare errors
Errors/anomalies noted in comparison, continue with next phase <Y/N>          :y
===============================================================================
23:07:36 Procedures complete
Converted 12497 dict, 9738 data files, 5 directories, 290924 records
Detected  7 errors, elasped time 02:17:59
Converted total of 22235 files
Compared  12497 dict, 9738 data files, 802911 records
Detected  0 errors, elapsed time 00:41:06
Compared total of 22235 files
Use the script '.\rdbmsConversion\rdbmsMove.bat' to reconfigure area
which will preserve originals and replace with converted directories
Process can be reversed by '.\rdbmsConversion\rdbmsRevert.bat' script
23:07:36 Process completed, total elasped 02:59:05
COMO completed 23:07:36  21 JAN 2007
Stage 3                      -           Convert STUBFILE files to VOC table entries and remove STUBFILE files
COMO rdbmsConversion_1_1169421232.8709 established 23:13:53  21 JAN 2007
23:13:53 21 JAN 2007 T24 RDBMS conversion version 2.0
RDBMS conversion option RDBMS_ALLOW_DUPLICATES configured
23:13:53 Options phase
Enter number of agents to use for process <default=2>                              :2
Enter database <XMLORACLE/XMLDB2/XMLMSSQL> <default=J4>                 :XMLDB2
Convert file data in ../mbdemo.data <Y/N> <default=Y>                             :N
Convert dict data in ../mbdemo.dict <Y/N> <default=N>                             :N
Convert jnl/help/arc files <Y/N> <default=N>                                            :N
Compare new database with original <Y/N> <default=N>                             :N
Convert stub files to voc table entries <Y/N> <default=N>                           :Y
Rescan original voc <Y/N> <default=N>                                                    :N
Options selected, Continue to scan phase Y/N <default=Y>                          :Y
23:14:10 Create work files
Create conversion directory .\rdbmsConversion
…… 
23:14:15 Scan phase
Skip scan of voc file 'E:\T24R6\tinymbdemo\mbdemo.run\VOC'
Scan file .\rdbmsConversion\DataFiles contains 7 entries
Scan file .\rdbmsConversion\DictFiles contains 0 entries
Scan file .\rdbmsConversion\OtherFiles contains 0 entries
Continuing
===============================================================================
23:14:16 Data Conversion phase
No files selected for conversion
Continuing
===============================================================================
23:14:17 VOC Conversion phase
23:14:18 Starting 2 voc conversion agents for 9713 converted data files
23:14:28 Voc Data       99 of     9713,        0 errors, total elapsed 00:00:10
23:14:43 Voc Data      399 of     9713,        0 errors, total elapsed 00:00:25
23:14:58 Voc Data      599 of     9713,        0 errors, total elapsed 00:00:40
23:15:13 Voc Data      999 of     9713,        0 errors, total elapsed 00:00:55
23:15:28 Voc Data     1499 of     9713,        0 errors, total elapsed 00:01:10
23:15:43 Voc Data     2399 of     9713,        0 errors, total elapsed 00:01:25
23:15:58 Voc Data     3599 of     9713,        0 errors, total elapsed 00:01:40
23:16:13 Voc Data     6799 of     9713,        0 errors, total elapsed 00:01:55
23:16:18 Agents 2 completed, 0 appears inactive, 0 still in progress
23:16:19 Starting 2 voc conversion agents for 12497 converted dict files
23:16:29 Voc Dict     1599 of    12497,        0 errors, total elapsed 00:00:10
23:16:39 Agents 2 completed, 0 appears inactive, 0 still in progress
23:16:39 Starting 2 voc conversion agents for 25 converted other files
23:16:49 Voc Other       25 of       25,        0 errors, total elapsed 00:00:10
23:16:49 Agents 2 completed, 0 appears inactive, 0 still in progress
23:16:49 Starting 2 voc conversion agents for 7 scanned data files
23:17:00 Voc Data        0 of        7,        0 errors, total elapsed 00:00:10
23:17:00 Agents 2 completed, 0 appears inactive, 0 still in progress
Skip scanned 'data' voc conversion, no entries
Skip scanned 'other' voc conversion, no entries
Skip scanned 'duplicate' data voc conversion, no entries
Skip scanned 'duplicate' dict voc conversion, no entries
VOC Convert 12497 dict entries, 9738 data entries, 0 errors, total elapsed 00:02:43
Continuing to stub removal phase
23:17:00 Removing StubFiles
 Processed 2000 stubfiles in 00:00:00
 Processed 4000 stubfiles in 00:00:00
 Processed 10000 stubfiles in 00:00:02
 Processed 14000 stubfiles in 00:00:01
 Processed 18000 stubfiles in 00:00:00
 Processed 20000 stubfiles in 00:00:00
 Processed 22000 stubfiles in 00:00:00
 Processed 24000 stubfiles in 00:00:00
 Processed 30000 stubfiles in 00:00:01
Processed 35881 stubfiles, removed 13646, total elapsed 00:00:14
Stub files can be regenerated if required using 'rdbmsRestoreStubs' command
===============================================================================
23:17:15 Procedures complete
Converted voc entries 12497 dict, 9738 data files
Detected    0 errors, elapsed time 00:02:43
Converted total of 22235 voc entries
Use the script '.\rdbmsConversion\rdbmsMove.bat' to reconfigure area
which will preserve originals and replace with converted directories
Process can be reversed by '.\rdbmsConversion\rdbmsRevert.bat' script
23:17:15 Process completed, total elasped 00:02:57
COMO completed 23:17:15  21 JAN 2007

Appendix B- Format of VOC table entry

The following table lists the format of a VOC entry for replacement of STUBFILE file information for FBNK.ACCOUNT.

Field Number

VoC Entry

Description

001

TABLE

VOC TABLE entry identifier

002

XMLORACLEInit

Database Driver Initialisation Function

003

D_F_ACCOUNT NOXMLSCHEMA

Dictionary table argument for database driver

004

acFBNK_ACCOUNT

Data table arguments for database driver

The format of the VOC TABLE entry also allows either the DICT or DATA definition to be a file path (absolute or relative) in which case the reference will operate similarly to the F pointer type.

The pseudo Inode value to be used for the record lock mechanism is derived from the Hash value of the following.

  • D_F_ACCOUNT Dictionary table, for a record lock taken on the Dictionary

  • acFBNK_ACCOUNT Data table, for a record lock taken on the Data table

The pseudo device value for the Dictionary or Data table will be the Hash value of the XMLORACLEInit Database Driver Initialisation Function. If the JBASE_DATABASE environment variable configured, the device number will be the Hash value of the specified string.

The VOC TABLE entry also allows the paths (absolute or relative) to be specified so that the usage of hash file versions of dictionaries can be continued, if required.

003 ../bnk.dict/F.ACCOUNT]D         - Path to dictionary hash file

Appendix C- Database Schema and Table space creation

This section provides the database schema and table space creation details for Oracle RDBMS.

Appendix D – Contents of rdbmsConversion subdirectory

Created for 'dict' file voc entries               .\rdbmsConversion\DictFiles
Created for 'data' file voc entries              .\rdbmsConversion\DataFiles
Created for 'other' file voc entries             .\rdbmsConversion\OtherFiles
Created for invalid file voc entries             .\rdbmsConversion\InvalidFiles
Created for part file voc entries                .\rdbmsConversion\PartFiles
Created for duplicate voc entries              .\rdbmsConversion\DuplicateFiles
Created for preservation voc entries          .\rdbmsConversion\eldVOC
Created to control conversion processes    .\rdbmsConversion\Control
Created to log conversion errors               .\rdbmsConversion\Errors
Created to log conversion data errors        .\rdbmsConversion\DataErrors
Created to confirm 'dir' type conversion      .\rdbmsConversion\ConvertedDir
Created to confirm 'dict' type conversion    .\rdbmsConversion\ConvertedDict
Created to confirm 'data' type conversion   .\rdbmsConversion\ConvertedData
Created to confirm 'other' type conversion  .\rdbmsConversion\ConvertedOther
Created to confirm dict path conversions    .\rdbmsConversion\DictNames
Created to retain original company             .\rdbmsConversion\EldCompanyFiles
Created to confirm new company               .\rdbmsConversion\NewCompanyFiles
Created to log comparison completed          .\rdbmsConversion\CompareFiles
Created to log comparison errors                .\rdbmsConversion\CompareErrors
Created to log voc conversion errors           .\rdbmsConversion\VocErrors
Created to record converted voc entries      .\rdbmsConversion\ConvertedVoc
Created to preserve stubfile                       .\rdbmsConversion\StubFiles

Appendix E-Troubleshooting Techniques

The conversion process will record the problems (if any) with the files processed. The rdbmsConversion directory contains various files, which record the progress and outcome of the conversion. You need to check InvalidFiles, DuplicateFiles, Errors and DataErrors to check for potential errors. Entries will have the ID of the VOC entry name and the associated record will have the details of the failure. Also, verify the <RDBMS>driver.log file for potential reasons for a transfer failure.

Appendix F - Conversions Requiring Codepage Translation

If your source data is not stored in UTF-8 format, then the rdbmsConversion need to translate it prior to copying it to the RDBMS database, for which you need to specify the corresponding codepage encoding of your data. This situation occurs most frequently for users that use the Latin alphabet but make use of the extended ASCII range to encode accented or special characters.


Bookmark Name Actions
Feedback
x