Data transfer into existing tables of the target database

  Previous topic Next topic JavaScript is required for the print function Mail us feedback on this topic! Mail us feedback on this topic!  

 

If radio button Into existing tables of the target data source is checked on panel 3. Data transfer of page Enabled actions then this page looks as follows:

 

data_transfer-existing_tables

 

In this case we can distribute data from the source table among the columns of one or more tables in the target database and perform complex operations before inserting them.

Assignments describing where the data should be inserted during data transfer can be determined on this page. Existing assignments are displayed in the table occupying most of this panel. This cannot be modified directly, editing items and creating new ones can be done using the fields Group name, Target table, Target field, Data to be inserted and checkbox Parse complex expressions located on the bottom. Click the button Save to save changes made to the item currently selected, Add to add it as a new one and Delete to remove it. Creating multiple entries with the same Group name, Target table and Target field values is not allowed.

 

Drop-down list Target table holds the names of the source database's tables. Choose the one you would like to insert data into. Then select the proper column in Target field.

 

Data to be inserted specifies what goes into the target field. This list has the column names of the source table (if we have selected it) between ${ (dollar sign + opening curly bracket) and } (closing curly bracket) signs (e.g. ${fieldName} instead of fieldName) so you can refer them unambiguously. Selecting such a column reference to be inserted causes values of this field to be put into the target field of selected target table.

Numerous other things may be provided in the input box Data to be inserted:

 

Contents of a field of the source table without alteration:

${fieldName}

Reference to column fieldName

 

References to the values from text boxes on page Configuration groups of the simple view (which in turn are parameters of the source and target connection strings and the otherwise unused Additional parameters):

${sourceServerName}

Source -> Server

${sourceDatabase}

Source -> Database

${sourceUsername}

Source -> Username

${sourcePassword}

Source -> Password

${sourceAdditionalParameter}

Source -> Additional parameter

${targetServerName}

Target -> Server

${targetDatabase}

Target -> Database

${targetUsername}

Target -> Username

${targetPassword}

Target -> Password

${targetAdditionalParameter}

Target -> Additional parameter

 

Constants:

13

Number

abcdefgh

Text

2010-11-12 13:14:15

Date

 

Expressions:

${fieldName} * 10

Mathematical operations can be performed on the contents of column fieldName if it's of a numeric type

"${fieldName}".Substring(0, 2)

Enclosing the reference ${fieldName} in quotation marks allows the use of member functions of Visual Basic's String class on it

CType("${fieldName}", DateTime).AddMonths(1)

If field fieldName contains dates, converting them to type DateTime allows the use of member functions of Visual Basic's DateTime class on them

getMeARandomNumber()

Visual Basic function are defined on page Own VB functions

 

SQL queries:

SQL(SELECT [xFieldName] FROM [tableName] WHERE [yFieldName] = ${zFieldName})

If column zFieldName is of a numeric type

SQL(SELECT [xFieldName] FROM [tableName] WHERE [yFieldName] = '${zFieldName}')

The reference has to be enclosed in apostrophes  if zFieldName's data type is text

SQL(SELECT [xFieldName] FROM [tableName] WHERE [yFieldName] LIKE '%${zFieldName}%')

ZFieldName is text once again

SQL(SELECT [xFieldName] FROM [tableName] WHERE [yFieldName] = {ts '${zFieldName}'})

ZFieldName contains dates so its reference must be enclosed in {ts ' (opening curly bracket + ts + space + apostrophe) and '} (apostrophe + closing curly bracket)

SQL queries need to be enclosed between SQL( and ) as seen above.

 

Querying auxiliary tables:

AUXTBL(firstAuxiliaryTable, [identifier] = 10, 2)

This command queries auxiliary table firstAuxiliaryTable for records having 10 in their identifier column and returns the value from the field with index 2 (meaning third as indexing starts with 0) of the result set's first row. If the result set has more than one row the rest is discarded.

First argument is the name of the auxiliary table to use. Keep in mind that the list of auxiliary tables always contains eight predefined, initially empty tables called virtualTable1, ..., virtualTable8 that can be used in the same way as tables of the source database marked as auxiliary.
Second argument (in the place of [identifier] = 10) is an expression applicable by .NET's DataTable.Select() as filter.
Third argument is the index or name of the desired column of the resulting row

 

Other:

${GetDate()}

Local timestamp

${GetUTCDate()}

UTC timestamp

 

Parse complex expressions must be checked if mathematical procedures or Visual Basic calls are used in the field Data to be inserted (as the table Expressions shows above). Leaving it unchecked leads to faulty behaviour. The problem may not become clear immediately in case of a target field with a textual data type as the unprocessed expression (something like "${fieldName}".Substring(0, 2)) gets inserted instead of the expected value without producing an error. Turning it on unnecessarily, on the other hand, should be avoided as it lowers performance.

 

Supplying a group name is only necessary in a certain number of cases. When the software inserts rows into target tables separate insert commands are built and ran for each table by default (meaning that data from a record of the source table takes as many transactions to get to its place as the number of tables we want to distribute them among, which equals the number of unique table names in the column Target table of the grid on page Data transfer (3)). Using group names causes the plugin to create separate insert commands for entries with different group names even if they are put into the same target table and field. This allows data from a single row of the source table to appear as several records in the same target table. (For further information see the example demonstrating the use of group names on the bottom of the page.)

 

 

An example

Set up connection parameters of the source and target databases and select the desired source table. Let's assume this is called Registry and its contents are:

 

Registry

ID

Name

Year of birth

1

Jane Doe

1986

2

John Doe

1949

3

James Doe

1986

 

We aim to distribute this data between two tables of the target database: Names and Years. These are empty tables, their ID columns contain unique, automatically incrementing values:

 

Names

ID

Name

 

Years

ID

Year of birth

 

To get this done, add the following entries to the grid on page Data transfer (3):

 

Group name

Target table

Target field

Data to be inserted

Complex

 

Names

Name

${Name}

NO

 

Years

Year of birth

${Year of birth}

NO

 

To replace the first names of people called John with their nicknames, modify this as follows:

 

Group name

Target table

Target field

Data to be inserted

Complex

 

Names

Name

"${Name}".Replace("John", "Johnny")

YES

 

Years

Year of birth

${Year of birth}

NO

 

Saving and executing this configuration will result in tables Names and Years looking like this:

 

Names

ID

Name

1

Jane Doe

2

Johnny Doe

3

James Doe

 

Years

ID

Year of birth

1

1986

2

1949

 

 

Another example

Source table is the same as that of the previous example but table Names is extended with a column (Year of birth ID) pointing to the corresponding ID in table Years to bind the appropriate years to names. So the target tables are:

 

Names

ID

Name

Year of birth ID

 

Years

ID

Year of birth

 

The grid on page Data transfer (3) has to be modified as follows:

 

Group name

Target table

Target field

Data to be inserted

Complex

 

Years

Year of birth

${Year of birth}

NO

 

Names

Name

${Name}

NO

 

Names

Year of birth ID

SQL(SELECT [ID] FROM [Years] WHERE [Year of birth] = ${Year of birth})

NO

 

Note that not only a new row has been added but the order of rows has changed as well: now the entry of table Years is on the top. This is because if a configuration uses multiple target tables data will be inserted into them in the order of their first occurrence in the grid. The year from a given row of the source table needs to be put into table Years, so that the SQL query in the newly added entry would find it.

Executing this configuration will result in:

 

Names

ID

Name

Year of birth ID

1

Jane Doe

1

2

John Doe

2

3

James Doe

1

 

Years

ID

Year of birth

1

1986

2

1949

 

 

 

An example demonstrating the use of groups

We have a table called Couples:

 

Couples

ID

Husband

Wife

1

John Doe

Jane Doe

2

James Doe

Judy Doe

 

Our task is to move all the names from Couples to the simple table below (ID is an automatically incrementing column):

 

Names

ID

Name

 

Each row of the source table has two pieces of information that need to be put into the same column of the same table. This is only possible if they are inserted in two separate transactions - creating two rows in the target table - which can be achieved by creating entries for them in the grid of page Data transfer (3)  with the same Target table and Target field but different Group name values.

 

Group name

Target table

Target field

Data to be inserted

Complex

First group

Names

Name

${Husband}

NO

Second group

Names

Name

${Wife}

NO

 

After running it will give the folowing results:

 

ID

Name

1

John Doe

2

Jane Doe

3

James Doe

4

Judy Doe

 

 

Warning:

The plugin only inserts rows into the target table that are not present yet. The second example shows that the value 1986 occurs only once in the table Years. This helps to avoid adding the same data over and over again if a configuration is executed multiple times.