By using this site, you acknowledge that you have read and understand our Cookie policy, Privacy policy and Terms .

Infosphere DataStage is adopted as ETL (Extract, Transform, Load) tool in many Teradata based data warehousing projects. With the Teradata ODBC and .NET data providers, you can also use the BI tools from Microsoft, i.e. SSIS.

In my previous post, I demonstrated how to install Teradata Toolkit & Utilities in Windows. Once we installed the ODBC driver and Teradata .NET data provider, you can use Teradata as data source or target in any SSIS package. In this post, I will create a SSIS project to load data from Teradata into a CSV file. It will be the same to load data into Teradata from other sources since one data source can also act as target/destination.

Prerequisites

Teradata ODBC Driver or Teradata .NET Data Provider

If you have not installed any of them, please install following the link below:

Connect to Teradata Virtual Machine Guest from Windows Host

SSIS (SQL Server Integration Service)

You can install Data Tools for Visual Studio to create ETL projects. I am using Microsoft SQL Server Data Tools – Business Intelligence for Visual Studio 2013 (https://www.microsoft.com/en-au/download/details.aspx?id=42313).

Create the ETL Project

Open Visual Studio and create a SSIS project named ‘SSIS_TeraData_Test’.

The project will looks like the following screenshot:
image

Create Connection Manager to Teradata

Right-click the folder Connection Managers; in the context menu, click ‘New Connection Manager’ to open the wizard.

You can select ODBC type but in my sample, I will choose ADO.NET as the connection manager type.

image

Click ‘Add..’'.’ button.

image

Click the ‘New…’ button to add a new connection manager.

image

Select ‘.NET Data Provider for Teradata’ as the Provider.

Specify the following connection properties:

  • Server name: 192.168.121.168 (Teradata instance address)
  • User: dbc
  • Password: dbc

image

Click the Test Connection button to ensure it can be successfully connected.

image

In the All tab, you can specify more connection properties. I will just leave them as default.

Click the ‘OK’ button to finish. You can find the connection manager in Connection Managers folder. It can be used by all the packages in the project.

image

Modify the Package

Open the package ‘Package.dtsx’ under SSIS Packages folder.

Drag the Data Flow Task from SSIS Toolbox window into the package design area.

image

Double click the data flow task and you can then design the steps in the flow.

In the Data Flow design area, drag the ADO NET Source into the workspace.

image

In the ADO Source Editor dialog, specify these properties:

  • Connect manager: the one created previously
  • Data access mode: SQL command
  • SQL command test:
SELECT    Emp.EmployeeID,
    Emp.EmployeeName,
    Dept.DepartmentName
    FROM TD_MS_SAMPLE_DB.Employee Emp
    INNER JOIN TD_MS_SAMPLE_DB.Department Dept
    ON Dept.DepartmentID = Emp.DepartmentID
    ORDER BY Emp.EmployeeID;

image

You can click Preview button to preview the data.

image

Back to the Editor, you can change the column names in Output Column in the Columns tab.

image

Click ‘OK’ to finish.

Drag Flat File Destination from the Other Destinations into the working area and connect it to ADO NET Source’s output link.

image

Double click the Flat File Destination to edit the properties.

In the Flat File Destination Editor, click New button to create another connection manager.

Select the Flat File Format as Delimited and then click OK to continue.

image

In the Flat File Connection Manager Editor, input the values as shown in the screenshot.

image

image

Click OK button to save this connection manager.

Back to the Flat File Destination Editor, click Mappings tab to map the columns.

image

Click OK button to save these configurations.

Until now, we have created the package to load employee data into a CSV file.

Run the Package

Right-click the package name in the Solution Explorer, and click Run Package to run the process.

image

From the above screenshot, we can know this data flow task was executed successfully and there were 9 rows inserted into the target file.

Navigate to C drive, you can find the created file TestTeradata.csv. Open the file, you can find all the employees were extracted to the file.

image

SSIS Data Flow Tasks

With SSIS Data Flow Task, you can do many transformations as you would do in any other ETL tools.

The followings are the available functions and all the data source/target types it supports.

imageimageimage

I am not covering SSIS details in this post as it requires hundreds of pages to illustrate. If you are interested, you can always visit Microsoft official site for all the details you need:

https://technet.microsoft.com/en-US/library/bb522537.aspx

info Last modified by Raymond at 5 months ago * This page is subject to Site terms.

More from Kontext

local_offer teradata local_offer SQL

visibility 6
thumb_up 0
access_time 2 days ago

OREPLACE functions in Teradata can be used to replace or remove characters from a string. OREPACE is Teradata's extension to ASNI SQL. The usual REPLACE function is not available. ANSI SQL REPLACE function REPLACE function is commonly implemented in many other SQL databases such as ...

open_in_new View open_in_new Code snippets

local_offer pyspark local_offer spark-2-x local_offer teradata local_offer SQL Server

visibility 59
thumb_up 0
access_time 17 days ago

In my previous article about  Connect to SQL Server in Spark (PySpark) , I mentioned the ways t...

open_in_new View open_in_new Spark + PySpark

local_offer teradata local_offer SQL

visibility 12
thumb_up 0
access_time 23 days ago

Extract sub string from a string is a common operation in data analytics. In Teradata, function SUBSTRING (SUBSTR) and REGEXP_SUBSTR are provided to achieve that. SUBSTR is used to extract string from a specified location while REGEXP_SUBSTR is used to extract string using regular expressions. ...

open_in_new View open_in_new Code snippets

local_offer teradata local_offer SQL

visibility 26
thumb_up 1
access_time 23 days ago

COALESCE function in Teradata returns NULL if all arguments evaluate to null; otherwise it returns the value of the first non-null argument. NULLIF is to used evaluate two expressions and returns NULL if the two arguments are equal otherwise if returns the first arguments. IS NULL i...

open_in_new View open_in_new Code snippets

info About author

comment Comments (4)

comment Add comment

Please log in or register to comment. account_circle Log in person_add Register
R
Raymondarrow_drop_down

I don't have SSIS in my current PC any more. However it should work.

Are there are any objects in your database? And also does your account have access to list the tables or views in the database?


format_quote

person Arul access_time 10 months ago
Re: Create ETL Project with Teradata through SSIS

after the configuration step instead of using SQL command have you tried tables or views in dropdown ? cause i have tried and i'm getting no tables or views could be loaded. 
reply Reply
account_circle Arul
after the configuration step instead of using SQL command have you tried tables or views in dropdown ? cause i have tried and i'm getting no tables or views could be loaded. 
reply Reply
account_circle Raymond

@Sasi Kumar

Apologies  for the late reply.

You can pass variables as you can do with any database, I.e. use variable expressions to prepare the statement.


format_quote

person Sasi Kumar access_time 3 years ago
Re:Create ETL Project with Teradata through SSIS

How to pass user variables in the query.
reply Reply
account_circle Sasi Kumar
How to pass user variables in the query.
reply Reply
Kontext dark theme mode

Dark theme mode

Dark theme mode is available on Kontext.

Learn more arrow_forward
Kontext Column

Kontext Column

Created for everyone to publish data, programming and cloud related articles. Follow three steps to create your columns.

Learn more arrow_forward
info Follow us on Twitter to get the latest article updates. Follow us