access_time 3y languageEnglish
more_vert

Load Data from Teradata in Spark (PySpark)

visibility 12,399 comment 2
In my article Connect to Teradata database through Python , I demonstrated about how to use Teradata python package or Teradata ODBC driver to connect to Teradata. In this article, I’m going to show you how to connect to Teradata through JDBC drivers so that you can load data directly into PySpark ...
info Last modified by Raymond 11m
thumb_up 1
comment Comments
4m link more_vert
Raymond Raymond
web_assetArticles 549
forumThreads 7
commentComments 168
account_circleProfile
#1501 Re: Load Data from Teradata in Spark (PySpark)

For the latest Teradata JDBC driver, there is only one JAR file required while earlier versions had two JAR file.

If you hit that error in Jupyter, it means you have not added Teradata JDBC driver path to classpath.

spark = SparkSession \
    .builder \
    .appName("Spark App") \
    .config("spark.jars", "/path/to/teradata/jdbc.jar,/path/to/another/jar.jar") \
    .getOrCreate()

format_quote

person venu access_time 4m
Re: Load Data from Teradata in Spark (PySpark)

Only found this file 'terajdbc4.jar'. 

Installed pyspark in Jupyter notebook. Set the class path in environment variable.

But still facing this issue: java.lang.ClassNotFoundException: com.teradata.jdbc.TeraDriver

4m link more_vert
V
venu
web_assetArticles 0
forumThreads 0
commentComments 1
account_circleProfile
#1500 Re: Load Data from Teradata in Spark (PySpark)

Only found this file 'terajdbc4.jar'. 

Installed pyspark in Jupyter notebook. Set the class path in environment variable.

But still facing this issue: java.lang.ClassNotFoundException: com.teradata.jdbc.TeraDriver

Forum discussions for column Spark.