arrow_back Spark - Save DataFrame to Hive Table

Raymond Raymond
Raymond
Stats
articleArticles 551
codeCode 3
imageDiagrams 51
descriptionNotebooks 0
chat_bubble_outlineThreads 8
commentComments 278
loyaltyKontext Points 6098
account_circle Profile
access_time 10 months ago link more_vert
#1708 Re: Spark - Save DataFrame to Hive Table

Hi Swapnil,

These frameworks are designed for batch upserts even they support ACID. If you want to use traditional UPDATE, DELETE and INSERT approach (OLTP), it would be better to use relational databases or nosql databases or alternatively use tools like JDBC to connect to Hive. 

format_quote

person Swapnil access_time 10 months ago
Re: Spark - Save DataFrame to Hive Table

HI  Reymond, Although hive/hudi supports ACID but we cannot spark.sql() dont support update ,merge or delete statements.so is there any workaround for submitting DML .
reply Reply
recommendMore from Kontext