arrow_back Spark - Save DataFrame to Hive Table

access_time 10 months ago link more_vert
#1704 Re: Spark - Save DataFrame to Hive Table

Thanks for the pointers

format_quote

person Raymond access_time 10 months ago
Re: Spark - Save DataFrame to Hive Table

Hive 3 does support ACID if table data is stored as ORC format and we can use INSERT, UPDATE and DELETE statements to manipulate the data as you can do with other databases. You can use SparkSession.sql to invoke these statements if necessary. 

Hudi and DeltaLake both support ACID too and can be used when working with data lake directly, for example, data stored as hudi and delta format in HDFS or S3, GCS, Azure Blob Storage, etc. 


recommendMore from Kontext