24 days ago
link
more_vert
articleArticles 549
codeCode 3
imageDiagrams 49
descriptionNotebooks 0
chat_bubble_outlineThreads 8
commentComments 268
loyaltyKontext Points 6058
account_circleProfile
#1800 Re: Hive ACID Inserts, Updates and Deletes with ORC
format_quote
person ugrite access_time 24 days ago
Re: Hive ACID Inserts, Updates and Deletes with ORC
Thank you very much for your article, it is very useful. As you mentioned in the conclusion that it should not use as the OLTP. Could you provide additional suggestions for the below questions?
What is the situation that we should consider to use this feature? Please give me an example.
As it executes as Map-Reduce, So if we want the better performance. Can we use Spark (Ex. PySpark) with the Delta library?
BRs,
Ugrite Hita
There are can be many different points. I will just provide some typical ones here:
If your use case has no low latency requirement, you can use it. For example, ingest events stored in Kafka into a persistent table in Hive.
Spark Delta uses different engine and usually can provide better performance but it really depends on your exact use cases. Delta Lake has the advantage of not relying on a metastore hence it is very easy to migrate from one platform to another.