hive
50 items tagged with "hive"
Articles
PySpark - Save DataFrame into Hive Table using insertInto
This code snippets provides one example of inserting data into Hive table using PySpark DataFrameWriter.insertInto API. `` DataFrameWriter.insertInto(tableName: str, overwrite: Optional[bool] = None) ` It takes two parameters: tableName - the table to insert data into; overwrite` - whether to overwrite existing data. By default, it won't overwrite existing data. This function uses position-based resolution for columns instead of column names.
Introduction to Hive Bucketed Table
Find Number of Rows of Hive Table via Scala
To find the number of rows/records in a Hive table, we can use Spark SQL count aggregation function: Hive SQL - Aggregate Functions Overview with Examples. This code snippet provide example of Scala code to implement the same. spark-shell is used directly for simplicity. The code snippet can also run Jupyter Notebooks or Zeppelin with Spark kernel. Alternatively, it can be compiled to jar file and then submit as job via spark-submit. !2022082315649-image.png
Start Hive Beeline CLI
This code snippet provides example to start Hive Beeline CLI in Linux. Beeline is the successor of Hive CLI. In the shell scripts, the environment variable $HIVE_HOME is the home folder of Hive installation in the system. In a cluster environment, it usually refers to the Hive client installation on an edge server. Output: `` $HIVE_HOME/bin/beeline -u jdbc:hive2:// Connecting to jdbc:hive2:// Hive Session ID = 65a40cd9-02ce-4965-93b6-cff9db461b70 Connected to: Apache Hive (version 3.1.3) Driver: Hive JDBC (version 3.1.3) Transaction isolation: TRANSACTIONREPEATABLEREAD Beeline version 3.1.3 by Apache Hive 0: jdbc:hive2://> ``
Hive - Retrieve Current User
This code snippet provides example of retrieving current user via current_user() function in HQL (Hive QL) code. Output: `` 0: jdbc:hive2://> select currentuser();OK+----------+| c0 |+----------+| kontext |+----------+ ``
Configure HiveServer2 to Enable Transactions (ACID Support)
Hive SQL - Merge Statement on ACID Tables
Hive supports standard ANSI SQL MERGE statement from version 2.2. However it can be only be applied to tables that support ACID transactions. To learn more about ACID support in Hive, refer to article: Hive ACID Inserts, Updates and Deletes with ORC. Sample table This code snippet merges into a sample table named testdb.crudtable. It has two records before the merge. !20220819124209-image.png The staging table was created using the following statements: `` create table crudtablestg (id int, value string, op string); insert into crudtablestg values (1,'AA','U'),(2,'B','D'),(3,'C', 'I'); ` It has one additional column named op to indicate the delta changes: U - updates D - deletes I - inserts (i.e. new records) Syntax ` MERGE INTO AS T USING AS S ON WHEN MATCHED [AND ] THEN UPDATE SET WHEN MATCHED [AND ] THEN DELETE WHEN NOT MATCHED [AND ] THEN INSERT VALUES `` Output After the merge, record 1 is updated; record 2 is deleted and record 3 is inserted into the table.
Spark Insert Data into Hive Tables
Hive ACID Inserts, Updates and Deletes with ORC
Hive SQL - Union data with UNION ALL and UNION DISTINCT
Hive SQL - Analytics with GROUP BY and GROUPING SETS, Cubes, Rollups
Hive SQL - Data Sampling using TABLESAMPLE
Extract Values from XML Column in Hive Tables
Hive SQL - Virtual Columns
Hive SQL - Cluster By and Distribute By
Hive SQL - Differences between Order By and Sort By
Hive SQL - Aggregate Functions Overview with Examples
List Tables in Hive Database
PySpark - Read from Hive Tables
Hive - Create External Table for Multiline CSV Files
Create Partitioned Hive Table
Spark - 保存DataFrame为Hive数据表
Hive - Rename Table
Python: Load Data from Hive
Apache Hive 3.1.2 Installation on Linux Guide
Error: Failed to load class org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver
Create Temporary Table - Hive SQL
Create Table as SELECT - Hive SQL
Create Bucketed Sorted Table - Hive SQL
Create Partitioned Table - Hive SQL
Create Table Stored as CSV, TSV, JSON Format - Hive SQL
Create Table with Parquet, Orc, Avro - Hive SQL
Create, Drop, and Truncate Table - Hive SQL
Create, Drop, Alter and Use Database - Hive SQL
Apache Hive 3.1.2 Installation on Windows 10
Hive: Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
Differences between Hive External and Internal (Managed) Tables
Schema Merging (Evolution) with Parquet in Spark and Hive
Select from dual in SQL / Hive
In Oracle database, you can select from dual table if you only want to return a one row result set. In many other databases, the query engine supports select directly from constant values without specifying a table name.
Select top N records in SQL / Hive
In different databases, the syntax of selecting top N records are slightly different. They may also differ from ISO standards.