Code python

Concatenate Columns in Spark DataFrame

Kontext Kontext visibility 161 comment 0 access_time 2 years ago language English

descriptionCode description

This code snippet provides one example of concatenating columns using a separator in Spark DataFrame. Function concat_ws is used directly. For Spark SQL version, refer to Spark SQL - Concatenate w/o Separator (concat_ws and concat).

Syntax of concat_ws

pyspark.sql.functions.concat_ws(sep: str, *cols: ColumnOrName)

Output:

+-----+--------+--------------+
| col1|    col2|     col1_col2|
+-----+--------+--------------+
|Hello| Kontext| Hello,Kontext|
|Hello|Big Data|Hello,Big Data|
+-----+--------+--------------+
fork_rightFork
more_vert
copyright This page is subject to Site terms.
comment Comments
No comments yet.

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts