Code python

Iterate through PySpark DataFrame Rows via foreach

Kontext Kontext visibility 1,685 comment 0 access_time 2 years ago language English

descriptionCode description

DataFrame.foreach can be used to iterate/loop through each row (pyspark.sql.types.Row) in a Spark DataFrame object and apply a function to all the rows. This method is a shorthand for DataFrame.rdd.foreach.

Note: Please be cautious when using this method especially if your DataFrame is big.


| col1|    col2|
|Hello| Kontext|
|Hello|Big Data|

col1=Hello, col2=Kontext
col1=Hello, col2=Big Data
copyright This page is subject to Site terms.
comment Comments
No comments yet.

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts