Code python

PySpark User Defined Functions (UDF)

Kontext Kontext visibility 87 comment 0 access_time 2 years ago language English

descriptionCode description

User defined functions (UDF) in PySpark can be used to extend built-in function library to provide extra functionality, for example, creating a function to extract values from XML, etc. 

This code snippet shows you how to implement an UDF in PySpark. It shows two slightly different approaches - one use udf decorator and another without. 

Output:

+--------------+-------+--------+--------+
|       str_col|int_col|str_len1|str_len2|
+--------------+-------+--------+--------+
|Hello Kontext!|    100|      14|      14|
|Hello Context!|    100|      14|      14|
+--------------+-------+--------+--------+
fork_rightFork
more_vert
copyright This page is subject to Site terms.
comment Comments
No comments yet.

Please log in or register to comment.

account_circle Log in person_add Register

Log in with external accounts