Sometime it is necessary to pass environment variables to Spark executors. To pass environment variable to executors, use setExecutorEnv function of SparkConf class. Code snippet In the following code snippet, an environment variable name ENV_NAME is set up with value ...
Spark provides rich APIs to save data frames to many different formats of files such as CSV, Parquet, Orc, Avro, etc. CSV is commonly used in data application though nowadays binary formats are getting momentum. In this article, I am going to show you how to save Spark data frame as CSV file in b...
It might be useful to push artifact or build variables to your Azure websites environment variables in Release pipelines.
At the moment, tags are managed by the site administrators and there are already a number of tags created. Refer to the following link for more details: Existing tags If you think there is other tags we need to add, please let us ...
Data analytics with Google Cloud Platform.
Data analytics with Microsoft Azure cloud platform.
Posts about Apache Sqoop, a tool designed for efficiently transferring bulk data between Apache Hadoop and structured datastores such as relational databases.
PowerShell, Bash, ksh, sh, Perl and etc.
General IT information for programming.