EMR - Expected schema-specific part at index : s3:
When reading data files from S3 in AWS EMR Spark or when submitting spark-submit
command, the following exception can popup:
java.net.URISyntaxException: Expected schema-specific part at index * : s3:
at org.apache.hadoop.fs.Path.initialize(Path.java:263)
Debug this issue
The error message itself already provides hint about where the issue occurs: the URI is not correct or invalid format.
Thus to fix this issue, please check anything related to URL in your spark-submit
command or Spark scripts.
For instance, the following command has spaces in the URL:
spark-submit --py-files "s3://mybucket/ spark/package.zip" ...
To resolve this issue, you can remove the space in the S3 path.
References
copyright
This page is subject to Site terms.
comment Comments
No comments yet.
Log in with external accounts
warning Please login first to view stats information.