How to fix ''java.lang.NoSuchMethodError" in oozie spark yarn submit?Spark 1.1.0 on cdh5.1.3 does not work in yarn-cluster modeSparkPi program keeps running under Yarn/Spark/Google Compute EngineError when deploying Spark from eclipse to YARN in cluster modeHortonworks Oozie Spark ActionHow to run spark-shell with YARN in client mode?Spark in Yarn Web UI not getting displayedRun spark python job with Oozie and Hue - Intercepting System.exit(1)Oozie workflow with spark application reports out of memorysubmitting PySpark app to spark on YARN in cluster modePyspark actions submitted with oozie failing: '[Errno 2] No such file or directory'
Having to constantly redo everything because I don't know how to do it?
Can a stressful Wish's Strength reduction be cured early by a Greater Restoration spell?
What is the proper markup for a Math operator in boldface?
Making a wall made from glass bricks
Does a return economy-class seat between London and San Francisco release 5.28 tonnes of CO2 equivalents?
Robots in a spaceship
Put my student loan in parents’ second mortgage - help?
On what to compliment someone with anorexia in order to improve their body image?
Head on collision of two black holes
What European countries have secret voting within the Legislature?
What is an acid trap
"I am [the / an] owner of a bookstore"?
Can a successful book series let the bad guy win?
Do the 26 richest billionaires own as much wealth as the poorest 3.8 billion people?
I just started should I accept a farewell lunch for a coworker I don't know?
Why wasn't EBCDIC designed with contiguous alphanumeric characters?
The Football Squad
When was this photo of Mission Dolores *actually* taken?
Reusable spacecraft: why still have fairings detach, instead of open/close?
How can I deal with extreme temperatures in a hotel room?
Simple logic puzzle
My colleague is constantly blaming me for his errors
pgfmath does not work
Can dual citizens open crypto exchange accounts where U.S. citizens are prohibited?
How to fix ''java.lang.NoSuchMethodError" in oozie spark yarn submit?
Spark 1.1.0 on cdh5.1.3 does not work in yarn-cluster modeSparkPi program keeps running under Yarn/Spark/Google Compute EngineError when deploying Spark from eclipse to YARN in cluster modeHortonworks Oozie Spark ActionHow to run spark-shell with YARN in client mode?Spark in Yarn Web UI not getting displayedRun spark python job with Oozie and Hue - Intercepting System.exit(1)Oozie workflow with spark application reports out of memorysubmitting PySpark app to spark on YARN in cluster modePyspark actions submitted with oozie failing: '[Errno 2] No such file or directory'
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I have a bare metal hadoop 3.1.2 cluster up and running and I've also installed Oozie 5.1.0 and managed to run shell example successfully.
But when I'm running Spark example (FileCopy) it indicates 'Success' in oozie, while file isn't really copied, and YARN stderr contains:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse.getResourceProfiles()Ljava/util/Map;
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:253)
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:234)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:201)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:180)
Do you have any idea why this may happen?
apache-spark hadoop oozie
asked Mar 25 at 14:55
Andrii SushkoAndrii Sushko
426 bronze badges
add a comment |
I have a bare metal hadoop 3.1.2 cluster up and running and I've also installed Oozie 5.1.0 and managed to run shell example successfully.
But when I'm running Spark example (FileCopy) it indicates 'Success' in oozie, while file isn't really copied, and YARN stderr contains:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse.getResourceProfiles()Ljava/util/Map;
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:253)
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:234)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:201)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:180)
Do you have any idea why this may happen?
apache-spark hadoop oozie
asked Mar 25 at 14:55
Andrii SushkoAndrii Sushko
426 bronze badges
add a comment |
I have a bare metal hadoop 3.1.2 cluster up and running and I've also installed Oozie 5.1.0 and managed to run shell example successfully.
But when I'm running Spark example (FileCopy) it indicates 'Success' in oozie, while file isn't really copied, and YARN stderr contains:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse.getResourceProfiles()Ljava/util/Map;
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:253)
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:234)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:201)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:180)
Do you have any idea why this may happen?
apache-spark hadoop oozie
asked Mar 25 at 14:55
Andrii SushkoAndrii Sushko
426 bronze badges
I have a bare metal hadoop 3.1.2 cluster up and running and I've also installed Oozie 5.1.0 and managed to run shell example successfully.
But when I'm running Spark example (FileCopy) it indicates 'Success' in oozie, while file isn't really copied, and YARN stderr contains:
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.hadoop.yarn.api.protocolrecords.RegisterApplicationMasterResponse.getResourceProfiles()Ljava/util/Map;
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:253)
at org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:234)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:201)
at org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:180)
Do you have any idea why this may happen?
apache-spark hadoop oozie
apache-spark hadoop oozie
asked Mar 25 at 14:55
Andrii SushkoAndrii Sushko
426 bronze badges
asked Mar 25 at 14:55
Andrii SushkoAndrii Sushko
426 bronze badges
asked Mar 25 at 14:55
Andrii SushkoAndrii Sushko
426 bronze badges
asked Mar 25 at 14:55
Andrii SushkoAndrii Sushko
426 bronze badges
asked Mar 25 at 14:55
Andrii SushkoAndrii Sushko
426 bronze badges
426 bronze badges
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Oozie 5.1.0 does not support Hadoop 3. Open issues:
- https://issues.apache.org/jira/browse/OOZIE-2973
- https://issues.apache.org/jira/browse/OOZIE-3255
answered Jun 3 at 23:08
attila_sattila_s
1259 bronze badges
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55340621%2fhow-to-fix-java-lang-nosuchmethoderror-in-oozie-spark-yarn-submit%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Oozie 5.1.0 does not support Hadoop 3. Open issues:
- https://issues.apache.org/jira/browse/OOZIE-2973
- https://issues.apache.org/jira/browse/OOZIE-3255
answered Jun 3 at 23:08
attila_sattila_s
1259 bronze badges
add a comment |
Oozie 5.1.0 does not support Hadoop 3. Open issues:
- https://issues.apache.org/jira/browse/OOZIE-2973
- https://issues.apache.org/jira/browse/OOZIE-3255
answered Jun 3 at 23:08
attila_sattila_s
1259 bronze badges
add a comment |
Oozie 5.1.0 does not support Hadoop 3. Open issues:
- https://issues.apache.org/jira/browse/OOZIE-2973
- https://issues.apache.org/jira/browse/OOZIE-3255
answered Jun 3 at 23:08
attila_sattila_s
1259 bronze badges
Oozie 5.1.0 does not support Hadoop 3. Open issues:
- https://issues.apache.org/jira/browse/OOZIE-2973
- https://issues.apache.org/jira/browse/OOZIE-3255
answered Jun 3 at 23:08
attila_sattila_s
1259 bronze badges
answered Jun 3 at 23:08
attila_sattila_s
1259 bronze badges
answered Jun 3 at 23:08
attila_sattila_s
1259 bronze badges
answered Jun 3 at 23:08
attila_sattila_s
1259 bronze badges
1259 bronze badges
add a comment |
add a comment |
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55340621%2fhow-to-fix-java-lang-nosuchmethoderror-in-oozie-spark-yarn-submit%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
