Tensorflow/nvidia/cuda docker mismatched versionsHow is Docker different from a virtual machine?Should I use Vagrant or Docker for creating an isolated environment?How to list containers in DockerHow to get a Docker container's IP address from the host?How to remove old Docker containersCopying files from Docker container to hostCopying files from host to Docker containerCan Tensorflow be installed alongside Theano?running ffmpeg with nvenc inside nvidia dockerUnable to run Distributed TensorFlow using V100 GPU
Why do we need explainable AI?
Is there a name for this metric: TN / (TN + FN)?
Can there be plants on the dark side of a tidally locked world?
Is Levitate supposed to basically disable a melee based enemy?
How does speed affect lift?
How do I make my fill-in-the-blank exercise more obvious?
MOSFET broke after attaching capacitor bank
How did Gollum know Sauron was gathering the Haradrim to make war?
One hour 10 min layover in Newark; International -> Domestic connection. Enough time to clear customs?
Punishment in pacifist society
How to encode a class with 24,000 categories?
Global variables and information security
How to find better food in airports
The little bee buzzes around the flower garden
Do I need to get a noble in order to win Splendor?
What exactly is a softlock?
What happens if I double Meddling Mage's 'enter the battlefield' trigger?
How does Harry wear the invisibility cloak?
Do 643,000 Americans go bankrupt every year due to medical bills?
Is it possible to observe space debris with Binoculars?
a harmful idea/plan
Why would a Intel 8080 chip be destroyed if +12 V is connected before -5 V?
When is it legal to castle moving the rook first?
Does immunity to damage from nonmagical attacks negate a rogue's Sneak Attack damage?
Tensorflow/nvidia/cuda docker mismatched versions
How is Docker different from a virtual machine?Should I use Vagrant or Docker for creating an isolated environment?How to list containers in DockerHow to get a Docker container's IP address from the host?How to remove old Docker containersCopying files from Docker container to hostCopying files from host to Docker containerCan Tensorflow be installed alongside Theano?running ffmpeg with nvenc inside nvidia dockerUnable to run Distributed TensorFlow using V100 GPU
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty,.everyoneloves__bot-mid-leaderboard:empty margin-bottom:0;
I am trying to use tensorflow and nvidia with docker, but hitting the following error:
docker run --runtime=nvidia -it --rm tensorflow/tensorflow:latest-gpu python -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
docker: Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused "process_linux.go:407: running prestart hook 1 caused \"error running hook: exit status 1, stdout: , stderr: exec command: [/usr/bin/nvidia-container-cli --load-kmods configure --ldconfig=@/sbin/ldconfig.real --device=all --compute --utility --require=cuda>=10.0 brand=tesla,driver>=384,driver<385 --pid=5393 /var/lib/docker/overlay2/......./merged]\nnvidia-container-cli: requirement error: unsatisfied condition: brand = tesla\n\""": unknown.
I get similar error when trying to run nvidia-smi:
docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
but when trying to run nvidia-smi with cuda:9.0-base, it works like a charm:
docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi
Do I need to make sure that cuda 10 works or I can run tensorflow with cuda 9? And how can I run the docker image of tensorflow with cuda:9.0-base? (still a docker newby).
Thanks a lot!
docker tensorflow cuda nvidia
asked Mar 28 at 3:20
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
add a comment |
I am trying to use tensorflow and nvidia with docker, but hitting the following error:
docker run --runtime=nvidia -it --rm tensorflow/tensorflow:latest-gpu python -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
docker: Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused "process_linux.go:407: running prestart hook 1 caused \"error running hook: exit status 1, stdout: , stderr: exec command: [/usr/bin/nvidia-container-cli --load-kmods configure --ldconfig=@/sbin/ldconfig.real --device=all --compute --utility --require=cuda>=10.0 brand=tesla,driver>=384,driver<385 --pid=5393 /var/lib/docker/overlay2/......./merged]\nnvidia-container-cli: requirement error: unsatisfied condition: brand = tesla\n\""": unknown.
I get similar error when trying to run nvidia-smi:
docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
but when trying to run nvidia-smi with cuda:9.0-base, it works like a charm:
docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi
Do I need to make sure that cuda 10 works or I can run tensorflow with cuda 9? And how can I run the docker image of tensorflow with cuda:9.0-base? (still a docker newby).
Thanks a lot!
docker tensorflow cuda nvidia
asked Mar 28 at 3:20
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
add a comment |
I am trying to use tensorflow and nvidia with docker, but hitting the following error:
docker run --runtime=nvidia -it --rm tensorflow/tensorflow:latest-gpu python -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
docker: Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused "process_linux.go:407: running prestart hook 1 caused \"error running hook: exit status 1, stdout: , stderr: exec command: [/usr/bin/nvidia-container-cli --load-kmods configure --ldconfig=@/sbin/ldconfig.real --device=all --compute --utility --require=cuda>=10.0 brand=tesla,driver>=384,driver<385 --pid=5393 /var/lib/docker/overlay2/......./merged]\nnvidia-container-cli: requirement error: unsatisfied condition: brand = tesla\n\""": unknown.
I get similar error when trying to run nvidia-smi:
docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
but when trying to run nvidia-smi with cuda:9.0-base, it works like a charm:
docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi
Do I need to make sure that cuda 10 works or I can run tensorflow with cuda 9? And how can I run the docker image of tensorflow with cuda:9.0-base? (still a docker newby).
Thanks a lot!
docker tensorflow cuda nvidia
asked Mar 28 at 3:20
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
I am trying to use tensorflow and nvidia with docker, but hitting the following error:
docker run --runtime=nvidia -it --rm tensorflow/tensorflow:latest-gpu python -c "import tensorflow as tf; tf.enable_eager_execution(); print(tf.reduce_sum(tf.random_normal([1000, 1000])))"
docker: Error response from daemon: OCI runtime create failed: container_linux.go:344: starting container process caused "process_linux.go:424: container init caused "process_linux.go:407: running prestart hook 1 caused \"error running hook: exit status 1, stdout: , stderr: exec command: [/usr/bin/nvidia-container-cli --load-kmods configure --ldconfig=@/sbin/ldconfig.real --device=all --compute --utility --require=cuda>=10.0 brand=tesla,driver>=384,driver<385 --pid=5393 /var/lib/docker/overlay2/......./merged]\nnvidia-container-cli: requirement error: unsatisfied condition: brand = tesla\n\""": unknown.
I get similar error when trying to run nvidia-smi:
docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
but when trying to run nvidia-smi with cuda:9.0-base, it works like a charm:
docker run --runtime=nvidia --rm nvidia/cuda:9.0-base nvidia-smi
Do I need to make sure that cuda 10 works or I can run tensorflow with cuda 9? And how can I run the docker image of tensorflow with cuda:9.0-base? (still a docker newby).
Thanks a lot!
docker tensorflow cuda nvidia
docker tensorflow cuda nvidia
asked Mar 28 at 3:20
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
asked Mar 28 at 3:20
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
asked Mar 28 at 3:20
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
asked Mar 28 at 3:20
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
asked Mar 28 at 3:20
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
6,41010 gold badges47 silver badges72 bronze badges
add a comment |
add a comment |
1 Answer
1
active
oldest
votes
Ok I think I am finally starting to figure out the mess on my machine.
The tensorflow image does NOT care about the cuda image version, it does not use the docker cuda image. It cares about my nvidia drivers, since it has CUDA integrated in the tensorflow image.
(The docker cuda image that is working with my current drivers, is cuda:9.0)
That meant I have to find a tensorflow image that is working with my drivers (390.116), or update the drivers.
I tried the same command with tensorflow:1.12.0-gpu-py3, and it didn't have any problems.
answered Mar 28 at 3:41
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
1
To add to your answer. The nvidia-driver that comes with each level of CUDA can be used with any container that has that level of CUDA or an older level of CUDA on it. Using a new nvidia-driver and an older CUDA level will work. Using a older nvidia-driver and newer CUDA level will not work.
– William D. Irons
Mar 28 at 14:33
1
tensorflow-gpu is linked against a particular CUDA version, so it does care about the cuda version. You must have the proper CUDA version, and you need a driver that is supportive of that CUDA version. Your docker image "does not care" because it has the necessary CUDA version already installed. A particular CUDA version has a minimum driver version required (scroll down to table 1), but it can be used with drivers that are newer than that.
– Robert Crovella
Mar 28 at 17:37
Yes, that's what I meant, but not explained correctly. Will edit my answer.
– Danail
Mar 28 at 20:07
add a comment |
Your Answer
StackExchange.ifUsing("editor", function ()
StackExchange.using("externalEditor", function ()
StackExchange.using("snippets", function ()
StackExchange.snippets.init();
);
);
, "code-snippets");
StackExchange.ready(function()
var channelOptions =
tags: "".split(" "),
id: "1"
;
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function()
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled)
StackExchange.using("snippets", function()
createEditor();
);
else
createEditor();
);
function createEditor()
StackExchange.prepareEditor(
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader:
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
,
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
);
);
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55389669%2ftensorflow-nvidia-cuda-docker-mismatched-versions%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
Ok I think I am finally starting to figure out the mess on my machine.
The tensorflow image does NOT care about the cuda image version, it does not use the docker cuda image. It cares about my nvidia drivers, since it has CUDA integrated in the tensorflow image.
(The docker cuda image that is working with my current drivers, is cuda:9.0)
That meant I have to find a tensorflow image that is working with my drivers (390.116), or update the drivers.
I tried the same command with tensorflow:1.12.0-gpu-py3, and it didn't have any problems.
answered Mar 28 at 3:41
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
1
To add to your answer. The nvidia-driver that comes with each level of CUDA can be used with any container that has that level of CUDA or an older level of CUDA on it. Using a new nvidia-driver and an older CUDA level will work. Using a older nvidia-driver and newer CUDA level will not work.
– William D. Irons
Mar 28 at 14:33
1
tensorflow-gpu is linked against a particular CUDA version, so it does care about the cuda version. You must have the proper CUDA version, and you need a driver that is supportive of that CUDA version. Your docker image "does not care" because it has the necessary CUDA version already installed. A particular CUDA version has a minimum driver version required (scroll down to table 1), but it can be used with drivers that are newer than that.
– Robert Crovella
Mar 28 at 17:37
Yes, that's what I meant, but not explained correctly. Will edit my answer.
– Danail
Mar 28 at 20:07
add a comment |
Ok I think I am finally starting to figure out the mess on my machine.
The tensorflow image does NOT care about the cuda image version, it does not use the docker cuda image. It cares about my nvidia drivers, since it has CUDA integrated in the tensorflow image.
(The docker cuda image that is working with my current drivers, is cuda:9.0)
That meant I have to find a tensorflow image that is working with my drivers (390.116), or update the drivers.
I tried the same command with tensorflow:1.12.0-gpu-py3, and it didn't have any problems.
answered Mar 28 at 3:41
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
1
To add to your answer. The nvidia-driver that comes with each level of CUDA can be used with any container that has that level of CUDA or an older level of CUDA on it. Using a new nvidia-driver and an older CUDA level will work. Using a older nvidia-driver and newer CUDA level will not work.
– William D. Irons
Mar 28 at 14:33
1
tensorflow-gpu is linked against a particular CUDA version, so it does care about the cuda version. You must have the proper CUDA version, and you need a driver that is supportive of that CUDA version. Your docker image "does not care" because it has the necessary CUDA version already installed. A particular CUDA version has a minimum driver version required (scroll down to table 1), but it can be used with drivers that are newer than that.
– Robert Crovella
Mar 28 at 17:37
Yes, that's what I meant, but not explained correctly. Will edit my answer.
– Danail
Mar 28 at 20:07
add a comment |
Ok I think I am finally starting to figure out the mess on my machine.
The tensorflow image does NOT care about the cuda image version, it does not use the docker cuda image. It cares about my nvidia drivers, since it has CUDA integrated in the tensorflow image.
(The docker cuda image that is working with my current drivers, is cuda:9.0)
That meant I have to find a tensorflow image that is working with my drivers (390.116), or update the drivers.
I tried the same command with tensorflow:1.12.0-gpu-py3, and it didn't have any problems.
answered Mar 28 at 3:41
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
Ok I think I am finally starting to figure out the mess on my machine.
The tensorflow image does NOT care about the cuda image version, it does not use the docker cuda image. It cares about my nvidia drivers, since it has CUDA integrated in the tensorflow image.
(The docker cuda image that is working with my current drivers, is cuda:9.0)
That meant I have to find a tensorflow image that is working with my drivers (390.116), or update the drivers.
I tried the same command with tensorflow:1.12.0-gpu-py3, and it didn't have any problems.
answered Mar 28 at 3:41
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
edited Mar 28 at 20:08
answered Mar 28 at 3:41
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
answered Mar 28 at 3:41
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
answered Mar 28 at 3:41
DanailDanail
6,41010 gold badges47 silver badges72 bronze badges
6,41010 gold badges47 silver badges72 bronze badges
1
To add to your answer. The nvidia-driver that comes with each level of CUDA can be used with any container that has that level of CUDA or an older level of CUDA on it. Using a new nvidia-driver and an older CUDA level will work. Using a older nvidia-driver and newer CUDA level will not work.
– William D. Irons
Mar 28 at 14:33
1
tensorflow-gpu is linked against a particular CUDA version, so it does care about the cuda version. You must have the proper CUDA version, and you need a driver that is supportive of that CUDA version. Your docker image "does not care" because it has the necessary CUDA version already installed. A particular CUDA version has a minimum driver version required (scroll down to table 1), but it can be used with drivers that are newer than that.
– Robert Crovella
Mar 28 at 17:37
Yes, that's what I meant, but not explained correctly. Will edit my answer.
– Danail
Mar 28 at 20:07
add a comment |
1
To add to your answer. The nvidia-driver that comes with each level of CUDA can be used with any container that has that level of CUDA or an older level of CUDA on it. Using a new nvidia-driver and an older CUDA level will work. Using a older nvidia-driver and newer CUDA level will not work.
– William D. Irons
Mar 28 at 14:33
1
tensorflow-gpu is linked against a particular CUDA version, so it does care about the cuda version. You must have the proper CUDA version, and you need a driver that is supportive of that CUDA version. Your docker image "does not care" because it has the necessary CUDA version already installed. A particular CUDA version has a minimum driver version required (scroll down to table 1), but it can be used with drivers that are newer than that.
– Robert Crovella
Mar 28 at 17:37
Yes, that's what I meant, but not explained correctly. Will edit my answer.
– Danail
Mar 28 at 20:07
1
1
To add to your answer. The nvidia-driver that comes with each level of CUDA can be used with any container that has that level of CUDA or an older level of CUDA on it. Using a new nvidia-driver and an older CUDA level will work. Using a older nvidia-driver and newer CUDA level will not work.
– William D. Irons
Mar 28 at 14:33
To add to your answer. The nvidia-driver that comes with each level of CUDA can be used with any container that has that level of CUDA or an older level of CUDA on it. Using a new nvidia-driver and an older CUDA level will work. Using a older nvidia-driver and newer CUDA level will not work.
– William D. Irons
Mar 28 at 14:33
1
1
tensorflow-gpu is linked against a particular CUDA version, so it does care about the cuda version. You must have the proper CUDA version, and you need a driver that is supportive of that CUDA version. Your docker image "does not care" because it has the necessary CUDA version already installed. A particular CUDA version has a minimum driver version required (scroll down to table 1), but it can be used with drivers that are newer than that.
– Robert Crovella
Mar 28 at 17:37
tensorflow-gpu is linked against a particular CUDA version, so it does care about the cuda version. You must have the proper CUDA version, and you need a driver that is supportive of that CUDA version. Your docker image "does not care" because it has the necessary CUDA version already installed. A particular CUDA version has a minimum driver version required (scroll down to table 1), but it can be used with drivers that are newer than that.
– Robert Crovella
Mar 28 at 17:37
Yes, that's what I meant, but not explained correctly. Will edit my answer.
– Danail
Mar 28 at 20:07
Yes, that's what I meant, but not explained correctly. Will edit my answer.
– Danail
Mar 28 at 20:07
add a comment |
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Got a question that you can’t ask on public Stack Overflow? Learn more about sharing private information with Stack Overflow for Teams.
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f55389669%2ftensorflow-nvidia-cuda-docker-mismatched-versions%23new-answer', 'question_page');
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
