/tags/tensorflow/Recent content in Tensorflow on My BlogHugoen-usSat, 10 Mar 2018 00:00:00 +0000/2018/03/10/horovod/Sat, 10 Mar 2018 00:00:00 +0000/2018/03/10/horovod/<!-- toc --> <p>[TOC]</p> <h1 id="官方介绍">官方介绍</h1> <p>Horovod is a distributed training framework for TensorFlow, Keras, and PyTorch. The goal of Horovod is to make distributed Deep Learning fast and easy to use.</p> <p>官方测试效果</p> <p><img alt="training" loading="lazy" src="/2018/03/10/horovod/horovod.png"></p> <h1 id="running-horovod">Running Horovod</h1> <p>The example commands below show how to run distributed training. See the <a href="https://github.com/uber/horovod/blob/master/docs/running.md">Running Horovod</a> page for more instructions, including RoCE/InfiniBand tweaks and tips for dealing with hangs.</p> <h2 id="1-单机4卡">1. 单机4卡:</h2> <pre tabindex="0"><code># docker nvidia-docker run -it 172.16.10.10:5000/horovod:0.12.1-tf1.8.0-py3.5 mpirun -np 4 -H localhost:4 python keras_mnist_advanced.py # singularity singularity shell --nv /scratch/containers/ubuntu.simg mpirun -np 4 -H localhost:4 python keras_mnist_advanced.py </code></pre><h2 id="2-多机多卡">2. 多机多卡:</h2> <pre tabindex="0"><code>$ mpirun -np 16 \ -H server1:4,server2:4,server3:4,server4:4 \ ... python train.py </code></pre><h2 id="3--完整-docker-使用horovod">3. <a href="https://github.com/uber/horovod/blob/master/docs/docker.md">完整 Docker 使用horovod </a></h2> <p>​</p>