Singularity

[TOC] 准备前提 首先你的pc要安装了singularity,下载好了cuda,cudnn,anaconda.而在集群上这些都已经准备好了. 制作基础系统镜像 mkdir ~/deepchem-build && cd ~/deepchem-build cp /atlas/backup/images/centos.def ./ singularity build --sandbox centos centos.def 准备下载好的应用. cp /atlas/backup/software/{Anaconda3-5.1.0-Linux-x86_64.sh, cudnn-9.1-linux-x64-v7.1.tgz } ./ cp /etc/yum.repos.d/base.repo ./ # base.repo 中是在安装软件过程中所依赖的绝大部分安装包和cuda, 用本地源提高速度节省时间. cat /etc/yum.repos.d/base.repo [development] name=development baseurl=ftp://172.16.10.10/centos7.2 gpgcheck=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7 [epel] name=epel baseurl=ftp://172.16.10.10/epel gpgcheck=0 [cuda] name=cuda baseurl=ftp://172.16.10.10/cuda/cuda gpgcheck=0 在镜像中安装应用. singularity shell -w centos cp base.repo /etc/yum.repos.d/ yum repolist yum -y install vim wget perl git curl bzip2 export LANG=en_US.UTF-8 #设定系统字符编码 # install Anaconda3 bash Anaconda3-5.1.0-Linux-x86_64.sh -f -b -p -s /usr/local/anaconda3 export PATH=/usr/local/anaconda3/bin:$PATH # install nvidia-driver cuda yum install cuda tar xf cudnn-9.1-linux-x64-v7.1.tgz -C /usr/local/ # test nvidia-smi nvcc --version # if you can not get gpu information by nvidia-smi # reload the nvidia modules lsmod | grep nvidia systemctl stop gmond rmmod nvidia_drm rmmod nvidia_modeset rmmod nvidia_uvm rmmod nvidia nvidia-smi #or cannot to reboot , can change the nvidia kernel module by hand rm -rf /lib/modules/3.10.0-327.el7.x86_64/extra/nvidia* cp /var/lib/dkms/nvidia/387.26/3.10.0-327.el7.x86_64/x86_64/module/nvidia* /lib/modules/3.10.0-327.el7.x86_64/extra/ nvidia-smi # add environment value cat /environment export LANG=en_US.UTF-8 export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH export PATH=/usr/local/anaconda3/bin:/usr/local/cuda/bin:$PATH # install tensorflow-gpu and keras and other modules can install by conda cat ~/.condarc channels: - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/menpo/ - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/bioconda/ - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/msys2/ - https://mirrors.tuna.tsinghua.edu.cn/anaconda/cloud/conda-forge/ - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/main/ - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/free/ - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/mro/ - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/pro/ - https://mirrors.tuna.tsinghua.edu.cn/anaconda/pkgs/r/ - defaults show_channel_urls: true conda install tensorflow-gpu==1.6.0 conda install keras-gpu conda install pytest pytest-html pytest-cov pydicom gevent numba autobahn pymongo redis rq txaio twisted conda install -c menpo opencv3 # install deepchem conda install -c deepchem -c rdkit -c conda-forge -c omnia deepchem-gpu=2.0.0 # install openslide and autobahn_autoreconnect etc. can only install by pip pip install openslide-python autobahn_autoreconnect wget http://ftp.gnu.org/pub/gnu/libiconv/libiconv-1.15.tar.gz && tar xf libiconv-1.15.tar.gz && cd libiconv-1.15 ./configure && make -j 12 && make install ln -sv /usr/local/lib/libiconv.so.2 /usr/lib/libiconv.so.2 ldconfig cd .. && rm -rf libiconv-1.15 conda clean -a yum clean all exit 生成只读镜像 ...

[TOC] 首先是下载源码，singularity下载地址：http://singularity.lbl.gov/all-releases 下载到本地之后，解压 tar xvf singularity-2.2.1.tar.gz mkdir /gensoft/singularity cd singularity-2.2.1/ ./configure make make install 到这里其实singularity已经安装完毕，下面是参照官网说明做的一个镜像 BootStrap: yum OSVersion: 7 MirrorURL: http://mirror.centos.org/centos/7/os/x86_64/ Include: yum [yhu@master ~]$sudo [root@7-2 singularity-2.2.1]#singularity shell –contain /tmp/Centos7.img Singularity: Invoking an interactive shell within container… Singularity.Centos7.img> id uid=0(root) gid=0(root) groups=0(root) Singularity.Centos7.img> ls / bin dev etc lib lost+found mnt proc run singularity sys usr boot environment home lib64 media opt root sbin srv tmp var Singularity.Centos7.img> exit exit ...

[TOC] 制作容器 选项一 导入docker容器 使用现成的docker容器，比如docker hub上面的tensorflow： sudo singularity create --size 4000 tensorflow.img sudo singularity import tensorflow.img docker://tensorflow/tensorflow:latest 选项二 自己制作singularity容器 制作一个singularity容器需要两步，需要在任何一台有root权限的linux计算机上进行： 1 创建空白镜像文件 sudo singularity create --size 1200 keras2.0_cpu.img 其中 size用来制定大小，单位是MB，上面将创建一个1.2GB的空白镜像文件。 2 接下来需要在这个镜像中写入系统和预装的软件 创建 keras2.0_cpu.def 文件，内容通过符号%分割成几个部分，分别代表不同的功能。 我们主要要写的内容是1） %setup； 2）%post # 基于docker的ubuntu镜像 BootStrap: docker From: ubuntu:16.04 %runscript echo "This is what happens when you run the container..." bash %setup # 1）这一步可以先设置好环境变量，通过wget等下载好必要的软件库等，注意这些指令都只是在host机器上运行，不是container里面。 # 内容请参考https://git.oschina.net/sg-ai/singularityimages/blob/master/keras-tf-1.0.1-gpu.setup %post # non interactive debian DEBIAN_FRONTEND=noninteractive # Install the necessary packages (from repo) apt-get update && apt-get install -y --no-install-recommends curl ca-certificates # 2）这些指令都在container里面运行，在这里写你要安装和设置的东西，可以使用apt install之类的安装需要的软件，这样就会被安装到容器里面去 # 内容请参考https://git.oschina.net/sg-ai/singularityimages/blob/master/keras-tf-1.0.1-gpu.post 接着运行： ...