12345qiupeng
/
gtsam
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

modernize containers 

use compose spec, use single image repo with tags for various configs, create a docker hub push script, clean up/slim down container files
release/4.3a0
Patel, Akash M 2024-07-08 22:35:49 -05:00
parent 5550537709
commit 24320deaaf
18 changed files with 267 additions and 185 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

45
containers/Containerfile Normal file
View File

@ -0,0 +1,45 @@
# base image off ubuntu image
ARG UBUNTU_TAG=22.04
FROM docker.io/ubuntu:${UBUNTU_TAG}
RUN apt-get update && apt-get install -y --no-install-recommends \
# dependencies
libboost-all-dev \
# optional dependencies
libtbb-dev \
python3-dev \
python3-pip \
python3-pyparsing \
python3-numpy \
# build dependencies
build-essential \
cmake \
# download dependencies
git \
ca-certificates && \
rm -rf /var/lib/apt/lists/*
# build flags
ARG GTSAM_GIT_TAG=4.2.0
ARG GTSAM_WITH_TBB=ON
ARG GTSAM_BUILD_PYTHON=ON
ARG CORES=4
# build and install gtsam
RUN mkdir -p /src/github/borglab && cd /src/github/borglab && \
git clone https://github.com/borglab/gtsam --depth 1 --branch ${GTSAM_GIT_TAG} && \
cd gtsam && \
mkdir build && \
cd build && \
cmake \
-DCMAKE_BUILD_TYPE=Release \
-DGTSAM_BUILD_TESTS=OFF \
-DGTSAM_WITH_TBB=${GTSAM_WITH_TBB} \
-DGTSAM_BUILD_PYTHON=${GTSAM_BUILD_PYTHON} \
.. && \
make -j${CORES} install && \
if [ "${GTSAM_BUILD_PYTHON}" = "ON" ] ; then \
make python-install; \
fi
CMD ["/bin/bash"]

127
containers/README.md Normal file
View File

@ -0,0 +1,127 @@
# GTSAM Containers
- container files to build images
- script to push images to a registry
- instructions to pull images and run containers
## Dependencies
- a container engine such as [`Docker Engine`](https://docs.docker.com/engine/install/)
## Pull from Docker Hub
Various GTSAM image configurations are available at [`docker.io/borglab/gtsam`](https://hub.docker.com/r/borglab/gtsam). Determine which [tag](https://hub.docker.com/r/borglab/gtsam/tags) you want and pull the image.
Example for pulling an image with GTSAM compiled with TBB and Python support on top of a base Ubuntu 22.04 image.
```bash
docker pull docker.io/borglab/gtsam:4.2.0-tbb-ON-python-ON_22.04
```
[`docker.io/borglab/gtsam-vnc`](https://hub.docker.com/r/borglab/gtsam-vnc) is also provided as an image with GTSAM that will run a VNC server to connect to.
## Using the images
### Just GTSAM
To start the image, execute
```bash
docker run -it borglab/gtsam:4.2.0-tbb-ON-python-OFF_22.04
```
after you will find yourself in a bash shell.
### GTSAM with Python wrapper
To use GTSAM via the python wrapper, similarly execute
```bash
docker run -it borglab/gtsam:4.2.0-tbb-ON-python-ON_22.04
```
and then launch `python3`:
```bash
python3
>>> import gtsam
>>> gtsam.Pose2(1,2,3)
(1, 2, 3)
```
### GTSAM with Python wrapper and VNC
First, start the image, which will run a VNC server on port 5900:
```bash
docker run -p 5900:5900 borglab/gtsam-vnc:4.2.0-tbb-ON-python-ON_22.04
```
Then open a remote VNC X client, for example:
#### Linux
```bash
sudo apt-get install tigervnc-viewer
xtigervncviewer :5900
```
#### Mac
The Finder's "Connect to Server..." with `vnc://127.0.0.1` does not work, for some reason. Using the free [VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/), enter `0.0.0.0:5900` as the server.
## Build images locally
### Build Dependencies
- a [Compose Spec](https://compose-spec.io/) implementation such as [docker-compose](https://docs.docker.com/compose/install/)
### `gtsam` image
#### `.env` file
- `GTSAM_GIT_TAG`: [git tag from the gtsam repo](https://github.com/borglab/gtsam/tags)
- `UBUNTU_TAG`: image tag provided by [ubuntu](https://hub.docker.com/_/ubuntu/tags) to base the image off of
- `GTSAM_WITH_TBB`: to build GTSAM with TBB, set to `ON`
- `GTSAM_BUILD_PYTHON`: to build python bindings, set to `ON`
- `CORES`: number of cores to compile with
#### Build `gtsam` image
```bash
docker compose build
```
### `gtsam-vnc` image
#### `gtsam-vnc/.env` file
- `GTSAM_TAG`: image tag provided by [gtsam](https://hub.docker.com/r/borglab/gtsam/tags)
#### Build `gtsam-vnc` image
```bash
docker compose --file gtsam-vnc/compose.yaml build
```
## Push to Docker Hub
Make sure you are logged in via: `docker login docker.io`.
### `gtsam` images
Specify the variables described in the `.env` file in the `hub_push.sh` script.
To push images to Docker Hub, run as follows:
```bash
./hub_push.sh
```
### `gtsam-vnc` images
Specify the variables described in the `gtsam-vnc/.env` file in the `gtsam-vnc/hub_push.sh` script.
To push images to Docker Hub, run as follows:
```bash
./gtsam-vnc/hub_push.sh
```

14
containers/compose.yaml Normal file
View File

@ -0,0 +1,14 @@
services:
gtsam:
build:
args:
UBUNTU_TAG: ${UBUNTU_TAG}
GTSAM_GIT_TAG: ${GTSAM_GIT_TAG}
GTSAM_WITH_TBB: ${GTSAM_WITH_TBB}
GTSAM_BUILD_PYTHON: ${GTSAM_BUILD_PYTHON}
CORES: ${CORES}
context: .
dockerfile: Containerfile
env_file:
- .env
image: gtsam:${GTSAM_GIT_TAG}-tbb-${GTSAM_WITH_TBB}-python-${GTSAM_BUILD_PYTHON}_${UBUNTU_TAG}

20
containers/gtsam-vnc/Containerfile Normal file
View File

@ -0,0 +1,20 @@
# This image connects to the host X-server via VNC to provide a Graphical User Interface for interaction.
# base image off gtsam image
ARG GTSAM_TAG=4.2.0-tbb-ON-python-ON_22.04
FROM docker.io/borglab/gtsam:${GTSAM_TAG}
RUN apt-get update && apt-get install -y --no-install-recommends \
# Things needed to get a python GUI
python3-tk \
python3-matplotlib \
# Install a VNC X-server, Frame buffer, and windows manager
x11vnc \
xvfb \
fluxbox \
# Finally, install wmctrl needed for bootstrap script
wmctrl \
rm -rf /var/lib/apt/lists/*
COPY bootstrap.sh /
CMD ["/bootstrap.sh"]

0
docker/ubuntu-gtsam-python-vnc/bootstrap.sh → containers/gtsam-vnc/bootstrap.sh Executable file → Normal file
View File

10
containers/gtsam-vnc/compose.yaml Normal file
View File

@ -0,0 +1,10 @@
services:
gtsam_vnc:
build:
args:
GTSAM_TAG: ${GTSAM_TAG}
context: .
dockerfile: Containerfile
env_file:
- .env
image: gtsam-vnc:${GTSAM_TAG}

19
containers/gtsam-vnc/hub_push.sh Normal file
View File

@ -0,0 +1,19 @@
#!/usr/bin/env bash
# A script to push images to Docker Hub
declare -a gtsam_tags=("4.2.0-tbb-ON-python-ON_22.04")
for gtsam_tag in "${gtsam_tags[@]}"; do
touch gtsam-vnc/.env
echo "GTSAM_TAG=${gtsam_tag}" > gtsam-vnc/.env
docker compose --file gtsam-vnc/compose.yaml build
docker tag gtsam-vnc:"${gtsam_tag}" \
docker.io/borglab/gtsam-vnc:"${gtsam_tag}"
docker push docker.io/borglab/gtsam-vnc:"${gtsam_tag}"
done

32
containers/hub_push.sh Normal file
View File

@ -0,0 +1,32 @@
#!/usr/bin/env bash
# A script to push images to Docker Hub
declare -a ubuntu_tags=("22.04")
declare -a gtsam_git_tags=("4.2.0")
declare -a gtsam_with_tbb_options=("OFF" "ON")
declare -a gtsam_build_python_options=("OFF" "ON")
for ubuntu_tag in "${ubuntu_tags[@]}"; do
for gtsam_git_tag in "${gtsam_git_tags[@]}"; do
for gtsam_with_tbb in "${gtsam_with_tbb_options[@]}"; do
for gtsam_build_python in "${gtsam_build_python_options[@]}"; do
touch .env
echo "UBUNTU_TAG=${ubuntu_tag}" > .env
echo "GTSAM_GIT_TAG=${gtsam_git_tag}" >> .env
echo "GTSAM_WITH_TBB=${gtsam_with_tbb}" >> .env
echo "GTSAM_BUILD_PYTHON=${gtsam_build_python}" >> .env
echo "CORES=4" >> .env
docker compose build
docker tag gtsam:"${gtsam_git_tag}-tbb-${gtsam_with_tbb}-python-${gtsam_build_python}_${ubuntu_tag}" \
docker.io/borglab/gtsam:"${gtsam_git_tag}-tbb-${gtsam_with_tbb}-python-${gtsam_build_python}_${ubuntu_tag}"
docker push docker.io/borglab/gtsam:"${gtsam_git_tag}-tbb-${gtsam_with_tbb}-python-${gtsam_build_python}_${ubuntu_tag}"
done
done
done
done

63
docker/README.md
View File

@ -1,63 +0,0 @@
# Instructions
# Images on Docker Hub
There are 4 images available on https://hub.docker.com/orgs/borglab/repositories:
- `borglab/ubuntu-boost-tbb`: 18.06 Linux (nicknamed `bionic`) base image, with Boost and TBB installed.
- `borglab/ubuntu-gtsam`: GTSAM Release version installed in `/usr/local`.
- `borglab/ubuntu-gtsam-python`: installed GTSAM with python wrapper.
- `borglab/ubuntu-gtsam-python-vnc`: image with GTSAM+python wrapper that will run a VNC server to connect to.
# Using the images
## Just GTSAM
To start the Docker image, execute
```bash
docker run -it borglab/ubuntu-gtsam:bionic
```
after you will find yourself in a bash shell, in the directory `/usr/src/gtsam/build`.
## GTSAM with Python wrapper
To use GTSAM via the python wrapper, similarly execute
```bash
docker run -it borglab/ubuntu-gtsam-python:bionic
```
and then launch `python3`:
```bash
python3
>>> import gtsam
>>> gtsam.Pose2(1,2,3)
(1, 2, 3)
```
## GTSAM with Python wrapper and VNC
First, start the docker image, which will run a VNC server on port 5900:
```bash
docker run -p 5900:5900 borglab/ubuntu-gtsam-python-vnc:bionic
```
Then open a remote VNC X client, for example:
### Linux
```bash
sudo apt-get install tigervnc-viewer
xtigervncviewer :5900
```
### Mac
The Finder's "Connect to Server..." with `vnc://127.0.0.1` does not work, for some reason. Using the free [VNC Viewer](https://www.realvnc.com/en/connect/download/viewer/), enter `0.0.0.0:5900` as the server.
# Re-building the images locally
To build all docker images, in order:
```bash
(cd ubuntu-boost-tbb && ./build.sh)
(cd ubuntu-gtsam && ./build.sh)
(cd ubuntu-gtsam-python && ./build.sh)
(cd ubuntu-gtsam-python-vnc && ./build.sh)
```
Note: building GTSAM can take a lot of memory because of the heavy templating. It is advisable to give Docker enough resources, e.g., 8GB, to avoid OOM errors while compiling.

19
docker/ubuntu-boost-tbb/Dockerfile
View File

@ -1,19 +0,0 @@
# Basic Ubuntu 18.04 image with Boost and TBB installed. To be used for building further downstream packages.
# Get the base Ubuntu image from Docker Hub
FROM ubuntu:bionic
# Disable GUI prompts
ENV DEBIAN_FRONTEND noninteractive
# Update apps on the base image
RUN apt-get -y update && apt-get -y install
# Install C++
RUN apt-get -y install build-essential apt-utils
# Install boost and cmake
RUN apt-get -y install libboost-all-dev cmake
# Install TBB
RUN apt-get -y install libtbb-dev

3
docker/ubuntu-boost-tbb/build.sh
View File

@ -1,3 +0,0 @@
# Build command for Docker image
# TODO(dellaert): use docker compose and/or cmake
docker build --no-cache -t borglab/ubuntu-boost-tbb:bionic .

20
docker/ubuntu-gtsam-python-vnc/Dockerfile
View File

@ -1,20 +0,0 @@
# This GTSAM image connects to the host X-server via VNC to provide a Graphical User Interface for interaction.
# Get the base Ubuntu/GTSAM image from Docker Hub
FROM borglab/ubuntu-gtsam-python:bionic
# Things needed to get a python GUI
ENV DEBIAN_FRONTEND noninteractive
RUN apt install -y python-tk
RUN python3 -m pip install matplotlib
# Install a VNC X-server, Frame buffer, and windows manager
RUN apt install -y x11vnc xvfb fluxbox
# Finally, install wmctrl needed for bootstrap script
RUN apt install -y wmctrl
# Copy bootstrap script and make sure it runs
COPY bootstrap.sh /
CMD '/bootstrap.sh'

4
docker/ubuntu-gtsam-python-vnc/build.sh
View File

@ -1,4 +0,0 @@
# Build command for Docker image
# TODO(dellaert): use docker compose and/or cmake
# Needs to be run in docker/ubuntu-gtsam-python-vnc directory
docker build -t borglab/ubuntu-gtsam-python-vnc:bionic .

5
docker/ubuntu-gtsam-python-vnc/vnc.sh
View File

@ -1,5 +0,0 @@
# After running this script, connect VNC client to 0.0.0.0:5900
docker run -it \
--workdir="/usr/src/gtsam" \
-p 5900:5900 \
borglab/ubuntu-gtsam-python-vnc:bionic

30
docker/ubuntu-gtsam-python/Dockerfile
View File

@ -1,30 +0,0 @@
# GTSAM Ubuntu image with Python wrapper support.
# Get the base Ubuntu/GTSAM image from Docker Hub
FROM borglab/ubuntu-gtsam:bionic
# Install pip
RUN apt-get install -y python3-pip python3-dev
# Run cmake again, now with python toolbox on
WORKDIR /usr/src/gtsam/build
RUN cmake \
-DCMAKE_BUILD_TYPE=Release \
-DGTSAM_WITH_EIGEN_MKL=OFF \
-DGTSAM_BUILD_EXAMPLES_ALWAYS=OFF \
-DGTSAM_BUILD_TIMING_ALWAYS=OFF \
-DGTSAM_BUILD_TESTS=OFF \
-DGTSAM_BUILD_PYTHON=ON \
-DGTSAM_PYTHON_VERSION=3\
..
# Build again, as ubuntu-gtsam image cleaned
RUN make -j4 install
RUN make python-install
RUN make clean
# Needed to run python wrapper:
RUN echo 'export PYTHONPATH=/usr/local/python/:$PYTHONPATH' >> /root/.bashrc
# Run bash
CMD ["bash"]

3
docker/ubuntu-gtsam-python/build.sh
View File

@ -1,3 +0,0 @@
# Build command for Docker image
# TODO(dellaert): use docker compose and/or cmake
docker build --no-cache -t borglab/ubuntu-gtsam-python:bionic .

35
docker/ubuntu-gtsam/Dockerfile
View File

@ -1,35 +0,0 @@
# Ubuntu image with GTSAM installed. Configured with Boost and TBB support.
# Get the base Ubuntu image from Docker Hub
FROM borglab/ubuntu-boost-tbb:bionic
# Install git
RUN apt-get update && \
apt-get install -y git
# Install compiler
RUN apt-get install -y build-essential
# Clone GTSAM (develop branch)
WORKDIR /usr/src/
RUN git clone --single-branch --branch develop https://github.com/borglab/gtsam.git
# Change to build directory. Will be created automatically.
WORKDIR /usr/src/gtsam/build
# Run cmake
RUN cmake \
-DCMAKE_BUILD_TYPE=Release \
-DGTSAM_WITH_EIGEN_MKL=OFF \
-DGTSAM_BUILD_EXAMPLES_ALWAYS=OFF \
-DGTSAM_BUILD_TIMING_ALWAYS=OFF \
-DGTSAM_BUILD_TESTS=OFF \
..
# Build
RUN make -j4 install && make clean
# Needed to link with GTSAM
RUN echo 'export LD_LIBRARY_PATH=/usr/local/lib:LD_LIBRARY_PATH' >> /root/.bashrc
# Run bash
CMD ["bash"]

3
docker/ubuntu-gtsam/build.sh
View File

@ -1,3 +0,0 @@
# Build command for Docker image
# TODO(dellaert): use docker compose and/or cmake
docker build --no-cache -t borglab/ubuntu-gtsam:bionic .