Installing OpenHSI on Linux

A tutorial on installing the openhsi library and camera SDKs on Ubuntu and Rasberry Pi OS machines.
Warning

This tutorial is a work in progress.

Requirements

  • Raspberry Pi Running Raspberry Pi OS (Raspbian) or Ubuntu 18.04 (using full 64 bit versions).
    • A Raspberry Pi 4 with 8 GB RAM is recommended.
  • Jetson development boards (E.g. Xavier) will also work. Some additional packages may be needed.
  • Debian based system
    • recommended using Ubuntu 18.04 and 20.04
Important

Using openhsi requires generating large datacubes which take up lots of RAM. It is recommended to get a Raspberry Pi 4 with as much RAM as possible, especially if you want to capture raw data. Otherwise, you will need to a processing_lvl that includes binning to reduce the datacube size.

A Raspberry Pi does not have the same processing power as a typical laptop/computer and so expect the framerate to drop slightly. You want a lightweight operating system so the camera can run as fast as possible. If you have 8 GB of RAM, definitely get the Raspbian 64 bit OS so you can use > 3 GB per process. Another benefit of running on ARM64 bit is that there are prebuilt OpenCV Python wheels - if your project requires opencv-python, then 64 bit is needed.

Note

The latest 64 bit Rasberry Pi OS (formerly Raspbian) is based on Debian 11 Bullseye (equivalent to Ubuntu 21+) and so some packages may not be updated to this platform yet. Also systemd is used to run scripts on startup.

Note

Tutorial assumes you have a working OS on your system.

Tip

Hyperspectral datacubes are really large so best to save them to an external SSD.

1. Python Environment

Installing Pyenv

The system python (on raspi in particular) is not always the most user friendly if any packages are not available pre-compiled, so using an alternative environment is recommended. A good way to get a independent python environment is using pyenv. Of course any modern python install (py>3.7) should work.

To install pyenv you can use the following commands:

sudo apt-get update; sudo apt-get install make build-essential libssl-dev zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev

curl https://pyenv.run | bash

echo 'export PATH="$HOME/.pyenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(pyenv init -)"' >> ~/.bashrc
echo 'eval "$(pyenv virtualenv-init -)"' >> ~/.bashrc
echo "export PYENV_VIRTUALENV_DISABLE_PROMPT=1" >>~/.bashrc

source ~/.bashrc

With pyenv ready, you can easily install essentially any version of Python, from a vanilla cpython too miniforge too pypy. For OpenHSI, Miniforge is a good option as the base environment for minimal effort. You can also get it without pyenv if you prefer at https://github.com/conda-forge/miniforge.

Note

https://pythonspeed.com/articles/conda-vs-pip/ - some notes on conda vs pip.

Install NodeJS

The Bokeh plotting library depends on nodeJS and is used to run OpenHSI interactive tools in a web browser.

sudo apt install nodejs

Miniforge with pyenv

pyenv install miniforge3

And then use conda later.

Miniforge - Manual

Miniforge3 is great lightweight option to get a new python environment up and running that will not interfere with the system one. You can download this from here https://github.com/conda-forge/miniforge#download and then install follow instructions.

wget "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh

cpython with pyenv

pyenv install 3.8.12

And the use pip later.

Install Jupyter Lab

All the interactive tools in openhsi require Jupyter Notebooks because it uses web tools. To install, follow the steps in https://jupyterlab.readthedocs.io/en/stable/getting_started/installation.html

In short, it boils down to conda install -c conda-forge jupyterlab or pip install jupyterlab.

You will also need some Jupyter Extensions. Navigate to the Jupyter Extensions tab and search for jupyterlab-manager, and jupyter_bokeh.

2. Install OpenHSI

via conda

Create a new environment and install OpenHSI (including python dependancies and 6SV, except for cameras). You can change python version as required as well.

pyenv activate miniforge3
conda create -n openhsi python=3.8 openhsi
conda activate openhsi

or using environment.yml in the main repo

conda env create -f environment.yml

via pip

Tip

If using raspi system python, some dependencies need to be installed using sudo apt install python-opencv etc.

To create a new environment and install OpenHSI.

pyenv virtualenv 3.8.12 openhsi
pyenv activate openhsi
pip install openhsi

Install 6SV (only needed for pip install)

git clone https://github.com/robintw/6S.git
cd 6S
cmake -D CMAKE_INSTALL_PREFIX=/usr/local .

3. Install Camera SDK

Install Lucid Vision Arena SDK (Sydney Photonics/Robonation OpenHSI)

The Robonation OpenHSI uses the detector from Lucid Vision Labs Inc. Their full SDK is required to use the sensor with the OpenHSI libary. This can be acquired from https://thinklucid.com/downloads-hub/.

To start the install, unzip the tar.gz package, i.e:

tar -xf  ArenaSDK_<version>_Linux_<platform>.tar.gz

And run this configiuration script/installer:

cd ArenaSDK_Linux_<platform>
sudo sh Arena_SDK_<platform>.conf
Important

You must keep the SDK files in the same location that you ran this command, or re-run the configiuration script. No files are copied or moved, just locations recorded for the arena python api to find.

And then install the arena python library (this is the same on a conda based or pip based python env):

pip install --no-deps arena_api-<version>-py3-none-any.whl

Enable Jumbo Packets

To ensure optimal performance you need to make sure your GigE link is setup for jumbo packets.

On Ubuntu system this can be done using (you may want to set this up to occur on startup):

    sudo ip link set eth0 mtu 9000
Note

eth0 should be the adapter name the camera is connected to, it may be different on your system.

Jumbo Packets on a Pi4

Using Raspberry Pi OS, some additional effort is required. https://support.thinklucid.com/knowledgebase/jumbo-frames-on-raspberry-pi/

Install Spinnaker SDK (FLIR based OpenHSI)

Download and install the Spinnaker SDK and python wheel. - https://www.flir.com.au/products/spinnaker-sdk/

Tip

On an Intel/Amd, you want amd64/x86_64 version on raspi you want aarch64 or arm64.

The 20.04 version has the follow dependancies:

sudo apt install libusb-1.0-0 libavcodec58 libavformat58 libswscale5 libswresample3 libavutil56 qt5-default

and then installing the FLIR SDK.

sudo sh install_spinnaker.sh
pip install --no-deps spinnaker_python-<your needed version>.whl

One of the prompts will ask you if you want to increase the USB buffer. Answer yes to this option.

More information on getting the FLIR SDK on a Raspberry Pi and Jetson is here https://www.flir.com.au/support-center/iis/machine-vision/application-note/using-spinnaker-on-arm-and-embedded-systems/

On Raspbian 64 bit

You will need to install Spinnaker however you’ll find that spinview-qt will not install because it is missing a dependency which we actually don’t require in openhsi. If you want to use a GUI called SpinView to view Camera output, then installing spinview-qt will let you use it but I haven’t yet figured out how on Raspbian 64 bit. It’s something to do with the QT5 package no longer being included in Debian 11+ package sources.

sudo apt install qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools

To proceed without spinview-qt, follow the hacks found in https://askubuntu.com/questions/1335184/qt5-default-not-in-ubuntu-21-04. You extract the offending package, change a line stating the dependency, then repackaged the files.

More information on getting the FLIR SDK on a Raspberry Pi and Jetson is here https://www.flir.com.au/support-center/iis/machine-vision/application-note/using-spinnaker-on-arm-and-embedded-systems/

Install Ximea SDK (Ximea based OpenHSI)

Navigate to https://www.ximea.com/support/wiki/apis/Python and click on the Linux installation instructions. This will bring you to the page https://www.ximea.com/support/wiki/apis/XIMEA_Linux_Software_Package#Installation which lists everything you need to install the Ximea API.

A 64 bit OS is required to run the GUI called XIMEA CamTool

wget https://www.ximea.com/downloads/recent/XIMEA_Linux_SP.tgz
tar xzf XIMEA_Linux_SP.tgz
cd package
./install
#./install -pcie # -pcie flag needed for PCIE cameras

If the build fails, try run the following first:

sudo apt-get update && sudo apt-get install build-essential linux-headers-"$(uname -r)"

Increase USB memory limit

This is from the XIMEA page https://www.ximea.com/support/wiki/apis/Raspberry_Pi_4_Benchmarks#Increase-USB-memory-limit

The default memory limit for buffers allocated for USB devices (usually 16MB) is too low for XIMEA USB cameras.This can be changed permanently following these steps:

  1. Open the file /boot/firmware/cmdline.txt in your preferred texteditor
  2. Append the following (in the same line): usbcore.usbfs_memory_mb=0
  3. Reboot the system
  4. Check if the changes are applied. The following command should print 0:
cat /sys/module/usbcore/parameters/usbfs_memory_mb

4. Additional Setup for sensor package on Raspberry Pi (optional)

4.1 Enable I2C and UART

Open the Raspberry Pi config tool.

sudo raspi-config

And enable I2C and UART. For more instructions see https://askubuntu.com/questions/1273700/enable-spi-and-i2c-on-ubuntu-20-04-raspberry-pi. After enabling them, add the user pi to the group.

sudo usermod -a -G dialout pi
sudo usermod -a -G i2c pi

Now, to use UART at high speed, we need to change a few more settings documented here https://www.abelectronics.co.uk/kb/article/1035/serial-port-setup-in-raspberry-pi-os

4.2 Disable serial login shell

remove console=serial0,115200 from /boot/cmdline.txt then reboot. This will allow us to run a higher baud rate.

4.3 Use PI0UART instead of miniUART

Add to end of file /boot/config.txt we need to add a configuration to /boot/config.txt.

sudo echo 'dtoverlay=disable-bt' >> /boot/config.txt

Then in the terminal

sudo systemctl disable hciuart
sudo reboot
Note

The trade off with allowing higher baud rates is no bluetooth.

4.4 Controlling GPIO pins

If you are running this on a Raspberry Pi 4, you’ll need to install the GPIO library. pip install RPi.GPIO

If you are running this on a Jetson, you’ll need the corresponding Jetson version. pip install Jetson.GPIO

4.5 Installing the Battery Hat software

To deploy the Raspberry Pi on a remote platform, it needs power. We use the X728 version 2.1 battery hat from Suptronics which lets us turn on and safe shutdown the Pi while indicating power left.

Since we are using Debian 11, we need to use systemd to enable /etc/rc-local which the battery hat scripts use. Follow the steps in https://www.linuxbabe.com/linux-server/how-to-enable-etcrc-local-with-systemd

The software for the battery hat can be found here: https://wiki.geekworm.com/X728-Software Be aware that you need to make some changes to get them to install Change python-smbus to python3-smbus.

sudo apt-get install python3-smbus
pip install smbus

Add PY_VERSION=3 on line 86 in x728-v2.1.sh to force the installer to use Python 3. Python 2 is deprecated so convert x728pld.py to Python 3 syntax. All the print statements need ( and ).

4.6 - Mounting SSD without sudo

The reason why we want to mount the SSD without sudo is that we can then write to it without sudo.

First, identify your SSD using sudo fdisk -l I followed the steps in https://blog.onetwentyseven001.com/mounting-without-sudo/index.html

Ready to go!

Congratulations, you now have a system that can run your openhsi!