Apple Silicon

The new MacBook Pro comes with a new Arm64 chip called Apple Silicon M1. This page includes some notes about compiling VisIt for Apple Silicon. The pcocess will be used to document what works and what does not, with a goal of eventually making build_visit work seamlessly on Apple Silicon.

The Intel build of VisIt works on Mac just fine and is automatically translated to Arm64 by Rosetta. Parallel even works.

Platform

The computer runs MacOS 11 Big Sur on Arm64.

% uname -a
Darwin arcee.local 20.1.0 Darwin Kernel Version 20.1.0: Sat Oct 31 00:07:10 PDT 2020; root:xnu-7195.50.7~2/RELEASE_ARM64_T8101 arm64

Xcode

When starting computer for the first time, is is missing some development tools so it is necessary to install Xcode to ensure that the basic tools such as clang, make, and git are available. After installing Xcode, it is necessary to accept the license.

sudo xcodebuild -license

Missing Tools

After Xcode is installed, some useful tools are still missing, which might affect build systems for some VisIt dependencies. We'll see.

• autoconf
• wget

git-lfs

Git LFS is needed to download some of VisIt's files but it is not installed by Xcode. Git LFS requires the go language. The Mac ARM 64 version of go can be obtained from:

https://golang.org/dl/go1.15.6.darwin-amd64.pkg

Installing Go may also require Rosetta2 to be installed to handle the installation program. Once Go has been installed, open a new terminal and make sure it works:

% go version
go version go1.15.6 darwin/amd64

Clone the Git LFS repo:

git clone git@github.com:git-lfs/git-lfs.git
cd git-lfs
git lfs install

ninja

Qt 6 wants to build using ninja.

git clone  git@github.com:ninja-build/ninja.git
/usr/local/cmake-3.19.2/bin/cmake -DCMAKE_INSTALL_PREFIX=/usr/local/ninja
make -j 8
sudo make install

CMake

I started using a new CMake version 3.19.2.

https://github.com/Kitware/CMake/releases/download/v3.19.2/cmake-3.19.2.tar.gz
tar xvf cmake-3.19.2.tar.gz
cd cmake-3.19.2
./configure --prefix=/usr/local/cmake-3.19.2
make -j 8
make install

MPI

Although build_visit will try to build an MPI (mpich), I like to have one installed globally too. Building OpenMPI appears to work.

tar xvf openmpi-4.0.2.tar.gz
cd openmpi-4.0.2
./configure CC=clang CXX=clang++ --prefix=/usr/local/openmpi-4.0.2 --enable-mpi1-compatibility --enable-mpi-cxx --enable-mpi-cxx-seek
make -j 8
sudo make install

Building VisIt

VisIt has a bunch of 3rd party dependencies that are built using the build_visit script. We will use that script to build VisIt and some subset of its dependencies, which may need to be updated in order to support building on Arm64. This work is being done at GitHub on the | apple_silicon branch.

I invoked build_visit using the following and it ran into problems building openssl because openssl's build wanted to build with "-arch i386" by default. I updated it to version 1.1.1i and it then built using "-arch arm64" by default.

env PATH=/usr/local/cmake-3.19.2/bin:$PATH ./build_visit --system-cmake --no-visit --thirdparty-path /Users/bjw/Development/thirdparty_shared/3.1.0 --no-thirdparty --zlib --openssl

Python

Next, I tried building Python. It did not work.

env PATH=/usr/local/cmake-3.19.2/bin:$PATH ./build_visit --system-cmake --no-visit --thirdparty-path /Users/bjw/Development/thirdparty_shared/3.1.0 --no-thirdparty --zlib --openssl --python

Extracting setuptools ...
Extracting cython ...
Extracting numpy ...
Installing setuptools (~1 min) ...
Could not install setuptools
numpy build failed. Bailing out.
Error in build process.  See build_visit_log for more information. If the error is unclear, please include build_visit_log in a  message to the visit-users@ornl.gov list.  You will probably  need to compress the build_visit_log using a program like gzip  so it will fit within the size limits for email attachments.
Log file full path:  /Users/bjw/Development/visit/tpbuilds/build_visit_log

After that failure, I changed to using --system-python for the time being. We'll see if it works later.

Qt

I added --qt to the build_visit command line and it built for a long time. Eventually, it failed due to a bad link line.

[182/185] CXX tools/gn/visual_studio_writer.o
[183/185] CXX tools/gn/xcode_writer.o
[184/185] AR gn_lib.a
[185/185] LINK gn
FAILED: gn 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++ -O3 -fdata-sections -ffunction-sections -Wl,-dead_strip -Wl,-S -mmacosx-version-min=10.9 -pthread -o gn tools/gn/gn_main.o  base.a gn_lib.a
ld: library not found for -lc++
clang: error: linker command failed with exit code 1 (use -v to see invocation)
ninja: build stopped: subcommand failed.
Project ERROR: GN build error!
make[3]: *** [sub-gn-pro-make_first] Error 3
make[2]: *** [sub-buildtools-make_first] Error 2
make[1]: *** [sub-src-make_first] Error 2
make: *** [module-qtwebengine-make_first] Error 2

I tried updating Qt to 5.15.2 and there were still some errors. First, there was a problem in a find_sdk.py script that did not support MacOS 11.0 in some regular expression parsing. I made some changes there and the build failed later in some other rules that appear to be related to zlib, which is weird since we build a zlib that we could be telling Qt to use.

/chromium 
ERROR at //third_party/zlib/BUILD.gn:100:9: Assertion failed.
        assert(false, "Unsupported ARM OS")
        ^-----
Unsupported ARM OS
See //tools/imagediff/BUILD.gn:51:7: which caused the file to be included.
      "//third_party/zlib",
      ^-------------------
Project ERROR: GN run error!
make[3]: *** [sub-gn_run-pro-make_first] Error 3
make[2]: *** [sub-core-make_first] Error 2
make[1]: *** [sub-src-make_first] Error 2
make: *** [module-qtwebengine-make_first] Error 2

I'm trying Qt 6.0.0 per some instructions at https://bugreports.qt.io/browse/QTBUG-85279. It appears to have some CMake build support.

env PATH=/usr/local/cmake-3.19.2/bin:$PATH QMAKE_APPLE_DEVICE_ARCHS=arm64 ./configure --prefix=/Users/bjw/Development/thirdparty_shared/3.1.0/qt/6.0.0/arm-apple-darwin20_clang

I noticed this warning during the build:

 CMake Warning at qtbase/cmake/QtAutoDetect.cmake:10 (message):
 The officially supported CMake generator for building Qt is Ninja.  You are
 using: 'Unix Makefiles' instead.  Thus, you might encounter issues.  Use at
 your own risk.

Ultimately the build fails.

[ 43%] Built target QGifPlugin
[ 43%] Automatic MOC for target QTuioTouchPlugin
[ 43%] Built target QTuioTouchPlugin_autogen
make[2]: *** No rule to make target `qtbase/src/plugins/generic/tuiotouch/CMakeFiles/QTuioTouchPlugin.dir/depend'.  Stop.
make[1]: *** [qtbase/src/plugins/generic/tuiotouch/CMakeFiles/QTuioTouchPlugin.dir/all] Error 2
make: *** [all] Error 2

I'm trying again with Ninja.

cd qt-*
env PATH=/usr/local/cmake-3.19.2/bin:/usr/local/ninja/bin:$PATH ./configure -- -DCMAKE_INSTALL_PREFIX=/Users/bjw/Development/thirdparty_shared/3.1.0/qt/6.0.0/arm-apple-darwin20_clang QMAKE_APPLE_DEVICE=arm64 -G Ninja
/usr/local/cmake-3.19.2/bin/cmake --build . --parallel
/usr/local/cmake-3.19.2/bin/cmake --install .

That appeared to work. We'll see.

VTK

VisIt uses VTK 8.1.0 and its build does not work with Qt 6.

CMake Error at GUISupport/Qt/CMakeLists.txt:72 (find_package):
  By not providing "FindQt5.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Qt5", but
  CMake did not find one.

  Could not find a package configuration file provided by "Qt5" with any of
  the following names:

    Qt5Config.cmake
    qt5-config.cmake

  Add the installation prefix of "Qt5" to CMAKE_PREFIX_PATH or set "Qt5_DIR"
  to a directory containing one of the above files.  If "Qt5" provides a
  separate development package or SDK, be sure it has been installed.


-- Configuring incomplete, errors occurred!
See also "/Users/bjw/Development/tpbuilds/VTK-8.1.0-build/CMakeFiles/CMakeOutput.log".
See also "/Users/bjw/Development/tpbuilds/VTK-8.1.0-build/CMakeFiles/CMakeError.log".