Developer Installation (Installation problems)

Hi :wave:
I’ve been working with Taichi on my Mac and Linux PC, and reached the point that I’m ready to go on GPU-cluster. However. I’m facing a lot of issues while installing Taichi in developer mode.

Following the instructions, I download the Taichi git repository as

git clone --recursive https://github.com/taichi-dev/taichi

cd taichi

I’m aiming for a Linux-based GPU-cluster with Nvidia GPUs but without GUI, thus I set the flags:

export TAICHI_CMAKE_ARGS="
  -DTI_WITH_VULKAN:BOOL=OFF
  -DTI_WITH_CUDA:BOOL=ON 
  -DTI_WITH_METAL:BOOL=OFF
  -DTI_WITH_OPENGL:BOOL=OFF
"
./build.py --shell

The script executes but fails to download a package for vulkan-1.3.236.0. I’m not sure why, but is easy to fix: The problem lies in the vulkan.py script as it cannot download the version 1.3.236.0 because it has been removed. I proceed to modify the script as:

# -*- coding: utf-8 -*-

# -- stdlib --
import os
import platform

# -- third party --
# -- own --
from .dep import download_dep
from .misc import banner, get_cache_home, path_prepend
from .python import path_prepend


# -- code --
@banner("Setup Vulkan 1.3.236.0")
def setup_vulkan():
    u = platform.uname()
    if u.system == "Linux":
        url = "https://sdk.lunarg.com/sdk/download/1.3.296.0/linux/vulkansdk-linux-x86_64-1.3.296.0.tar.xz"
        prefix = get_cache_home() / "vulkan-1.3.296.0"
        download_dep(url, prefix, strip=1)
        sdk = prefix / "x86_64"
        os.environ["VULKAN_SDK"] = str(sdk)
        path_prepend("PATH", sdk / "bin")
        path_prepend("LD_LIBRARY_PATH", sdk / "lib")
        os.environ["VK_LAYER_PATH"] = str(sdk / "etc" / "vulkan" / "explicit_layer.d")
    # elif (u.system, u.machine) == ("Darwin", "arm64"):
    # elif (u.system, u.machine) == ("Darwin", "x86_64"):
    elif (u.system, u.machine) == ("Windows", "AMD64"):
        url = "https://sdk.lunarg.com/sdk/download/1.3.236.0/windows/VulkanSDK-1.3.236.0-Installer.exe"
        prefix = get_cache_home() / "vulkan-1.3.236.0"
        download_dep(
            url,
            prefix,
            elevate=True,
            args=[
                "--accept-licenses",
                "--default-answer",
                "--confirm-command",
                "--root",
                prefix,
                "install",
                "com.lunarg.vulkan.sdl2",
                "com.lunarg.vulkan.glm",
                "com.lunarg.vulkan.volk",
                "com.lunarg.vulkan.vma",
                # 'com.lunarg.vulkan.debug',
            ],
        )
        os.environ["VULKAN_SDK"] = str(prefix)
        os.environ["VK_SDK_PATH"] = str(prefix)
        os.environ["VK_LAYER_PATH"] = str(prefix / "Bin")
        path_prepend("PATH", prefix / "Bin")
    else:
        return

and rerun ./build.py --shell command. The build executable complete its task correctly. Now, I’m intending to install the project within the build enviroment by using

python3 setup.py develop

But during the construction I a new error message that reads:

[Taichi Build] (master) taichi $ python setup.py develop
[]
['taichi', 'taichi.graph', 'taichi.tools', 'taichi.aot', 'taichi._snode', 'taichi.profiler', 'taichi.lang', 'taichi._ti_module', 'taichi.ui', 'taichi.types', 'taichi.math', 'taichi.ad', 'taichi._lib', 'taichi.sparse', 'taichi.linalg', 'taichi.algorithms', 'taichi.aot.conventions', 'taichi.aot.conventions.gfxruntime140', 'taichi.lang.simt', 'taichi.lang.ast']
/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/setuptools/dist.py:759: SetuptoolsDeprecationWarning: License classifiers are deprecated.
!!

        ********************************************************************************
        Please consider removing the following classifiers in favor of a SPDX license expression:

        License :: OSI Approved :: Apache Software License

        See https://packaging.python.org/en/latest/guides/writing-pyproject-toml/#license for details.
        ********************************************************************************

!!
  self._finalize_license_expression()
/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/setuptools/_distutils/cmd.py:90: DevelopDeprecationWarning: develop command is deprecated.
!!

        ********************************************************************************
        Please avoid running ``setup.py`` and ``develop``.
        Instead, use standards-based tools like pip or uv.

        By 2025-Oct-31, you need to update your project and remove deprecated calls
        or your builds will no longer be supported.

        See https://github.com/pypa/setuptools/issues/917 for details.
        ********************************************************************************

!!
  self.initialize_options()


--------------------------------------------------------------------------------
-- Trying 'Ninja' generator
--------------------------------
---------------------------
----------------------
-----------------
------------
-------
--
Not searching for unused variables given on the command line.
CMake Deprecation Warning at CMakeLists.txt:1 (cmake_minimum_required):
  Compatibility with CMake < 3.10 will be removed from a future version of
  CMake.

  Update the VERSION argument <min> value.  Or, use the <min>...<max> syntax
  to tell CMake that the project requires at least <min> but has been updated
  to work with policies introduced by <max> or earlier.


-- The C compiler identification is Clang 18.1.8
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /gpfs/softs/easybuild/2024a/software/Clang/18.1.8-GCCcore-13.3.0/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- The CXX compiler identification is Clang 18.1.8
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /gpfs/softs/easybuild/2024a/software/Clang/18.1.8-GCCcore-13.3.0/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Configuring done (0.9s)
-- Generating done (0.0s)
-- Build files have been written to: /gpfs/home/cenaero/mdiaz/Depots/taichi/_cmake_test_compile/build
--
-------
------------
-----------------
----------------------
---------------------------
--------------------------------
-- Trying 'Ninja' generator - success
--------------------------------------------------------------------------------

Configuring Project
  Working directory:
    /gpfs/home/cenaero/mdiaz/Depots/taichi/_skbuild/linux-x86_64-3.12/cmake-build
  Command:
    /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/cmake/data/bin/cmake /gpfs/home/cenaero/mdiaz/Depots/taichi -G Ninja -DCMAKE_MAKE_PROGRAM:FILEPATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/bin/ninja --no-warn-unused-cli -DCMAKE_INSTALL_PREFIX:PATH=/gpfs/home/cenaero/mdiaz/Depots/taichi/_skbuild/linux-x86_64-3.12/cmake-install -DPYTHON_VERSION_STRING:STRING=3.12.11 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/bin/python -DPYTHON_INCLUDE_DIR:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/include/python3.12 -DPYTHON_LIBRARY:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/libpython3.12.so -DPython_EXECUTABLE:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/bin/python -DPython_ROOT_DIR:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12 -DPython_FIND_REGISTRY:STRING=NEVER -DPython_INCLUDE_DIR:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/include/python3.12 -DPython_NumPy_INCLUDE_DIRS:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/numpy/core/include -DPython3_EXECUTABLE:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/bin/python -DPython3_ROOT_DIR:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12 -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_INCLUDE_DIR:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/include/python3.12 -DPython3_NumPy_INCLUDE_DIRS:PATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/numpy/core/include -DCMAKE_MAKE_PROGRAM:FILEPATH=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/bin/ninja -DTI_WITH_VULKAN:BOOL=OFF -DTI_WITH_CUDA:BOOL=ON -DTI_WITH_METAL:BOOL=OFF -DTI_WITH_OPENGL:BOOL=OFF -DCLANG_EXECUTABLE=/gpfs/softs/easybuild/2024a/software/Clang/18.1.8-GCCcore-13.3.0/bin/clang -DCMAKE_C_COMPILER_LAUNCHER=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/sccache-v041/bin/sccache -DCMAKE_CXX_COMPILER_LAUNCHER=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/sccache-v041/bin/sccache -DTI_VERSION_MAJOR=1 -DTI_VERSION_MINOR=8 -DTI_VERSION_PATCH=0 -DCMAKE_BUILD_TYPE:STRING=Release

Not searching for unused variables given on the command line.
Taichi Version 1.8.0
       Commit a1d99c9c4cef9de5f9b633a515d227677ffab9a0
-- Python: Using /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/bin/python as the interpreter
    version: 3.12.11
    include: /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/include/python3.12
    library: /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/libpython3.12.so
    numpy include: /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/numpy/core/include
CMake Warning (dev) at /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/pybind11/share/cmake/pybind11/FindPythonLibsNew.cmake:101 (message):
  Policy CMP0148 is not set: The FindPythonInterp and FindPythonLibs modules
  are removed.  Run "cmake --help-policy CMP0148" for policy details.  Use
  the cmake_policy command to set the policy and suppress this warning, or
  preferably upgrade to using FindPython, either by calling it explicitly
  before pybind11, or by setting PYBIND11_FINDPYTHON ON before pybind11.
Call Stack (most recent call first):
  /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/pybind11/share/cmake/pybind11/pybind11Tools.cmake:50 (find_package)
  /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/pybind11/share/cmake/pybind11/pybind11Common.cmake:228 (include)
  /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/pybind11/share/cmake/pybind11/pybind11Config.cmake:250 (include)
  cmake/PythonNumpyPybind11.cmake:15 (find_package)
  CMakeLists.txt:118 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Found pybind11: /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/mambaforge/envs/3.12/lib/python3.12/site-packages/pybind11/include (found version "2.13.6")
Using C++ compiler: /gpfs/softs/easybuild/2024a/software/Clang/18.1.8-GCCcore-13.3.0/bin/clang++
Clang compiler detected. Using std=c++17.
Building for processor x86_64
Setting -march=nehalem for x86_64 processors
Using float32 (single) precision as real
CMake Warning at cmake/TaichiCore.cmake:68 (message):
  external/glad submodule not detected.  Settings TI_WITH_OPENGL to OFF.
Call Stack (most recent call first):
  CMakeLists.txt:130 (include)


Getting LLVM_DIR=/gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/llvm15 from the environment variable
-- Found LLVM 15.0.4
-- Using LLVMConfig.cmake in: /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/llvm15/lib/cmake/llvm
LLVM include dirs /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/llvm15/include
LLVM library dirs /gpfs/home/cenaero/mdiaz/.cache/ti-build-cache/llvm15/lib
CMake Error at cmake/TaichiCore.cmake:290 (add_subdirectory):
  The source directory

    /gpfs/home/cenaero/mdiaz/Depots/taichi/external/SPIRV-Tools

  does not contain a CMakeLists.txt file.
Call Stack (most recent call first):
  CMakeLists.txt:130 (include)
...

Clearly the problem now is that the git submodule(s) are not initialized. Thus, I call

git submodule init
git submodule update

Once the modules are loaded I call python3 setup.py develop again and managed to install taichi-1.8.0. However, as I call the example

[Taichi Build] (master) taichi $ python python/taichi/examples/simulation/mpm128.py
[Taichi] version 1.8.0, llvm 15.0.4, commit a1d99c9c, linux, python 3.12.11
[Taichi] Starting on arch=x64
[W 07/01/25 15:47:34.509 3516996] [llvm_context.cpp:load@263] Bitcode loading error message:
Unknown attribute kind (91) (Producer: 'LLVM18.1.8' Reader: 'LLVM 15.0.4')
[E 07/01/25 15:47:34.509 3516996] [llvm_context.cpp:load@265] Failed to load bitcode=/gpfs/home/cenaero/mdiaz/Depots/taichi/python/taichi/_lib/runtime/runtime_x64.bc


Program aborted due to an unhandled Error:
Unknown attribute kind (91) (Producer: 'LLVM18.1.8' Reader: 'LLVM 15.0.4')
Aborted (core dumped)

I observe that I have inconsistent versions of LLVM code produced.
On my machine I have LLVM-18 and Clang-18, but for some reason the installation introduced it’s own LLVM-15 package.

Question: How can I modify the build.py internal calls to avoid using and downloading the llvm-15 package and instead use the llvm-18.1.8 currently installed in the system?