在使用ChatGLM进行Lora训练的时候，遇到一个问题：

CUDA SETUP: CUDA version lower than 11 are currently not supported for LLM.int8(). You will be only to use 8-bit optimizers and quantization routines!!
CUDA SETUP: CUDA runtime path found: /data/cuda/cuda-10.2/cuda/lib64/libcudart.so
CUDA SETUP: Highest compute capability among GPUs detected: 7.5
CUDA SETUP: Detected CUDA version 102
CUDA SETUP: Required library version not found: libbitsandbytes_cuda102.so. Maybe you need to compile it from source?
CUDA SETUP: Defaulting to libbitsandbytes_cpu.so...

================================================ERROR=====================================
CUDA SETUP: CUDA detection failed! Possible reasons:

CUDA driver not installed
CUDA not installed
You have multiple conflicting CUDA libraries
Required library not pre-compiled for this bitsandbytes release!
CUDA SETUP: If you compiled from source, try again with make CUDA_VERSION=DETECTED_CUDA_VERSION for example, make CUDA_VERSION=113.
CUDA SETUP: The CUDA version for the compile might depend on your conda install. Inspect CUDA version via conda list | grep cuda.
================================================================================
CUDA SETUP: Something unexpected happened. Please compile from source:
git clone git@github.com:TimDettmers/bitsandbytes.git

看起来是cuda环境的问题，也就是libbitsandbytes_cuda102没有编译出来，解决办法是：

git clone git@github.com:TimDettmers/bitsandbytes.git
cd bitsandbytes
export CUDA_HOME=/usr/local/cuda-12.1 && make cuda12x CUDA_VERSION=121
export CUDA_HOME=/usr/local/cuda-12.1 && make cuda12x_nomatmul CUDA_VERSION=121
CUDA_VERSION=121 && python setup.py install

this will generate 2 files one with approximately 7mb and other with 14mb

libbitsandbytes_cuda121.so
libbitsandbytes_cuda121_nocublaslt.so
Copy those files to

python3.9/site-packages/bitsandbytes/
Restart the env and try again

这样的话就可以产生需要的so。

扫码手机观看或分享：