Overview
- 记录error处理方法
mkl Cannot load libmkl_avx2.so or libmkl_def.so
运行faiss的时候遇到Cannot load libmkl_avx2.so or libmkl_def.so.问题
- 无需重新安装, 添加命令
export LD_PRELOAD=/<option your path>/anaconda2/lib/libmkl_rt.so
到~/.bashrc中
- 问题: 是因为环境配置过程中mkl库与其他库的依赖关系出现混乱导致的, 即mkl动态库对应有多处路径
- LD_PRELOAD是个环境变量, 用于动态库的加载, 动态库加载的优先级最高, 一般情况下, 其加载顺序为
LD_PRELOAD > LD_LIBRARY_PATH > /etc/ld.so.cache> /lib > /usr/lib
因此使用最高级别的直接指定, 由于是anaconda中的python内部报错, 所以指定anaconda/lib
报错如下
OverflowError: cannot serialize a bytes object larger than 4 GiB
解决方法
pickle dump OverflowError
pickle.dump(d, open("file", 'w'), protocol=4)
官方文档
Protocol version 0 is the original “human-readable” protocol and is backwards compatible with earlier versions of Python.
Protocol version 1 is an old binary format which is also compatible with earlier versions of Python.
Protocol version 2 was introduced in Python 2.3. It provides much more efficient pickling of new-style classes. Refer to PEP 307 for information about improvements brought by protocol 2.
Protocol version 3 was added in Python 3.0. It has explicit support for bytes objects and cannot be unpickled by Python 2.x. This is the default protocol, and the recommended protocol when compatibility with other Python 3 versions is required.
Protocol version 4 was added in Python 3.4. It adds support for very large objects, pickling more kinds of objects, and some data format optimizations. Refer to PEP 3154 for information about improvements brought by protocol 4.
Reference
转载请注明来源, from goldandrabbit.github.io
