Skip to content

[多进程优化] 在脚本中尝试加入对numpy多线程限制 #779

@giansha

Description

@giansha

在exp_anomaly_detection.py中,下面二值化会消耗较多时间并且numpy占用的大量线程空转,进而导致其余进程无法分配到CPU资源(其余各种numpy计算同理):

...
# (3) evaluation on the test set
pred = (test_energy > threshold).astype(int)
...

在脚本中可尝试加入对numpy多线程限制以提高多个python进程中CPU计算的效率:

export CUDA_VISIBLE_DEVICES=xx

python -u run.py \
  --task_name anomaly_detection \
  --is_training 1 \
...
export CUDA_VISIBLE_DEVICES=xx

export OMP_NUM_THREADS=16
export OPENBLAS_NUM_THREADS=16
export MKL_NUM_THREADS=16
export VECLIB_MAXIMUM_THREADS=16
export NUMEXPR_NUM_THREADS=16

python -u run.py \
  --task_name anomaly_detection \
  --is_training 1 \
...

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions