:<<!
[script description]: use neural machine translation model to translate
[dataset]: multi domain DE-EN dataset
[base model]: WMT19 DE-EN
!
# this line speed up faiss. base nmt dosent need faiss,
# we set this environment variable here just for fair comparison.
export OMP_WAIT_POLICY=PASSIVE
PROJECT_PATH=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )/../..
DATA_PATH=$PROJECT_PATH/data-bin/name
BASE_MODEL=$PROJECT_PATH/pretrain-models/name/checkpoint_best.pt
CUDA_VISIBLE_DEVICES=0 python $PROJECT_PATH/fairseq_cli/generate.py $DATA_PATH \
--task translation \
--path $BASE_MODEL \
--dataset-impl mmap \
--beam 4 --lenpen 0.6 --max-len-a 1.2 --max-len-b 10 --source-lang en --target-lang zh \
--gen-subset test \
--model-overrides "{'eval_bleu': False, 'required_seq_len_multiple':1, 'load_alignments': False}" \
--max-tokens 2048 \
--scoring sacrebleu \
--tokenizer moses --remove-bpe \
Traceback (most recent call last):
File "/home/jncbs/huzf/knn-box/knnbox-scripts/base-nmt/../../fairseq_cli/generate.py", line 383, in <module>
cli_main()
File "/home/jncbs/huzf/knn-box/knnbox-scripts/base-nmt/../../fairseq_cli/generate.py", line 379, in cli_main
main(args)
File "/home/jncbs/huzf/knn-box/knnbox-scripts/base-nmt/../../fairseq_cli/generate.py", line 41, in main
return _main(args, sys.stdout)
File "/home/jncbs/huzf/knn-box/knnbox-scripts/base-nmt/../../fairseq_cli/generate.py", line 94, in _main
num_shards=args.checkpoint_shard_count,
File "/home/jncbs/huzf/knn-box/fairseq/checkpoint_utils.py", line 270, in load_model_ensemble
num_shards,
File "/home/jncbs/huzf/knn-box/fairseq/checkpoint_utils.py", line 296, in load_model_ensemble_and_task
state = load_checkpoint_to_cpu(filename, arg_overrides)
File "/home/jncbs/huzf/knn-box/fairseq/checkpoint_utils.py", line 243, in load_checkpoint_to_cpu
setattr(args, arg_name, arg_val)
AttributeError: 'NoneType' object has no attribute 'eval_bleu'
以下是我所使用的指令:
其中,
DATA_PATH和BASE_MODEL用的是我自己训练的。运行指令报错:请问使用其他模型是需要像#32 一样自己重新写吗?