Skip to content

如何使用其他模型 #34

@zhmingzhaung

Description

@zhmingzhaung

以下是我所使用的指令:

:<<!
[script description]: use neural machine translation model to translate 
[dataset]: multi domain DE-EN dataset
[base model]: WMT19 DE-EN
!
# this line speed up faiss. base nmt dosent need faiss, 
# we set this environment variable here just for fair comparison.
export OMP_WAIT_POLICY=PASSIVE

PROJECT_PATH=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd )/../..
DATA_PATH=$PROJECT_PATH/data-bin/name
BASE_MODEL=$PROJECT_PATH/pretrain-models/name/checkpoint_best.pt

CUDA_VISIBLE_DEVICES=0 python $PROJECT_PATH/fairseq_cli/generate.py $DATA_PATH \
--task translation \
--path $BASE_MODEL \
--dataset-impl mmap \
--beam 4 --lenpen 0.6 --max-len-a 1.2 --max-len-b 10 --source-lang en --target-lang zh \
--gen-subset test \
--model-overrides "{'eval_bleu': False, 'required_seq_len_multiple':1, 'load_alignments': False}" \
--max-tokens 2048 \
--scoring sacrebleu \
--tokenizer moses --remove-bpe \

其中,DATA_PATHBASE_MODEL用的是我自己训练的。运行指令报错:

Traceback (most recent call last):
  File "/home/jncbs/huzf/knn-box/knnbox-scripts/base-nmt/../../fairseq_cli/generate.py", line 383, in <module>
    cli_main()
  File "/home/jncbs/huzf/knn-box/knnbox-scripts/base-nmt/../../fairseq_cli/generate.py", line 379, in cli_main
    main(args)
  File "/home/jncbs/huzf/knn-box/knnbox-scripts/base-nmt/../../fairseq_cli/generate.py", line 41, in main
    return _main(args, sys.stdout)
  File "/home/jncbs/huzf/knn-box/knnbox-scripts/base-nmt/../../fairseq_cli/generate.py", line 94, in _main
    num_shards=args.checkpoint_shard_count,
  File "/home/jncbs/huzf/knn-box/fairseq/checkpoint_utils.py", line 270, in load_model_ensemble
    num_shards,
  File "/home/jncbs/huzf/knn-box/fairseq/checkpoint_utils.py", line 296, in load_model_ensemble_and_task
    state = load_checkpoint_to_cpu(filename, arg_overrides)
  File "/home/jncbs/huzf/knn-box/fairseq/checkpoint_utils.py", line 243, in load_checkpoint_to_cpu
    setattr(args, arg_name, arg_val)
AttributeError: 'NoneType' object has no attribute 'eval_bleu'

请问使用其他模型是需要像#32 一样自己重新写吗?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions