Skip to content

计算torch.nn.MultiheadAttention的FLOPS,涉及到了NonDynamicallyQuantizableLinear=0 FLOPS  #60

@CFZ1

Description

@CFZ1

pytorch版本1.10.0,每次计算到torch.nn.MultiheadAttention,都是这样汇报的:

      (attn): MultiheadAttention(
        263.17 K = 0.5576% Params, 104.86 MMACs = 0.007% MACs, 211 MFLOPS = 0.0071% FLOPs
        (out_proj): NonDynamicallyQuantizableLinear(65.79 K = 0.1394% Params, 0 MACs = 0% MACs, 0 FLOPS = 0% FLOPs, in_features=256, out_features=256, bias=True)
      )

请问这里的NonDynamicallyQuantizableLinear=0 FLOPS 没问题吗?需要自己手动计算NonDynamicallyQuantizableLinear的FLOPS,然后加上吗?谢谢

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions