module_adapter: Fix memory leak #21122

Job	Run time
LP64-WIP	3m 34s
build-windows (-d, lnl)	6m 41s
build-windows (ptl)	6m 50s
build-windows (mtl lnl)	7m 36s
manifest-check	51s
build-windows (tgl tgl-h)	8m 3s
build-windows (imx8 imx8x imx8m imx8ulp)	9m 9s
build-linux (zmain, imx8 imx8x imx8m imx8ulp)	6m 40s
build-linux (mnfst, mtl lnl)	5m 19s
build-linux (zmain, mtl lnl)	5m 47s
build-linux (mnfst, ptl)	4m 7s
build-linux (mnfst, tgl tgl-h)	5m 3s
build-linux (-d, mnfst, lnl)	4m 18s
build-linux (zmain, tgl tgl-h)	5m 49s
build-linux (zmain, ptl)	4m 25s
build-linux (mnfst, --all)	15m 33s
build-linux (mnfst, imx8 imx8x imx8m imx8ulp)	6m 5s
compare-linux-win	1m 46s
	1h 47m 36s

Provide feedback