Add document to clarify BKM of user space packages: alsa-lib, alsa-ucm-conf, PulseAudio, GNOME etc

[mengdonglin]

Add document to clarify Best Known Method of user space packages: alsa-lib, alsa-ucm-conf, PulseAudio, GNOME etc.

Based on the document, we'll maintain ‘BKM packages’ based on documented upstream tag + fixes being upstreamed for each program.

The BKM packages would be provided externally as Debian packages as well as source code with a branch for non-Debian distros. This will require the use of clones in thesofproject (as done for alsa-lib already).

See also internal issue GL 221

[libinyang]

@mengdonglin You mean we will add document in https://github.com/thesofproject/sof-docs, right?

[libinyang]

State summary of user space which we may need to upgrade:

1. linux-firmware
2. alsa-lib ( we need to upgrade alsa-lib to support BootSequence)
3. alsa-utils (we need to update alsa-utils to support BootSequence)
4. pulseaudio: the Ubuntu20.04 pulseaudio has some bugs. The solution is not finalized.
5. gnome sound setting: (we need to upgrade gnome-control-center to support output/input auto switch when audio device (such as heaphone) is hotplugged)
6. UCM: QA are using https://github.com/thesofproject/alsa-ucm-conf/ master branch + PR (some PRs are from alsa repo)

[plbossart]

@libinyang sounds good but can we start adding tags/versions for each, as well as the list of PRs needed on top?

[libinyang]

@plbossart Yes, I will add the tags and PRs for them.

[libinyang]

1. linux-firmware
2. alsa-lib: b13b9116050357b39c5f4964f83cd45b3da1b1d8 ('Release v1.2.3.1')
3. alsa-utils: 247d9e336d56d223f1e51dd164c39d82af5a5523 (tag: v1.2.3)
4. pulseaudio: the recipe is: 2ae94c141ff4d76d60a66a71d80acefbba3966d1('device-port: queue CARD CHANGE event before update default sink')
   https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/283
   https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/295 (not finalized)
5. gnome control center: d9ea45354ef684542877643b96f4fd012863d1a6
6. UCM: https://github.com/alsa-project/alsa-ucm-conf/ the recipe is 450bebd628975ffab153978b80f1493100910e53 + PR32 + PR36

[plbossart]

1. linux-firmware

what tag?

2. alsa-lib: b13b9116050357b39c5f4964f83cd45b3da1b1d8 ('Release v1.2.3.1')
3. alsa-utils: 247d9e336d56d223f1e51dd164c39d82af5a5523 (tag: v1.2.3)
4. pulseaudio: the recipe is: 2ae94c141ff4d76d60a66a71d80acefbba3966d1('device-port: queue CARD CHANGE event before update default sink')
   https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/283

this is merged already?

https://gitlab.freedesktop.org/pulseaudio/pulseaudio/-/merge_requests/295 (not finalized)
5. gnome control center: d9ea45354ef684542877643b96f4fd012863d1a6

what about https://gitlab.gnome.org/GNOME/libgnome-volume-control/-/merge_requests/10

6. UCM: https://github.com/alsa-project/alsa-ucm-conf/ the recipe is 450bebd628975ffab153978b80f1493100910e53 + PR32 + PR36

[marc-hb]

A Debian source package is traditionally made of an upstream.orig.tar.bz2 + debian/patches/*.patch files and other debian/* control files packages in a second, downstream "debian diff" tarball. Lucky us, for alsa-lib (and others?) Debian provides something much better than tarballs: git!

While downloading the above, apt-get source alsa-lib prints this awesome advice:

Reading package lists... Done
NOTICE: 'alsa-lib' packaging is maintained in the 'Git' version control system at:
https://salsa.debian.org/alsa-team/alsa-lib.git
Please use:
git clone https://salsa.debian.org/alsa-team/alsa-lib.git
to retrieve the latest (possibly unreleased) updates to the package.

Get:1 http://  /pub/mirrors/ubuntu focal-updates/main alsa-lib 1.2.2-2.1ubuntu1 (dsc) [2,007 B]
Get:2 http://  /pub/mirrors/ubuntu focal-updates/main alsa-lib 1.2.2-2.1ubuntu1 (tar) [1,031 kB]
Get:3 http://  /pub/mirrors/ubuntu focal-updates/main alsa-lib 1.2.2-2.1ubuntu1 (diff) [30.4 kB]

So let's follow this advice: delete the above and start over:

mkdir DebianBuild && cd DebianBuild # debuild creates .deb files in the parent directory
git clone https://salsa.debian.org/alsa-team/alsa-lib.git

Optional: git merge or git cherry-pick. For instance let's fix the super annoying thesofproject/sof#2543 ASRC1.0 regression that breaks sof/scripts/build-tools.sh -t

git remote add upstream  git://git.alsa-project.org/alsa-lib.git
git fetch upstream
git cherry-pick  v1.2.2-6-gcaf77a93cef5
git merge sof/has-a-better-alsa-lib

After testing this branch can now be shared somewhere like https://github.com/thesofproject/alsa-lib/ubuntu20

One-command build:

./debian/rules clean
debuild --build=binary --unsigned-source --unsigned-changes --diff-ignore --tar-ignore 

This will typically fails the first time because of some missing dependencies. apt-get install them, rinse repeat:

time debuild -b -us -uc -i -I

dpkg-deb: building package 'libasound2' in '../libasound2_1.2.2-2.1_amd64.deb'.
dpkg-deb: building package 'libasound2-dev' in '../libasound2-dev_1.2.2-2.1_amd64.deb'.
dpkg-deb: building package 'libasound2-data' in '../libasound2-data_1.2.2-2.1_all.deb'.
dpkg-deb: building package 'libasound2-doc' in '../libasound2-doc_1.2.2-2.1_all.deb'.
dpkg-deb: building package 'libasound2-plugin-smixer' in '../libasound2-plugin-smixer_1.2.2-2.1_amd64.deb'.
dpkg-deb: building package 'libatopology2' in '../libatopology2_1.2.2-2.1_amd64.deb'.

Install the ../*.deb just created

sudo debi

More info at https://help.ubuntu.com/community/UpdatingADeb

https://salsa.debian.org/alsa-team/ has alsa-utils too, I'll prepare a branch for alsa-utils too if there's explicit approval for the above approach.

TODO: figure out a safer and less scary "clean" command than git reset --hard && git clean -fdx. make distclean goes a long way but falls a bit short.
UPDATE: ./debian/rules clean seems good enough.

[marc-hb]

alsa-lib: b13b9116050357b39c5f4964f83cd45b3da1b1d8 ('Release v1.2.3.1')

I merged this with the debian branch, fixed two minor issues and shared it there:

git clone  https://github.com/marc-hb/alsa-lib.git   -b v1.2.3.1+debian
cd alsa-lib
time debuild -b -us -uc -i -I

These three lines are now enough for me to build .deb files for v1.2.3.1, let me know if any issue.

[plbossart]

alsa-lib: b13b9116050357b39c5f4964f83cd45b3da1b1d8 ('Release v1.2.3.1')

I merged this with the debian branch, fixed two minor issues and shared it there:

git clone  git@github.com:marc-hb/alsa-lib   -b v1.2.3.1+debian
cd alsa-lib
time debuild -b -us -uc -i -I

These three lines are now enough for me to build .deb files for v1.2.3.1, let me know if any issue.

Doesn't work for me with 1.2.3.2 (actually commit 32addac948a61c4770f9cdf459fa29879602df38)

Conflicts left and right and errors on compilation.

debuild --build=binary --unsigned-source --unsigned-changes --diff-ignore --tar-ignore 
 dpkg-buildpackage -us -uc -ui --build=binary --build=binary --diff-ignore --tar-ignore
dpkg-buildpackage: info: source package alsa-lib
dpkg-buildpackage: info: source version 1.2.2-2.1
dpkg-buildpackage: info: source distribution unstable
dpkg-buildpackage: info: source changed by Matthias Klose <doko@debian.org>
 dpkg-source --diff-ignore --tar-ignore --before-build .
dpkg-buildpackage: info: host architecture amd64
dpkg-source: info: using patch list from debian/patches/series
dpkg-source: info: applying 0001-Enabled-extended-namehints-in-alsa.conf.patch
dpkg-source: info: applying python3.8.diff
patching file configure.ac
Hunk #1 FAILED at 423.
1 out of 1 hunk FAILED
dpkg-source: info: the patch has fuzz which is not allowed, or is malformed
dpkg-source: info: if patch 'python3.8.diff' is correctly applied by quilt, use 'quilt refresh' to update it
dpkg-source: info: restoring quilt backup files for python3.8.diff
dpkg-source: error: LC_ALL=C patch -t -F 0 -N -p1 -u -V never -E -b -B .pc/python3.8.diff/ --reject-file=- < alsa-lib/debian/patches/python3.8.diff subprocess returned exit status 1
dpkg-buildpackage: error: dpkg-source --diff-ignore --tar-ignore --before-build . subprocess returned exit status 2
debuild: fatal error at line 1182:
dpkg-buildpackage -us -uc -ui --build=binary --build=binary --diff-ignore --tar-ignore failed

To be clear, the goal is to use early versions or SHA1s, not necessarily released versions of alsa-lib.

[marc-hb]

git clone git@github.com:marc-hb/alsa-lib -b v1.2.3.1+debian

Sorry I meant: git clone https://github.com/marc-hb/alsa-lib.git -b v1.2.3.1+debian

Doesn't work for me with 1.2.3.2 (actually commit 32addac948a61c4770f9cdf459fa29879602df38)

That's a plain, upstream ALSA commit with nothing I did. It doesn't even have any debian/ subdirectory:

time debuild -b -us -uc -i -I
debuild: fatal error at line 679:
cannot find readable debian/changelog anywhere!
Are you in the source code tree?

You seem to have some stale debian/ subdirectory from some past experiment? Run the clone git command above from scratch in a fresh, brand new subdirectory.

To be clear, the goal is to use early versions or SHA1s, not necessarily released versions of alsa-lib.

I just git merged my v1.2.3.1+debian branch with the very latest v1.2.3.2-4-g877bdf95fdfd and everything built fine.

[plbossart]

@marc-hb i didn't use your branch because I couldn't figure out why you used an older version - I tried with the instructions above with git fetch/merge and that's how I got the merge issue.

I appreciate the help but I don't want to use something that someone else created with some magic - a script is all I need/want.

[marc-hb]

@marc-hb i didn't use your branch because I couldn't figure out why you used an older version

I used the tag listed by @libinyang in the requirements in the second comment. As tested above you can merge any later version and it will Just Work. I'll push the result soon.

I tried with the instructions above with git fetch/merge and that's how I got the merge issue.

The git command failed (and you didn't notice) because I had git@github.com:marc-hb/alsa-lib as a remote which works only for me. Fixed, try again and it will Just Work.

I appreciate the help but I don't want to use something that someone else created with some magic - a script is all I need/want.

Try the two commands above. Repeating them here:

git clone  https://github.com/marc-hb/alsa-lib.git   -b v1.2.3.1+debian
cd alsa-lib
time debuild -b -us -uc -i -I

The "magic" is merely combining debian packaging with upstream ALSA and fixing two minor issues. The only alternative is re-implementing debian packaging from scratch which is already available in a git repo - why would anyone do that?