Buildability on CentOS-7 and Windows using PCRE2 sources

[jerinphilip]

Experimentation in several repositories lead me to believe we have to take a look at the ExternalProject build of PCRE2 (from sources).

I'm trying to tinker with a GitHub action that allows me to release py3.6, py3.7 manylinux wheels eventually to get batteries included bergamot package into pypa. Here's a teaser.

However, the CentOS-7 based image which generates manylinux wheels seems to be not working well with ssplit-cpp. Please find the build full log at:

https://github.com/jerinphilip/lemonade/runs/4912697746?check_suite_focus=true

PCRE2 from source build log: relevant bits (click to expand)

  [ 75%] Building C object CMakeFiles/pcre2-8-static.dir/src/pcre2_xclass.c.o
  [ 77%] Linking C static library libpcre2-8.a
  [ 77%] Built target pcre2-8-static
  [ 80%] Building C object CMakeFiles/pcre2-posix-static.dir/src/pcre2posix.c.o
  [ 83%] Linking C static library libpcre2-posix.a
  [ 83%] Built target pcre2-posix-static
  [ 86%] Building C object CMakeFiles/pcre2grep.dir/src/pcre2grep.c.o
  [ 88%] Linking C executable pcre2grep
  [ 88%] Built target pcre2grep
  [ 91%] Building C object CMakeFiles/pcre2test.dir/src/pcre2test.c.o
  [ 94%] Linking C executable pcre2test
  [ 94%] Built target pcre2test
  [ 97%] Building C object CMakeFiles/pcre2_jit_test.dir/src/pcre2_jit_test.c.o
  [100%] Linking C executable pcre2_jit_test
  [100%] Built target pcre2_jit_test
  [  2%] Performing install step for 'pcre2'
  Consolidate compiler generated dependencies of target pcre2-8-static
  [ 77%] Built target pcre2-8-static
  Consolidate compiler generated dependencies of target pcre2-posix-static
  [ 83%] Built target pcre2-posix-static
  Consolidate compiler generated dependencies of target pcre2grep
  [ 88%] Built target pcre2grep
  Consolidate compiler generated dependencies of target pcre2test
  [ 94%] Built target pcre2test
  Consolidate compiler generated dependencies of target pcre2_jit_test
  [100%] Built target pcre2_jit_test

Weirdly in the last the build is failing with the following error message.

  [ 98%] Linking CXX static library libbergamot-translator.a
  [ 98%] Built target bergamot-translator
  [100%] Building CXX object bindings/python/CMakeFiles/_bergamot.dir/bergamot.cpp.o
  gmake[3]: *** No rule to make target `lib/libpcre2-8.a', needed by `../lib.linux-x86_64-3.6/bergamot/_bergamot.cpython-36m-x86_64-linux-gnu.so'.  Stop.
  gmake[2]: *** [bindings/python/CMakeFiles/_bergamot.dir/all] Error 2
  gmake[1]: *** [bindings/python/CMakeFiles/_bergamot.dir/rule] Error 2
  gmake: *** [_bergamot] Error 2

Note that the existing build using ExternalProject compiles with Ubuntu and MacOS and fails with CentOS. We are also aware of issues with Windows when attempting to switch to compile PCRE2 from sources (we're working around this by using a vcpkg provision of pcre2 and using the not from source ExternalProject right now).

Since there are other issues affecting Windows compilation as well, I was wondering if it's worthwhile to pool the efforts and fix the problem here so as to provide a neat means to get the targets (when loaded from ExternalProject built from source) all the way up via CMake.

Thanks in advance,

[ugermann]

Hmm... https://pythonspeed.com/articles/centos-8-is-dead/ Let me dig around for a Docker image to test compilation that I don't have to perform CPR on to get it working.

[ugermann]

I think I fixed the issue. @jerinphilip Can you try if it works now?

[ugermann]

Hold that: regression test failed, but I think I'm onto the right thing ...

[jerinphilip]

@ugermann I have jerinphilip/bergamot-translator#60 ready to provide feedback.

[ugermann]

It's a mess https://gitlab.kitware.com/cmake/cmake/-/issues/23365

[ugermann]

@jerinphilip I'm not able to run the workflows in jerinphilip/bergamot-translator#60.

[Build/Ubuntu 20.04 / py3.8-3   ] ⭐  Run Set up Python
[Build/Ubuntu 20.04 / py3.8-3   ]   ☁  git clone 'https://github.com/actions/setup-python' # ref=v2
[Build/Ubuntu 20.04 / py3.8-3   ]   cloning https://github.com/actions/setup-python to /home/germann/.cache/act/actions-setup-python@v2
[Build/Ubuntu 20.04 / py3.8-3   ] Unable to resolve v2: reference not found
DEBU[0010] reference not found                          
DEBU[0010] reference not found                          
[Build/Ubuntu 20.04 / py3.8-3   ]   ❌  Failure - Set up Python

[jerinphilip]

I'm not sure where the error logs are coming from in your message, but looks like manylinux build passed after I point to current main:
https://github.com/jerinphilip/bergamot-translator/runs/5991984231?check_suite_focus=true

[ugermann]

The error log comes from running

act -v -W .github/workflows/build.yml

locally. That's nectos/act.