Skip to content

CI for forward-porting GC3 patches to GC4#147

Draft
ddeclerck wants to merge 7 commits intoOCamlPro:gitside-masterfrom
ddeclerck:gc3_to_gc4
Draft

CI for forward-porting GC3 patches to GC4#147
ddeclerck wants to merge 7 commits intoOCamlPro:gitside-masterfrom
ddeclerck:gc3_to_gc4

Conversation

@ddeclerck
Copy link
Collaborator

Follow-up of #146.

@ddeclerck ddeclerck changed the title Dummy commit CI for forward-porting GC3 patches to GC4 May 29, 2024
@ddeclerck ddeclerck force-pushed the gc3_to_gc4 branch 3 times, most recently from 6900d8d to 1c95bd0 Compare May 30, 2024 21:21
# check so reporting as portability problem is only noise.
# This has the effect of redirecting some error messages to stdout.
# to be moved to the Makefile - currently only usable for bootstrap,
# but should be done on autogen, too
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A, that old TODO...

Copy link
Collaborator

@GitMensch GitMensch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest to wrap the commits again. From what I've inspected we need one refactor for integrating 4.x logic (you've spotted that well) nicely.

snprintf (file_open_env, (size_t)COB_FILE_MAX, "%s%s", "IO_", s);
if ((file_open_io_env = cob_get_env (file_open_env, NULL)) == NULL) {
snprintf (file_open_env, (size_t)COB_FILE_MAX, "%s%s", "io_", s);
if (f != NULL) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When/why should f be null here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because some functions (open_cbl_file, cob_sys_delete_file, ...) were "improved" to perform file mapping in GC3 (rev 3944), by calling the cob_chk_file_mapping function, which does not take a cob_file argument in GC3 but does in GC4, and that function in turn calls cob_chk_file_env. Since these functions (open_cbl_file, cob_sys_delete_file, ...) do not use a cob_file object, I resorted to passing NULL and coping with that...

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this okay for you @GitMensch ?

@GitMensch
Copy link
Collaborator

oops, hope I haven't broken the gitignore in f36dcda - if not then we likely should apply that to the gcos3x branch as well.

@ddeclerck
Copy link
Collaborator Author

ddeclerck commented May 31, 2024

I suggest to wrap the commits again. From what I've inspected we need one refactor for integrating 4.x logic (you've spotted that well) nicely.

Saw your message a bit late, added another commit in the meantime 😅

By wrapping up you mean, committing to SVN ? (after doing the requested modifications of course)

@GitMensch
Copy link
Collaborator

GitMensch commented Jun 1, 2024 via email

@ddeclerck
Copy link
Collaborator Author

I tend to be overly "conservative". Indeed this piece of code is barely modified afterwards, so I'll do the refactoring.

@ddeclerck
Copy link
Collaborator Author

Is this okay to merge (@GitMensch) ?

@GitMensch
Copy link
Collaborator

I'll try to review this (late) evening.

@GitMensch
Copy link
Collaborator

looks_absolute should use "src", not file_open_name directly (merge issue?)

"apply_file_paths" should get that via argument as well and have a function comment that it writes to the global buffer. Then add a Changelog "extracted from xyz and also used in abc" to finish that last commit.

We either have to remember for later that we need to add a testcase for the new use or (potentially easier) also include it in the last commit as well.

@ddeclerck
Copy link
Collaborator Author

looks_absolute should use "src", not file_open_name directly (merge issue?)

This change is introduced in a later commit (3993).

"apply_file_paths" should get that via argument as well and have a function comment that it writes to the global buffer. Then add a Changelog "extracted from xyz and also used in abc" to finish that last commit.

Alright ; as for its output, should it write it through its argument or directly to file_open_name ?

We either have to remember for later that we need to add a testcase for the new use or (potentially easier) also include it in the last commit as well.

By "new use", de you mean the fact that we apply file paths to the complex case ?

@GitMensch
Copy link
Collaborator

By "new use", de you mean the fact that we apply file paths to the complex case ?

yes

This change is introduced in a later commit (3993).

good catch - then it is fine to leave as is; if you don't expect any big problem it would be nice to merge that in this bunch to commit that together, but a later bunch is fine as well

Alright ; as for its output, should it write it through its argument or directly to file_open_name ?

Depends on how other functions do it - it is best for now to mimic that (once the merge is completed we may revisit that part, but there are "some" commits left until we get there).

@ddeclerck
Copy link
Collaborator Author

I made the necessary changes.

This change is introduced in a later commit (3993).

good catch - then it is fine to leave as is; if you don't expect any big problem it would be nice to merge that in this bunch to commit that together, but a later bunch is fine as well

I find it more convenient to merge consecutive commits. If that's okay for you I could add to the current batch the next eligible commits until 3993 (that would be 6 commits: 3973, 3979, 3988, 3989, 3992 and 3993).

@GitMensch
Copy link
Collaborator

That batch is good to go :-)

@ddeclerck
Copy link
Collaborator Author

That batch is good to go :-)

Merged in SVN ;)

I see the next commits deal with translation files. Checking the history, it seems those files are usually just copied "as-is" from the GC3 branch to the trunk; is this correct ?

@GitMensch
Copy link
Collaborator

I see the next commits deal with translation files. Checking the history, it seems those files are usually just copied "as-is" from the GC3 branch to the trunk; is this correct ?

No, only new files are copied, the others left as-is; before a release I regenerate the files but the files are nearly completely maintained by the translation project.
So just record the merge, then copy over new files and svn add them, if there are any.

And of course

@ddeclerck
Copy link
Collaborator Author

ddeclerck commented Jun 8, 2024

Alright.

And of course

Hope this unfinished sentence did not have any vital info 😅

@GitMensch
Copy link
Collaborator

I can't remember any important info missing there.

@codecov-commenter
Copy link

codecov-commenter commented Jun 8, 2024

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 91.85668% with 25 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.41%. Comparing base (14e2140) to head (7eea649).
⚠️ Report is 20 commits behind head on gc4.

Files with missing lines Patch % Lines
libcob/move.c 90.04% 9 Missing and 12 partials ⚠️
cobc/tree.c 94.91% 0 Missing and 3 partials ⚠️
libcob/termio.c 90.00% 0 Missing and 1 partial ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##              gc4     #147      +/-   ##
==========================================
- Coverage   66.20%   63.41%   -2.79%     
==========================================
  Files          39       40       +1     
  Lines       69061    72197    +3136     
  Branches    19164    20185    +1021     
==========================================
+ Hits        45723    45785      +62     
- Misses      16147    19270    +3123     
+ Partials     7191     7142      -49     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@ddeclerck
Copy link
Collaborator Author

Quick question: I sometimes see alternative code for GC4 in #if 0 blocks; I guess I should implement those and drop the other branch, correct ?

I'm talking about those:

#if 0 /* TODO for 4.0: set the attributes from the field given outside on the stack */
		output ("cob_field *cob_fret, const int cob_pam");
#else
		output ("cob_field **cob_fret, const int cob_pam");
#endif

@GitMensch
Copy link
Collaborator

That's quite a bunch - any reason to not merge upstream?
Open issues you are aware of or special adjustments needed?

[we really need to get to commits that have someone else in the ChangeLogs...]

@ddeclerck
Copy link
Collaborator Author

ddeclerck commented Jun 12, 2024

That's quite a bunch - any reason to not merge upstream? Open issues you are aware of or special adjustments needed?

No good reason. It may be many commits, but the first batch had way more lines (this one is only +1,162 −904).
Anyways, I'll merge upstream if that's okay with you.

[we really need to get to commits that have someone else in the ChangeLogs...]

I'm looking forward to reaching commit 4614 - our first contribution to GC3 ;)

@GitMensch
Copy link
Collaborator

GitMensch commented Jul 29, 2025

Just a note for a minor issue I've stumbled over when inspecting the last missing "unexpected diff" from this branch: at least run_extensions.at (likely other test sources as well) have $COMPILE -std=default which can be sed -i to $COMPILE and ] -> ] and

], [], [

which may be adjusted to

], [],
[
AT_CLEANUP

AT_SETUP

that will be

AT_CLEANUP


AT_SETUP

we may gather all those changes to a single "cleanup testsuite" commit (and may also find uses of $COBC that should be $COMPILE when inspecting that)

@GitMensch
Copy link
Collaborator

OK, so the only difference for this bunch is that previously "LENGTH OF something" was always pretty-printed without leading zero, while it now is printed as a signed binary.

I'm fine with adjusting the expected output in the testsuite for that...

And checked with MF: if there is no ODOSLIDE enabled, then the LENGTH OF seems compile-time computed and printed as digit without leading zero or sign; if ODOSLIDE is enabled, then it is printed as a binary (as GC3.3, but with an additional zero instead of the sign) - so I think we really can adjust the testcase's printing here, please do that while I reinspect the bunch.

@GitMensch
Copy link
Collaborator

One note: We should adjust GC3 and merge to GC4 (or possibly the other way around, seems like GC4 has a warning there):

MF (with ODOSLIDE) and IBM do not allow the DEPENDING ON item to have a variable position - those have to be before any ODO fields:

** ODO object ODO-2 must have fixed location

@GitMensch
Copy link
Collaborator

... but the biggest issue is that -fodoslide seems to be not functional with GC3.2 - as the actual values are wrong.
It would be really good if you can have a look at that and provide a patch (or backport) for 3.3, if the result is the same:

********
TEST WITHOUT ODO: 'A01  B  CD'
DATA1=A
DEP1=0
DEP2=1
DATA21(1)=B
DATA3=CD
TEST WITH ODO: 'A01BC ' Len:+000000006
ODO-DATA1=A.
ODO-DEP1=0.
ODO-DEP2=1.
ODO-DATA21(1)=B.
ODO-DATA3=C .
TEST WITH ODO: 'A12BCDE ' Len:+000000008
ODO-DATA1=A.
ODO-DEP1=1.
ODO-DEP2=2.
ODO-DATA11(1)=B.
ODO-DATA21(1)=C.
ODO-DATA21(2)=D.
ODO-DATA3=E .
TEST WITH ODO: 'A23BCDEF  ' Len:+000000010
ODO-DATA1=A.
ODO-DEP1=2.
ODO-DEP2=3.
ODO-DATA11(1)=B.
ODO-DATA11(2)=C.
ODO-DATA21(1)=D.
ODO-DATA21(2)=E.
ODO-DATA21(3)=F.
ODO-DATA3=  .
********
TEST WITH ODO, SEPERATED: 'A01BCD'
ODO-DATA1=A.
ODO-DEP1=0.
ODO-DEP2=1.
ODO-DATA21(1)=B.
ODO-DATA3=CD.
********
Slided ODO : '2123123End'
Slided ODO : '2126123456End'

(compare to the expected output https://github.com/ddeclerck/gnucobol/blob/7cb4b9eae33e8cd94a983491a0e413a976a86878/tests/testsuite.src/run_extensions.at#L1774-L1813 - MF and GC4 with this PR are identical, apart from how the binary is presented)

Copy link
Collaborator

@GitMensch GitMensch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apart from fixing the failures by adjusting the expected output of LENGTH OF to be a native binary (=what it currently displays) that bunch seems fine;

as noted, tree.c is currently "strange", merging the fix in the bunch would be good

also: the additional warning for positions of ODO-items should go to 3.x and possibly be an error in both branches (not sure if the current "only GC allows that" is useful); even more important - if 3.x still has the same bad output, that needs to be fixed.

@ddeclerck
Copy link
Collaborator Author

as noted, tree.c is currently "strange", merging the fix in the bunch would be good

I possibly missed that - what is strange in tree.c (appart from formatting isues) ?

@GitMensch
Copy link
Collaborator

tree.c: just formatting issues

@GitMensch
Copy link
Collaborator

Seems like we need first a CI adjustment 3.x->4.x, then a rebase to get current CI definitions (with increased MSYS2 timeout and ideally IBM LE + most BE) in.

Copy link
Collaborator

@GitMensch GitMensch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apart from some more GC4 testsuite cleanups (can be done later), that looks good

@ddeclerck
Copy link
Collaborator Author

apart from some more GC4 testsuite cleanups (can be done later), that looks good

Yeah I was already doing more adjustments when you wrote this ;)

@ddeclerck
Copy link
Collaborator Author

If that's okay I'll merge that and create a new batch with the newer 3.x commits (5531, 5537, 5540-5542, 5549-5553).

@GitMensch
Copy link
Collaborator

sure, time for upstreaming this batch - again: well done!

@ddeclerck
Copy link
Collaborator Author

ddeclerck commented Jul 30, 2025

About to merge 5541 & 5542 - are you fine with the current formatting of the code in these two commits ?
(well, you did make formatting adjustments in 5542 so I guess yes)

@GitMensch
Copy link
Collaborator

Just to say: current bunch is good for upstream.

You may want to include ibm ci afterwards, then check its state before going on merging and/or inspect the ODO issues in 3.x instead, first.

@ddeclerck
Copy link
Collaborator Author

Just to say: current bunch is good for upstream.

Great ; it was just missing the C89/C23 compat fixes (5552, 5553) .

You may want to include ibm ci afterwards, then check its state before going on merging and/or inspect the ODO issues in 3.x instead, first.

Actually I had merged the IBM CI from 3.X - just forgot to change the target branch... Changed it here so we check how it behaves.
After this indeed I'll check those ODO issues - though I just have 24 hours before I take a long (and mostly offline) vacation ;)

@ddeclerck
Copy link
Collaborator Author

ddeclerck commented Jul 31, 2025

And obviously, we have failures on IBM PPC & Z...
I'll try to identify which commit caused that.
Funny thing these failures do not occur on 3.x.

@GitMensch
Copy link
Collaborator

GitMensch commented Jul 31, 2025

Those failures look older - I'd suggest to get the current bunch upstream (maybe with a new TODO note on PPC + BE), then leaving the IBM stuff for 4.x until you are back.

ppc: the align definition stuff seems wrong - needs to check configure and the use of the macros in the headers (possibly something broken on the header split)

big-endian: obviously some errors related to that, this is possibly also older (the last time this was verified was around 18 months ago; when Ron left 4.x passed all tests on a similar environment [but that was also before we integrated newer tests])

@ddeclerck
Copy link
Collaborator Author

ddeclerck commented Jul 31, 2025

Those failures look older - I'd suggest to get the current bunch upstream, then leaving the IBM stuff until you are back.

I confirm this is older than the current batch.
So I'll merge the current batch.

ppc: the align definition stuff seems wrong - needs to check configure and the use of the macros in the headers (possibly something broken on the header split)

Just to let you know, all the PPC failures are due to this __unaligned keyword (a run with this attribute boldy removed works). So appart from this, PPC is fine.

big-endian: obviously some errors related to that, this is possibly also older (the last time this was verified was around 18 months ago; when Ron left 4.x passed all tests on a similar environment [but that was also before we integrated newer tests])

Yup, the S390x errors all seem caused by big-endian - note we also have some in 3.x.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants