summaryrefslogtreecommitdiff
path: root/ci
AgeCommit message (Collapse)AuthorFilesLines
2025-05-12Merge branch 'jc/ci-skip-unavailable-external-software'Junio C Hamano1-13/+8
Further refinement on CI messages when an optional external software is unavailable (e.g. due to third-party service outage). * jc/ci-skip-unavailable-external-software: ci: download JGit from maven, not eclipse.org ci: update the message for unavailble third-party software
2025-05-05Merge branch 'kn/meson-hdr-check'Junio C Hamano1-1/+1
Add an equivalent to "make hdr-check" target to meson based builds. * kn/meson-hdr-check: makefile/meson: add 'check-headers' as alias for 'hdr-check' meson: add support for 'hdr-check' meson: rename 'third_party_sources' to 'third_party_excludes' meson: move headers definition from 'contrib/coccinelle' coccinelle: meson: rename variables to be more specific ci/github: install git before checking out the repository
2025-05-05ci: fix aggregation of test results with MesonPatrick Steinhardt1-0/+1
Our CI needs to be aware of the location of the test output directory so that it knows where to find test results. Some of our CI jobs achieve this by setting the `TEST_OUTPUT_DIRECTORY` environment variable, which ensures that the output will be written to that directory. Other jobs, especially on GitHub Workflows, don't set that environment variable and instead expect test results to be located in the source directory in "t/". The latter logic does not work with Meson though, as the test results are not written into the source directory by default, but instead into the build directory. As such, any job that uses Meson without setting the environment variable will be unable to locate and aggregate results. Fix this by explicitly setting the test output directory when we set up the Meson build directory. Like this, we can easily default to "t/" in the source directory when the value hasn't been set explicitly. Reported-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-05-05ci: stop linking the `prove` cacheJohannes Schindelin2-10/+0
It is not useful because we do not have any persisted directory anymore, not since dropping our Travis CI support. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-25ci: download JGit from maven, not eclipse.orgJunio C Hamano1-1/+1
As Matthias Sohn, JGit maintainer, recommends, update the JGit download link from repo.eclipse.org to a one in maven.org Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-25ci: update the message for unavailble third-party softwareJunio C Hamano1-12/+7
An earlier fix added an extra message immediately after failing to download a third-party package. But near the end of the script, their availability is checked again and given a message. Remove the new ones added with a recent fix, as they are redundant. If we were to add more places to download these software (e.g. for other platforms we currently do not download them on), the existing warnning near the end of the script will also trigger. While at it, as Dscho suggests, rewrite the WARNING: label on the warning message to ::warning::, which presumably should be shown a bit more prominently in the CI summary. Suggested-by: Johannes Schindelin <Johannes.Schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-24Merge branch 'jc/ci-skip-unavailable-external-software'Junio C Hamano1-7/+20
Make sure outage of third-party sites that supply P4, Git-LFS, and JGit we use for testing would not prevent our CI jobs from running at all. * jc/ci-skip-unavailable-external-software: ci: skip unavailable external software
2025-04-24Merge branch 'jc/ci-skip-unavailable-external-software' into maint-2.49Junio C Hamano1-7/+20
Make sure outage of third-party sites that supply P4, Git-LFS, and JGit we use for testing would not prevent our CI jobs from running at all. * jc/ci-skip-unavailable-external-software: ci: skip unavailable external software
2025-04-24Merge branch 'js/ci-fedora-gawk' into maint-2.49Junio C Hamano1-1/+1
Work around CI breakage due to fedora base image getting updated. * js/ci-fedora-gawk: ci(pedantic): ensure that awk is installed
2025-04-24ci: skip unavailable external softwareJunio C Hamano1-7/+20
The ci/install-dependencies.sh script used in a very early phase of our CI jobs downloads Perforce, Git-LFS, and JGit, used for running the test scripts. The test framework is prepared to properly skip the tests that depend on these external software, but the CI script is unnecessarily strict (due to its use of "set -e" in ci/lib.sh) and fails the entire CI run before even starting to test the rest of the system. Notice a failure to download to any of these external software, but keep going. We need to be careful about cleaning after a failed wget, as a later part of the script that does: if type jgit >/dev/null 2>&1 then echo "$(tput setaf 6)JGit Version$(tput sgr0)" jgit version else echo >&2 "WARNING: JGit wasn't installed, see above for clues why" fi will (surprise!) succeed running "type jgit", and then fail with "jgit version", taking the whole thing down due to "set -e". Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-23makefile/meson: add 'check-headers' as alias for 'hdr-check'Karthik Nayak1-1/+1
The 'hdr-check' target in Meson and makefile is used to check if headers can be compiled individually. The naming however isn't readable as 'hdr' is not a common shortforme for 'header', neither is it an abbreviation. Let's introduce 'check-headers' as an alternative target for 'hdr-check' and add a `TODO` to deprecate the latter after 2 releases. Since this is an internal tool, we can use a shorter deprecation cycle. Change existing usage of 'hdr-check' in 'ci/run-static-analysis.sh' to also use 'check-headers'. Signed-off-by: Karthik Nayak <karthik.188@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-17Merge branch 'en/merge-recursive-debug'Junio C Hamano1-1/+0
Remove remnants of the recursive merge strategy backend, which was superseded by the ort merge strategy. * en/merge-recursive-debug: builtin/{merge,rebase,revert}: remove GIT_TEST_MERGE_ALGORITHM tests: remove GIT_TEST_MERGE_ALGORITHM and test_expect_merge_algorithm merge-recursive.[ch]: thoroughly debug these merge, sequencer: switch recursive merges over to ort sequencer: switch non-recursive merges over to ort merge-ort: enable diff-algorithms other than histogram builtin/merge-recursive: switch to using merge_ort_generic() checkout: replace merge_trees() with merge_ort_nonrecursive()
2025-04-16Merge branch 'js/ci-fedora-gawk'Junio C Hamano1-1/+1
Work around CI breakage due to fedora base image getting updated. * js/ci-fedora-gawk: ci(pedantic): ensure that awk is installed
2025-04-16Merge branch 'js/ci-github-update-ubuntu'Junio C Hamano1-1/+1
Adjust to the deprecation of use of Ubuntu 20.04 GitHub Actions CI. * js/ci-github-update-ubuntu: ci: upgrade `sparse` to supported build agents
2025-04-16ci(pedantic): ensure that awk is installedJohannes Schindelin1-1/+1
The image pointed to by the fedora:latest tag has moved from fedora 41 to 42. The fedora 41 container images have awk installed while the fedora 42 images do not. That change is most likely just part of reducing the size of the base container images. In both AlmaLinux and Fedora (as well as other RHEL derivatives/relatives), awk is provided by the gawk package. On Fedora, `dnf install awk` would work, by using the package filelist data to determine that /usr/bin/awk is provided by gawk and installs gawk as a result. On AlmaLinux (8 & 9, by quick testing by Todd), that is not the case and you'd need to use `dnf install gawk` or `dnf install '*bin/awk'` to get it installed. Having said that, awk _is_ included in the current AlmaLinux 8 and 9 images, so it isn't strictly needed. But it's probably better to be explicit that we need it installed, as a defense against some future change to the AlmaLinux container removing awk. Because we know that on both of these distros, our scripts that call for 'awk' had been using 'gawk' that was installed as part of the base image, let's make sure that we explicitly install 'gawk'. If the image already has it, it would be a no-op that does not cause breakage. Suggested-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-09ci: upgrade `sparse` to supported build agentsJohannes Schindelin1-1/+1
The `sparse` job still uses the `ubuntu-20.04` runner pool, but that pool is about to go away, so let's stop using it. There is no `sparse-22.04` artifact provided by the "Build sparse for Ubuntu" Azure Pipeline, but that is not necessary anyway because Ubuntu 22.04 has the `sparse` package: https://packages.ubuntu.com/jammy/sparse Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-08tests: remove GIT_TEST_MERGE_ALGORITHM and test_expect_merge_algorithmElijah Newren1-1/+0
Both of these existed to allow us to reuse all the merge-related tests in the testsuite while easily flipping between the 'recursive' and the 'ort' backends. Now that we have removed merge-recursive and remapped 'recursive' to mean 'ort', we don't need this scaffolding anymore. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-04-08Merge branch 'ps/ci-meson-check-build-docs'Junio C Hamano1-6/+21
CI update. * ps/ci-meson-check-build-docs: ci: perform build and smoke tests for Meson docs
2025-03-21ci: add build checking for side-effects in assert() callsElijah Newren2-0/+20
It is a big no-no to have side-effects in an assertion, because if the assert() is compiled out, you don't get that side-effect, leading to the code behaving differently. That can be a large headache to debug. We have roughly 566 assert() calls in our codebase (my grep might have picked up things that aren't actually assert() calls, but most appeared to be). All but 9 of them can be determined by gcc to be free of side effects with a clever redefine of assert() provided by Bruno De Fraine (from https://stackoverflow.com/questions/10593492/catching-assert-with-side-effects), who upon request has graciously placed his two-liner into the public domain without warranty of any kind. The current 9 assert() calls flagged by this clever redefinition of assert() appear to me to be free of side effects as well, but are too complicated for a compiler/linker to figure that since each assertion involves some kind of function call. Add a CI job which will find and report these possibly problematic assertions, and have the job suggest to the user that they replace these with ASSERT() calls. Example output from running: ``` ERROR: The compiler could not verify the following assert() calls are free of side-effects. Please replace with ASSERT() calls. /home/newren/floss/git/diffcore-rename.c:1409 assert(!dir_rename_count || strmap_empty(dir_rename_count)); /home/newren/floss/git/merge-ort.c:1645 assert(renames->deferred[side].trivial_merges_okay && !strset_contains(&renames->deferred[side].target_dirs, path)); /home/newren/floss/git/merge-ort.c:794 assert(omittable_hint == (!starts_with(type_short_descriptions[type], "CONFLICT") && !starts_with(type_short_descriptions[type], "ERROR")) || type == CONFLICT_DIR_RENAME_SUGGESTED); /home/newren/floss/git/merge-recursive.c:1200 assert(!merge_remote_util(commit)); /home/newren/floss/git/object-file.c:2709 assert(would_convert_to_git_filter_fd(istate, path)); /home/newren/floss/git/parallel-checkout.c:280 assert(is_eligible_for_parallel_checkout(pc_item->ce, &pc_item->ca)); /home/newren/floss/git/scalar.c:244 assert(have_fsmonitor_support()); /home/newren/floss/git/scalar.c:254 assert(have_fsmonitor_support()); /home/newren/floss/git/sequencer.c:4968 assert(!(opts->signoff || opts->no_commit || opts->record_origin || should_edit(opts) || opts->committer_date_is_author_date || opts->ignore_date)); ``` Note that if there are possibly problematic assertions, not necessarily all of them will be shown in a single run, because the compiler errors may include something like "ld: ... more undefined references to `not_supposed_to_survive' follow" instead of listing each individually. But in such cases, once you clean up a few that are shown in your first run, subsequent runs will show (some of) the ones that remain, allowing you to iteratively remove them all. Helped-by: Bruno De Fraine <defraine@gmail.com> Signed-off-by: Elijah Newren <newren@gmail.com> Acked-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-03-12ci: perform build and smoke tests for Meson docsPatrick Steinhardt1-6/+21
Our "documentation" CI job performs a couple of tests against our documentation. Part of these tests is to check whether documentation builds at all and whether it spits out the expected set of files. We don't yet have such a test for Meson, which means that we wouldn't notice at all if building the documentation were to break. As a result, breakages as fixed by 87eccc3a81d (meson: fix building technical and howto docs, 2025-03-02) are easy to go unnoticed. Address this test gap by starting to build both manpages and HTML sites as part of the CI job. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-02-20ci: exercise credential helpersPatrick Steinhardt2-1/+7
Wire up credential helpers in our CI runs so that we can rest assured that they compile and (if tests are available) function correctly. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-02-20ci: fix propagating UTF-8 test locale in musl-based Meson jobPatrick Steinhardt1-2/+2
The musl-based Meson job is supposed to explicitly specify the UTF-8 locale used for testing, which has been introduced with 84bb5eeace7 (ci: switch linux-musl to use Meson, 2025-01-28). That commit had two issues though: - We continue to refer to "linux-musl", even though the job has been renamed in the same commit to "linux-musl-meson". - We use the wrong option name to specify the locale. This was not noticed though due to the first issue. Fix both of these issues by fixing both the job and option naems. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-02-06Merge branch 'ps/zlib-ng'Junio C Hamano3-7/+5
The code paths to interact with zlib has been cleaned up in preparation for building with zlib-ng. * ps/zlib-ng: ci: make "linux-musl" job use zlib-ng ci: switch linux-musl to use Meson compat/zlib: allow use of zlib-ng as backend git-zlib: cast away potential constness of `next_in` pointer compat/zlib: provide stubs for `deflateSetHeader()` compat/zlib: provide `deflateBound()` shim centrally git-compat-util: move include of "compat/zlib.h" into "git-zlib.h" compat: introduce new "zlib.h" header git-compat-util: drop `z_const` define compat: drop `uncompress2()` compatibility shim
2025-02-06Merge branch 'ps/ci-misc-updates'Junio C Hamano3-37/+8
CI updates (containerization, dropping stale ones, etc.). * ps/ci-misc-updates: ci: remove stale code for Azure Pipelines ci: use latest Ubuntu release ci: stop special-casing for Ubuntu 16.04 gitlab-ci: add linux32 job testing against i386 gitlab-ci: remove the "linux-old" job github: simplify computation of the job's distro github: convert all Linux jobs to be containerized github: adapt containerized jobs to be rootless t7422: fix flaky test caused by buffered stdout t0060: fix EBUSY in MinGW when setting up runtime prefix
2025-02-03Merge branch 'ps/build-meson-fixes'Junio C Hamano1-1/+3
More build fixes and enhancements on meson based build procedure. * ps/build-meson-fixes: ci: wire up Visual Studio build with Meson ci: raise error when Meson generates warnings meson: fix compilation with Visual Studio meson: make the CSPRNG backend configurable meson: wire up fuzzers meson: wire up generation of distribution archive meson: wire up development environments meson: fix dependencies for generated headers meson: populate project version via GIT-VERSION-GEN GIT-VERSION-GEN: allow running without input and output files GIT-VERSION-GEN: simplify computing the dirty marker
2025-02-03Merge branch 'ps/3.0-remote-deprecation'Junio C Hamano2-6/+2
Following the procedure we established to introduce breaking changes for Git 3.0, allow an early opt-in for removing support of $GIT_DIR/branches/ and $GIT_DIR/remotes/ directories to configure remotes. * ps/3.0-remote-deprecation: remote: announce removal of "branches/" and "remotes/" builtin/pack-redundant: remove subcommand with breaking changes ci: repurpose "linux-gcc" job for deprecations ci: merge linux-gcc-default into linux-gcc Makefile: wire up build option for deprecated features
2025-01-28ci: make "linux-musl" job use zlib-ngPatrick Steinhardt1-1/+1
We don't yet have any test coverage for the new zlib-ng backend as part of our CI. Add it by installing zlib-ng in Alpine Linux, which causes Meson to pick it up automatically. Note that we are somewhat limited with regards to where we run that job: Debian-based distributions don't have zlib-ng in their repositories, Fedora has it but doesn't run tests, and Alma Linux doesn't have the package either. Alpine Linux does have it available and is running our test suite, which is why it was picked. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-28ci: switch linux-musl to use MesonPatrick Steinhardt3-6/+4
Switch over the "linux-musl" job to use Meson instead of Makefiles. This is done due to multiple reasons: - It simplifies our CI infrastructure a bit as we don't have to manually specify a couple of build options anymore. - It verifies that Meson detects and sets those build options automatically. - It makes it easier for us to wire up a new CI job using zlib-ng as backend. One platform compatibility that Meson cannot easily detect automatically is the `GIT_TEST_UTF8_LOCALE` variable used in tests. Wire up a build option for it, which we set via a new "MESONFLAGS" environment variable. Note that we also drop the CC variable, which is set to "gcc". We already default to GCC when CC is unset in "ci/lib.sh", so this is not needed. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22ci: raise error when Meson generates warningsPatrick Steinhardt1-0/+1
Meson prints warnings in several cases, like for example when using a feature supported by the current version of Meson, but not yet supported by the minimum required version as declared by the project. These warnings will not cause the setup to fail by default, which makes it quite easy to miss them. Improve this by passing `--fatal-meson-warnings` to `meson setup` so that our CI jobs will fail on warnings. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22meson: wire up fuzzersPatrick Steinhardt1-1/+2
Meson does not yet know to build our fuzzers. Introduce a new build option "fuzzers" and wire up the fuzzers in case it is enabled. Adapt our CI jobs so that they build the fuzzers by default. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22ci: repurpose "linux-gcc" job for deprecationsPatrick Steinhardt1-1/+2
The "linux-gcc" job isn't all that interesting by itself and can be considered more or less the "standard" job: it is running with a reasonably up-to-date image and uses GCC as a compiler, both of which we already cover in other jobs. There is one exception though: we change the default branch to be "main" instead of "master", so it is forging ahead a bit into the future to make sure that this change does not cause havoc. So let's expand on this a bit and also add the new "WITH_BREAKING_CHANGES" flag to the mix. Rename the job to "linux-breaking-changes" accordingly. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-22ci: merge linux-gcc-default into linux-gccPatrick Steinhardt1-5/+0
The "linux-gcc-default" job is mostly doing the same as the "linux-gcc" job, except for a couple of minor differences: - We use an explicit GCC version instead of the default version provided by the distribution. We have other jobs that test with "gcc-8", making this distinction pointless. - We don't set up the Python version explicitly, and instead use the default Python version. Python 2 has been end-of-life for quite a while now though, making this distinction less interesting. - We set up the default branch name to be "main" in "linux-gcc". We have other testcases that don't and also some that explicitly use "master". - We use "ubuntu:20.04" in one job and "ubuntu:latest" in another. We already have a couple other jobs testing these respectively. So overall, the job does not add much to our test coverage. Drop the "linux-gcc-default" job and adapt "linux-gcc" to start using the default GCC compiler, effectively merging those two jobs into one. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10ci: remove stale code for Azure PipelinesPatrick Steinhardt2-25/+1
Support for Azure Pipelines has been retired in 6081d3898f (ci: retire the Azure Pipelines definition, 2020-04-11) in favor of GitHub Actions. Our CI library still has some infrastructure left for Azure though that is now unused. Remove it. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10ci: stop special-casing for Ubuntu 16.04Patrick Steinhardt1-8/+1
With c85bcb5de1 (gitlab-ci: switch from Ubuntu 16.04 to 20.04, 2024-10-31) we have adapted the last CI job to stop using Ubuntu 16.04 in favor of Ubuntu 20.04. Remove the special-casing we still have in our CI scripts. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10gitlab-ci: add linux32 job testing against i386Patrick Steinhardt1-1/+1
Add another job to GitLab CI that tests against the i386 architecture. This job is equivalent to the same job in GitHub Workflows. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10github: simplify computation of the job's distroPatrick Steinhardt2-2/+4
We explicitly list the distro of Linux-based jobs, but it is equivalent to the name of the image in almost all cases, except that colons are replaced with dashes. Drop the redundant information and massage it in our CI scripts, which is equivalent to how we do it in GitLab CI. There are a couple of exceptions: - The "linux32" job, whose distro name is different than the image name. This is handled by adapting all sites to use the new name. - The "alpine" and "fedora" jobs, neither of which specify a tag for their image. This is handled by adding the "latest" tag. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-10github: adapt containerized jobs to be rootlessPatrick Steinhardt1-1/+1
The containerized jobs in GitHub Actions run as root, giving them special permissions to for example delete files even when the user shouldn't be able to due to file permissions. This limitation keeps us from using containerized jobs for most of our Ubuntu-based jobs as it causes a number of tests to fail. Adapt the jobs to create a separate user that executes the test suite. This follows similar infrastructure that we already have in GitLab CI. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-01-02Merge branch 'jk/lsan-race-ignore-false-positive'Junio C Hamano1-1/+0
CI jobs that run threaded programs under LSan has been giving false positives from time to time, which has been worked around. This is an alternative to the jk/lsan-race-with-barrier topic with much smaller change to the production code. * jk/lsan-race-ignore-false-positive: test-lib: ignore leaks in the sanitizer's thread code test-lib: check leak logs for presence of DEDUP_TOKEN test-lib: simplify leak-log checking test-lib: rely on logs to detect leaks Revert barrier-based LSan threading race workaround
2025-01-01Revert barrier-based LSan threading race workaroundJunio C Hamano1-1/+0
The extra "barrier" approach was too much code whose sole purpose was to work around a race that is not even ours (i.e. in LSan's teardown code). In preparation for queuing a solution taking a much-less-invasive approach, let's revert them.
2025-01-01Merge branch 'jk/lsan-race-with-barrier'Junio C Hamano1-0/+1
CI jobs that run threaded programs under LSan has been giving false positives from time to time, which has been worked around. * jk/lsan-race-with-barrier: grep: work around LSan threading race with barrier index-pack: work around LSan threading race with barrier thread-utils: introduce optional barrier type Revert "index-pack: spawn threads atomically" test-lib: use individual lsan dir for --stress runs
2025-01-01Merge branch 'ps/weak-sha1-for-tail-sum-fix'Junio C Hamano1-0/+1
An earlier "csum-file checksum does not have to be computed with sha1dc" topic had a few code paths that had initialized an implementation of a hash function to be used by an unmatching hash by mistake, which have been corrected. * ps/weak-sha1-for-tail-sum-fix: ci: exercise unsafe OpenSSL backend builtin/fast-import: fix segfault with unsafe SHA1 backend bulk-checkin: fix segfault with unsafe SHA1 backend
2024-12-30ci: exercise unsafe OpenSSL backendPatrick Steinhardt1-0/+1
In the preceding commit we have fixed a segfault when using an unsafe SHA1 backend that is different from the safe one. This segfault only went by unnoticed because we never set up an unsafe backend in our CI systems. Fix this ommission by setting `OPENSSL_SHA1_UNSAFE` in our TEST-vars job. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-30thread-utils: introduce optional barrier typeJeff King1-0/+1
One thread primitive we don't yet support is a barrier: it waits for all threads to reach a synchronization point before letting any of them continue. This would be useful for avoiding the LSan race we see in index-pack (and other places) by having all threads complete their initialization before any of them start to do real work. POSIX introduced a pthread_barrier_t in 2004, which does what we want. But if we want to rely on it: 1. Our Windows pthread emulation would need a new set of wrapper functions. There's a Synchronization Barrier primitive there, which was introduced in Windows 8 (which is old enough for us to depend on). 2. macOS (and possibly other systems) has pthreads but not pthread_barrier_t. So there we'd have to implement our own barrier based on the mutex and cond primitives. Those are do-able, but since we only care about avoiding races in our LSan builds, there's an easier way: make it a noop on systems without a native pthread barrier. This patch introduces a "maybe_thread_barrier" API. The clunky name (rather than just using pthread_barrier directly) should hopefully clue people in that on some systems it will do nothing. It's wired to a Makefile knob which has to be triggered manually, and we enable it for the linux-leaks CI jobs (since we know we'll have it there). There are some other possible options: - we could turn it on all the time for Linux systems based on uname. But we really only care about it for LSan builds, and there is no need to add extra code to regular builds. - we could turn it on only for LSan builds. But that would break builds on non-Linux platforms (like macOS) that otherwise should support sanitizers. - we could trigger only on the combination of Linux and LSan together. This isn't too hard to do, but the uname check isn't completely accurate. It is really about what your libc supports, and non-glibc systems might not have it (though at least musl seems to). So we'd risk breaking builds on those systems, which would need to add a new knob. Though the upside would be that running local "make SANITIZE=leak test" would be protected automatically. And of course none of this protects LSan runs from races on systems without pthread barriers. It's probably OK in practice to protect only our CI jobs, though. The race is rare-ish and most leak-checking happens through CI. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-23Merge branch 'ps/ci-meson'Junio C Hamano4-15/+39
The meson-build procedure is integrated into CI to catch and prevent bitrotting. * ps/ci-meson: ci: wire up Meson builds t: introduce compatibility options to clar-based tests t: fix out-of-tree tests for some git-p4 tests Makefile: detect missing Meson tests meson: detect missing tests at configure time t/unit-tests: rename clar-based unit tests to have a common prefix Makefile: drop -DSUPPRESS_ANNOTATED_LEAKS ci/lib: support custom output directories when creating test artifacts
2024-12-23Merge branch 'ps/ci-gitlab-update'Junio C Hamano1-5/+6
GitLab CI updates. * ps/ci-gitlab-update: ci/lib: fix "CI setup" sections with GitLab CI ci/lib: do not interpret escape sequences in `group ()` arguments ci/lib: remove duplicate trap to end "CI setup" group gitlab-ci: update macOS images to Sonoma
2024-12-15Merge branch 'ps/build'Junio C Hamano1-1/+1
Build procedure update plus introduction of Meson based builds. * ps/build: (24 commits) Introduce support for the Meson build system Documentation: add comparison of build systems t: allow overriding build dir t: better support for out-of-tree builds Documentation: extract script to generate a list of mergetools Documentation: teach "cmd-list.perl" about out-of-tree builds Documentation: allow sourcing generated includes from separate dir Makefile: simplify building of templates Makefile: write absolute program path into bin-wrappers Makefile: allow "bin-wrappers/" directory to exist Makefile: refactor generators to be PWD-independent Makefile: extract script to generate gitweb.js Makefile: extract script to generate gitweb.cgi Makefile: extract script to massage Python scripts Makefile: extract script to massage Shell scripts Makefile: use "generate-perl.sh" to massage Perl library Makefile: extract script to massage Perl scripts Makefile: consistently use PERL_PATH Makefile: generate doc versions via GIT-VERSION-GEN Makefile: generate "git.rc" via GIT-VERSION-GEN ...
2024-12-13Merge branch 'es/oss-fuzz'Junio C Hamano1-1/+12
Backport oss-fuzz tests for us to our codebase. * es/oss-fuzz: fuzz: port fuzz-url-decode-mem from OSS-Fuzz fuzz: port fuzz-parse-attr-line from OSS-Fuzz fuzz: port fuzz-credential-from-url-gently from OSS-Fuzz
2024-12-13ci: wire up Meson buildsPatrick Steinhardt4-9/+33
Wire up CI builds for both GitLab and GitHub that use the Meson build system. While the setup is mostly trivial, one gotcha is the test output directory used to be in "t/", but now it is contained in the build directory. To unify the logic across Makefile- and Meson-based builds we explicitly set up the `TEST_OUTPUT_DIRECTORY` variable so that it is the same for both build systems. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-13ci/lib: support custom output directories when creating test artifactsPatrick Steinhardt1-6/+6
Update `create_failed_test_artifacts ()` so that it can handle arbitrary test output directories. This fixes creation of these artifacts for macOS on GitLab CI, which uses a separate output directory already. This will also be used by our out-of-tree builds with Meson. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-12ci/lib: fix "CI setup" sections with GitLab CIPatrick Steinhardt1-2/+2
Whenever we source "ci/lib.sh" we wrap the directives in a separate group so that they can easily be collapsed in the web UI. And as we source the script multiple times during a single CI run we thus end up with the same section name reused multiple times, as well. This is broken on GitLab CI though, where reusing the same group name is not supported. The consequence is that only the last of these sections can be collapsed. Fix this issue by including the name of the sourcing script in the group's name. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>