summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-10-16refs/files: remove "name exist?" check in lock_ref_oid_basic()Ævar Arnfjörð Bjarmason1-24/+24
In lock_ref_oid_basic() we'll happily lock a reference that doesn't exist yet. That's normal, and is how references are initially born, but we don't need to retain checks here in lock_ref_oid_basic() about the state of the ref, when what we're checking is either checked already, or something we're about to discover by trying to lock the ref with raceproof_create_file(). The one exception is the caller in files_reflog_expire(), who passes us a "type" to find out if the reference is a symref or not. We can move the that logic over to that caller, which can now defer its discovery of whether or not the ref is a symref until it's needed. In the preceding commit an exhaustive regression test was added for that case in a new test in "t1417-reflog-updateref.sh". The improved diagnostics here were added in 5b2d8d6f218 (lock_ref_sha1_basic(): improve diagnostics for ref D/F conflicts, 2015-05-11), and then much of the surrounding code went away recently in my 245fbba46d6 (refs/files: remove unused "errno == EISDIR" code, 2021-08-23). The refs_resolve_ref_unsafe() code being removed here looks like it should be tasked with doing that, but it's actually redundant to other code. The reason for that is as noted in 245fbba46d6 this once widely used function now only has a handful of callers left, which all handle this case themselves. To the extent that we're racy between their check and ours removing this check actually improves the situation, as we'll be doing fewer things between the not-under-lock initial check and acquiring the lock. Why this is OK for all the remaining callers of lock_ref_oid_basic() is noted below. There are only two of those callers: * "git branch -[cm] <oldbranch> <newbranch>": In files_copy_or_rename_ref() we'll call this when we copy or rename refs via rename_ref() and copy_ref(). but only after we've checked if the refname exists already via its own call to refs_resolve_ref_unsafe() and refs_rename_ref_available(). As the updated comment to the latter here notes neither of those are actually needed. If we delete not only this code but also refs_rename_ref_available() we'll do just fine, we'll just emit a less friendly error message if e.g. "git branch -m A B/C" would have a D/F conflict with a "B" file. Actually we'd probably die before that in case reflogs for the branch existed, i.e. when the try to rename() or copy_file() the relevant reflog, since if we've got a D/F conflict with a branch name we'll probably also have the same with its reflogs (but not necessarily, we might have reflogs, but it might not). As some #leftoverbits that code seems buggy to me, i.e. the reflog "protocol" should be to get a lock on the main ref, and then perform ref and/or reflog operations. That code dates back to c976d415e53 (git-branch: add options and tests for branch renaming, 2006-11-28) and probably pre-dated the solidifying of that convention. But in any case, that edge case is not our bug or problem right now. * "git reflog expire <ref>": In files_reflog_expire() we'll call this without previous ref existence checking in files-backend.c, but that code is in turn called by code that's just finished checking if the refname whose reflog we're expiring exists. See ae35e16cd43 (reflog expire: don't lock reflogs using previously seen OID, 2021-08-23) for the current state of that code, and 5e6f003ca8a (reflog_expire(): ignore --updateref for symbolic references, 2015-03-03) for the code we'd break if we only did a "update = !!ref" here, which is covered by the aforementioned regression test in "t1417-reflog-updateref.sh". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-16reflog tests: add --updateref testsÆvar Arnfjörð Bjarmason1-0/+65
Add tests that cover blindspots in "git reflog delete --updateref" behavior. Before this change removing the "type & REF_ISSYMREF" check added in 5e6f003ca8a (reflog_expire(): ignore --updateref for symbolic references, 2015-03-03) would not fail any tests. The "--updateref" option was added in 55f10565371 (git-reflog: add option --updateref to write the last reflog sha1 into the ref, 2008-02-22) for use in git-stash.sh, see e25d5f9c82e (git-stash: add new 'drop' subcommand, 2008-02-22). Even though the regression test I need is just the "C" case here, let's test all these combinations for good measure. I started out doing these as a for-loop, but I think this is more readable. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-16refs API: make refs_rename_ref_available() staticÆvar Arnfjörð Bjarmason3-33/+29
Move the refs_rename_ref_available() function into "refs/files-backend.c". It is file-backend specific. This function was added in 5fe7d825da8 (refs.c: pass a list of names to skip to is_refname_available, 2014-05-01) as rename_ref_available() and was only ever used in this one file-backend specific codepath. So let's move it there. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-16refs API: make parse_loose_ref_contents() not set errnoHan-Wen Nienhuys3-16/+29
Change the parse_loose_ref_contents() function to stop setting "errno" and failure, and to instead pass up a "failure_errno" via a parameter. This requires changing its callers to do the same. The EINVAL error from parse_loose_ref_contents is used in files-backend to create a custom error message. In untangling this we discovered a tricky edge case. The refs_read_special_head() function was relying on parse_loose_ref_contents() setting EINVAL. By converting it to use "saved_errno" we can migrate away from "errno" in this part of the code entirely, and do away with an existing "save_errno" pattern, its only purpose was to not clobber the "errno" we previously needed at the end of files_read_raw_ref(). Let's assert that we can do that by not having files_read_raw_ref() itself operate on *failure_errno in addition to passing it on. Instead we'll assert that if we return non-zero we actually do set errno, thus assuring ourselves and callers that they can trust the resulting "failure_errno". Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-16refs API: make refs_read_raw_ref() not set errnoHan-Wen Nienhuys4-18/+29
Add a "failure_errno" to refs_read_raw_ref(), his allows refs_werrres_ref_unsafe() to pass along its "failure_errno", as a first step before its own callers are migrated to pass it further up the chain. We are leaving out out the refs_read_special_head() in refs_read_raw_ref() for now, as noted in a subsequent commit moving it to "failure_errno" will require some special consideration. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-16refs API: add a version of refs_resolve_ref_unsafe() with "errno"Ævar Arnfjörð Bjarmason2-9/+39
Add a new refs_werrres_ref_unsafe() function, which is like refs_resolve_ref_unsafe() except that it explicitly saves away the "errno" to a passed-in parameter, the refs_resolve_ref_unsafe() then becomes a wrapper for it. In subsequent commits we'll migrate code over to it, before finally making "refs_resolve_ref_unsafe()" with an "errno" parameter the canonical version, so this this function exists only so that we can incrementally migrate callers, it will be going away in a subsequent commit. As the added comment notes has a rather tortured name to be the same length as "refs_resolve_ref_unsafe", to avoid churn as we won't need to re-indent the argument lists, similarly the documentation and structure of it in refs.h is designed to minimize a diff in a subsequent commit, where that documentation will be added to the new refs_resolve_ref_unsafe(). At the end of this migration the "meaningful errno" TODO item left in 76d70dc0c63 (refs.c: make resolve_ref_unsafe set errno to something meaningful on error, 2014-06-20) will be resolved. As can be seen from the use of refs_read_raw_ref() we'll also need to convert some functions that the new refs_werrres_ref_unsafe() itself calls to take this "failure_errno". That will be done in subsequent commits. Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-16branch tests: test for errno propagating on failing readHan-Wen Nienhuys1-0/+22
Add a test for "git branch" to cover the case where .git/refs is symlinked. To check availability, refs_verify_refname_available() will run refs_read_raw_ref() on each prefix, leading to a read() from .git/refs (which is a directory). It would probably be more robust to re-issue the lstat() as a normal stat(), in which case, we would fall back to the directory case, but for now let's just test for the existing behavior as-is. This test covers a regression in a commit that only ever made it to "next", see [1]. 1. http://lore.kernel.org/git/pull.1068.git.git.1629203489546.gitgitgadget@gmail.com Signed-off-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15gc: perform incremental repack when implictly enabledGlen Choo2-7/+26
builtin/gc.c has two ways of checking if multi-pack-index is enabled: - git_config_get_bool() in incremental_repack_auto_condition() - the_repository->settings.core_multi_pack_index in maintenance_task_incremental_repack() The two implementations have existed since the incremental-repack task was introduced in e841a79a13 (maintenance: add incremental-repack auto condition, 2020-09-25). These two values can diverge because prepare_repo_settings() enables the feature in the_repository->settings by default. In the case where core.multiPackIndex is not set in the config, the auto condition would fail, causing the incremental-repack task to not be run. Because we always want to consider the default values, we should always use the_repository->settings. Standardize on using the_repository->settings.core_multi_pack_index to check if multi-pack-index is enabled. Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15fsck: verify multi-pack-index when implictly enabledGlen Choo2-2/+5
Like the previous commit, change fsck to check the "core_multi_pack_index" variable set in "repo-settings.c" instead of reading the "core.multiPackIndex" config variable. This fixes a bug where we wouldn't verify midx if the config key was missing. This bug was introduced in 18e449f86b (midx: enable core.multiPackIndex by default, 2020-09-25) where core.multiPackIndex was turned on by default. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15fsck: verify commit graph when implicitly enabledGlen Choo3-3/+29
Change fsck to check the "core_commit_graph" variable set in "repo-settings.c" instead of reading the "core.commitGraph" variable. This fixes a bug where we wouldn't verify the commit-graph if the config key was missing. This bug was introduced in 31b1de6a09 (commit-graph: turn on commit-graph by default, 2019-08-13), where core.commitGraph was turned on by default. Add tests to "t5318-commit-graph.sh" to verify that fsck checks the commit-graph as expected for the 3 values of core.commitGraph. Also, disable GIT_TEST_COMMIT_GRAPH in t/t0410-partial-clone.sh because some test cases use fsck in ways that assume that commit-graph checking is disabled. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Glen Choo <chooglen@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15Merge branch 'ab/ignore-replace-while-working-on-commit-graph' into ↵Junio C Hamano3-11/+16
gc/use-repo-settings * ab/ignore-replace-while-working-on-commit-graph: commit-graph: don't consider "replace" objects with "verify" commit-graph tests: fix another graph_git_two_modes() helper commit-graph tests: fix error-hiding graph_git_two_modes() helper
2021-10-15pkt-line.[ch]: remove unused packet_read_line_buf()Ævar Arnfjörð Bjarmason6-36/+13
This function was added in 4981fe750b1 (pkt-line: share buffer/descriptor reading implementation, 2013-02-23), but in 01f9ec64c8a (Use packet_reader instead of packet_read_line, 2018-12-29) the code that was using it was removed. Since it's being removed we can in turn remove the "src" and "src_len" arguments to packet_read(), all the remaining users just passed a NULL/NULL pair to it. That function is only a thin wrapper for packet_read_with_status() which still needs those arguments, but for the thin packet_read() convenience wrapper we can do away with it for now. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15pkt-line.[ch]: remove unused packet_buf_write_len()Ævar Arnfjörð Bjarmason2-17/+0
This function was added in f1f4d8acf40 (pkt-line: add packet_buf_write_len function, 2018-03-15) for use in 0f1dc53f45d (remote-curl: implement stateless-connect command, 2018-03-15). In a97d00799a1 (remote-curl: use post_rpc() for protocol v2 also, 2019-02-21) that only user of it went away, let's remove it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15midx.c: guard against commit_lock_file() failuresTaylor Blau1-1/+2
When writing a MIDX, we atomically move the new MIDX into place via commit_lock_file(), but do not check to see if that call was successful. Make sure that we do check in order to prevent us from incorrectly reporting that we wrote a new MIDX if we actually encountered an error. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15midx.c: lookup MIDX by object directory during repackTaylor Blau1-4/+1
Apply similar treatment as in the last commit to the MIDX `repack` operation. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15midx.c: lookup MIDX by object directory during expireTaylor Blau1-4/+3
Before a new MIDX can be written, expire_midx_packs() first loads the existing MIDX, figures out which packs can be expired, and then writes a new MIDX based on that information. In order to load the existing MIDX, it uses load_multi_pack_index(), which mmaps the multi-pack-index file, but does not store the resulting `struct multi_pack_index *` in the object store. write_midx_internal() also needs to open the existing MIDX, and it does so by iterating the results of get_multi_pack_index(), so that it reuses the same pointer held by the object store. But before it can move the new MIDX into place, it close_object_store() to munmap() the multi-pack-index file to accommodate platforms like Windows which don't allow overwriting files which are memory mapped. That's where things get weird. Since expire_midx_packs has its own *separate* memory mapped copy of the MIDX, the MIDX file is still memory mapped! Interestingly, this doesn't seem to cause a problem in our tests. (I believe that this has much more to do with my own lack of familiarity with Windows than it does a lack of coverage in our tests). In any case, we can side-step the whole issue by teaching expire_midx_packs() to use the `struct multi_pack_index` pointer it found via the object store instead of maintain its own copy. That way, when write_midx_internal() calls `close_object_store()`, we know that there are no memory mapped copies of the MIDX laying around. A couple of other small notes about this patch: - As far as I can tell, passing `local == 1` to the call to load_multi_pack_index() was an error, since object_dir could be an alternate. But it doesn't matter, since even though we write `m->local = 1`, we never read that field back later on. - Setting `m = NULL` after write_midx_internal() was likely to prevent a double-free back from when that function took a `struct multi_pack_index *` that it called close_midx() on itself. We can rely on write_midx_internal() to call that for us now. Finally, this enforces the same "the value of --object-dir must be the local object store, or an alternate" rule from f57a739691 (midx: avoid opening multiple MIDXs when writing, 2021-09-01) to the `expire` sub-command, too. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15midx.c: extract MIDX lookup by object_dirTaylor Blau1-10/+17
The first thing that write_midx_internal() does is load the MIDX corresponding to the given object directory, if one is present. Prepare for other functions in midx.c to do the same thing by extracting that operation out to a small helper function. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15Merge branch 'tb/repack-write-midx' into tb/fix-midx-rename-while-mappedJunio C Hamano14-56/+723
* tb/repack-write-midx: test-read-midx: fix leak of bitmap_index struct builtin/repack.c: pass `--refs-snapshot` when writing bitmaps builtin/repack.c: make largest pack preferred builtin/repack.c: support writing a MIDX while repacking builtin/repack.c: extract showing progress to a variable builtin/repack.c: rename variables that deal with non-kept packs builtin/repack.c: keep track of existing packs unconditionally midx: preliminary support for `--refs-snapshot` builtin/multi-pack-index.c: support `--stdin-packs` mode midx: expose `write_midx_file_only()` publicly
2021-10-15grep/pcre2: fix an edge case concerning ascii patterns and UTF-8 dataHamza Mahfooz2-2/+52
If we attempt to grep non-ascii log message text with an ascii pattern, we run into the following issue: $ git log --color --author='.var.*Bjar' -1 origin/master | grep ^Author grep: (standard input): binary file matches So, to fix this teach the grep code to use PCRE2_UTF, as long as the log output is encoded in UTF-8. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Hamza Mahfooz <someguy@effective-light.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15doc lint: make "lint-docs" non-.PHONYÆvar Arnfjörð Bjarmason3-8/+64
Speed up the "lint-docs" target by making it non-.PHONY. Similar to my c234e8a0ecf (Makefile: make the "sparse" target non-.PHONY, 2021-09-23). We'll now create empty files corresponding to a dependency graph for each of these lint scripts. This speeds things up a bit[1], and makes the output correspond to any in-tree changes we have: $ touch git-add.txt; make lint-docs; make lint-docs GEN cmd-list.made GEN doc.dep LINT GITLINK git-add.txt LINT MAN END git-add.txt LINT MAN SEC git-add.txt make: Nothing to be done for 'lint-docs'. As with the "sparse" target changes this has a hard dependency on the use of ".DELETE_ON_ERROR" in the Makefile, added here in db10fc6c09f (doc: simplify Makefile using .DELETE_ON_ERROR, 2021-05-21). This method also depends on the output for us emitting any errors on STDERR (fixed in a preceding commit), as well us these scripts exiting with non-zero on any errors (which they were already doing). 1. $ git show HEAD~:Documentation/Makefile >Makefile.old $ hyperfine --warmup 2 -L f ",.old" 'make -j1 -f Makefile{f} lint-docs' Benchmark #1: make -j1 -f Makefile lint-docs Time (mean ± σ): 60.8 ms ± 1.4 ms [User: 58.7 ms, System: 2.5 ms] Range (min … max): 58.9 ms … 64.0 ms 48 runs Benchmark #2: make -j1 -f Makefile.old lint-docs Time (mean ± σ): 84.0 ms ± 1.5 ms [User: 78.6 ms, System: 5.7 ms] Range (min … max): 81.8 ms … 87.8 ms 35 runs Summary 'make -j1 -f Makefile lint-docs' ran 1.38 ± 0.04 times faster than 'make -j1 -f Makefile.old lint-docs' Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15doc build: speed up "make lint-docs"Ævar Arnfjörð Bjarmason1-1/+1
Extend the trick we use to speed up the "clean" target to also extend to the "lint-docs" target. See 54df87555b1 (Documentation/Makefile: conditionally include doc.dep, 2020-12-08) for the "clean" implementation. The "doc-lint" target only depends on *.txt files, so we don't need to generate GIT-VERSION-FILE etc. if that's all we're doing. This makes the "make lint-docs" target more than 2x as fast: $ git show HEAD~:Documentation/Makefile >Makefile.old $ hyperfine -L f ",.old" 'make -f Makefile{f} lint-docs' Benchmark #1: make -f Makefile lint-docs Time (mean ± σ): 100.2 ms ± 1.3 ms [User: 93.7 ms, System: 6.7 ms] Range (min … max): 98.4 ms … 103.1 ms 29 runs Benchmark #2: make -f Makefile.old lint-docs Time (mean ± σ): 220.0 ms ± 20.0 ms [User: 206.0 ms, System: 18.0 ms] Range (min … max): 206.6 ms … 267.5 ms 11 runs Summary 'make -f Makefile lint-docs' ran 2.19 ± 0.20 times faster than 'make -f Makefile.old lint-docs' Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15doc lint: emit errors on STDERRÆvar Arnfjörð Bjarmason3-4/+4
Have all of the scripts invoked by "make check-docs" emit their output on STDERR. This does not currently matter due to the way we're invoking them, but will in a subsequent change. It's a good idea to do this in any case for consistency with other tools we invoke. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15doc lint: fix error-hiding regressionÆvar Arnfjörð Bjarmason1-3/+3
Fix the broken "make lint-docs" (or "make check-docs" at the top-level) target, which has been broken since my cafd9828e89 (doc lint: lint and fix missing "GIT" end sections, 2021-04-09). The CI for "seen" is emitting an error about a broken gitlink, but due to there being 3x scripts chained via ";" instead of "&&" we're not carrying forward the non-zero exit code. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15tests: stop using top-level "README" and "COPYING" filesÆvar Arnfjörð Bjarmason4-14/+19
In 459b8d22e54 (tests: do not borrow from COPYING and README from the real source, 2015-02-15) tests that used "lib-diff.sh" (called "diff-lib.sh" then) were made to stop relying on the top-level COPYING file, but we still had other tests that referenced it. Let's move them over to use the "COPYING_test_data" utility function introduced in the preceding commit, and in the case of the one test that needed the "README" file use a ROT 13 version of that "COPYING" test data. That test added in afd222967c6 (Extend testing git-mv for renaming of subdirectories, 2006-07-26) just needs more test data that's not the same as the "COPYING" test data, so a ROT 13 version will do. This change removes the last references to ../{README,COPYING} in the test suite. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15"lib-diff" tests: make "README" and "COPYING" test data smallerÆvar Arnfjörð Bjarmason10-419/+34
Follow-up the change in 459b8d22e54 (tests: do not borrow from COPYING and README from the real source, 2015-02-15) by not shipping a full copy of older versions of the top-level "COPYING" and "README" files. The tests that use them just need the small blurb at the top of "COPYING" as test data, or mock data that's dissimilar. Let's provide that with a "COPYING_test_data" function instead. We're not replacing this with some other generic test data (e.g. "lorum ipsum") because these tests require test file header to be the old "COPYING" file. See e.g. "t4003-diff-rename-1.sh" which changes the file, and then does full "test_cmp" comparisons on the resulting "git diff" output. This change only changes tests that used the "lib-diff.sh" library, but splits up what they need into a new "lib-diff-data.sh". A subsequent commit will change related tests that were missed in 459b8d22e54. For the test in "t4008-diff-break-rewrite.sh" the "README" file can go away in favor of echoing the line "some dissimilar content" to a file in the one test that needed it. The point of that test is to start with files "A" and "B", and then have A be more similar to the state of "B" than to its old version (by copying over the content from the "COPYING" file). Just comparing the pre-image of "some dissimilar content" and later a munged version of the "COPYING" output serves that purpose. While we're at it get rid of a stray "echo $tree" debugging line added in 15d061b435a ([PATCH] Fix the way diffcore-rename records unremoved source., 2005-05-27), and stop calling "hash-object" to get the hash of an object we've just added to the index. We can instead extract that information from the index itself with "rev-parse". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15commit-graph: don't consider "replace" objects with "verify"Ævar Arnfjörð Bjarmason2-1/+2
Extend the code added in d6538246d3d (commit-graph: not compatible with replace objects, 2018-08-20) which ignored replace objects in the "write" command to ignore it in the "verify" command too. We can just move this assignment to the cmd_commit_graph(), it dispatches to "write" and "verify", and we're unlikely to ever get a sub-command that would like to consider replace refs. This will make tests added in eddc1f556cd (mktag tests: test update-ref and reachable fsck, 2021-06-17) pass in combination with the "GIT_TEST_COMMIT_GRAPH" mode added in 859fdc0c3cf (commit-graph: define GIT_TEST_COMMIT_GRAPH, 2018-08-29), except that mode is currently broken (but is being fixed concurrently). See the discussion starting at [1]. 1. https://lore.kernel.org/git/87wnmihswp.fsf@evledraar.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15commit-graph tests: fix another graph_git_two_modes() helperÆvar Arnfjörð Bjarmason1-8/+12
In 135a7123755 (commit-graph: add --split option to builtin, 2019-06-18) this function was copy/pasted to the split commit-graph tests, as in the preceding commit we need to fix this to use &&-chaining, so it won't be hiding errors. Unlike its sister function in "t5318-commit-graph.sh", which we got lucky with, this one was hiding a real test failure. A tests added in c523035cbd8 (commit-graph: allow cross-alternate chains, 2019-06-18) has never worked as intended. Unlike most other graph_git_behavior uses in this file it clones the repository into a sub-directory, so we'll need to refer to "commits/6" as "origin/commits/6". It's not easy to simply move the "graph_git_behavior" to the test above it, since it itself spawns a "test_expect_success". Let's instead add support to "graph_git_behavior()" and "graph_git_two_modes()" to pass a "-C" argument to git. We also need to add a "test -d fork" here, because otherwise we'll fail on e.g.: GIT_SKIP_TESTS=t5324.13 ./t5324-split-commit-graph.sh Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15commit-graph tests: fix error-hiding graph_git_two_modes() helperÆvar Arnfjörð Bjarmason1-2/+2
The graph_git_two_modes() helper added in 177722b3442 (commit: integrate commit graph with commit parsing, 2018-04-10) didn't &&-chain its "git commit-graph" invocations, which as can be seen with SANITIZE=leak will happily mark tests as passing if both of these commands die, since test_cmp() will be comparing two empty files. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-15test-lib.sh: try to re-chmod & retry on failed trash removalÆvar Arnfjörð Bjarmason1-1/+13
Try to re-chmod the trash directory on startup if we fail to "rm -rf" it. This fixes problems where the test leaves the trash directory behind in a bad permission state for whatever reason. This fixes an interaction between [1] where t0004-unwritable.sh was made to use "test_when_finished" for cleanup, and [2] which added the "--immediate" mode. If a test in this file failed when running with "--immediate" we wouldn't run the "test_when_finished" block, which re-chmods the ".git/objects" directory (see [1]). This can be demonstrated as e.g. (output snipped for less verbosity): $ ./t0004-unwritable.sh --run=3 --immediate ok 1 # skip setup (--run) ok 2 # skip write-tree should notice unwritable repository (--run) not ok 3 - commit should notice unwritable repository [...] $ ./t0004-unwritable.sh --run=3 --immediate rm: cannot remove '[...]/trash directory.t0004-unwritable/.git/objects/info': Permission denied FATAL: Cannot prepare test area [...] Instead of some version of reverting [1] let's make the test-lib.sh resilient to this edge-case, it will happen due to [1], but also e.g. if the relevant "test-lib.sh" process is kill -9'd during the test run. We should try harder to recover in this case. If we fail to remove the test directory let's retry after (re-)chmod-ing it. This doesn't need to be guarded by something that's equivalent to "POSIXPERM" since if we don't support "chmod" we were about to fail anyway. Let's also discard any error output from (a possibly nonexisting) "chmod", we'll fail on the subsequent "rm -rf" anyway, likewise for the first "rm -rf" invocation, we don't want to get the "cannot remove" output if we can get around it with the "chmod", but we do want any error output from the second "rm -rf", in case that doesn't fix the issue. The lack of &&-chaining between the "chmod" and "rm -rf" is intentional, if we fail the first "rm -rf", can't chmod, but then succeed the second time around that's what we were hoping for. We just want to nuke the directory, not carry forward every possible error code or error message. 1. dbda967684d (t0004 (unwritable files): simplify error handling, 2010-09-06) 2. b586744a864 (test: skip clean-up when running under --immediate mode, 2011-06-27) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-14Thirteenth batchJunio C Hamano1-0/+22
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-14test-lib.sh: use "Bail out!" syntax on bad SANITIZE=leak useÆvar Arnfjörð Bjarmason1-4/+14
Improve the "GIT_TEST_PASSING_SANITIZE_LEAK=true" test mode added in 956d2e4639b (tests: add a test mode for SANITIZE=leak, run it in CI, 2021-09-23) to use a TAP "Bail out!" message when exiting. This will cause the test run to exit immediately under a TAP consumer like "prove(1)". See 614fe015212 (test-lib: bail out when "-v" used under "prove", 2016-10-22) for the initial introduction of "Bail out!" to the --verbose being amended here. Before this compiling with "SANITIZE=" and running the tests with "prove(1)" would cause all the tests to be run to the end (output trimmed for fewer columns): $ GIT_TEST_PASSING_SANITIZE_LEAK=true make rm -f -r 'test-results' *** prove *** t0000-basic.sh ......... Dubious, test returned 1 (wstat 256, 0x100) No subtests run t0001-init.sh .......... Dubious, test returned 1 (wstat 256, 0x100) No subtests run [...output where we list every single t[0-9]*.sh file as failing snipped] Whereas now we'll fail early, like this ("->" line wrapping added): $ GIT_TEST_PASSING_SANITIZE_LEAK=true make [...] t0000-basic.sh ..................................... Bailout called. Further testing stopped: -> GIT_TEST_PASSING_SANITIZE_LEAK=true has no effect except when compiled with SANITIZE=leak FAILED--Further testing stopped: GIT_TEST_PASSING_SANITIZE_LEAK=true has no effect except -> when compiled with SANITIZE=leak make: *** [Makefile:53: prove] Error 1 This change also adds a red color to the "Bailout called" line, as we're now using "say_color error". That improves existing output in the case of e.g.: $ prove -j8 t[0-9]*.sh :: -v Bailout called. Further testing stopped: verbose mode forbidden under TAP harness; try --verbose-log FAILED--Further testing stopped: verbose mode forbidden under TAP harness; try --verbose-log We don't need to have a "Bail out! " prefix when we're not running under a TAP consumer (i.e. if test -n "$HARNESS_ACTIVE"), but let's not make the output conditional on that. Showing it under e.g.: $ GIT_TEST_PASSING_SANITIZE_LEAK=true ./t0095-bloom.sh Bail out! GIT_TEST_PASSING_SANITIZE_LEAK=true has no effect except when compiled with SANITIZE=leak Doesn't harm anything, and I don't think the (small) complexity of only adding this if we're under "$HARNESS_ACTIVE" is worth it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-14test-lib.sh: de-duplicate error() teardown codeÆvar Arnfjörð Bjarmason1-3/+7
De-duplicate the "finalize_junit_xml; GIT_EXIT_OK=t; exit 1" code shared between the "error()" and "--immediate on failure" code paths, in preparation for adding a third user in a subsequent commit. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-13Merge branch 'ja/doc-status-types-and-copies'Junio C Hamano2-11/+14
A few kinds of changes "git status" can show were not documented. * ja/doc-status-types-and-copies: Documentation/git-status: mention how to detect copies Documentation/git-status: document porcelain status T (typechange) Documentation/diff-format: state in which cases porcelain status is T Documentation/git-status: remove impossible porcelain status DR and DC
2021-10-13Merge branch 'ab/make-sparse-for-real'Junio C Hamano2-3/+7
Prevent "make sparse" from running for the source files that haven't been modified. * ab/make-sparse-for-real: Makefile: make the "sparse" target non-.PHONY
2021-10-13Merge branch 'ab/align-parse-options-help'Junio C Hamano8-26/+132
When "git cmd -h" shows more than one line of usage text (e.g. the cmd subcommand may take sub-sub-command), parse-options API learned to align these lines, even across i18n/l10n. * ab/align-parse-options-help: parse-options: properly align continued usage output git rev-parse --parseopt tests: add more usagestr tests send-pack: properly use parse_options() API for usage string parse-options API users: align usage output in C-strings
2021-10-13Merge branch 'ab/help-config-vars'Junio C Hamano7-68/+166
Teach "git help -c" into helping the command line completion of configuration variables. * ab/help-config-vars: help: move column config discovery to help.c library help / completion: make "git help" do the hard work help tests: test --config-for-completion option & output help: simplify by moving to OPT_CMDMODE() help: correct logic error in combining --all and --guides help: correct logic error in combining --all and --config help tests: add test for --config output help: correct usage & behavior of "git help --guides" help: correct the usage string in -h and documentation
2021-10-13Merge branch 'jh/builtin-fsmonitor-part1'Junio C Hamano13-198/+587
Built-in fsmonitor (part 1). * jh/builtin-fsmonitor-part1: t/helper/simple-ipc: convert test-simple-ipc to use start_bg_command run-command: create start_bg_command simple-ipc/ipc-win32: add Windows ACL to named pipe simple-ipc/ipc-win32: add trace2 debugging simple-ipc: move definition of ipc_active_state outside of ifdef simple-ipc: preparations for supporting binary messages. trace2: add trace2_child_ready() to report on background children
2021-10-13Merge branch 'ab/config-based-hooks-1'Junio C Hamano19-100/+127
Mostly preliminary clean-up in the hook API. * ab/config-based-hooks-1: hook-list.h: add a generated list of hooks, like config-list.h hook.c users: use "hook_exists()" instead of "find_hook()" hook.c: add a hook_exists() wrapper and use it in bugreport.c hook.[ch]: move find_hook() from run-command.c to hook.c Makefile: remove an out-of-date comment Makefile: don't perform "mv $@+ $@" dance for $(GENERATED_H) Makefile: stop hardcoding {command,config}-list.h Makefile: mark "check" target as .PHONY
2021-10-13Merge branch 'ab/lib-subtest'Junio C Hamano2-330/+213
Updates to the tests in t0000 to test the test framework. * ab/lib-subtest: test-lib tests: get rid of copy/pasted mock test code test-lib tests: assert 1 exit code, not non-zero test-lib tests: refactor common part of check_sub_test_lib_test*() test-lib tests: avoid subshell for "test_cmp" for readability test-lib tests: don't provide a description for the sub-tests test-lib tests: split up "write and run" into two functions test-lib tests: move "run_sub_test" to a new lib-subtest.sh
2021-10-13Merge branch 'en/removing-untracked-fixes'Junio C Hamano23-74/+366
Various fixes in code paths that move untracked files away to make room. * en/removing-untracked-fixes: Documentation: call out commands that nuke untracked files/directories Comment important codepaths regarding nuking untracked files/dirs unpack-trees: avoid nuking untracked dir in way of locally deleted file unpack-trees: avoid nuking untracked dir in way of unmerged file Change unpack_trees' 'reset' flag into an enum Remove ignored files by default when they are in the way unpack-trees: make dir an internal-only struct unpack-trees: introduce preserve_ignored to unpack_trees_options read-tree, merge-recursive: overwrite ignored files by default checkout, read-tree: fix leak of unpack_trees_options.dir t2500: add various tests for nuking untracked files
2021-10-13Merge branch 'mt/grep-submodule-textconv'Junio C Hamano2-3/+106
"git grep --recurse-submodules" takes trees and blobs from the submodule repository, but the textconv settings when processing a blob from the submodule is not taken from the submodule repository. A test is added to demonstrate the issue, without fixing it. * mt/grep-submodule-textconv: grep: demonstrate bug with textconv attributes and submodules
2021-10-13Merge branch 'ds/add-rm-with-sparse-index'Junio C Hamano13-51/+504
"git add", "git mv", and "git rm" have been adjusted to avoid updating paths outside of the sparse-checkout definition unless the user specifies a "--sparse" option. * ds/add-rm-with-sparse-index: advice: update message to suggest '--sparse' mv: refuse to move sparse paths rm: skip sparse paths with missing SKIP_WORKTREE rm: add --sparse option add: update --renormalize to skip sparse paths add: update --chmod to skip sparse paths add: implement the --sparse option add: skip tracked paths outside sparse-checkout cone add: fail when adding an untracked sparse file dir: fix pattern matching on dirs dir: select directories correctly t1092: behavior for adding sparse files t3705: test that 'sparse_entry' is unstaged
2021-10-13doc: add bundle-format to TECH_DOCSTodd Zullinger1-0/+1
A link to the bundle-format was added in 5c8273d57c (bundle doc: rewrite the "DESCRIPTION" section, 2021-07-31). Ensure `technical/bundle-format.html` is created to avoid a broken link in `git-bundle.html`. Signed-off-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-13mergetools/xxdiff: prevent segfaults from stopping difftoolDavid Aguilar1-0/+7
Users often use "git difftool HEAD^" to review their work, and have "mergetool.prompt" set to false so that difftool does not prompt them before diffing each file. This is very convenient because users can see all their diffs by reviewing the xxdiff windows one at a time. A problem occurs when xxdiff encounters some binary files. It can segfault and return exit code 128, which is special-cased by git-difftool-helper as being an extraordinary situation that aborts the process. Suppress the exit code from xxdiff in its diff_cmd() implementation when we see exit code 128 so that the GIT_EXTERNAL_DIFF loop continues on uninterrupted to the next file rather than aborting when it encounters the first binary file. Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-13sequencer: fix a memory leak in do_reset()Ævar Arnfjörð Bjarmason1-2/+2
Fix a memory leak introduced in 9055e401dd6 (sequencer: introduce new commands to reset the revision, 2018-04-25), which called setup_unpack_trees_porcelain() without a corresponding call to clear_unpack_trees_porcelain(). This introduces a change in behavior in that we now start calling clear_unpack_trees_porcelain() even without having called the setup_unpack_trees_porcelain(). That's OK, that clear function, like most others, will accept a zero'd out struct. This inches us closer to passing various tests in "t34*.sh" (e.g. "t3434-rebase-i18n.sh"), but because they have so many other memory leaks in revisions.c this doesn't make any test file or even a single test pass. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-13sequencer: add a "goto cleanup" to do_reset()Ævar Arnfjörð Bjarmason1-19/+13
Restructure code that's mostly added in 9055e401dd6 (sequencer: introduce new commands to reset the revision, 2018-04-25) to avoid code duplication, and to make freeing other resources easier in a subsequent commit. It's safe to initialize "tree_desc" to be zero'd out in order to unconditionally free desc.buffer, it won't be initialized on the first couple of "goto"'s. There are three earlier "return"'s in this function which should probably be made to use this new "cleanup" too, per [1] it looks like they're leaving behind stale locks. But let's not try to fix every potential bug here now, I'm just trying to narrowly plug a memory leak. 1. https://lore.kernel.org/git/CABPp-BH=3DP-dXRCphY53-3eZd1TU8h5GY_M12nnbEGm-UYB9Q@mail.gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-13ci(windows): ensure that we do not pick up random executablesJohannes Schindelin1-3/+3
On the Windows build agents, a lot of programs are installed, and added to the PATH automatically. One such program is Git for Windows, and due to the way it is set up, unfortunately its copy of `gpg.exe` is also reachable via the PATH. This usually does not pose any problems. To the contrary, it even allows us to test the GPG parts of Git's test suite even if `gpg.exe` is not delivered as part of `git-sdk-64-minimal`, the minimal subset of Git for Windows' SDK that we use in the CI builds to compile Git. However, every once in a while we build a new MSYS2 runtime, which means that there is a mismatch between the copy in `git-sdk-64-minimal` and the copy in C:\Program Files\Git\usr\bin. When that happens we hit the dreaded problem where only one `msys-2.0.dll` is expected to be in the PATH, and things start to fail. Let's avoid all of this by restricting the PATH to the minimal set. This is actually done by `git-sdk-64-minimal`'s `/etc/profile`, and we just have to source this file manually (one would expect that it is sourced automatically, but the Bash steps in Azure Pipelines/GitHub workflows are explicitly run using `--noprofile`, hence the need for doing this explicitly). Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-13ssh signing: clarify trustlevel usage in docsFabian Stelzer1-3/+1
facca53ac added verification for ssh signatures but incorrectly described the usage of gpg.minTrustLevel. While the verifications trustlevel is stil set to fully or undefined depending on if the key is known or not it has no effect on the verification result. Unknown keys will always fail verification. This commit updates the docs to match this behaviour. Signed-off-by: Fabian Stelzer <fs@gigacodes.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12signature-format.txt: explain and illustrate multi-line headersJunio C Hamano1-4/+20
A signature attached to a signed commit, and the contents of the commit that merged a signed tag, are both recorded as a value of an object header field as a multi-line value, and are subject to the formatting convention for multi-line values in the headers, with a leading SP signaling that the rest of the line is a continuation of the previous line. Most notably, an empty line in such a multi-line value would result in a line with a sole SP on it. Examples in the signature-format technical documentation include a few of these cases but we did not show these otherwise invisible SPs in the example. These trailing spaces cannot be seen on display or on paper, and forces the readers to look for them in their editors or pagers, even if we added them to the document. Extend the overview section to explain the multi-line value formatting and highlight these otherwise invisible SPs by inventing the "a dollar-sign at the end of line that appears after SP merely signals that there is a SP there, and the dollar-sign itself does not appear in the real file" notation, inspired by "cat -e" output, to help readers to learn exactly where such "a single SP that is originally an empty line" appears in the examples. Reported-by: Rob Browning <rlb@defaultvalue.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-10-12urlmatch: add underscore to URL_HOST_CHARSJeff King2-2/+2
When parsing a URL to normalize it, we allow hostnames to contain only dot (".") or dash ("-"), plus brackets and colons for IPv6 literals. This matches the old URL standard in RFC 1738, which says: host = hostname | hostnumber hostname = *[ domainlabel "." ] toplabel domainlabel = alphadigit | alphadigit *[ alphadigit | "-" ] alphadigit But this was later updated by RFC 3986, which is more liberal: host = IP-literal / IPv4address / reg-name reg-name = *( unreserved / pct-encoded / sub-delims ) unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~" While names with underscore in them are not common and possibly violate some DNS rules, they do work in practice, and we will happily contact them over http://, git://, or ssh://. It seems odd to ignore them for purposes of URL matching, especially when the URL RFC seems to allow them. There shouldn't be any downside here. It's not a syntactically significant character in a URL, so we won't be confused about parsing; we'd have simply rejected such a URL previously (the test here checks the url code directly, but the obvious user-visible effect would be failing to match credential.http://foo_bar.example.com.helper, or similar config in http.<url>.*). Arguably we'd want to allow tilde ("~") here, too. There's likewise probably no downside, but I didn't add it simply because it seems like an even less likely character to appear in a hostname. Reported-by: Alex Waite <alex@waite.eu> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>