summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-06-28submodule: refrain from filtering GIT_CONFIG_COUNTJonathan Tan1-1/+2
14111fc492 ("git: submodule honor -c credential.* from command line", 2016-03-01) taught Git to pass through the GIT_CONFIG_PARAMETERS environment variable when invoking a subprocess on behalf of a submodule. But when d8d77153ea ("config: allow specifying config entries via envvar pairs", 2021-01-15) introduced support for GIT_CONFIG_COUNT (and its associated GIT_CONFIG_KEY_? and GIT_CONFIG_VALUE_?), the subprocess mechanism wasn't updated to also pass through these variables. Since they are conceptually the same (d8d77153ea was written to address a shortcoming of GIT_CONFIG_PARAMETERS), update the submodule subprocess mechanism to also pass through GIT_CONFIG_COUNT. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28promisor-remote: support per-repository configJonathan Tan4-46/+82
Instead of using global variables to store promisor remote information, store this config in struct repository instead, and add repository-agnostic non-static functions corresponding to the existing non-static functions that only work on the_repository. The actual lazy-fetching of missing objects currently does not work on repositories other than the_repository, and will still not work after this commit, so add a BUG message explaining this. A subsequent commit will remove this limitation. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28repository: move global r_f_p_c to repo structJonathan Tan5-20/+23
Move repository_format_partial_clone, which is currently a global variable, into struct repository. (Full support for per-repository partial clone config will be done in a subsequent commit - this is split into its own commit because of the extent of the changes needed.) The new repo-specific variable cannot be set in check_repository_format_gently() (as is currently), because that function does not know which repo it is operating on (or even whether the value is important); therefore this responsibility is delegated to the outermost caller that knows. Of all the outermost callers that know (found by looking at all functions that call clear_repository_format()), I looked at those that either read from the main Git directory or write into a struct repository. These callers have been modified accordingly (write to the_repository in the former case and write to the given struct repository in the latter case). Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> Reviewed-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28git-p4: fix failed submit by skip non-text data filesdorgon.chang1-2/+5
If the submit contain binary files, it will throw exception and stop submit when try to append diff line description. This commit will skip non-text data files when exception UnicodeDecodeError thrown. The skip will not affect actual submit files in the resulting cl, the diff line description will only appear in submit template, so you can review what changed before actully submit to p4. I don't know if add any message here will be helpful for users, so I choose to just skip binary content, since it already append filename previously. Signed-off-by: dorgon.chang <dorgonman@hotmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28show-branch tests: add missing testsÆvar Arnfjörð Bjarmason1-0/+61
Add missing tests for --remotes, --list and --merge-base. These are not exhaustive, but better than the nothing we have now. There were some tests for this command added in f76412ed6db ([PATCH] Add 'git show-branch'., 2005-08-21) has never been properly tested, namely for the --all option in t6432-merge-recursive-space-options.sh, and some of --merge-base and --independent in t6010-merge-base.sh. This fixes a few more blind spots, but there's still a lot of behavior that's not tested for. These new tests show the odd (and possibly unintentional) behavior of --merge-base with one argument, and how its output is the same as "git merge-base" with N bases in this particular case. See the test added in f621a8454d1 (git-merge-base/git-show-branch --merge-base: Documentation and test, 2009-08-05) for a case where the two aren't the same. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28show-branch: don't <COLOR></RESET> for space charactersÆvar Arnfjörð Bjarmason2-3/+36
Change the colored output introduced in ab07ba2a24 (show-branch: color the commit status signs, 2009-04-22) to not color and reset each individual space character we use for padding. The intent is to color just the "!", "+" etc. characters. This makes the output easier to test, so let's do that now. The test would be much more verbose without a color/reset for each space character. Since the coloring cycles through colors we previously had a "rainbow of space characters". In theory this breaks things for anyone who's relying on the exact colored output of show-branch, in practice I'd think anyone parsing it isn't actively turning on the colored output. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28mktag tests: test fast-exportÆvar Arnfjörð Bjarmason1-0/+8
Pass the bad tags we've created in the mktag tests through fast-export, it will die on the bad object or ref, let's make sure that happens. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28mktag tests: test for-each-refÆvar Arnfjörð Bjarmason1-0/+13
Add a "for-each-ref" for all the mktag tests. This test would have caught the segfault which was fixed in c6854508808 (ref-filter: fix NULL check for parse object failure, 2021-04-01). Let's make sure we test that code more exhaustively. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28mktag tests: test update-ref and reachable fsckÆvar Arnfjörð Bjarmason1-1/+20
Extend the mktag tests to pass the created bad tag through update-ref and fsck. The reason for passing it through update-ref is to guard against it having a segfault as for-each-ref did before c6854508808 (ref-filter: fix NULL check for parse object failure, 2021-04-01). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28mktag tests: test hash-object --literally and unreachable fsckÆvar Arnfjörð Bjarmason1-8/+38
Extend the mktag tests to pass the tag we've created through both hash-object --literally and fsck. This checks that fsck itself will not complain about certain invalid content if a reachable tip isn't involved. Due to how fsck works and walks the graph the failure will be different if the object is reachable, so we might succeed before we've created the ref. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28merge-ort: add prefetching for content mergesElijah Newren2-1/+51
Commit 7fbbcb21b1 ("diff: batch fetching of missing blobs", 2019-04-05) introduced batching of fetching missing blobs, so that the diff machinery would have one fetch subprocess grab N blobs instead of N processes each grabbing 1. However, the diff machinery is not the only thing in a merge that needs to work on blobs. The 3-way content merges need them as well. Rather than download all the blobs 1 at a time, prefetch all the blobs needed for regular content merges. This does not cover all possible paths in merge-ort that might need to download blobs. Others include: - The blob_unchanged() calls to avoid modify/delete conflicts (when blob renormalization results in an "unchanged" file) - Preliminary content merges needed for rename/add and rename/rename(2to1) style conflicts. (Both of these types of conflicts can result in nested conflict markers from the need to do two levels of content merging; the first happens before our new prefetch_for_content_merges() function.) The first of these wouldn't be an extreme amount of work to support, and even the second could be theoretically supported in batching, but all of these cases seem unusual to me, and this is a minor performance optimization anyway; in the worst case we only get some of the fetches batched and have a few additional one-off fetches. So for now, just handle the regular 3-way content merges in our prefetching. For the testcase from the previous commit, the number of downloaded objects remains at 63, but this drops the number of fetches needed from 32 down to 20, a sizeable reduction. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28diffcore-rename: use a different prefetch for basename comparisonsElijah Newren2-20/+85
merge-ort was designed to minimize the amount of data needed and used, and several changes were made to diffcore-rename to take advantage of extra metadata to enable this data minimization (particularly the relevant_sources variable for skipping "irrelevant" renames). This effort obviously succeeded in drastically reducing computation times, but should also theoretically allow partial clones to download much less information. Previously, though, the "prefetch" command used in diffcore-rename had never been modified and downloaded many blobs that were unnecessary for merge-ort. This commit corrects that. When doing basename comparisons, we want to fetch only the objects that will be used for basename comparisons. If after basename fetching this leaves us with no more relevant sources (or no more destinations), then we won't need to do the full inexact rename detection and can skip downloading additional source and destination files. Even if we have to do that later full inexact rename detection, irrelevant sources are culled after basename matching and before the full inexact rename detection, so we can still avoid downloading the blobs for irrelevant sources. Rename prefetch() to inexact_prefetch(), and introduce a new basename_prefetch() to take advantage of this. If we modify the testcase from commit 557ac0350d ("merge-ort: begin performance work; instrument with trace2_region_* calls", 2021-01-23) to pass --sparse --filter=blob:none to the clone command, and use the new trace2 "fetch_count" output from a few commits ago to track both the number of fetch subcommands invoked and the number of objects fetched across all those fetches, then for the mega-renames testcase we observe the following: BEFORE this commit, rebasing 35 patches: strategy # of fetches total # of objects fetched --------- ------------ -------------------------- recursive 62 11423 ort 30 11391 AFTER this commit, rebasing the same 35 patches: ort 32 63 This means that the new code only needs to download less than 2 blobs per patch being rebased. That is especially interesting given that the repository at the start only had approximately half a dozen TOTAL blobs downloaded to start with (because the default sparse-checkout of just the toplevel directory was in use). So, for this particular linux kernel testcase that involved ~26,000 renames on the upstream side (drivers/ -> pilots/) across which 35 patches were being rebased, this change reduces the number of blobs that need to be downloaded by a factor of ~180. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28diffcore-rename: allow different missing_object_cb functionsElijah Newren1-19/+39
estimate_similarity() was setting up a diff_populate_filespec_options every time it was called, requiring the caller of estimate_similarity() to pass in some data needed to set up this option. Currently the needed data consisted of a single variable (skip_unmodified), but we want to also have the different estimate_similarity() callsites start using different missing_object_cb functions as well. Rather than also passing that data in, just have the caller pass in the whole diff_populate_filespec_options, and reduce the number of times we need to set it up. As a side note, this also drops the number of calls to has_promisor_remote() dramatically. If L is the number of basename paths to compare, M is the number of inexact sources, and N is the number of inexact destinations, then the number of calls to has_promisor_remote() drops from L+M*N down to at most 2 -- one for each of the sites that calls estimate_similarity(). has_promisor_remote() is a very fast function so this almost certainly has no measurable performance impact, but it seems cleaner to avoid calling that function so many times. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-28t6421: add tests checking for excessive object downloads during mergeElijah Newren1-0/+440
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-21l10n: fixed tripple-letter typosJiang Xin6-11/+11
Andrei found that the word "shallow" has an extra letter "l" in "po/zh_CN.po". There are similar typos in other l10n files. Reported-by: Andrei Rybak <rybak.a.v@gmail.com> Signed-off-by: Jiang Xin <worldhello.net@gmail.com>
2021-06-19pull: trivial whitespace style fixFelipe Contreras1-3/+3
Two spaces unaligned to anything is not part of the coding-style. A single tab is. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-19pull: trivial cleanupFelipe Contreras1-4/+2
There's no need to store ran_ff. Now it's obvious from the conditionals. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-19pull: cleanup autostash checkFelipe Contreras1-9/+7
Currently "git pull --rebase" takes a shortcut in the case a fast-forward merge is possible; run_merge() is called with --ff-only. However, "git merge" didn't have an --autostash option, so, when "git pull --rebase --autostash" was called *and* the fast-forward merge shortcut was taken, then the pull failed. This was fixed in commit f15e7cf5cc (pull: ff --rebase --autostash works in dirty repo, 2017-06-01) by simply skipping the fast-forward merge shortcut. Later on "git merge" learned the --autostash option [a03b55530a (merge: teach --autostash option, 2020-04-07)], and so did "git pull" [d9f15d37f1 (pull: pass --autostash to merge, 2020-04-07)]. Therefore it's not necessary to skip the fast-forward merge shortcut anymore when called with --rebase --autostash. Let's always take the fast-forward merge shortcut by essentially reverting f15e7cf5cc. Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-19completion: bash: fix late declaration of __git_cmd_idxFabian Wermelinger1-0/+1
A recent update to contrib/completion/git-completion.bash causes bash to fail auto complete custom commands that are wrapped with __git_func_wrap. Declaring __git_cmd_idx=0 inside __git_func_wrap resolves the issue. Signed-off-by: Fabian Wermelinger <fabianw@mavt.ethz.ch> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-19t: use portable wrapper for readlink(1)Jeff King3-3/+9
Not all systems have a readlink program available for use by the shell. This causes t3210 to fail on at least AIX. Let's provide a perl one-liner to do the same thing, and use it there. I also updated calls in t9802. Nobody reported failure there, but it's the same issue. Presumably nobody actually tests with p4 on AIX in the first place (if it is even available there). I left the use of readlink in the "--valgrind" setup in test-lib.sh, as valgrind isn't available on exotic platforms anyway (and I didn't want to increase dependencies between test-lib.sh and test-lib-functions.sh). There's one other curious case. Commit d2addc3b96 (t7800: readlink may not be available, 2016-05-31) fixed a similar case. We can't use our wrapper function there, though, as it's inside a sub-script triggered by Git. It uses a slightly different technique ("ls" piped to "sed"). I chose not to use that here as it gives confusing "ls -l" output if the file is unexpectedly not a symlink (which is OK for its limited use, but potentially confusing for general use within the test suite). The perl version emits the empty string. Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-17test: refactor to use "get_abbrev_oid" to get abbrev oidJiang Xin15-52/+72
Add new function "get_abbrev_oid" to get abbrev object ID. This function has a default value which helps to prepare a nonempty replace pattern for sed command. An empty replace pattern may cause sed fail to allocate memory. Refactor function "make_user_friendly_and_stable_output" to use "get_abbrev_oid" to get abbrev object ID. Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-17test: refactor to use "test_commit" to create commitsJiang Xin2-38/+12
Refactor function "create_commits_in" to use "test_commit" to create commit. Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-17test: compare raw output, not mangle tabs and spacesJiang Xin31-966/+972
Before comparing with the expect file, we used to call function "make_user_friendly_and_stable_output" to filter out trailing spaces in output. Ævar recommends using pattern "s/Z$//" to prepare expect file, and then compare it with raw output. Since we have fixed the issue of occasionally missing the clear-to-eol suffix when displaying sideband #2 messages, it is safe and stable to test against raw output. Suggested-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-17sideband: don't lose clear-to-eol at packet boundaryJiang Xin1-0/+23
When "demultiplex_sideband()" sees a nonempty message ending with CR or LF on the sideband #2, it adds "suffix" string to clear to the end of the current line, which helps when relaying a progress display whose records are terminated with CRs. But if it sees a single LF, no clear-to-end suffix should be appended, because this single LF is used to end the progress display by moving to the next line, and the final progress display above should be preserved. However, the code forgot that depending on the length of the payload line, such a CR may fall exactly at the packet boundary and the number of bytes before the CR from the beginning of the packet could be zero. In such a case, the message that was terminated by the CR were leftover in the "scratch" buffer in the previous call to the function and we still need to clear to the end of the current line. Helped-by: Junio C Hamano <gitster@pobox.com> Helped-by: Nicolas Pitre <nico@fluxnic.net> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-17t6020: fix incompatible parameter expansionJiang Xin1-19/+31
Ævar reported that the function `make_user_friendly_and_stable_output()` failed on a i386 box (gcc45) in the gcc farm boxes with error: sed: couldn't re-allocate memory It turns out that older versions of bash (4.3) or dash (0.5.7) cannot evaluate expression like `${A%${A#???????}}` used to get the leading 7 characters of variable A. Replace the incompatible parameter expansion so that t6020 works on older version of bash or dash. Reported-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Jiang Xin <zhiyou.jx@alibaba-inc.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-16userdiff: add support for C# record typesJulian Verdurmen1-1/+1
Records are added in C# 9 Code example : public record Person(string FirstName, string LastName); For more information, see: * https://docs.microsoft.com/en-us/dotnet/csharp/whats-new/csharp-9 Signed-off-by: Julian Verdurmen <julian.verdurmen@outlook.com> Reviewed-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-16promisor-remote: output trace2 statistics for number of objects fetchedElijah Newren1-2/+5
Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-16*: fix typosDerrick Stolee2-2/+1
These typos were found while searching the codebase for gendered pronouns. In the case of t9300-fast-import.sh, remove a confusing comment that is unnecessary to the understanding of the test. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-16comments: avoid using the gender of our usersFelipe Contreras6-8/+8
We generally avoid specifying the gender of our users in order to be more inclusive, but sometimes a few slip by due to habit. Since by doing a little bit of rewording we can avoid this irrelevant detail, let's do so. Inspired-by: Derrick Stolee <dstolee@microsoft.com> Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-16doc: avoid using the gender of other peopleFelipe Contreras3-6/+5
Using gendered pronouns for an anonymous person applies a gender where none is known and further excludes readers who do not use gendered pronouns. Avoid such examples in the documentation by using "they" or passive voice to avoid the need for a pronoun. Inspired-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Felipe Contreras <felipe.contreras@gmail.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-16gc tests: add a test for the "pre-auto-gc" hookÆvar Arnfjörð Bjarmason1-0/+46
Add a missing test for the behavior of the pre-auto-gc hook added in 0b85d92661e (Documentation/hooks: add pre-auto-gc hook, 2008-04-02). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-16pre-commit hook tests: don't leave "actual" nonexisting on failureÆvar Arnfjörð Bjarmason1-0/+1
Start by creating an "actual" file in a core.hooksPath test that has the hook echoing to the "actual" file. We later test_cmp that file to see what hooks were run. If we fail to run our hook(s) we'll have an empty list of hooks for the test_cmp instead of a nonexisting file. For the logic of this test that makes more sense. See 867ad08a261 (hooks: allow customizing where the hook directory is, 2016-05-04) for the commit that added these tests. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-15graph: improve grammar of "invalid color" error messageAlex Henrie1-1/+1
Without the "d", it sounds like a command, not an error, and is liable to be translated incorrectly. Signed-off-by: Alex Henrie <alexhenrie24@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-15show-branch tests: modernize test codeÆvar Arnfjörð Bjarmason1-52/+40
Modernize test code added in ce567d1867a (Add test to show that show-branch misses out the 8th column, 2008-07-23) and 11ee57bc4c4 (sort_in_topological_order(): avoid setting a commit flag, 2008-07-23) to use test helpers. I'm renaming "out" to "actual" for consistency with other tests, and introducing a "branches.sorted" file in the setup, to make it clear that it's important that the list be sorted in this particular way. The "show-branch" output is indented with spaces, which would cause complaints under "git show --check" with an indented here-doc block. Let's prefix the lines with "> " to work around that, and to make it clear that the leading whitespace is important. We can also get rid of the hardcoding of "main" added here in 334afbc76fb (tests: mark tests relying on the current default for `init.defaultBranch`, 2020-11-18). For this test we're setting up an "initial" commit anyway, and now that we've moved over to test_commit we can reference that instead. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-15show-branch tests: rename the one "show-branch" test fileÆvar Arnfjörð Bjarmason1-1/+1
Rename the only *show-branch* test file to indicate that more tests belong it in than just the one-off octopus test it now contains. The test was initially added in ce567d1867a (Add test to show that show-branch misses out the 8th column, 2008-07-23) and 11ee57bc4c4 (sort_in_topological_order(): avoid setting a commit flag, 2008-07-23). Those two add almost the same content, one with a test_expect_success and the other a test_expect_failure (a bug being tested for was fixed on one of the branches). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-15builtin/checkout--worker: zero-initialise struct to avoid MSAN complaintsAndrzej Hunt1-1/+1
report_result() sends a struct to the parent process, but that struct would contain uninitialised padding bytes. Running this code under MSAN rightly triggers a warning - but we don't particularly care about this warning because we control the receiving code, and we therefore know that those padding bytes won't be read on the receiving end. We could simply suppress this warning under MSAN with the approporiate ifdef'd attributes, but a less intrusive solution is to 0-initialise the struct, which guarantees that the padding will also be initialised. Interestingly, in the error-case branch, we only try to copy the first two members of pc_item_result, by copying only PC_ITEM_RESULT_BASE_SIZE bytes. However PC_ITEM_RESULT_BASE_SIZE is defined as 'offsetof(the_last_member)', which means that we're copying padding bytes after the end of the second last member. We could avoid doing this by redefining PC_ITEM_RESULT_BASE_SIZE as 'offsetof(second_last_member) + sizeof(second_last_member)', but there's no huge benefit to doing so (and this patch silences the MSAN warning in this scenario either way). MSAN output from t2080 (partially interleaved due to the parallel work :) ): Uninitialized bytes in __interceptor_write at offset 12 inside [0x7fff37d83408, 160) ==23279==WARNING: MemorySanitizer: use-of-uninitialized-value Uninitialized bytes in __interceptor_write at offset 12 inside [0x7ffdb8a07ec8, 160) ==23280==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0xd5ac28 in xwrite /home/ahunt/git/git/wrapper.c:256:8 #1 0xd5b327 in write_in_full /home/ahunt/git/git/wrapper.c:311:21 #2 0xb0a8c4 in do_packet_write /home/ahunt/git/git/pkt-line.c:221:6 #3 0xb0a5fd in packet_write /home/ahunt/git/git/pkt-line.c:242:6 #4 0x4f7441 in report_result /home/ahunt/git/git/builtin/checkout--worker.c:69:2 #5 0x4f6be6 in worker_loop /home/ahunt/git/git/builtin/checkout--worker.c:100:3 #6 0x4f68d3 in cmd_checkout__worker /home/ahunt/git/git/builtin/checkout--worker.c:143:2 #7 0x4a1e76 in run_builtin /home/ahunt/git/git/git.c:461:11 #8 0x49e1e7 in handle_builtin /home/ahunt/git/git/git.c:714:3 #9 0x4a0c08 in run_argv /home/ahunt/git/git/git.c:781:4 #10 0x49d5a8 in cmd_main /home/ahunt/git/git/git.c:912:19 #11 0x7974da in main /home/ahunt/git/git/common-main.c:52:11 #12 0x7f8778114349 in __libc_start_main (/lib64/libc.so.6+0x24349) #13 0x421bd9 in _start /home/abuild/rpmbuild/BUILD/glibc-2.26/csu/../sysdeps/x86_64/start.S:120 Uninitialized value was created by an allocation of 'res' in the stack frame of function 'report_result' #0 0x4f72c0 in report_result /home/ahunt/git/git/builtin/checkout--worker.c:55 SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/ahunt/git/git/wrapper.c:256:8 in xwrite Exiting #0 0xd5ac28 in xwrite /home/ahunt/git/git/wrapper.c:256:8 #1 0xd5b327 in write_in_full /home/ahunt/git/git/wrapper.c:311:21 #2 0xb0a8c4 in do_packet_write /home/ahunt/git/git/pkt-line.c:221:6 #3 0xb0a5fd in packet_write /home/ahunt/git/git/pkt-line.c:242:6 #4 0x4f7441 in report_result /home/ahunt/git/git/builtin/checkout--worker.c:69:2 #5 0x4f6be6 in worker_loop /home/ahunt/git/git/builtin/checkout--worker.c:100:3 #6 0x4f68d3 in cmd_checkout__worker /home/ahunt/git/git/builtin/checkout--worker.c:143:2 #7 0x4a1e76 in run_builtin /home/ahunt/git/git/git.c:461:11 #8 0x49e1e7 in handle_builtin /home/ahunt/git/git/git.c:714:3 #9 0x4a0c08 in run_argv /home/ahunt/git/git/git.c:781:4 #10 0x49d5a8 in cmd_main /home/ahunt/git/git/git.c:912:19 #11 0x7974da in main /home/ahunt/git/git/common-main.c:52:11 #12 0x7f2749a0e349 in __libc_start_main (/lib64/libc.so.6+0x24349) #13 0x421bd9 in _start /home/abuild/rpmbuild/BUILD/glibc-2.26/csu/../sysdeps/x86_64/start.S:120 Uninitialized value was created by an allocation of 'res' in the stack frame of function 'report_result' #0 0x4f72c0 in report_result /home/ahunt/git/git/builtin/checkout--worker.c:55 SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/ahunt/git/git/wrapper.c:256:8 in xwrite Signed-off-by: Andrzej Hunt <andrzej@ahunt.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-15split-index: use oideq instead of memcmp to compare object_id'sAndrzej Hunt1-1/+2
cache_entry contains an object_id, and compare_ce_content() would include that field when calling memcmp on a subset of the cache_entry. Depending on which hashing algorithm is being used, only part of object_id.hash is actually being used, therefore including it in a memcmp() is incorrect. Instead we choose to exclude the object_id when calling memcmp(), and call oideq() separately. This issue was found when running t1700-split-index with MSAN, see MSAN output below (on my machine, offset 76 corresponds to 4 bytes after the start of object_id.hash). Uninitialized bytes in MemcmpInterceptorCommon at offset 76 inside [0x7f60e7c00118, 92) ==27914==WARNING: MemorySanitizer: use-of-uninitialized-value #0 0x4524ee in memcmp /home/abuild/rpmbuild/BUILD/llvm-11.0.0.src/build/../projects/compiler-rt/lib/msan/../sanitizer_common/sanitizer_common_interceptors.inc:873:10 #1 0xc867ae in compare_ce_content /home/ahunt/git/git/split-index.c:208:8 #2 0xc859fb in prepare_to_write_split_index /home/ahunt/git/git/split-index.c:336:9 #3 0xb4bbca in write_split_index /home/ahunt/git/git/read-cache.c:3107:2 #4 0xb42b4d in write_locked_index /home/ahunt/git/git/read-cache.c:3295:8 #5 0x638058 in try_merge_strategy /home/ahunt/git/git/builtin/merge.c:758:7 #6 0x63057f in cmd_merge /home/ahunt/git/git/builtin/merge.c:1663:9 #7 0x4a1e76 in run_builtin /home/ahunt/git/git/git.c:461:11 #8 0x49e1e7 in handle_builtin /home/ahunt/git/git/git.c:714:3 #9 0x4a0c08 in run_argv /home/ahunt/git/git/git.c:781:4 #10 0x49d5a8 in cmd_main /home/ahunt/git/git/git.c:912:19 #11 0x7974da in main /home/ahunt/git/git/common-main.c:52:11 #12 0x7f60e928e349 in __libc_start_main (/lib64/libc.so.6+0x24349) #13 0x421bd9 in _start /home/abuild/rpmbuild/BUILD/glibc-2.26/csu/../sysdeps/x86_64/start.S:120 Uninitialized value was stored to memory at #0 0x447eb9 in __msan_memcpy /home/abuild/rpmbuild/BUILD/llvm-11.0.0.src/build/../projects/compiler-rt/lib/msan/msan_interceptors.cpp:1558:3 #1 0xb4d1e6 in dup_cache_entry /home/ahunt/git/git/read-cache.c:3457:2 #2 0xd214fa in add_entry /home/ahunt/git/git/unpack-trees.c:215:18 #3 0xd1fae0 in keep_entry /home/ahunt/git/git/unpack-trees.c:2276:2 #4 0xd1ff9e in twoway_merge /home/ahunt/git/git/unpack-trees.c:2504:11 #5 0xd27028 in call_unpack_fn /home/ahunt/git/git/unpack-trees.c:593:12 #6 0xd2443d in unpack_nondirectories /home/ahunt/git/git/unpack-trees.c:1106:12 #7 0xd19435 in unpack_callback /home/ahunt/git/git/unpack-trees.c:1306:6 #8 0xd0d7ff in traverse_trees /home/ahunt/git/git/tree-walk.c:532:17 #9 0xd1773a in unpack_trees /home/ahunt/git/git/unpack-trees.c:1683:9 #10 0xdc6370 in checkout /home/ahunt/git/git/merge-ort.c:3590:8 #11 0xdc51c3 in merge_switch_to_result /home/ahunt/git/git/merge-ort.c:3728:7 #12 0xa195a9 in merge_ort_recursive /home/ahunt/git/git/merge-ort-wrappers.c:58:2 #13 0x637fff in try_merge_strategy /home/ahunt/git/git/builtin/merge.c:751:12 #14 0x63057f in cmd_merge /home/ahunt/git/git/builtin/merge.c:1663:9 #15 0x4a1e76 in run_builtin /home/ahunt/git/git/git.c:461:11 #16 0x49e1e7 in handle_builtin /home/ahunt/git/git/git.c:714:3 #17 0x4a0c08 in run_argv /home/ahunt/git/git/git.c:781:4 #18 0x49d5a8 in cmd_main /home/ahunt/git/git/git.c:912:19 #19 0x7974da in main /home/ahunt/git/git/common-main.c:52:11 Uninitialized value was created by a heap allocation #0 0x44e73d in malloc /home/abuild/rpmbuild/BUILD/llvm-11.0.0.src/build/../projects/compiler-rt/lib/msan/msan_interceptors.cpp:901:3 #1 0xd592f6 in do_xmalloc /home/ahunt/git/git/wrapper.c:41:8 #2 0xd59248 in xmalloc /home/ahunt/git/git/wrapper.c:62:9 #3 0xa17088 in mem_pool_alloc_block /home/ahunt/git/git/mem-pool.c:22:6 #4 0xa16f78 in mem_pool_init /home/ahunt/git/git/mem-pool.c:44:3 #5 0xb481b8 in load_all_cache_entries /home/ahunt/git/git/read-cache.c #6 0xb44d40 in do_read_index /home/ahunt/git/git/read-cache.c:2298:17 #7 0xb48a1b in read_index_from /home/ahunt/git/git/read-cache.c:2389:8 #8 0xbd5a0b in repo_read_index /home/ahunt/git/git/repository.c:276:8 #9 0xb4bcaf in repo_read_index_unmerged /home/ahunt/git/git/read-cache.c:3326:2 #10 0x62ed26 in cmd_merge /home/ahunt/git/git/builtin/merge.c:1362:6 #11 0x4a1e76 in run_builtin /home/ahunt/git/git/git.c:461:11 #12 0x49e1e7 in handle_builtin /home/ahunt/git/git/git.c:714:3 #13 0x4a0c08 in run_argv /home/ahunt/git/git/git.c:781:4 #14 0x49d5a8 in cmd_main /home/ahunt/git/git/git.c:912:19 #15 0x7974da in main /home/ahunt/git/git/common-main.c:52:11 #16 0x7f60e928e349 in __libc_start_main (/lib64/libc.so.6+0x24349) SUMMARY: MemorySanitizer: use-of-uninitialized-value /home/abuild/rpmbuild/BUILD/llvm-11.0.0.src/build/../projects/compiler-rt/lib/msan/../sanitizer_common/sanitizer_common_interceptors.inc:873:10 in memcmp Exiting Signed-off-by: Andrzej Hunt <andrzej@ahunt.org> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-15mktag tests: invert --no-strict testÆvar Arnfjörð Bjarmason1-0/+2
Change the mktag --no-strict test to actually test success under --no-strict, that test was added in 06ce79152be (mktag: add a --[no-]strict option, 2021-01-06). It doesn't make sense to check that we have the same failure except when we want --no-strict, by doing that we're assuming that the behavior will be different under --no-strict, bun nothing was testing for that. We should instead assert that --strict is the same as --no-strict, except in the cases where we've declared that it's not. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-15mktag tests: parse out options in helperÆvar Arnfjörð Bjarmason1-12/+31
Change check_verify_failure() helper to parse out options from $@. This makes it easier to add new options in the future. See 06ce79152be (mktag: add a --[no-]strict option, 2021-01-06) for the initial implementation. Let's also replace "" quotes with '' for the test body, the varables we need are eval'd into the body, so there's no need for the quoting confusion. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-15subtree: fix assumption about the directory separatorJohannes Schindelin1-3/+4
On Windows, both forward and backslash are valid separators. In 22d550749361 (subtree: don't fuss with PATH, 2021-04-27), however, we added code that assumes that it can only be the forward slash. Let's fix that. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-15subtree: fix the GIT_EXEC_PATH sanity check to work on WindowsJohannes Schindelin1-1/+4
In 22d550749361 (subtree: don't fuss with PATH, 2021-04-27), `git subtree` was broken thoroughly on Windows. The reason is that it assumes Unix semantics, where `PATH` is colon-separated, and it assumes that `$GIT_EXEC_PATH:` is a verbatim prefix of `$PATH`. Neither are true, the latter in particular because `GIT_EXEC_PATH` is a Windows-style path, while `PATH` is a Unix-style path list. Let's make extra certain that `$GIT_EXEC_PATH` and the first component of `$PATH` refer to different entities before erroring out. We do that by using the `test <path1> -ef <path2>` command that verifies that the inode of `<path1>` and of `<path2>` is the same. Sadly, this construct is non-portable, according to https://pubs.opengroup.org/onlinepubs/009695399/utilities/test.html. However, it does not matter in practice because we still first look whether `$GIT_EXEC_PREFIX` is string-identical to the first component of `$PATH`. This will give us the expected result everywhere but in Git for Windows, and Git for Windows' own Bash _does_ handle the `-ef` operator. Just in case that we _do_ need to show the error message _and_ are running in a shell that lacks support for `-ef`, we simply suppress the error output for that part. This fixes https://github.com/git-for-windows/git/issues/3260 Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-15bitmaps: don't recurse into trees already in the bitmapJeff King3-0/+22
If an object is already mentioned in a reachability bitmap we are building, then by definition so are all of the objects it can reach. We have an optimization to stop traversing commits when we see they are already in the bitmap, but we don't do the same for trees. It's generally unavoidable to recurse into trees for commits not yet covered by bitmaps (since most commits generally do have unique top-level trees). But they usually have subtrees that are shared with other commits (i.e., all of the subtrees the commit _didn't_ touch). And some of those commits (and their trees) may be covered by the bitmap. Usually this isn't _too_ big a deal, because we'll visit those subtrees only once in total for the whole walk. But if you have a large number of unbitmapped commits, and if your tree is big, then you may end up opening a lot of sub-trees for no good reason. We can use the same optimization we do for commits here: when we are about to open a tree, see if it's in the bitmap (either the one we are building, or the "seen" bitmap which covers the UNINTERESTING side of the bitmap when doing a set-difference). This works especially well because we'll visit all commits before hitting any trees. So even in a history like: A -- B if "A" has a bitmap on disk but "B" doesn't, we'll already have OR-ed in the results from A before looking at B's tree (so we really will only look at trees touched by B). For most repositories, the timings produced by p5310 are unspectacular. Here's linux.git: Test HEAD^ HEAD -------------------------------------------------------------------- 5310.4: simulated clone 6.00(5.90+0.10) 5.98(5.90+0.08) -0.3% 5310.5: simulated fetch 2.98(5.45+0.18) 2.85(5.31+0.18) -4.4% 5310.7: rev-list (commits) 0.32(0.29+0.03) 0.33(0.30+0.03) +3.1% 5310.8: rev-list (objects) 1.48(1.44+0.03) 1.49(1.44+0.05) +0.7% Any improvement there is within the noise (the +3.1% on test 7 has to be noise, since we are not recursing into trees, and thus the new code isn't even run). The results for git.git are likewise uninteresting. But here are numbers from some other real-world repositories (that are not public). This one's tree is comparable in size to linux.git, but has ~16k refs (and so less complete bitmap coverage): Test HEAD^ HEAD ------------------------------------------------------------------------- 5310.4: simulated clone 38.34(39.86+0.74) 33.95(35.53+0.76) -11.5% 5310.5: simulated fetch 2.29(6.31+0.35) 2.20(5.97+0.41) -3.9% 5310.7: rev-list (commits) 0.99(0.86+0.13) 0.96(0.85+0.11) -3.0% 5310.8: rev-list (objects) 11.32(11.04+0.27) 6.59(6.37+0.21) -41.8% And here's another with a very large tree (~340k entries), and a fairly large number of refs (~10k): Test HEAD^ HEAD ------------------------------------------------------------------------- 5310.3: simulated clone 53.83(54.71+1.54) 39.77(40.76+1.50) -26.1% 5310.4: simulated fetch 19.91(20.11+0.56) 19.79(19.98+0.67) -0.6% 5310.6: rev-list (commits) 0.54(0.44+0.11) 0.51(0.43+0.07) -5.6% 5310.7: rev-list (objects) 24.32(23.59+0.73) 9.85(9.49+0.36) -59.5% This patch provides substantial improvements in these larger cases, and have any drawbacks for smaller ones (the cost of the bitmap check is quite small compared to an actual tree traversal). Note that we have to add a version of revision.c's include_check callback which handles non-commits. We could possibly consolidate this into a single callback for all objects types, as there's only one user of the feature which would need converted (pack-bitmap.c:should_include). That would in theory let us avoid duplicating any logic. But when I tried it, the code ended up much worse to read, with lots of repeated "if it's a commit do this, otherwise do that". Having two separate callbacks splits that naturally, and matches the existing split of show_commit/show_object callbacks. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-14The second batchJunio C Hamano1-2/+48
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-06-14Merge branch 'fc/doc-build-cleanup'Junio C Hamano1-48/+29
Preparatory build procedure clean-up for documentation. * fc/doc-build-cleanup: doc: avoid using rm directly doc: simplify Makefile using .DELETE_ON_ERROR doc: remove unnecessary rm instances doc: improve asciidoc dependencies doc: refactor common asciidoc dependencies
2021-06-14Merge branch 'ab/test-lib-updates'Junio C Hamano14-119/+89
Test clean-up. * ab/test-lib-updates: test-lib: split up and deprecate test_create_repo() test-lib: do not show advice about init.defaultBranch under --verbose test-lib: reformat argument list in test_create_repo() submodule tests: use symbolic-ref --short to discover branch name test-lib functions: add --printf option to test_commit describe tests: convert setup to use test_commit test-lib functions: add an --annotated option to "test_commit" test-lib-functions: document test_commit --no-tag test-lib-functions: reword "test_commit --append" docs test-lib tests: remove dead GIT_TEST_FRAMEWORK_SELFTEST variable test-lib: bring $remove_trash out of retirement
2021-06-14Merge branch 'dd/honor-users-tar-in-tests'Junio C Hamano2-4/+4
Test portability fix. * dd/honor-users-tar-in-tests: t: use configured TAR instead of tar
2021-06-14Merge branch 'ps/rev-list-object-type-filter'Junio C Hamano1-1/+1
Message update. * ps/rev-list-object-type-filter: help: fix small typo in error message
2021-06-14Merge branch 'ab/trace2-squelch-gcc-warning'Junio C Hamano1-10/+8
Workaround compiler warnings. * ab/trace2-squelch-gcc-warning: trace2: refactor to avoid gcc warning under -O3
2021-06-14Merge branch 'so/log-m-implies-p'Junio C Hamano12-25/+200
The "-m" option in "git log -m" that does not specify which format, if any, of diff is desired did not have any visible effect; it now implies some form of diff (by default "--patch") is produced. * so/log-m-implies-p: diff-merges: let "-m" imply "-p" diff-merges: rename "combined_imply_patch" to "merges_imply_patch" stash list: stop passing "-m" to "git log" git-svn: stop passing "-m" to "git rev-list" diff-merges: move specific diff-index "-m" handling to diff-index t4013: test "git diff-index -m" t4013: test "git diff-tree -m" t4013: test "git log -m --stat" t4013: test "git log -m --raw" t4013: test that "-m" alone has no effect in "git log"
2021-06-14Merge branch 'en/ort-perf-batch-11'Junio C Hamano8-37/+1804
Optimize out repeated rename detection in a sequence of mergy operations. * en/ort-perf-batch-11: merge-ort, diffcore-rename: employ cached renames when possible merge-ort: handle interactions of caching and rename/rename(1to1) cases merge-ort: add helper functions for using cached renames merge-ort: preserve cached renames for the appropriate side merge-ort: avoid accidental API mis-use merge-ort: add code to check for whether cached renames can be reused merge-ort: populate caches of rename detection results merge-ort: add data structures for in-memory caching of rename detection t6429: testcases for remembering renames fast-rebase: write conflict state to working tree, index, and HEAD fast-rebase: change assert() to BUG() Documentation/technical: describe remembering renames optimization t6423: rename file within directory that other side renamed