summaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2021-09-30difftool: refactor dir-diff to write files using helper functionsDavid Aguilar1-22/+28
Add a helpers function to handle the unlinking and writing of the dir-diff submodule and symlink stand-in files. Use the helpers to implement the guts of the hashmap loops. This eliminate duplicate code and safeguards the submodules hashmap loop against the symlink-chasing behavior that 5bafb3576a (difftool: fix symlink-file writing in dir-diff mode, 2021-09-22) addressed. The submodules loop should not strictly require the unlink() call that this is introducing to them, but it does not necessarily hurt them either beyond the cost of the extra unlink(). Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-30difftool: create a tmpdir path without repeated slashesDavid Aguilar2-26/+31
The paths generated by difftool are passed to user-facing diff tools. Using paths with repeated slashes in them is a cosmetic blemish that is exposed to users and can be avoided. Use a strbuf to create the buffer used for the dir-diff tmpdir. Strip trailing slashes from the value read from TMPDIR to avoid repeated slashes in the generated paths. Adjust the error handling to avoid leaking strbufs and to avoid returning -1 to cmd_main(). Signed-off-by: David Aguilar <davvid@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-29grep: refactor next_match() and match_one_pattern() for external useHamza Mahfooz2-30/+58
These changes are made in preparation of, the colorization support for the "git log" subcommands that, rely on regex functionality (i.e. "--author", "--committer" and "--grep"). These changes are necessary primarily because match_one_pattern() expects header lines to be prefixed, however, in pretty, the prefixes are stripped from the lines because the name-email pairs need to go through additional parsing, before they can be printed and because next_match() doesn't handle the case of "ctx == GREP_CONTEXT_HEAD" at all. So, teach next_match() how to handle the new case and move match_one_pattern()'s core logic to headerless_match_one_pattern() while preserving match_one_pattern()'s uses that depend on the additional processing. Signed-off-by: Hamza Mahfooz <someguy@effective-light.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-29grep: demonstrate bug with textconv attributes and submodulesMatheus Tavares2-3/+106
In some circumstances, "git grep --textconv --recurse-submodules" ignores the textconv attributes from the submodules and erroneously applies the attributes defined in the superproject on the submodules' files. The textconv cache is also saved on the superproject, even for submodule objects. A fix for these problems will probably require at least three changes: - Some textconv and attributes functions (as well as their callees) will have to be adjusted to work with arbitrary repositories. Note that "fill_textconv()", for example, already receives a "struct repository" but it writes the textconv cache using "write_loose_object()", which implicitly works on "the_repository". - grep.c functions will have to call textconv/userdiff routines passing the "repo" field from "struct grep_source" instead of the one from "struct grep_opt". The latter always points to "the_repository" on "git grep" executions (see its initialization in builtin/grep.c), but the former points to the correct repository that each source (an object, file, or buffer) comes from. - "userdiff_find_by_path()" might need to use a different attributes stack for each repository it works on or reset its internal static stack when the repository is changed throughout the calls. For now, let's add some tests to demonstrate these problems, and also update a NEEDSWORK comment in grep.h that mentions this bug to reference the added tests. Signed-off-by: Matheus Tavares <matheus.bernardino@usp.br> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28builtin/repack.c: make largest pack preferredTaylor Blau6-3/+78
When repacking into a geometric series and writing a multi-pack bitmap, it is beneficial to have the largest resulting pack be the preferred object source in the bitmap's MIDX, since selecting the large packs can lead to fewer broken delta chains and better compression. Teach 'git repack' to identify this pack and pass it to the MIDX write machinery in order to mark it as preferred. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28builtin/repack.c: support writing a MIDX while repackingTaylor Blau5-24/+234
Teach `git repack` a new `--write-midx` option for callers that wish to persist a multi-pack index in their repository while repacking. There are two existing alternatives to this new flag, but they don't cover our particular use-case. These alternatives are: - Call 'git multi-pack-index write' after running 'git repack', or - Set 'GIT_TEST_MULTI_PACK_INDEX=1' in your environment when running 'git repack'. The former works, but introduces a gap in bitmap coverage between repacking and writing a new MIDX (since the repack may have deleted a pack included in the existing MIDX, invalidating it altogether). Setting the 'GIT_TEST_' environment variable is obviously unsupported. In fact, even if it were supported officially, it still wouldn't work, because it generates the MIDX *after* redundant packs have been dropped, leading to the same issue as above. Introduce a new option which eliminates this race by teaching `git repack` to generate the MIDX at the critical point: after the new packs have been written and moved into place, but before the redundant packs have been removed. This option is compatible with `git repack`'s '--bitmap' option (it changes the interpretation to be: "write a bitmap corresponding to the MIDX after one has been generated"). There is a little bit of additional noise in the patch below to avoid repeating ourselves when selecting which packs to delete. Instead of a single loop as before (where we iterate over 'existing_packs', decide if a pack is worth deleting, and if so, delete it), we have two loops (the first where we decide which ones are worth deleting, and the second where we actually do the deleting). This makes it so we have a single check we can make consistently when (1) telling the MIDX which packs we want to exclude, and (2) actually unlinking the redundant packs. There is also a tiny change to short-circuit the body of write_midx_included_packs() when no packs remain in the case of an empty repository. The MIDX code does not handle this, so avoid trying to generate a MIDX covering zero packs in the first place. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28builtin/repack.c: extract showing progress to a variableTaylor Blau1-1/+2
We only ask whether stderr is a tty before calling 'prune_packed_objects()', but the subsequent patch will add another use. Extract this check into a variable so that both can use it without having to call 'isatty()' twice. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28builtin/repack.c: rename variables that deal with non-kept packsTaylor Blau1-11/+11
The new variable `existing_kept_packs` (and corresponding parameter `fname_kept_list`) added by the previous patch make it seem like `existing_packs` and `fname_list` are each subsets of the other two respectively. In reality, each pair is disjoint: one stores the packs without .keep files, and the other stores the packs with .keep files. Rename each to more clearly reflect this. Suggested-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28builtin/repack.c: keep track of existing packs unconditionallyTaylor Blau1-25/+31
In order to be able to write a multi-pack index during repacking, `git repack` must keep track of which packs it wants to write into the MIDX. This set is the union of existing packs which will not be deleted, new pack(s) generated as a result of the repack, and .keep packs. Prior to this patch, `git repack` populated the list of existing packs only when repacking all-into-one (i.e., with `-A` or `-a`), but we will soon need to know this list when repacking when writing a MIDX without a-i-o. Populate the list of existing packs unconditionally, and guard removing packs from that list only when repacking a-i-o. Additionally, keep track of filenames of kept packs separately, since this, too, will be used in an upcoming patch. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28midx: preliminary support for `--refs-snapshot`Taylor Blau6-13/+164
To figure out which commits we can write a bitmap for, the multi-pack index/bitmap code does a reachability traversal, marking any commit which can be found in the MIDX as eligible to receive a bitmap. This approach will cause a problem when multi-pack bitmaps are able to be generated from `git repack`, since the reference tips can change during the repack. Even though we ignore commits that don't exist in the MIDX (when doing a scan of the ref tips), it's possible that a commit in the MIDX reaches something that isn't. This can happen when a multi-pack index contains some pack which refers to loose objects (e.g., if a pack was pushed after starting the repack but before generating the MIDX which depends on an object which is stored as loose in the repository, and by definition isn't included in the multi-pack index). By taking a snapshot of the references before we start repacking, we can close that race window. In the above scenario (where we have a packed object pointing at a loose one), we'll either (a) take a snapshot of the references before seeing the packed one, or (b) take it after, at which point we can guarantee that the loose object will be packed and included in the MIDX. This patch does just that. It writes a temporary "reference snapshot", which is a list of OIDs that are at the ref tips before writing a multi-pack bitmap. References that are "preferred" (i.e,. are a suffix of at least one value of the 'pack.preferBitmapTips' configuration) are marked with a special '+'. The format is simple: one line per commit at each tip, with an optional '+' at the beginning (for preferred references, as described above). When provided, the reference snapshot is used to drive bitmap selection instead of the MIDX code doing its own traversal. When it isn't provided, the usual traversal takes place instead. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28builtin/multi-pack-index.c: support `--stdin-packs` modeTaylor Blau3-0/+46
To power a new `--write-midx` mode, `git repack` will want to write a multi-pack index containing a certain set of packs in the repository. This new option will be used by `git repack` to write a MIDX which contains only the packs which will survive after the repack (that is, it will exclude any packs which are about to be deleted). This patch effectively exposes the function implemented in the previous commit via the `git multi-pack-index` builtin. An alternative approach would have been to call that function from the `git repack` builtin directly, but this introduces awkward problems around closing and reopening the object store, so the MIDX will be written out-of-process. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28midx: expose `write_midx_file_only()` publiclyTaylor Blau2-8/+58
Expose a variant of the write_midx_file() function which ignores packs that aren't included in an explicit "allow" list. This will be used in an upcoming patch to power a new `--stdin-packs` mode of `git multi-pack-index write` for callers that only want to include certain packs in a MIDX (and ignore any packs which may have happened to enter the repository independently, e.g., from pushes). Those patches will provide test coverage for this new function. Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28Makefile: restrict -Wpedantic and -Wno-pedantic-ms-format betterCarlo Marcelo Arenas Belón2-2/+11
6a8cbc41ba (developer: enable pedantic by default, 2021-09-03) enables pedantic mode in as many compilers as possible to help gather feedback on future tightening, so lets do so. -Wpedantic is missing in some really old gcc 4 versions so lets restrict it to gcc5 and clang4 (it does work in clang3 AFAIK, but it will be unlikely that a developer will use such an old compiler anyway). MinGW gcc is the only one which has -Wno-pedantic-ms-format, and while that is available also in older compilers, the Windows SDK provides gcc10 so lets aim for that. Note that in order to target the flag to only Windows, additional changes were needed in config.mak.uname to propagate the OS detection which also did some minor refactoring, but which is functionaly equivalent. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Carlo Marcelo Arenas Belón <carenas@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28parse-options.h: move PARSE_OPT_SHELL_EVAL between enumsÆvar Arnfjörð Bjarmason1-1/+1
Fix a bad landmine of a bug which has been with us ever since PARSE_OPT_SHELL_EVAL was added in 47e9cd28f8a (parseopt: wrap rev-parse --parseopt usage for eval consumption, 2010-06-12). It's an argument to parse_options() and should therefore be in "enum parse_opt_flags", but it was added to the per-option "enum parse_opt_option_flags" by mistake. Therefore as soon as we'd have an enum member in the former that reached its value of "1 << 8" we'd run into a seemingly bizarre bug where that new option would turn on the unrelated PARSE_OPT_SHELL_EVAL in "git rev-parse --parseopt" by proxy. I manually checked that no other enum members suffered from such overlap, by setting the values to non-overlapping values, and making the relevant codepaths BUG() out if the given value was above/below the expected (excluding flags=0 in the case of "enum parse_opt_flags"). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28doc: fix capitalization in "git status --porcelain=v2" descriptionOrgad Shaneh1-1/+1
The summary line had xy, while the description (and other sub-sections) has XY. Signed-off-by: Orgad Shaneh <orgads@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28Merge branch 'jk/ref-paranoia' into jt/no-abuse-alternate-odb-for-submodulesJunio C Hamano92-472/+964
* jk/ref-paranoia: (71 commits) refs: drop "broken" flag from for_each_fullref_in() ref-filter: drop broken-ref code entirely ref-filter: stop setting FILTER_REFS_INCLUDE_BROKEN repack, prune: drop GIT_REF_PARANOIA settings refs: turn on GIT_REF_PARANOIA by default refs: omit dangling symrefs when using GIT_REF_PARANOIA refs: add DO_FOR_EACH_OMIT_DANGLING_SYMREFS flag refs-internal.h: reorganize DO_FOR_EACH_* flag documentation refs-internal.h: move DO_FOR_EACH_* flags next to each other t5312: be more assertive about command failure t5312: test non-destructive repack t5312: create bogus ref as necessary t5312: drop "verbose" helper t5600: provide detached HEAD for corruption failures t5516: don't use HEAD ref for invalid ref-deletion tests t7900: clean up some more broken refs The eighth batch t0000: avoid masking git exit value through pipes tree-diff: fix leak when not HAVE_ALLOCA_H pack-revindex.h: correct the time complexity descriptions ...
2021-09-28refs/ref-cache.[ch]: remove "incomplete" from create_dir_entry()Ævar Arnfjörð Bjarmason3-9/+7
Remove the now-unused "incomplete" parameter from create_dir_entry(), all its callers specify it as "1", so let's drop the "incomplete=0" case. The last caller to use it was search_for_subdir(), but that code was removed in the preceding commit. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28refs/ref-cache.c: remove "mkdir" parameter from find_containing_dir()Ævar Arnfjörð Bjarmason1-24/+12
Remove the "mkdir" parameter from the find_containing_dir() function, the add_ref_entry() function removed in the preceding commit was its last user. Since "mkdir" is always "0" we can also remove the parameter from search_for_subdir(), which in turn means that we can delete most of that function. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28refs/ref-cache.[ch]: remove unused add_ref_entry()Ævar Arnfjörð Bjarmason2-16/+0
This function has not been used since 9dd389f3d8d (packed_ref_store: get rid of the `ref_cache` entirely, 2017-09-25). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28refs/ref-cache.[ch]: remove unused remove_entry_from_dir()Ævar Arnfjörð Bjarmason2-51/+0
This function was missed in 9939b33d6a3 (packed-backend: rip out some now-unused code, 2017-09-08), and has been orphaned since then. Let's delete it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28refs.[ch]: remove unused ref_storage_backend_exists()Ævar Arnfjörð Bjarmason2-7/+0
This function was added in 3dce444f178 (refs: add a backend method structure, 2016-09-04), but has never been used by anything. The only caller that might care uses find_ref_storage_backend() directly. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28config.c: remove unused git_config_key_is_valid()Ævar Arnfjörð Bjarmason2-27/+8
The git_config_key_is_valid() function got left behind in a refactoring in a9bcf6586d1 (alias: use the early config machinery to expand aliases, 2017-06-14), It previously had two users when it was added in 9e9de18f1ad (config: silence warnings for command names with invalid keys, 2015-08-24), and after 6a1e1bc0a15 (pager: use callbacks instead of configset, 2016-09-12) only one remained. By removing it we can get rid of the "quiet" branches in this function, as well as cases where "store_key" is NULL, for which there are no other users. Out of the 5 callers of git_config_parse_key() only one needs to pass a non-NULL "size_t *baselen_", so we could remove the third parameter from the public interface. I did not find that potential simplification to be worthwhile. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28string-list.[ch]: remove string_list_init() compatibility functionÆvar Arnfjörð Bjarmason3-15/+2
Remove this function left over to accommodate in-flight changes, see 770fedaf9fb (string-list.[ch]: add a string_list_init_{nodup,dup}(), 2021-07-01) for the recent change to add "string_list_init_{nodup,dup}()" initializers. There was only one user of the API left in remote-curl.c. I don't know why I didn't include this change to remote-curl.c in bc40dfb10a0 (string-list.h users: change to use *_{nodup,dup}(), 2021-07-01), perhaps I just missed it. In any case, let's change that one user to use the new API, as of writing this there are no in-flight changes that use, so this seems like a good time to drop this before we get any new users of this compatibility API. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28The ninth batchJunio C Hamano1-0/+5
Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28Merge branch 'en/typofixes'Junio C Hamano2-2/+2
Typofixes. * en/typofixes: merge-ort: fix completely wrong comment trace2.h: fix trivial comment typo
2021-09-28Merge branch 'cb/unicode-14'Junio C Hamano1-15/+29
The unicode character width table (used for output alignment) has been updated. * cb/unicode-14: unicode: update the width tables to Unicode 14
2021-09-28Merge branch 'jk/reduce-malloc-in-v2-servers'Junio C Hamano4-59/+164
Code cleanup to limit memory consumption and tighten protocol message parsing. * jk/reduce-malloc-in-v2-servers: ls-refs: reject unknown arguments serve: reject commands used as capabilities serve: reject bogus v2 "command=ls-refs=foo" docs/protocol-v2: clarify some ls-refs ref-prefix details ls-refs: ignore very long ref-prefix counts serve: drop "keys" strvec serve: provide "receive" function for session-id capability serve: provide "receive" function for object-format capability serve: add "receive" method for v2 capabilities table serve: return capability "value" from get_capability() serve: rename is_command() to parse_command()
2021-09-28advice: update message to suggest '--sparse'Derrick Stolee4-14/+24
The previous changes modified the behavior of 'git add', 'git rm', and 'git mv' to not adjust paths outside the sparse-checkout cone, even if they exist in the working tree and their cache entries lack the SKIP_WORKTREE bit. The intention is to warn users that they are doing something potentially dangerous. The '--sparse' option was added to each command to allow careful users the same ability they had before. To improve the discoverability of this new functionality, add a message to advice.updateSparsePath that mentions the existence of the option. The previous set of changes also modified the purpose of this message to include possibly a list of paths instead of only a list of pathspecs. Make the warning message more clear about this new behavior. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28mv: refuse to move sparse pathsDerrick Stolee2-9/+229
Since cmd_mv() does not operate on cache entries and instead directly checks the filesystem, we can only use path_in_sparse_checkout() as a mechanism for seeing if a path is sparse or not. Be sure to skip returning a failure if '-k' is specified. To ensure that the advice around sparse paths is the only reason a move failed, be sure to check this as the very last thing before inserting into the src_for_dst list. The tests cover a variety of cases such as whether the target is tracked or untracked, and whether the source or destination are in or outside of the sparse-checkout definition. Helped-by: Matheus Tavares Bernardino <matheus.bernardino@usp.br> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28rm: skip sparse paths with missing SKIP_WORKTREEDerrick Stolee2-3/+20
If a path does not match the sparse-checkout cone but is somehow missing the SKIP_WORKTREE bit, then 'git rm' currently succeeds in removing the file. One reason a user might be in this situation is a merge conflict outside of the sparse-checkout cone. Removing such a file might be problematic for users who are not sure what they are doing. Add a check to path_in_sparse_checkout() when 'git rm' is checking if a path should be considered for deletion. Of course, this check is ignored if the '--sparse' option is specified, allowing users who accept the risks to continue with the removal. This also removes a confusing behavior where a user asks for a directory to be removed, but only the entries that are within the sparse-checkout definition are removed. Now, 'git rm <dir>' will fail without '--sparse' and will succeed in removing all contained paths with '--sparse'. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28rm: add --sparse optionDerrick Stolee3-2/+24
As we did previously in 'git add', add a '--sparse' option to 'git rm' that allows modifying paths outside of the sparse-checkout definition. The existing checks in 'git rm' are restricted to tracked files that have the SKIP_WORKTREE bit in the current index. Future changes will cause 'git rm' to reject removing paths outside of the sparse-checkout definition, even if they are untracked or do not have the SKIP_WORKTREE bit. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28add: update --renormalize to skip sparse pathsDerrick Stolee2-2/+14
We added checks for path_in_sparse_checkout() to portions of 'git add' that add warnings and prevent stagins a modification, but we skipped the --renormalize mode. Update renormalize_tracked_files() to ignore cache entries whose path is outside of the sparse-checkout cone (unless --sparse is provided). Add a test in t3705. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28add: update --chmod to skip sparse pathsDerrick Stolee2-2/+12
We added checks for path_in_sparse_checkout() to portions of 'git add' that add warnings and prevent staging a modification, but we skipped the --chmod mode. Update chmod_pathspec() to ignore cache entries whose path is outside of the sparse-checkout cone (unless --sparse is provided). Add a test in t3705. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28add: implement the --sparse optionDerrick Stolee4-24/+43
We previously modified 'git add' to refuse updating index entries outside of the sparse-checkout cone. This is justified to prevent users from accidentally getting into a confusing state when Git removes those files from the working tree at some later point. Unfortunately, this caused some workflows that were previously possible to become impossible, especially around merge conflicts outside of the sparse-checkout cone. These were documented in tests within t1092. We now re-enable these workflows using a new '--sparse' option to 'git add'. This allows users to signal "Yes, I do know what I'm doing with these files," and accept the consequences of the files leaving the worktree later. We delay updating the advice message until implementing a similar option in 'git rm' and 'git mv'. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28add: skip tracked paths outside sparse-checkout coneDerrick Stolee5-10/+34
When 'git add' adds a tracked file that is outside of the sparse-checkout cone, it checks the SKIP_WORKTREE bit to see if the file exists outside of the sparse-checkout cone. This is usually correct, except in the case of a merge conflict outside of the cone. Modify add_pathspec_matched_against_index() to be more careful about paths by checking the sparse-checkout patterns in addition to the SKIP_WORKTREE bit. This causes 'git add' to no longer allow files outside of the cone that removed the SKIP_WORKTREE bit due to a merge conflict. With only this change, users will only be able to add the file after adding the file to the sparse-checkout cone. A later change will allow users to force adding even though the file is outside of the sparse-checkout cone. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28add: fail when adding an untracked sparse fileDerrick Stolee2-9/+42
The add_files() method in builtin/add.c takes a set of untracked files that are being added by the input pathspec and inserts them into the index. If these files are outside of the sparse-checkout cone, then they gain the SKIP_WORKTREE bit at some point. However, this was not checked before inserting into the index, so these files are added even though we want to avoid modifying the index outside of the sparse-checkout cone. Add a check within add_files() for these files and write the advice about files outside of the sparse-checkout cone. This behavior change modifies some existing tests within t1092. These tests intended to document how a user could interact with the existing behavior in place. Many of these tests need to be marked as expecting failure. A future change will allow these tests to pass by adding a flag to 'git add' that allows users to modify index entries outside of the sparse-checkout cone. The 'submodule handling' test is intended to document what happens to directories that contain a submodule when the sparse index is enabled. It is not trying to say that users should be able to add submodules outside of the sparse-checkout cone, so that test can be modified to avoid that operation. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28dir: fix pattern matching on dirsDerrick Stolee1-1/+1
Within match_pathname(), one successful matching category happens when the pattern is equal to its non-wildcard prefix. At this point, we have checked that the input 'pathname' matches the pattern up to the prefix length, and then we subtraced that length from both 'patternlen' and 'namelen'. In the case of a directory match, this prefix match should be sufficient. However, the success condition only cared about _exact_ equality here. Instead, we should allow any path that agrees on this prefix in the case of PATTERN_FLAG_MUSTBEDIR. This case was not tested before because of the way unpack_trees() would match a parent directory before visiting the contained paths. This approach is changing, so we must change this comparison. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28dir: select directories correctlyDerrick Stolee1-5/+49
When matching a path against a list of patterns, the ones that require a directory match previously did not work when a filename is specified. This was fine when all pattern-matching was done within methods such as unpack_trees() that check a directory before recursing into the contained files. However, other commands will start matching individual files against pattern lists without that recursive approach. The last_matching_pattern_from_list() logic performs some checks on the filetype of a path within the index when the PATTERN_FLAG_MUSTBEDIR flag is set. This works great when setting SKIP_WORKTREE bits within unpack_trees(), but doesn't work well when passing an arbitrary path such as a file within a matching directory. We extract the logic around determining the file type, but attempt to avoid checking the filesystem if the parent directory already matches the sparse-checkout patterns. The new path_matches_dir_pattern() method includes a 'path_parent' parameter that is used to store the parent directory of 'pathname' between multiple pattern matching tests. This is loaded lazily, only on the first pattern it finds that has the PATTERN_FLAG_MUSTBEDIR flag. If we find that a path has a parent directory, we start by checking to see if that parent directory matches the pattern. If so, then we do not need to query the index for the type (which can be expensive). If we find that the parent does not match, then we still must check the type from the index for the given pathname. Note that this does not affect cone mode pattern matching, but instead the more general -- and slower -- full pattern set. Thus, this does not affect the sparse index. Helped-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-28t1092: behavior for adding sparse filesDerrick Stolee1-0/+28
Add some tests to demonstrate the current behavior around adding files outside of the sparse-checkout cone. Currently, untracked files are handled differently from tracked files. A future change will make these cases be handled the same way. Further expand checking that a failed 'git add' does not stage changes to the index. Signed-off-by: Derrick Stolee <dstolee@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27maintenance: fix test t7900-maintenance.shLénaïc Huard1-2/+1
Commit b681b191 introduced the support of systemd timers for git maintenance. A test is leveraging the `systemd-analyze verify` utility to verify the correctness of the systemd unit files generated by git. But on some systems, although the `systemd-analyze` tool is installed and supports the `verify` subcommand, it fails with some permission errors. So, instead of only checking if the `verify` subcommand exists, a more reliable way of detecting whether `systemd-analyze verify` can be used is to try to use it. The SYSTEMD_ANALYZE prerequisite is now trying to run `systemd-analyze verify` on a systemd unit file which is shipped by systemd itself. We can reasonably think that, on systemd hosts, this file is present and valid. Signed-off-by: Lénaïc Huard <lenaic@lhuard.fr> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27builtin/blame.c: refactor commit_info_init() to COMMIT_INFO_INIT macroÆvar Arnfjörð Bjarmason1-17/+13
Remove the commit_info_init() function addded in ea02ffa3857 (mailmap: simplify map_user() interface, 2013-01-05) and instead initialize the "struct commit_info" with a macro. This is the more idiomatic pattern in the codebase, and doesn't leave us wondering when we see the *_init() function if this struct needs more complex initialization than a macro can provide. The get_commit_info() function is only called by the three callers being changed here immediately after initializing the struct with the macros, so by moving the initialization to the callers we don't need to do it in get_commit_info() anymore. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27daemon.c: refactor hostinfo_init() to HOSTINFO_INIT macroÆvar Arnfjörð Bjarmason1-12/+7
Remove the hostinfo_init() function added in 01cec54e135 (daemon: deglobalize hostname information, 2015-03-07) and instead initialize the "struct hostinfo" with a macro. This is the more idiomatic pattern in the codebase, and doesn't leave us wondering when we see the *_init() function if this struct needs more complex initialization than a macro can provide. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27Merge branch 'ab/designated-initializers' into ab/designated-initializers-moreJunio C Hamano29-54/+68
* ab/designated-initializers: cbtree.h: define cb_init() in terms of CBTREE_INIT *.h: move some *_INIT to designated initializers *.h _INIT macros: don't specify fields equal to 0 *.[ch] *_INIT macros: use { 0 } for a "zero out" idiom submodule-config.h: remove unused SUBMODULE_INIT macro
2021-09-27cbtree.h: define cb_init() in terms of CBTREE_INITÆvar Arnfjörð Bjarmason1-2/+3
Use the same pattern for cb_init() as the one established in the recent refactoring of other such patterns in 5726a6b4012 (*.c *_init(): define in terms of corresponding *_INIT macro, 2021-07-01). It has been pointed out[1] that we could perhaps use this C99 replacement of using a compound literal for all of these: *t = (struct cb_tree){ 0 }; But let's just stick to the existing pattern established in 5726a6b4012 for now, we can leave another weather balloon for some other time. 1. http://lore.kernel.org/git/ef724a3a-a4b8-65d3-c928-13a7d78f189a@gmail.com Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27*.h: move some *_INIT to designated initializersÆvar Arnfjörð Bjarmason13-18/+42
Move various *_INIT macros to use designated initializers. This helps readability. I've only picked those leftover macros that were not touched by another in-flight series of mine which changed others, but also how initialization was done. In the case of SUBMODULE_ALTERNATE_SETUP_INIT I've left an explicit initialization of "error_mode", even though SUBMODULE_ALTERNATE_ERROR_IGNORE itself is defined as "0". Let's not peek under the hood and assume that enum fields we know the value of will stay at "0". The change to "TESTSUITE_INIT" in "t/helper/test-run-command.c" was part of an earlier on-list version[1] of c90be786da9 (test-tool run-command: fix flip-flop init pattern, 2021-09-11). 1. https://lore.kernel.org/git/patch-1.1-0aa4523ab6e-20210909T130849Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27*.h _INIT macros: don't specify fields equal to 0Ævar Arnfjörð Bjarmason7-11/+7
Change the initialization of "struct strbuf" changed in cbc0f81d96f (strbuf: use designated initializers in STRBUF_INIT, 2017-07-10) to omit specifying "alloc" and "len", as we do with other "alloc" and "len" (or "nr") in similar structs. Let's likewise omit the explicit initialization of all fields in the "struct ipc_client_connect_option" struct added in 59c7b88198a (simple-ipc: add win32 implementation, 2021-03-15). Do the same for a few other initializers, e.g. STRVEC_INIT and CACHE_DEF_INIT. Finally, start incrementally changing the same pattern in "t/helper/test-run-command.c". This change was part of an earlier on-list version[1] of c90be786da9 (test-tool run-command: fix flip-flop init pattern, 2021-09-11). 1. https://lore.kernel.org/git/patch-1.1-0aa4523ab6e-20210909T130849Z-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27*.[ch] *_INIT macros: use { 0 } for a "zero out" idiomÆvar Arnfjörð Bjarmason13-24/+21
In C it isn't required to specify that all members of a struct are zero'd out to 0, NULL or '\0', just providing a "{ 0 }" will accomplish that. Let's also change code that provided N zero'd fields to just provide one, and change e.g. "{ NULL }" to "{ 0 }" for consistency. I.e. even if the first member is a pointer let's use "0" instead of "NULL". The point of using "0" consistently is to pick one, and to not have the reader wonder why we're not using the same pattern everywhere. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27submodule-config.h: remove unused SUBMODULE_INIT macroÆvar Arnfjörð Bjarmason1-4/+0
This macro was added and used in c68f8375760 (implement fetching of moved submodules, 2017-10-16) but its last user went away in be76c212823 (fetch: ensure submodule objects fetched, 2018-12-06). Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27Makefile: pass -Wno-pendantic under GENERATE_COMPILATION_DATABASE=yesÆvar Arnfjörð Bjarmason1-0/+1
The same bug fixed in the "COMPUTE_HEADER_DEPENDENCIES=auto" mode in the preceding commit was also present with "GENERATE_COMPILATION_DATABASE=yes". Let's fix it so it works again with "DEVOPTS=1". Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2021-09-27Documentation: call out commands that nuke untracked files/directoriesElijah Newren3-5/+8
Some commands have traditionally also removed untracked files (or directories) that were in the way of a tracked file we needed. Document these cases. Signed-off-by: Elijah Newren <newren@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>