summaryrefslogtreecommitdiff
path: root/t
AgeCommit message (Collapse)AuthorFilesLines
2019-03-04tests: let --stress-limit=<N> imply --stressJohannes Schindelin2-1/+2
It does not make much sense that running a test with --stress-limit=<N> seemingly ignores that option because it does not stress test at all. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-03remote-curl: use post_rpc() for protocol v2 alsoJonathan Tan1-1/+32
When transmitting and receiving POSTs for protocol v0 and v1, remote-curl uses post_rpc() (and associated functions), but when doing the same for protocol v2, it uses a separate set of functions (proxy_rpc() and others). Besides duplication of code, this has caused at least one bug: the auth retry mechanism that was implemented in v0/v1 was not implemented in v2. To fix this issue and avoid it in the future, make remote-curl also use post_rpc() when handling protocol v2. Because line lengths are written to the HTTP request in protocol v2 (unlike in protocol v0/v1), this necessitates changes in post_rpc() and some of the functions it uses; perform these changes too. A test has been included to ensure that the code for both the unchunked and chunked variants of the HTTP request is exercised. Note: stateless_connect() has been updated to use the lower-level packet reading functions instead of struct packet_reader. The low-level control is necessary here because we cannot change the destination buffer of struct packet_reader while it is being used; struct packet_buffer has a peeking mechanism which relies on the destination buffer being present in between a peek and a read. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-03-01bisect: make diff-tree output prettierJeff King1-3/+3
After completing a bisection, we print out the commit we found using an internal version of diff-tree. The result is aesthetically lacking: - it shows a raw diff, which is generally less informative for human readers than "--stat --summary" (which we already decided was nice for humans in format-patch's output). - by not abbreviating hashes, the result is likely to wrap on most people's terminals - we don't use "-r", so if the commit touched files in a directory, you only get to see the top-level directory mentioned - we don't specify "--cc" or similar, so merges print nothing (not even the commit message!) Even though bisect might be driven by scripts, there's no reason to consider this part of the output as machine-readable (if anything, the initial "$hash is the first bad commit" might be parsed, but we won't touch that here). Let's make it prettier and more informative for a human reading the output. While we're tweaking the options, let's also switch to using the diff "ui" config. If we're accepting that this is human-readable output, then we should respect the user's options for how to display it. Note that we have to touch a few tests in t6030. These check bisection in a corrupted repository (it's missing a subtree). They didn't fail with the previous code, because it didn't actually recurse far enough in the diff to find the broken tree. But now we'll see the corruption and complain. Adjusting the tests to expect the die() is the best fix. We still confirm that we're able to bisect within the broken repo. And we'll still print "$hash is the first bad commit" as usual before dying; showing that is a reasonable outcome in a corrupt repository (and was what might happen already, if the root tree was corrupt). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-24diff: reuse diff setup for --no-index caseJeff King1-0/+8
When "--no-index" is in effect (or implied by the arguments), git-diff jumps early to a special code path to perform that diff. This means we miss out on some settings like enabling --ext-diff and --textconv by default. Let's jump to the no-index path _after_ we've done more setup on rev.diffopt. Since some of the options don't affect us (e.g., items related to the index), let's re-order the setup into two blocks (see the in-code comments). Note that we also need to stop re-initializing the diffopt struct in diff_no_index(). This should not be necessary, as it will already have been initialized by cmd_diff() (and there are no other callers). That in turn lets us drop the "repository" argument from diff_no_index (which never made much sense, since the whole point is that you don't need a repository). Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22Merge branch 'ab/bsd-fixes'Junio C Hamano2-5/+5
Test portability fix. * ab/bsd-fixes: commit-graph tests: fix unportable "dd" invocation tests: fix unportable "\?" and "\+" regex syntax
2019-02-22Merge branch 'ab/workaround-dash-bug-in-test'Junio C Hamano1-0/+1
* ab/workaround-dash-bug-in-test: tests: avoid syntax triggering old dash bug
2019-02-22trace2: t/helper/test-trace2, t0210.sh, t0211.sh, t0212.shJeff Hostetler9-0/+1174
Create unit tests for Trace2. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22trace2: create new combined trace facilityJeff Hostetler3-0/+7
Create a new unified tracing facility for git. The eventual intent is to replace the current trace_printf* and trace_performance* routines with a unified set of git_trace2* routines. In addition to the usual printf-style API, trace2 provides higer-level event verbs with fixed-fields allowing structured data to be written. This makes post-processing and analysis easier for external tools. Trace2 defines 3 output targets. These are set using the environment variables "GIT_TR2", "GIT_TR2_PERF", and "GIT_TR2_EVENT". These may be set to "1" or to an absolute pathname (just like the current GIT_TRACE). * GIT_TR2 is intended to be a replacement for GIT_TRACE and logs command summary data. * GIT_TR2_PERF is intended as a replacement for GIT_TRACE_PERFORMANCE. It extends the output with columns for the command process, thread, repo, absolute and relative elapsed times. It reports events for child process start/stop, thread start/stop, and per-thread function nesting. * GIT_TR2_EVENT is a new structured format. It writes event data as a series of JSON records. Calls to trace2 functions log to any of the 3 output targets enabled without the need to call different trace_printf* or trace_performance* routines. Signed-off-by: Jeff Hostetler <jeffhost@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22commit-graph tests: test a graph that's too smallÆvar Arnfjörð Bjarmason1-0/+6
Use the recently split-up components of the corrupt_graph_and_verify() function to assert that we error on graphs that are too small. The error was added in 2a2e32bdc5 ("commit-graph: implement git commit-graph read", 2018-04-10), but there was no test for it. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22commit-graph tests: split up corrupt_graph_and_verify()Ævar Arnfjörð Bjarmason1-6/+16
Split up the corrupt_graph_and_verify() function added in d9b9f8a6fd ("commit-graph: verify catches corrupt signature", 2018-06-27) into its logical components of setting up the test itself, doing the corruption in a particular way with "dd", and then finally testing that stderr is what we expect. This allows for re-using everything except the now slimmer corrupt_graph_and_verify() to corrupt the graph in a way that doesn't involve inserting a given byte sequence at a given position, e.g. truncating it entirely to a custom value. Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22format-patch: notice failure to open cover letter for writingJunio C Hamano1-0/+6
The make_cover_letter() function is supposed to open a new file for writing, and let the caller write into it via FILE *rev->diffopt.file but because the function does not return anything, the caller does not bother checking the return value. Make sure it dies, instead of keep going with a NULL output filestream and relying on it to cause a crash, when it fails to open the file. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-22commit-graph tests: fix unportable "dd" invocationÆvar Arnfjörð Bjarmason1-1/+1
Change an unportable invocation of "dd" with count=0, that wanted to truncate the commit-graph file. In POSIX it is unspecified what happens when count=0 is provided[1]. The NetBSD "dd" behavior differs from GNU (and seemingly other BSDs), which has left this test broken since d2b86fbaa1 ("commit-graph: fix buffer read-overflow", 2019-01-15). Copying from /dev/null would seek/truncate to seek=$zero_pos and stop immediately after that (without being able to copy anything), which is the right way to truncate the file. 1. http://pubs.opengroup.org/onlinepubs/9699919799/utilities/dd.html Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Helped-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-21tests: fix unportable "\?" and "\+" regex syntaxÆvar Arnfjörð Bjarmason1-4/+4
Fix widely supported but non-POSIX basic regex syntax introduced in [1] and [2]. On GNU, NetBSD and FreeBSD the following works: $ echo xy >f $ grep 'xy\?' f; echo $? xy 0 The same goes for "\+". The "?" and "+" syntax is not in the BRE syntax, just in ERE, but on some implementations it can be invoked by prefixing the meta-operator with "\", but not on OpenBSD: $ uname -a OpenBSD obsd.my.domain 6.2 GENERIC#132 amd64 $ grep --version grep version 0.9 $ grep 'xy\?' f; echo $? 1 Let's fix this by moving to ERE syntax instead, where "?" and "+" are universally supported: $ grep -E 'xy?' f; echo $? xy 0 1. 2ed5c8e174 ("describe: setup working tree for --dirty", 2019-02-03) 2. c801170b0c ("t6120: test for describe with a bare repository", 2019-02-03) Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-19Merge branch 'js/test-tool-gen-nuls'Junio C Hamano4-7/+24
* js/test-tool-gen-nuls: tests: teach the test-tool to generate NUL bytes and use it
2019-02-19Merge branch 'mk/t5562-no-input-to-too-large-an-input-test'Junio C Hamano1-2/+2
* mk/t5562-no-input-to-too-large-an-input-test: t5562: do not depend on /dev/zero Revert "t5562: replace /dev/zero with a pipe from generate_zero_bytes"
2019-02-19t5562: do not reuse output filesMax Kirillov1-4/+4
Some expected failures of git-http-backend leaves running its children (receive-pack or upload-pack) which still hold opened descriptors to act.err and with some probability they live long enough to write there their failure messages after next test has already truncated the files. This causes occasional failures of the test script. Avoid the issue by using separated output and error file for each test, apprending the test number to their name. Reported-by: Carlo Arenas <carenas@gmail.com> Helped-by: Carlo Arenas <carenas@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Max Kirillov <max@max630.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-19tests: teach the test-tool to generate NUL bytes and use itJohannes Schindelin4-7/+24
In cc95bc2025 (t5562: replace /dev/zero with a pipe from generate_zero_bytes, 2019-02-09), we replaced usage of /dev/zero (which is not available on NonStop, apparently) by a Perl script snippet to generate NUL bytes. Sadly, it does not seem to work on NonStop, as t5562 reportedly hangs. Worse, this also hangs in the Ubuntu 16.04 agents of the CI builds on Azure Pipelines: for some reason, the Perl script snippet that is run via `generate_zero_bytes` in t5562's 'CONTENT_LENGTH overflow ssite_t' test case tries to write out an infinite amount of NUL bytes unless a broken pipe is encountered, that snippet never encounters the broken pipe, and keeps going until the build times out. Oddly enough, this does not reproduce on the Windows and macOS agents, nor in a local Ubuntu 18.04. This developer tried for a day to figure out the exact circumstances under which this hang happens, to no avail, the details remain a mystery. In the end, though, what counts is that this here change incidentally fixes that hang (maybe also on NonStop?). Even more positively, it gets rid of yet another unnecessary Perl invocation. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-19t5562: do not depend on /dev/zeroMax Kirillov1-1/+1
It was reported [1] that NonStop platform does not have /dev/zero. The test uses /dev/zero as a dummy input. Passing case (http-backed failed because of too big input size) should not be reading anything from it. If http-backend would erroneously try to read any data returning EOF probably would be even safer than providing some meaningless data. Replace /dev/zero with /dev/null to avoid issues with platforms which do not have /dev/zero. [1] https://public-inbox.org/git/20190209185930.5256-4-randall.s.becker@rogers.com/ Reported-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Max Kirillov <max@max630.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-19Revert "t5562: replace /dev/zero with a pipe from generate_zero_bytes"Junio C Hamano1-2/+2
Revert cc95bc20 ("t5562: replace /dev/zero with a pipe from generate_zero_bytes", 2019-02-09), as not feeding anything to the command is a better way to test it.
2019-02-15read-cache: add post-index-change hookBen Peart1-0/+144
Add a post-index-change hook that is invoked after the index is written in do_write_locked_index(). This hook is meant primarily for notification, and cannot affect the outcome of git commands that trigger the index write. The hook is passed a flag to indicate whether the working directory was updated or not and a flag indicating if a skip-worktree bit could have changed. These flags enable the hook to optimize its response to the index change notification. Signed-off-by: Ben Peart <benpeart@microsoft.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14t5304: rename "sha1" variables to "oid"Jeff King1-12/+12
Let's make the script less jarring to read in a post-sha1 world by using more hash-agnostic variable names. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14prune: use bitmaps for reachability traversalJeff King1-0/+11
Pruning generally has to traverse the whole commit graph in order to see which objects are reachable. This is the exact problem that reachability bitmaps were meant to solve, so let's use them (if they're available, of course). Here are timings on git.git: Test HEAD^ HEAD ------------------------------------------------------------------------ 5304.6: prune with bitmaps 3.65(3.56+0.09) 1.01(0.92+0.08) -72.3% And on linux.git: Test HEAD^ HEAD -------------------------------------------------------------------------- 5304.6: prune with bitmaps 35.05(34.79+0.23) 3.00(2.78+0.21) -91.4% The tests show a pretty optimal case, as we'll have just repacked and should have pretty good coverage of all refs with our bitmaps. But that's actually pretty realistic: normally prune is run via "gc" right after repacking. A few notes on the implementation: - the change is actually in reachable.c, so it would improve reachability traversals by "reflog expire --stale-fix", as well. Those aren't performed regularly, though (a normal "git gc" doesn't use --stale-fix), so they're not really worth measuring. There's a low chance of regressing that caller, since the use of bitmaps is totally transparent from the caller's perspective. - The bitmap case could actually get away without creating a "struct object", and instead the caller could just look up each object id in the bitmap result. However, this would be a marginal improvement in runtime, and it would make the callers much more complicated. They'd have to handle both the bitmap and non-bitmap cases separately, and in the case of git-prune, we'd also have to tweak prune_shallow(), which relies on our SEEN flags. - Because we do create real object structs, we go through a few contortions to create ones of the right type. This isn't strictly necessary (lookup_unknown_object() would suffice), but it's more memory efficient to use the correct types, since we already know them. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14prune: lazily perform reachability traversalJeff King2-0/+36
The general strategy of "git prune" is to do a full reachability walk, then for each loose object see if we found it in our walk. But if we don't have any loose objects, we don't need to do the expensive walk in the first place. This patch postpones that walk until the first time we need to see its results. Note that this is really a specific case of a more general optimization, which is that we could traverse only far enough to find the object under consideration (i.e., stop the traversal when we find it, then pick up again when asked about the next object, etc). That could save us in some instances from having to do a full walk. But it's actually a bit tricky to do with our traversal code, and you'd need to do a full walk anyway if you have even a single unreachable object (which you generally do, if any objects are actually left after running git-repack). So in practice this lazy-load of the full walk catches one easy but common case (i.e., you've just repacked via git-gc, and there's nothing unreachable). The perf script is fairly contrived, but it does show off the improvement: Test HEAD^ HEAD ------------------------------------------------------------------------- 5304.4: prune with no objects 3.66(3.60+0.05) 0.00(0.00+0.00) -100.0% and would let us know if we accidentally regress this optimization. Note also that we need to take special care with prune_shallow(), which relies on us having performed the traversal. So this optimization can only kick in for a non-shallow repository. Since this is easy to get wrong and is not covered by existing tests, let's add an extra test to t5304 that covers this case explicitly. Signed-off-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14Merge branch 'jc/no-grepping-for-strerror-in-tests'Junio C Hamano1-1/+1
* jc/no-grepping-for-strerror-in-tests: t1404: do not rely on the exact phrasing of strerror()
2019-02-14Merge branch 'jt/fetch-v2-sideband'Junio C Hamano1-0/+1
"git fetch" and "git upload-pack" learned to send all exchange over the sideband channel while talking the v2 protocol. * jt/fetch-v2-sideband: t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through Apache
2019-02-14t/lib-httpd: pass GIT_TEST_SIDEBAND_ALL through ApacheTodd Zullinger1-0/+1
07c3c2aa16 ("tests: define GIT_TEST_SIDEBAND_ALL", 2019-01-16) added GIT_TEST_SIDEBAND_ALL to the apache.conf PassEnv list. Avoid warnings from Apache when the variable is unset, as we do for GIT_VALGRIND* and GIT_TRACE, from f628825481 ("t/lib-httpd: handle running under --valgrind", 2012-07-24) and 89c57ab3f0 ("t: pass GIT_TRACE through Apache", 2015-03-13), respectively. Signed-off-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-14t1404: do not rely on the exact phrasing of strerror()Junio C Hamano1-1/+1
Not even in C locale, it is wrong to expect that the exact phrasing "File exists" is used to show EEXIST. Reported-by: Randall S. Becker <rsbecker@nexbridge.com> Helped-by: Duy Nguyen <pclouds@gmail.com> Helped-by: Jeff King <peff@peff.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-13Merge branch 'ab/rebase-test-fix'Junio C Hamano1-3/+6
* ab/rebase-test-fix: rebase: fix regression in rebase.useBuiltin=false test mode
2019-02-13Merge branch 'rb/no-dev-zero-in-test'Junio C Hamano3-3/+16
* rb/no-dev-zero-in-test: t5562: replace /dev/zero with a pipe from generate_zero_bytes t5318: replace use of /dev/zero with generate_zero_bytes test-lib-functions.sh: add generate_zero_bytes function
2019-02-13Merge branch 'sg/stress-test'Junio C Hamano3-4/+23
Test improvement. * sg/stress-test: test-lib: fix non-portable pattern bracket expressions test-lib: make '--stress' more bisect-friendly
2019-02-13Merge branch 'kd/t0028-octal-del-is-377-not-777'Junio C Hamano1-4/+4
Test fix. * kd/t0028-octal-del-is-377-not-777: t0028: fix wrong octal values for BOM in setup
2019-02-13rebase: fix regression in rebase.useBuiltin=false test modeÆvar Arnfjörð Bjarmason1-3/+6
Fix a recently introduced regression in c762aada1a ("rebase -x: sanity check command", 2019-01-29) triggered when running the tests with GIT_TEST_REBASE_USE_BUILTIN=false. See 62c23938fa ("tests: add a special setup where rebase.useBuiltin is off", 2018-11-14) for how that test mode works. As discussed on-list[1] it's not worth it to implement the sanity check in the legacy rebase code, we plan to remove it after the 2.21 release. So let's do the bare minimum to make the tests pass under the GIT_TEST_REBASE_USE_BUILTIN=false special setup. 1. https://public-inbox.org/git/xmqqva1nbeno.fsf@gitster-ct.c.googlers.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-13tests: avoid syntax triggering old dash bugÆvar Arnfjörð Bjarmason1-0/+1
Avoid a bug in dash that's been fixed ever since its ec2c84d ("[PARSER] Fix clobbering of checkkwd", 2011-03-15)[1] first released with dash v0.5.7 in July 2011. This failing test was introduced in 5f9674243d ("config: add --expiry-date", 2017-11-18). This fixes 1/2 tests failing on Debian Lenny & Squeeze. The other failure is due to 1b42f45255 ("git-svn: apply "svn.pathnameencoding" before URL encoding", 2016-02-09). The dash bug is triggered by this test because the heredoc contains a command embedded in "$()" with a "{}" block coming right after it. Refactoring the "$()" to e.g. be a variable that was set earlier will also work around it, but let's instead break up the "EOF" and the "{}". An earlier version of this patch[2] mitigated the issue by breaking the "$()" out of the "{}" block, that worked, but just because it broke up the "EOF" and "{}" block. Putting e.g. "echo &&" between the two would also work. 1. https://git.kernel.org/pub/scm/utils/dash/dash.git/ 2. https://public-inbox.org/git/20181127164253.9832-1-avarab@gmail.com/ Signed-off-by: Ævar Arnfjörð Bjarmason <avarab@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-13read-cache.c: fix writing "link" index ext with null base oidNguyễn Thái Ngọc Duy1-0/+18
Since commit 7db118303a (unpack_trees: fix breakage when o->src_index != o->dst_index - 2018-04-23) and changes in merge code to use separate index_state for source and destination, when doing a merge with split index activated, we may run into this line in unpack_trees(): o->result.split_index = init_split_index(&o->result); This is by itself not wrong. But this split index information is not fully populated (and it's only so when move_cache_to_base_index() is called, aka force splitting the index, or loading index_state from a file). Both "base_oid" and "base" in this case remain null. So when writing the main index down, we link to this index with null oid (default value after init_split_index()), which also means "no split index" internally. This triggers an incorrect base index refresh: warning: could not freshen shared index '.../sharedindex.0{40}' This patch makes sure we will not refresh null base_oid (because the file is never there). It also makes sure not to write "link" extension with null base_oid in the first place (no point having it at all). Read code already has protection against null base_oid. There is also another side fix in remove_split_index() that causes a crash when doing "git update-index --no-split-index" when base_oid in the index file is null. In this case we will not load istate->split_index->base but we dereference it anyway and are rewarded with a segfault. This should not happen anymore, but it's still wrong to dereference a potential NULL pointer, especially when we do check for NULL pointer in the next code. Reported-by: Luke Diamand <luke@diamand.org> Signed-off-by: Nguyễn Thái Ngọc Duy <pclouds@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-13t5562: replace /dev/zero with a pipe from generate_zero_bytesRandall S. Becker1-2/+2
To help platforms that lack /dev/zero (e.g. NonStop), replace use of /dev/zero to feed "git http-backend" with a pipe of output from the generate_zero_bytes helper. Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-13t5318: replace use of /dev/zero with generate_zero_bytesRandall S. Becker1-1/+1
There are platforms (e.g. NonStop) that lack /dev/zero; use the generate_zero_bytes helper we just introduced to append stream of NULs at the end of the file. The original, even though it uses "dd seek=... count=..." to make it look like it is overwriting the middle part of an existing file, has truncated the file before this step with another use of "dd", which may make it tricky to see why this rewrite is a correct one. Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-12test-lib-functions.sh: add generate_zero_bytes functionRandall S. Becker1-0/+13
t5318 and t5562 used /dev/zero, which is not portable. This function provides both a fixed block of NUL bytes and an infinite stream of NULs. Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-11utf8: handle systems that don't write BOM for UTF-16brian m. carlson1-5/+29
When serializing UTF-16 (and UTF-32), there are three possible ways to write the stream. One can write the data with a BOM in either big-endian or little-endian format, or one can write the data without a BOM in big-endian format. Most systems' iconv implementations choose to write it with a BOM in some endianness, since this is the most foolproof, and it is resistant to misinterpretation on Windows, where UTF-16 and the little-endian serialization are very common. For compatibility with Windows and to avoid accidental misuse there, Git always wants to write UTF-16 with a BOM, and will refuse to read UTF-16 without it. However, musl's iconv implementation writes UTF-16 without a BOM, relying on the user to interpret it as big-endian. This causes t0028 and the related functionality to fail, since Git won't read the file without a BOM. Add a Makefile and #define knob, ICONV_OMITS_BOM, that can be set if the iconv implementation has this behavior. When set, Git will write a BOM manually for UTF-16 and UTF-32 and then force the data to be written in UTF-16BE or UTF-32BE. We choose big-endian behavior here because the tests use the raw "UTF-16" encoding, which will be big-endian when the implementation requires this knob to be set. Update the tests to detect this case and write test data with an added BOM if necessary. Always write the BOM in the tests in big-endian format, since all iconv implementations that omit a BOM must use big-endian serialization according to the Unicode standard. Preserve the existing behavior for systems which do not have this knob enabled, since they may use optimized implementations, including defaulting to the native endianness, which may improve performance. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-11t0028: fix wrong octal values for BOM in setupKevin Daudt1-4/+4
The setup code uses octal values with printf to generate a BOM for UTF-16/32 BE/LE. It specifically uses '\777' to emit a 0xff byte. This relies on the fact that most shells truncate the value above 0o377. Ash however interprets '\777' as '\77' + a literal '7', resulting in an invalid BOM. Fix this by using the proper value of 0xff: '\377'. Signed-off-by: Kevin Daudt <me@ikke.info> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-11test-lib: fix non-portable pattern bracket expressionsSZEDER Gábor2-3/+3
Use a '!' character to start a non-matching pattern bracket expression, as specified by POSIX in Shell Command Language section 2.13.1 Patterns Matching a Single Character [1]. I used '^' instead in three places in the previous three commits, to verify that the arguments of the '--stress=' and '--stress-limit=' options and the values of various '*_PORT' environment variables are valid numbers. With certain shells, at least with dash (upstream and in Ubuntu 14.04) and mksh, this led to various undesired behaviors: # error message in case of a valid number $ ~/src/dash/src/dash ./t3903-stash.sh --stress=8 error: --stress=<N> requires the number of jobs to run # not the expected error message $ ~/src/dash/src/dash ./t3903-stash.sh --stress=foo ./t3903-stash.sh: 238: test: Illegal number: foo # no error message at all?! $ mksh ./t3903-stash.sh --stress=foo $ echo $? 0 Some other shells, e.g. Bash (even in posix mode), ksh, dash in Ubuntu 16.04 or later, are apparently happy to accept '^' just as well. [1] http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html#tag_02_13 Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-11test: caution on our version of 'yes'Junio C Hamano2-1/+14
During a review of a patch, we noticed that we use our own imitation of 'yes' with the limit of 99 lines. It is very tempting to lift this arbitrary limit, but the limit is there for a reason. Add an in-code comment to prevent future developers from wasting their time. Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-08Merge branch 'ld/git-p4-remove-flakey-test'Junio C Hamano1-27/+0
A flakey "p4" test has been removed. * ld/git-p4-remove-flakey-test: git-p4: remove ticket expiry test
2019-02-08Merge branch 'js/rebase-i-redo-exec-fix'Junio C Hamano1-3/+0
For "rebase -i --reschedule-failed-exec", we do not want the "-y" shortcut after all. * js/rebase-i-redo-exec-fix: Revert "rebase: introduce a shortcut for --reschedule-failed-exec"
2019-02-08Merge branch 'js/smart-http-detect-remote-error'Junio C Hamano4-0/+13
Some errors from the other side coming over smart HTTP transport were not noticed, which has been corrected. * js/smart-http-detect-remote-error: t5551: test server-side ERR packet remote-curl: tighten "version 2" check for smart-http remote-curl: refactor smart-http discovery
2019-02-08Merge branch 'tz/gpg-test-fix'Junio C Hamano1-2/+1
Test fix. * tz/gpg-test-fix: t/lib-gpg: drop redundant killing of gpg-agent t/lib-gpg: quote path to ${GNUPGHOME}/trustlist.txt
2019-02-08Merge branch 'os/rebase-runs-post-checkout-hook'Junio C Hamano1-1/+1
Test fix. * os/rebase-runs-post-checkout-hook: t5403: correct bash ambiguous redirect error in subtest 8 by quoting $GIT_DIR
2019-02-08submodule--helper: teach config subcommand --unsetDenton Liu1-0/+9
This teaches submodule--helper config the --unset option, which removes the specified configuration key from the .gitmodule file. Signed-off-by: Denton Liu <liu.denton@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-08test-lib: make '--stress' more bisect-friendlySZEDER Gábor2-2/+21
Let's suppose that a test somehow becomes flaky between 'master' and 'pu', and tends to fail within the first 50 repetitions when run with '--stress'. In such a case we could use 'git bisect' to find the culprit: if the test script fails with '--stress', then the commit is definitely bad, but if it survives, say, 300 repetitions, then we could consider it good with reasonable confidence. Unfortunately, all this could only be done manually, because '--stress' would run the test script repeatedly for all eternity on a good commit, and it would exit with success even when it found a failure on a bad commit. So let's make '--stress' usable with 'git bisect run': - Make it exit with failure if a failure is found. - Add the '--stress-limit=<N>' option to repeat the test script at most N times in each of the parallel jobs, and exit with success when the limit is reached. And then we could simply run something like: $ git bisect start origin/pu master $ git bisect run sh -c 'make && cd t && ./t1234-foo.sh --stress --stress-limit=300' Sure, as a brand new feature it won't be any useful right now, but in a release or three most cooking topics will already contain this, so we could automatically bisect at least newly introduced flakiness. Signed-off-by: SZEDER Gábor <szeder.dev@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-08t5403: correct bash ambiguous redirect error in subtest 8 by quoting $GIT_DIRRandall S. Becker1-1/+1
The embedded blanks in the full path of the test git repository cased bash to generate an ambugious redirect error. Signed-off-by: Randall S. Becker <rsbecker@nexbridge.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2019-02-08t/lib-gpg: drop redundant killing of gpg-agentTodd Zullinger1-1/+0
In 53fc999306 ("gpg-interface t: extend the existing GPG tests with GPGSM", 2018-07-20), the gpgconf call which kills gpg-agent was copied from the existing gpg setup code. The reason for killing gpg-agent is given in 29ff1f8f74 ("t: lib-gpg: flush gpg agent on startup", 2017-07-20): When running gpg-relevant tests, a gpg-daemon is spawned for each GNUPGHOME used. This daemon may stay running after the test and cache file descriptors for the trash directories, even after the trash directory is removed. This leads to ENOENT errors when attempting to create files if tests are run multiple times. Add a cleanup script to force flushing the gpg-agent for that GNUPGHOME (if any) before setting up the GPG relevant-environment. Killing gpg-agent once per test is sufficient. Signed-off-by: Todd Zullinger <tmz@pobox.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>