| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The provider chain so far holds the diff-hunks store in front of the
terminal builtin computation. Open it to external processes: a pair on
a path whose driver configures diff.<driver>.process is answered by a
long-running process speaking a pkt-line protocol (following the filter
process protocol), registered at the head of the chain and consulted
before the store and before any blob is loaded.
The protocol starts with the smallest request that can carry an answer:
object names alone. A request is the pathname and the pair's
old-oid/new-oid, with no content. The process answers with hunk lines,
with a zero-hunk success that asserts the blobs equivalent (trailing
newlines included), or with status=need-content, on which the pair
falls through to the builtin answer. This serves the two shapes that
need no content pushed to them: a cache keyed on the blob pair, and a
process that fetches the blobs itself (for example over "git cat-file
--batch"). A pair whose side is not a stored blob carries a NULL id;
the provider sends no request and passes it. Because Git holds no
content for the exchange, the answer is used as sent: hunks are
validated for order, overlap, lockstep alignment, and magnitude, then
replayed without the normalization xdiff applies to diffs it computes
itself. The magnitude bound is the blobs' sizes, read from the object
database without loading content: a blob of N bytes holds at most N
lines.
Because the process's answer is authoritative, it outranks the store,
and its head-of-chain position says so. A pair the process answers
never reaches the store and is never recorded, so nothing it produces
enters the store, which holds the builtin answer only. A request it
does not answer, whether need-content, a missing capability, or a
missing id, passes down the chain to the builtin answer, which is what
the store serves, so the store may serve such a pair and a warming run
may record it. Entries recorded before a process was configured are
not purged; a pair the process answers ignores them, and "git diff-hunks
clear" discards them.
The provider gates itself per request. The driver is looked up by the
old-side path, so a renamed file resolves to the same driver, and by
the repository-relative path, so a diff.relative run from a
subdirectory names the pair the same way. Options the process is never
told about select no process: the whitespace-ignoring options, -I,
--anchored, and an algorithm forced by option or configuration (blame
routes its algorithm through xdl_opts, so --histogram is covered). The
request gains its last field, the path; the consumers change only by
filling it, and neither names the process.
The provider's state is its repository's pool of running processes,
keyed by the configured command, so drivers sharing a command share a
process, a submodule speaks to its own, and releasing the provider
(from repo_clear()) stops them. The pool owns a copy of each command
string, so an entry outlives a config re-read. A command that fails
stays as an entry that is not retried: its request and every later one
pass, so the store may serve the path for the rest of the command.
A protocol error in a response never kills the command. The response
is read through a packet reader gentle about framing, so an error takes
one path: a single warning, the process stopped and marked failed, and
the builtin diff for the rest of the command. That covers garbage
bytes, a truncated response, an empty packet, a bare status, and an
unrecognized status. Semantically invalid coordinates cost only their
pair: the response is drained, the pair is computed, and the process
stays alive. A path the protocol cannot carry (an embedded newline, or
one too long for a packet) falls back per path rather than costing the
command its process. The handshake keeps one fatal check: a process
that announces a capability Git did not request aborts the command, as
the long-running filter protocol does.
Consulting is allowed per command, following the allow_textconv
precedent. "git diff", "git log" and "git show", and "git blame" set
allow_diff_process; the plumbing diff commands and the interactive-patch
machinery never set it, so scripted and staging output stays builtin.
The options adjust the flag:
- --no-ext-diff clears it and --ext-diff sets it;
- --diff-process and --no-diff-process set and clear it alone, leaving
external diff drivers as they were;
- format-patch clears it unconditionally, so a generated patch applies
for recipients without the process;
- range-diff passes --no-ext-diff to the "git log" it compares.
git blame and the summary formats consult the process. For blame, a
pair reported equivalent emits no hunks, so the whole commit passes to
its parent. In the stat formats such a pair sums to a zero-count entry,
which the "nothing changed" rule omits, as under -w. The subprocess is
long-running: one startup cost across a traversal, one round-trip per
consulted pair. Answers travel in struct xdl_hunk, new in
xdiff-interface.h, holding xdiff's 1-based coordinates; nothing feeds
them back to xdiff, since only coordinate consumers consult.
A content-carrying request is the natural extension: it would serve
sides that are not stored blobs and processes that want content pushed
to them, and bring patch output and log -L's range tracking to the same
answer. As it stands, a process's answers show in blame and the summary
formats while patch output stays builtin.
t4080 exercises the protocol, the per-command gate, and the error paths:
- each adversarial response shape warns and falls back to builtin, the
request log proving which failures disable the process and which keep
it alive (a malformed hunk line, coordinates past the blob size, a
count overflowing strtol(), overlapping or misaligned hunks, an
unrecognized status, a bare status, an empty packet, a mid-response
crash, and raw garbage);
- a capability-less process and status=abort degrade without noise, and
a failed start warns once and returns the path to the store;
- a trailing token on a hunk line is ignored, pinning field
appendability;
- positive consults for git diff, git show, and diff-tree under
--ext-diff and --diff-process; textconv output and gitlink sides are
never identified; a diff.relative run consults by the repo-relative
path;
- the equivalence answer is pinned from both consumers, and a warming
run past a deferring process records the pair for a later read.
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Teach builtin_diffstat() to consult the hunk provider interface through
diff_provider_consult(), new here: the consult-only entry that answers
without loading content or computing, so it never returns
DIFF_PROVIDER_ERROR. On an answer, the summing callback accumulates the
provided counts directly into the diffstat entry; the blobs were already
loaded for the binary check, so an answer saves the diff run, not the
content load (blame, taught next, skips its loads too). On an
unanswered outcome it computes as before and, with a writer attached,
records what it computed; on unanswered-no-record it computes without
recording.
The provider behind the consult is the diff-hunks store, registered in
front of the terminal builtin computation. Its consult serves a
recorded pair through diff_hunks_replay(), which validates the sequence
before any hunk reaches the callback, so direct accumulation is safe.
The request gains the pair's object ids and the diff options read by the
exclusions below. A side whose bytes are not a stored blob, such as a
working-tree file or a gitlink, has a NULL id; the store passes it by and
the terminal provider computes it. diff_provider_emit_hunks() walks the
same chain, so blame's requests follow these rules the moment blame
supplies identity. The walk also insists, as a BUG check, that a
request's diff options belong to the repository whose chain it walks.
Each exclusion lives with the provider whose key cannot express it. -I
patterns and --anchored shape the diff outside the store key, and break
detection (-B) rescores the pair outside it; the store's consult maps
all three to stop-no-record, so such a request is neither served nor
recorded for any consumer. The consumer-side guard the recording commit
carried for those three comes out here. The compile-time assert on
xpparam_t's layout sits next to that decision, forcing an explicit
keying decision whenever a diff parameter is added. The stat consumer
keeps only the exclusion that is not about the key: --ignore-blank-lines
is part of the key but coalesces hunks differently between the
text-emitting and coordinate-callback paths, so the consumer returns
before consulting. A "log -L" range-scoped stat neither reads nor
records; the line-range filter computes it as before.
"git diff", "git log", "git show", and "git diff-tree" with the --stat,
--numstat, and --shortstat formats consult the interface. Reading is
controlled by core.diffHunks.
An answer is invisible in the output, so the store counts the pairs it
serves and the consultations it cannot, and diff_hunks_read_stats()
reports both; the stat path emits the hits as a trace2 "read-hits" datum
for tests and tuning. The counters live on the store because only the
store knows whether a consultation reached it, and none of its exclusion
legs reaches the replay, so none counts as a miss.
Extend t4220 with the read half:
- output parity with and without the store, at several context lengths
and both directions, and reversed pairs keying apart;
- the consultation made visible through the read-hits datum, and the
trim-divergent pair correct at every context;
- the settings that must bypass the store doing so in both directions
(-I, -B, --anchored, --ignore-blank-lines), asserted through the trace
rather than output parity alone, which a coincidentally equal count
could satisfy;
- a driver-forced algorithm keying apart rather than bypassing: it is
part of the key, so a read under it misses the default entries and a
warm records under its own.
A "log -L" range-scoped stat neither reads nor records.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
The diff-hunks store has a writer, but nothing fills it. Teach
builtin_diffstat() to do so: on a warming run (a writer is attached), a
modified pair's stat is produced by collecting the pair's hunk
coordinates instead of emitting text, the counts are summed from those
hunks, and the pair is recorded. A run without a writer is unchanged,
and nothing reads the store yet; the read side arrives next.
The store records one context-free entry per pair, and only for a
trim-stable pair: one whose zero-context trimmed diff (what blame will
read) and untrimmed diff (whose counts a nonzero-context stat matches)
are identical. The warming path computes both and hands them to
diff_hunks_writer_record_stable(), new here, which records only when
they agree; a divergent pair is never recorded and every consumer
computes it. The warming run displays the counts it shows a store-less
run: the trimmed ones, since xdi_diff trims at zero context, while the
untrimmed counts serve only the stability comparison.
Not everything the stat path computes may be recorded.
--ignore-blank-lines is part of the key, but it coalesces hunks
differently between the text-emitting and coordinate-callback paths, so
a recorded entry would not match a store-less run's --stat. -I
patterns, --anchored, and break detection (-B) shape the diff outside
the key entirely; the guard for those three sits in this consumer for
now and moves into the store's own provider when it registers, next. A
"log -L" range-scoped stat is not the whole-pair diff the key describes,
so it does not record. Recording also requires both sides to be valid
regular files whose blobs the key can name: a working-tree side,
textconv output, or a gitlink has no usable id.
"git diff", "git log", "git show", and "git diff-tree" with the --stat,
--numstat, and --shortstat formats attach a writer when writing is
enabled and flush it when the traversal finishes, so a warming run such
as
GIT_DIFF_HUNKS_WRITE=1 git log --all --stat >/dev/null
fills the cache as a side effect of the diff work the command already
does. Writing is controlled by diffHunks.write and GIT_DIFF_HUNKS_WRITE.
Add the write half of t4220:
- ordinary commands never create the store, and creation is gated off
by default, the environment overriding the config;
- a warming run builds a store that verifies, and a second refreshes it
in place;
- a warming run displays parity at zero context on a trim-divergent
pair, committed as a fixture (small synthetic pairs cannot diverge:
minimal diffs add and delete equal counts, and trimming preserves
that);
- binary and mode-only pairs do not break the writer;
- a corrupt store is discarded at seed;
- verify and clear run against the files a warming run builds.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Teach -G to only search the line ranges specified by -L.
Teaching -S is left as future work, so it still matches the entire
file even if -L is specified.
Rather than being part of diff.c's builtin implementations, the
diffcore-pickaxe functionality interacts with xdiff-interface as a
separate component. Add a sibling to xdi_diff_outf(), called
diff_emit_line_ranges(), that limits emitted lines to the given line
ranges.
Use diff_emit_line_ranges() when searching text if line ranges have
been specified. If textconv is enabled, use normal diffing instead of
diff_emit_line_ranges() since line range tracking relies on the line
coordinates of the original, pre-textconv file.
Update documentation and add tests accordingly.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Reuse the line_range_filter in builtin_checkdiff() so -L supports
the --check option.
Add orig_hunk_fn field similar to orig_line_fn that forwards
xdiff_emit_hunk_fn calls when we flush filtered hunks. This is necessary
because --check relies on receiving calls to its checkdiff_consume_hunk
function for managing state.
Document and ungate the newly enabled option, and add tests verifying
the new behavior.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Reuse the line_range_filter in builtin_diffstat() so -L supports
the stat formats and add tests verifying the new behavior.
Ungate the newly enabled options and drop "yet" from the generic
-L rejection message ("does not yet support the requested diff
format"). Some rejected formats do not fit -L at all, so "yet"
wrongly implies they are all awaiting support.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Extract logic for initializing the line-range filter and running a diff
for a specific line range. This logic is needed for any diff that
targets a line range independent of the current patch display path.
The subsequent commits use this logic to enable additional line range
targeted diff modes.
No logical behavior change.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Currently, diff's line-range filter implements its own method for
emitting diff hunk headers. This mostly matches what xdiff itself
outputs, but there is a discrepancy for postimage or preimage sides
with 0 line changes. For a side with no lines (count 0), the begin is
the line before the change. The header omits the line count of 1.
Rather than fix this case in the line-range implementation, expose the
function xdiff uses to emit its headers. Reusing it keeps the header
format consistent with and without -L.
Update test scenarios and fixtures to reflect the now consistent header
format.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Currently, the diff line-range filter buffers preimage removal lines
until a postimage line arrives. That line's number confirms whether the
preimage line falls in a relevant range. However, storing preimage
lines in a separate buffer is unnecessary. Worse, the logic has a bug:
a preimage line outside the target range is included when it
immediately follows an in-range postimage line.
Preimage lines will always precede their postimage counterpart both in
content line number and emission order from xdiff's line callback
function. So preimage lines can share the postimage buffer. The filter
flushes them based on whether the postimage lines fall within the
target range.
Remove logic related to storing preimage lines in a separate "removal"
buffer and prepending them to the accumulating_hunk's line buffer.
Instead, store those lines in the accumulating_hunk's line_buffer
immediately and flush everything as appropriate based on postimage
line numbers that arrive. This resolves the bug by construction.
Also, calculate the old and new line counts for the diff hunk header
when flushing rather than storing counters in line_range_filter to
simplify state management further.
Add a test to t/t4211-line-log.sh that verifies the preimage line
emission bug is fixed.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
diff's line-range filtering logic uses the line_range_callback
struct to represent filtering state. However, this name does not
clearly reflect the role it plays. This is especially relevant as
we expand diff's line-range filtering to work with more options,
including --stat and -G.
Also, line_range_callback's fields are terse, while the comment
explaining line_range_callback is verbose and out of place compared to
its surroundings.
Rename line_range_callback to line_range_filter, and replace the verbose
comment with a concise one, instead preferring descriptive field and
variable names that are self-explanatory over comments.
No logical behavior change. Some fields are grouped under a new struct
in the newly renamed line_range_filter. Everything else is just a
rename.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
The help text for the '-l' option of 'git diff' has been updated.
* en/diff-l-opt-help:
diff: avoid misleading statement about -l option
|
|
The 'pack-objects' and delta-encoding code paths have been updated to
use 'size_t' instead of 'unsigned long' for object sizes and offset
limits, avoiding potential truncation issues on 64-bit Windows.
* js/pack-objects-delta-size-t:
packfile: widen `unpack_object_header_buffer()` to `size_t`
git-zlib: widen `git_deflate_bound()` to `size_t`
t/helper/test-pack-deltas: widen `do_compress()`'s maxsize local to `size_t`
http-push: widen `start_put()`'s size local from `ssize_t` to `size_t`
diff: widen `deflate_it()`'s bound local from int to `size_t`
archive-zip: widen `zlib_deflate_raw()`'s maxsize local to `size_t`
packfile, git-zlib: widen `use_pack()` and zstream avail fields to `size_t`
delta: widen `create_delta()` and `diff_delta()` to `size_t`
pack-objects: widen `mem_usage` and `try_delta()`'s out-param to `size_t`
pack-objects: widen `free_unpacked()` return to `size_t`
pack-objects: widen delta-cache accounting to `size_t`
delta: widen `create_delta_index()` parameter to `size_t`
diff-delta: widen `struct delta_index`' size fields to `size_t`
|
|
In commit 6623a528e00b (doc: clarify documentation for rename/copy
limits, 2021-07-15), the wording around rename limit options and config
variables were updated to point out that only the quadratic portion of
rename detection (or "exhaustive portion of rename/copy detection" as
used in that commit) was limited by these options, because exact rename
detection and basename-guided rename detection (which both run in time
linear in the number of files) still run before this limit is checked.
However, the short help message wasn't updated at the time; update it
too.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Fixes a pre-existing silent narrowing from `git_deflate_bound()`'s
`unsigned long` return into an `int` local: anything past 2 GiB has
always wrapped negative here and then been re-extended to `size_t`
inside `xmalloc()`. Also prep for the upcoming `git_deflate_bound()`
widening to `size_t`, which would extend the narrowing further if
`bound` stayed `int`.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Last stop in the delta-encoding API widening for >4 GiB blobs on
Windows: with `create_delta_index()` done in the prior commit and
`create_delta()`/`diff_delta()` finished here, every byte count that
crosses delta.h is now `size_t`. The struct fields they store into have
been `size_t` since the diff-delta struct widening.
The API change must move with all callers in the same commit (the build
only passes when every `&delta_size` matches the new `size_t*`). Caller
updates are kept minimal:
* builtin/pack-objects.c `get_delta()` and `try_delta()`: widen only
the local `delta_size` variable; the surrounding unsigned-long
locals and their `cast_size_t_to_ulong()` shims are out of scope
here and will be cleaned up in their own commits.
* builtin/fast-import.c, diff.c, t/helper/test-pack-deltas.c:
keep the local unsigned-long delta size (each feeds a still-
unsigned-long downstream consumer: zlib's `avail_in`,
`deflate_it()`, the test helper's own `do_compress()`), and bridge
via a temporary `size_t` plus `cast_size_t_to_ulong()`. The new
casts are paid back in later topics that widen those consumers.
* t/helper/test-delta.c: widen the local outright (no downstream
consumer beyond the test's own `out_size`, which is already
`size_t`).
Note that GCC struggles a bit to figure out that `deltalen` is always
initialized before it is used; To help it along, we initialize it to 0.
This work-around will go away in a later patch series when `deltalen`
can be widened to `size_t`.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
The diff.c:is_conflict_marker() and rerere.c:is_cmarker() functions
implement duplicate logic for identifying conflict marker lines
(lines that begin with a run of '<', '=', '>', and '|' characters).
diff.c's original version from 049540435f (diff --check: detect
leftover conflict markers, 2008-06-26) accepts any whitespace (such
as a newline) immediately following '<<<<<<<' and '>>>>>>>', whereas
rerere.c's version from 191f241717 (rerere: prepare for customizable
conflict marker length, 2010-01-16) strictly requires a space
character (' ') after them.
Implement is_conflict_marker_line() in merge-ll.c to serve as a
replacement for both, and update diff.c and rerere.c to use the new
helper. The unified helper intentionally adopts rerere's stricter
rule, as the conflicts generated by Git always show the "ours" and
"theirs" labels after these markers separated by a space.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
We'd like to add more logic to git_hash_init(), but many callers skip it
and call algop->init_fn() directly. Let's make sure we're consistently
using the wrapper by adding a coccinelle rule.
Besides the coccinelle file itself, this is a purely mechanical
conversion based on the patch it generates. There should be no bare
init_fn() calls left (except for the one in the wrapper).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
* jk/hash-algo-leak-fixes:
hash: add platform-specific discard functions
hash: fix memory leak copying sha256 gcrypt handles
http: discard hash in dumb-http http_object_request
check_stream_oid(): discard hash on read error
patch-id: discard hash when done
csum-file: provide a function to release checkpoints
csum-file: always finalize or discard hash
hash: add discard primitive
csum-file: drop discard_hashfile()
|
|
When computing a patch-id, we have a flush_one_hunk() helper that calls
git_hash_final() on our running hunk git_hash_ctx, and then
reinitializes that context for the next hunk.
When we run out of hunks to look at, we return, discarding the
git_hash_ctx. This can cause a leak if the hash implementation we are
using allocates any memory during its initialization. This includes
OpenSSL >= 3.0, for both SHA-1 and SHA-256. Normally we would not use
SHA-1 here at all, as we only recommend using non-DC implementations for
the "unsafe" variant (and patch-id, though they probably _could_ use the
unsafe variant, were never taught to do so).
But it is certainly a problem for SHA-256, which you can see with:
make SANITIZE=leak \
OPENSSL_SHA256=1 \
GIT_TEST_DEFAULT_HASH=sha256 \
test
That results in leak failures of 60 scripts, 57 of which are fixed by
this patch (basically anything which runs rebase will hit this case).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
* js/objects-larger-than-4gb-on-windows-more:
odb: use size_t for object_info.sizep and the size APIs
packfile,delta: drop the `cast_size_t_to_ulong()` wrappers
pack-objects: use size_t for in-core object sizes
packfile: widen unpack_entry()'s size out-parameter to size_t
pack-objects(check_pack_inflate()): use size_t instead of unsigned long
patch-delta: use size_t for sizes
compat/msvc: use _chsize_s for ftruncate
|
|
When `js/objects-larger-than-4gb-on-windows` widened the streaming,
index-pack and unpack-objects code paths, in the interest of keeping the
patches somewhat reasonably-sized, it left the public ODB API still
typed in `unsigned long`. In particular `struct object_info::sizep` and
the four wrappers built on top of it (`odb_read_object`,
`odb_read_object_peeled`, `odb_read_object_info`, `odb_pretend_object`)
still return the unpacked size through `unsigned long *`, so on Windows
`cat-file -s` and the `git add` / `git status` paths for a >4 GiB blob
silently cap at 4 GiB.
Widen the field and the four wrappers. The previous commits already
widened the `unpack_entry()` cascade and pack-objects' in-core size
accessors, so most of the cascade arrives here with no further work: the
temporary shims in `packed_object_info_with_index_pos()` and in
`unpack_entry()`'s delta-base recovery path go away, the two
`SET_SIZE(entry, cast_size_t_to_ulong(canonical_size))` calls in
`check_object()` and the matching one in `drop_reused_delta()` collapse
to plain `SET_SIZE`, and `oe_get_size_slow()`'s tail
`cast_size_t_to_ulong()` is gone too.
What remains narrow are the boundaries this series does not
intend to touch: the diff, blame, textconv and fast-import machinery.
Even so, this patch is unfortunately quite large.
Assisted-by: Opus 4.7
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Many core configuration variables have been migrated from global
variables into 'repo_config_values' to tie them to a specific
repository instance, avoiding cross-repository state leakage.
* ob/more-repo-config-values:
environment: move "warn_on_object_refname_ambiguity" into `struct repo_config_values`
environment: move "sparse_expect_files_outside_of_patterns" into `struct repo_config_values`
environment: move "core_sparse_checkout_cone" into `struct repo_config_values`
environment: move "precomposed_unicode" into `struct repo_config_values`
environment: move "pack_compression_level" into `struct repo_config_values`
environment: move `zlib_compression_level` into `struct repo_config_values`
environment: move "check_stat" into `struct repo_config_values`
environment: move "trust_ctime" into `struct repo_config_values`
|
|
The `zlib_compression_level` configuration is currently stored in the
global variable `zlib_compression_level`, which makes it shared across
repository instances within a single process.
Store it instead in `repo_config_values`, where eagerly‑parsed
repository configuration lives. `zlib_compression_level` is parsed
eagerly because it determines compression behaviour for objects and
packs – core operations where a lazy parse could lead to unpredictable
results and hinder libification. This preserves the existing
eager‑parsing behavior while tying the value to the repository it
was read from, avoiding cross‑repository state leakage and continuing
the effort to reduce reliance on global configuration state.
Update all references to use `repo_config_values()`.
Mentored-by: Christian Couder <christian.couder@gmail.com>
Mentored-by: Usman Akinyemi <usmanakinyemi202@gmail.com>
Signed-off-by: Olamide Caleb Bello <belkid98@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
The command line parser for "git diff" learned a few options take
only non-negative integers.
* mm/diff-U-takes-no-negative-values:
parse-options: clarify what "negated" means for PARSE_OPT_NONEG
xdiff: guard against negative context lengths
diff: reject negative values for -U/--unified
diff: reject negative values for --inter-hunk-context
|
|
Passing a negative value to -U is silently accepted and produces
corrupt unified diff output with malformed hunk headers:
$ git log -1 -p -U-500 -- GIT-VERSION-GEN | grep '^@@'
@@ -503,999- +503,999- @@
Line 503 of a 106-line file, count "999-" is not a valid integer.
The config variable diff.context already rejects negative values, but
the command line callback diff_opt_unified() uses strtol() with no
range check.
Change the type of diff_options.context and its static default from
int to unsigned int, matching the change to interhunkcontext in the
previous commit. The type change requires reworking the callback and
config parsing to validate in a local variable before assigning to
the now-unsigned field.
Unlike --inter-hunk-context which could be converted to OPT_UNSIGNED,
-U needs OPT_CALLBACK_F for PARSE_OPT_OPTARG (bare -U with no value
enables patch output). Add a range check in the callback instead.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Negative values for --inter-hunk-context produce structurally invalid
diff output with overlapping hunks:
$ git log -1 -p -U3 --inter-hunk-context=-100 791aeddfa2 \
-- git-compat-util.h | grep '^@@'
@@ -110,6 +110,9 @@
@@ -115,6 +118,9 @@
@@ -116,6 +122,7 @@
Hunk 1 covers lines 110-115, hunk 2 starts at 115 (overlap), hunk 3
starts at 116 (overlaps both). The resulting patch cannot be applied.
The config variable diff.interHunkContext already rejects negative
values, but the command line option does not.
Change the type of diff_options.interhunkcontext and its static
default from int to unsigned int, and switch the option parser from
OPT_INTEGER_F to OPT_UNSIGNED. This rejects negative values at parse
time via git_parse_unsigned() and enforces the correct type at compile
time via BARF_UNLESS_UNSIGNED.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
f85b49f3d4a (diff: improve scaling of filenames in diffstat to handle
UTF-8 chars, 2026-01-16) introduced a loop in show_stats() that calls
utf8_width() repeatedly to skip leading characters until the displayed
width fits. However, utf8_width() can return problematic values:
- For invalid UTF-8 sequences, pick_one_utf8_char() sets the name
pointer to NULL and utf8_width() returns 0. Since name_len does
not change, the loop iterates once more and pick_one_utf8_char()
dereferences the NULL pointer, crashing.
- For control characters, utf8_width() returns -1, so name_len
grows when it is expected to shrink. This can cause the loop to
consume more characters than the string contains, reading past
the trailing NUL.
By default, fill_print_name() will C-quote filenames which escapes
control characters and invalid bytes to printable text. That avoids
this bug from being triggered; however, with core.quotePath=false,
most characters are no longer escaped (though some control characters
still are) and raw bytes can reach this code.
Add tests exercising both failure modes with core.quotePath=false and
a narrow --stat-name-width to force truncation: one with a bare 0xC0
byte (invalid UTF-8 lead byte, triggers NULL deref) and one with
several C1 control characters (repeats of 0xC2 0x9F, causing
the loop to read past the end of the string). The second test
reliably catches the out-of-bounds read when run under ASan, though
it may pass silently without sanitizers.
Fix both issues by introducing utf8_ish_width(), a thin wrapper
around utf8_width() that guarantees the pointer always advances and
the returned width is never negative:
- On invalid UTF-8 it restores the pointer, advances by one byte,
and returns width 1 (matching the strlen()-based fallback used
by utf8_strwidth()).
- On a control character it returns 0 (matching utf8_strnwidth()
which skips them).
Also add a "&& *name" guard to the while-loop condition so it
terminates at end-of-string even when utf8_strwidth()'s strlen()
fallback causes name_len to exceed the sum of per-character widths.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
The way the "git log -L<range>:<file>" feature is bolted onto the
log/diff machinery is being reworked a bit to make the feature
compatible with more diff options, like -S/G.
* mm/line-log-use-standard-diff-output:
doc: note that -L supports patch formatting and pickaxe options
t4211: add tests for -L with standard diff options
line-log: route -L output through the standard diff pipeline
line-log: fix crash when combined with pickaxe options
|
|
`git log -L` has always bypassed the standard diff pipeline.
`dump_diff_hacky()` in line-log.c hand-rolls its own diff headers and
hunk output, which means most diff formatting options are silently
ignored. A NEEDSWORK comment has acknowledged this since the feature
was introduced:
/*
* NEEDSWORK: manually building a diff here is not the Right
* Thing(tm). log -L should be built into the diff pipeline.
*/
Remove `dump_diff_hacky()` and its helpers and route -L output through
`builtin_diff()` / `fn_out_consume()`, the same path used by `git diff`
and `git log -p`. The mechanism is a pair of callback wrappers that sit
between `xdi_diff_outf()` and `fn_out_consume()`, filtering xdiff's
output to only the tracked line ranges. To ensure xdiff emits all lines
within each range as context, the context length is inflated to span the
largest range.
Wire up the `-L` implies `--patch` default in revision setup rather
than forcing it at output time, so `line_log_print()` is just
`diffcore_std()` + `diff_flush()` with no format save/restore.
Rename detection is a no-op since pairs are already resolved during
the history walk in `queue_diffs()`, but running `diffcore_std()`
means `-S`/`-G` (pickaxe), `--orderfile`, and `--diff-filter` now
work with `-L`, and `diff_resolve_rename_copy()` sets pair statuses
correctly without manual assignment.
Switch `diff_filepair_dup()` from `xmalloc` to `xcalloc` so that new
fields (including `line_ranges`) are zero-initialized by default.
As a result, diff formatting options that were previously silently
ignored (e.g. --word-diff, --no-prefix, -w, --color-moved) now work
with -L, and output gains `index` lines, `new file mode` headers, and
funcname context in `@@` headers. This is a user-visible output change:
tools that parse -L output may need to handle the additional header
lines.
The context-length inflation means xdiff may process more output than
needed for very wide line ranges, but benchmarks on files up to 7800
lines show no measurable regression.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
"git diff --no-index --find-object=<object-name>" outside a
repository of course wouldn't be able to find the object and died
while parsing the command line, which is made to die in a bit more
user-friendly way.
* mm/diff-no-index-find-object:
diff: fix crash with --find-object outside repository
|
|
"git log --graph --stat" did not count the display width of colored
graph part of its own output correctly, which has been corrected.
* lp/diff-stat-utf8-display-width-fix:
t4052: test for diffstat width when prefix contains ANSI escape codes
diff: handle ANSI escape codes in prefix when calculating diffstat width
|
|
Mark the marge-ort codebase to prevent more uses of the_repository
from getting added.
* en/merge-ort-almost-wo-the-repository:
replay: prevent the_repository from coming back
merge-ort: prevent the_repository from coming back
merge-ort: replace the_hash_algo with opt->repo->hash_algo
merge-ort: replace the_repository with opt->repo
merge-ort: pass repository to write_tree()
merge,diff: remove the_repository check before prefetching blobs
|
|
When "git diff --find-object=<oid>" is run outside a git repository,
the option parsing callback eagerly resolves the OID via
repo_get_oid(), which reaches get_main_ref_store() and hits a BUG()
assertion because no repository has been set up.
Check startup_info->have_repository before attempting to resolve the
OID, and return a user-friendly error instead.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
It does not make much sense to apply the "incomplete-line"
whitespace rule to symbolic links, whose contents almost always
lack the final newline. "git apply" and "git diff" are now taught
to exclude them for a change to symbolic links.
* jc/whitespace-incomplete-line:
whitespace: symbolic links usually lack LF at the end
|
|
The diffstat width is calculated by taking the terminal width and
incorrectly subtracting the `strlen()` of `line_prefix`, instead of the
actual display width of `line_prefix`, which may contain ANSI escape
codes (e.g., ANSI-colored strings in `log --graph --stat`).
Utilize the display width instead, obtained via `utf8_strnwidth()` with
the flag `skip_ansi`.
Signed-off-by: LorenzoPegorari <lorenzo.pegorari2002@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Prefetching of blobs from promisor remotes was added to diff in
7fbbcb21b162 (diff: batch fetching of missing blobs, 2019-04-05). In
that commit,
https://lore.kernel.org/git/20190405170934.20441-1-jonathantanmy@google.com/
was squashed into
https://lore.kernel.org/git/44de02e584f449481e6fb00cf35d74adf0192e9d.1553895166.git.jonathantanmy@google.com/
without the extra explanation about the squashed changes being added to
the commit message; in particular, this explanation from that first link
is absent:
> Also, prefetch only if the repository being diffed is the_repository
> (because we do not support lazy fetching for any other repository
> anyway).
Then, later, this checking was spread from diff.c to diffcore-rename.c
and diffcore-break.c by 95acf11a3dc3 (diff: restrict when prefetching
occurs, 2020-04-07) and then further split in d331dd3b0c82
(diffcore-rename: allow different missing_object_cb functions,
2021-06-22). I also copied the logic from prefetching blobs from
diff.c to merge-ort.c in 2bff554b23e8 (merge-ort: add prefetching for
content merges, 2021-06-22).
The reason for all these checks was noted above -- we only supported
lazy fetching for the_repository. However, that changed with
ef830cc43412 (promisor-remote: teach lazy-fetch in any repo,
2021-06-17), so these checks are now unnecessary. Remove them.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
ISO C23 redefines strchr and friends that tradiotionally took
a const pointer and returned a non-const pointer derived from it to
preserve constness (i.e., if you ask for a substring in a const
string, you get a const pointer to the substring). Update code
paths that used non-const pointer to receive their results that did
not have to be non-const to adjust.
* cf/c23-const-preserving-strchr-updates-0:
gpg-interface: remove an unnecessary NULL initialization
global: constify some pointers that are not written to
|
|
For a patch that touches a symbolic link, it is perfectly normal
that the contents ends with "\ No newline at end of file". The
checks introduced recently to detect incomplete lines (i.e., a text
file that lack the newline on its final line) should not trigger.
Disable the check early for symbolic links, both in "git apply" and
"git diff" and test them. For "git apply", we check only when the
postimage is a symbolic link regardless of the preimage, and we only
care about preimage when applying in reverse. Similarly, "git diff"
would warn only when the postimage is a symbolic link, or the
preimage when running "git diff -R".
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
The recent glibc 2.43 release had the following change listed in its
NEWS file:
For ISO C23, the functions bsearch, memchr, strchr, strpbrk, strrchr,
strstr, wcschr, wcspbrk, wcsrchr, wcsstr and wmemchr that return
pointers into their input arrays now have definitions as macros that
return a pointer to a const-qualified type when the input argument is
a pointer to a const-qualified type.
When compiling with GCC 15, which defaults to -std=gnu23, this causes
many warnings like this:
merge-ort.c: In function ‘apply_directory_rename_modifications’:
merge-ort.c:2734:36: warning: initialization discards ‘const’ qualifier from pointer target type [-Wdiscarded-qualifiers]
2734 | char *last_slash = strrchr(cur_path, '/');
| ^~~~~~~
This patch fixes the more obvious ones by making them const when we do
not write to the returned pointer.
Signed-off-by: Collin Funk <collin.funk1@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
The computation of column width made by "git diff --stat" was
confused when pathnames contain non-ASCII characters.
* lp/diff-stat-utf8-display-width-fix:
t4073: add test for diffstat paths length when containing UTF-8 chars
diff: improve scaling of filenames in diffstat to handle UTF-8 chars
|
|
The `show_stats()` function tries to scale the filenames in the diffstat to
ensure they don't exceed the given `name-width`. It does so by calculating
the "display width" of the characters to be dropped, but then advances the
filename pointer by that number of bytes.
However, the "display width" of a character is not always equal to its byte
count. The result is that sometimes, when displaying UTF-8 characters,
filenames exceed the given `name-width`, and frequently the bytes of the
UTF-8 characters are truncated.
The following is an example of the issue, where the 2 files are "HelloHi" and
"Hello你好", and `name-width=6`:
...oHi | 0
...<BD><A0>好 | 0
Make the filename pointer move by the actual number of bytes of the
characters to drop from the filename, rather than their display width, using
the `utf8_width()` function.
Force `len` to not be less than 0 (this happens if the given `name-width` is
2 or less), otherwise an infinite loop is entered.
Signed-off-by: LorenzoPegorari <lorenzo.pegorari2002@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
When computing a diff in a partial clone, there is a chance that we
could trigger a prefetch of missing objects at the same time as we are
freeing entries from the global diff queue. This is difficult to
reproduce, as we need to have some objects be freed from the queue
before triggering the prefetch of missing objects. There is a new test
in t4067 that does trigger the segmentation fault that results in this
case.
The fix is to set the queue pointer to NULL after it is freed, and then
to be careful about NULL values in the prefetch.
The more elaborate explanation is that within diffcore_std(), we may
skip the initial prefetch due to the output format (--name-only in the
test) and go straight to diffcore_skip_stat_unmatch(). In that method,
the index entries that have been invalidated by path changes show up as
entries but may be deleted because they are not actually content diffs
and only newer timestamps than expected. As those entries are deleted,
later entries are checked with diff_filespec_check_stat_unmatch(), which
uses diff_queued_diff_prefetch() as the missing_object_cb in its diff
options. That can trigger downloading missing objects if the appropriate
scenario occurs to trigger a call to diff_popoulate_filespec(). It's
finally within that callback to diff_queued_diff_prefetch() that the
segfault occurs.
The test was hard to find because it required some real differences,
some not-different files that had a newer modified time, and the order
of those files alphabetically was important to trigger the deletion
before the prefetch was triggered.
I briefly considered a "lock" member for the diff queue, but it was a
much larger diff and introduced many more possible error scenarios.
Signed-off-by: Derrick Stolee <stolee@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Halve the memory consumed by artificial filepairs created during
"git diff --find-copioes-harder", also making the operation run
faster.
* rs/diff-index-find-copies-harder-optim:
diff-index: don't queue unchanged filepairs with diff_change()
|
|
Both "git apply" and "git diff" learn a new whitespace error class,
"incomplete-line".
* jc/whitespace-incomplete-line:
attr: enable incomplete-line whitespace error for this project
diff: highlight and error out on incomplete lines
apply: check and fix incomplete lines
whitespace: allocate a few more bits and define WS_INCOMPLETE_LINE
apply: revamp the parsing of incomplete lines
diff: update the way rewrite diff handles incomplete lines
diff: call emit_callback ecbdata everywhere
diff: refactor output of incomplete line
diff: keep track of the type of the last line seen
diff: correct suppress_blank_empty hack
diff: emit_line_ws_markup() if/else style fix
whitespace: correct bit assignment comments
|
|
diff_cache() queues unchanged filepairs if the flag find_copies_harder
is set, and uses diff_change() for that. This function allocates a
filespec for each side, does a few other things that are unnecessary for
unchanged filepairs and always sets the diff_flag has_changes, which is
simply misleading in this case.
Add a new streamlined function for queuing unchanged filepairs and
use it in show_modified(), which is called by diff_cache() via
oneway_diff() and do_oneway_diff(). It allocates only a single filespec
for each filepair and uses it twice with reference counting. This has a
measurable effect if there are a lot of them, like in the Linux repo:
Benchmark 1: ./git_v2.52.0 -C ../linux diff --cached --find-copies-harder
Time (mean ± σ): 31.8 ms ± 0.2 ms [User: 24.2 ms, System: 6.3 ms]
Range (min … max): 31.5 ms … 32.3 ms 85 runs
Benchmark 2: ./git -C ../linux diff --cached --find-copies-harder
Time (mean ± σ): 23.9 ms ± 0.2 ms [User: 18.1 ms, System: 4.6 ms]
Range (min … max): 23.5 ms … 24.4 ms 111 runs
Summary
./git -C ../linux diff --cached --find-copies-harder ran
1.33 ± 0.01 times faster than ./git_v2.52.0 -C ../linux diff --cached --find-copies-harder
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
"git blame" learns "--diff-algorithm=<algo>" option.
* ad/blame-diff-algorithm:
blame: make diff algorithm configurable
xdiff: add 'minimal' to XDF_DIFF_ALGORITHM_MASK
|
|
As "git diff --quiet" only cares about the existence of any
changes, disable rename/copy detection to skip more expensive
processing whose result will be discarded anyway.
* rs/diff-quiet-no-rename:
diff: disable rename detection with --quiet
|
|
The XDF_DIFF_ALGORITHM_MASK bit mask only includes bits for the patience
and histogram diffs, not for the minimal one. This means that when
reseting the diff algorithm to the default one, one needs to separately
clear the bit for the minimal diff. There are places in the code that fail
to do that: merge-ort.c and builtin/merge-file.c.
Add the XDF_NEED_MINIMAL bit to the bit mask, and remove the separate
clearing of this bit in the places where it hasn't been forgotten.
Signed-off-by: Antonin Delpeuch <antonin@delpeuch.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Teach "git diff" to highlight "\ No newline at end of file" message
as a whitespace error when incomplete-line whitespace error class is
in effect. Thanks to the previous refactoring of complete rewrite
code path, we can do this at a single place.
Unlike whitespace errors in the payload where we need to annotate in
line, possibly using colors, the line that has whitespace problems,
we have a dedicated line already that can serve as the error
message, so paint it as a whitespace error message.
Also teach "git diff --check" to notice incomplete lines as
whitespace errors and report when incomplete-line whitespace error
class is in effect.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
|
Reserve a few more bits in the diff flags word to be used for future
whitespace rules. Add WS_INCOMPLETE_LINE without implementing the
behaviour (yet).
Signed-off-by: Junio C Hamano <gitster@pobox.com>
|
