| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The figures in the README came from a single render of linux.git's tip,
which is an unusually large merge, and from a box that still had load on
it from the previous benchmark. Re-measured on an idle machine over 60
random commits: 12.6 ms to 2.0 ms rather than 14.83 to 3.13.
Also note that the esi-diff fragment does not change, which is the point
worth knowing: fragments carry no subject line and so never ask for
decorations at all.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
01 stopped add_ref_decoration() inflating every tag object, but left two
odb_read_object_info() calls per ref behind: one to classify the ref
target so lookup_object_by_type() can be called, and one to classify
what the ref peels to. Each is a binary search through the pack index.
On linux.git that index is 389 MB across 11.8 million objects, and a
commit page walks 946 refs, so it paid about 1900 of those searches per
request. Profiling put add_ref_decoration() at 78% of the whole frame
render, with odb_read_object_info() 76% and find_pack_entry() 52%.
Neither lookup is necessary. packed-refs only records a peeled line for
refs that point at a tag object, so peeled_oid being present already
says the type is OBJ_TAG. And the peeled object is only ever used as a
key for the decoration; nothing reads its type, so lookup_unknown_object()
gives the same struct a later lookup_commit() would fill in.
Frame render with cgit's cache off, on an i7-7700:
linux.git 14.83 ms -> 3.13 ms
git.git 6.49 ms -> 2.23 ms
A control run serving the same objects through a repository with one ref
instead of 946 costs 1.7 ms, so this closes most of the remaining gap.
Verified byte-identical: git log --decorate over 20000 linux.git commits
and the whole of git.git, for-each-ref and tag -l with peeled fields,
and 6000 cgit commit and diff pages. git's t4202-log, t3200-branch,
t6300-for-each-ref and t7004-tag pass, 976 tests between them, and
cgit's own suite passes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
Preloads ESI fragments so the expensive half of every commit page is
already cached before anyone asks for it. Because fragments are shared by
every repository holding the same objects, walking one fork preloads the
diffs for all of them.
Three modes. "crawl" follows links from a starting URL the way a scraper
would, queueing commit and diff pages ahead of everything else. "commits"
walks one repository's log pages; it is denser, but cgit paginates the log
by walking the revision list from the start each time, so it slows down as
the offset grows. "oids" reads object ids from stdin and has no pagination
to walk, which makes it the one to use for a bulk preload: fed from
git rev-list it measured 249 req/s against 68 for the log walk.
The reported hit rate counts only the pages the crawler asks for. ESI
fragments are fetched by Varnish itself and a client cannot see whether
they hit, so a healthy run shows a low client-side hit rate together with
roughly half the expected backend requests.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
02-cgit-output-buffering.patch was cut from a working tree that already
carried the ESI work, so it also contained the enable-esi config hook,
cache_key(), the repolink and cgit_fileurl fragment hunks and the
memrchr fix. That contradicted its README and made it fail to apply on
any tree that already had ESI, which is every tree in this repository.
Regenerate it with only the output buffering change: the html.c buffer
itself and the flushes in cache.c, filter.c, cgit.c and ui-shared.c.
Verified to apply cleanly both to stock cgit and on top of the ESI
commits here.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
Two independent performance patches found while measuring the ESI split,
both of which help whether or not a cache sits in front of cgit.
01-git-peel-decoration.patch applies to the git submodule. git's
add_ref_decoration() calls parse_object() on every annotated tag to
follow it to its commit, inflating all of them, even though the ref
backend already hands the callback a peeled object id when packed-refs
has one. Using that takes commit and log pages on git.git from about
6 ms to about 2 ms. Output is byte-identical and git's own t4202-log,
t3200-branch and t6300-for-each-ref pass.
02-cgit-output-buffering.patch applies to cgit. html_raw() did one
write(2) per call: a 1.7 MB diff took 99151 writes averaging 18 bytes.
A 64 KB buffer takes that to 27. The buffer has to be flushed wherever
fd 1 is redirected or handed to another writer, which is the cache slot,
exec filters, and the end of the HTTP headers, because git's archive
code writes snapshots and clone data straight to fd 1.
Kept as patches rather than applied because the first belongs upstream
in git, and the second touches enough of cgit's output path to be worth
landing on its own terms.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
Turns on ESI processing for frames carrying Surrogate-Control, hashes
esi-diff fragments on their query string so that every fork sharing a
commit shares one cached rendering, and refuses fragment requests that
do not come from the ESI processor so they do not become one more thing
for crawlers to enumerate.
Frames get a short TTL on purpose: they are cheap to render and there
are a lot of them, and they should not crowd fragments out of the cache.
cgit-esi.vtc exercises the VCL against a fake cgit backend, asking for
the same commit through two repository names and checking that the
backend saw exactly one fragment request.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
The rendered diff is the expensive part of a commit page, and its output
depends only on the objects involved. Everything around it depends on the
repository and the branch the page was reached through. A commit that
lives in many forks is therefore rendered once per fork per branch, which
is what makes crawlers so expensive on a host with many forks of the same
repository.
With "enable-esi=1", the commit and diff pages become a cheap frame: the
document head, page header, tabs, commit info and diff controls are
rendered as before, but where the diff would go cgit emits
<esi:include src='/<repo>/esi-diff/<path>?id=<oid>&id2=<oid>&dt=<n>'/>
and the response carries a Surrogate-Control header announcing ESI/1.0.
The new "esi-diff" page renders just the diffstat and the diff, with no
layout and with links relative to the repository root that carry no
branch; the frame emits a matching <base href> so the browser resolves
them into whichever fork it came through. Object ids are always resolved
to full hashes, so symbolic names do not multiply fragments.
Because the fragment's output is repository neutral, cgit's own cache
keys it on the diff parameters rather than the raw URL, and the cache in
front is expected to do the same. Rename following is a per repository
setting that changes the diff, so that case renders inline.
With enable-esi unset, output is unchanged.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
memrchr() is not in POSIX and is missing on macOS, where it breaks the
build of cgit_set_title_from_path(). Scan backwards for the separator by
hand instead; the path components handed to this function are short
enough that the difference does not matter.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Update to git version v2.54.0, this requires changes for these
upstream commits:
* d9ecf268ef3f69130fa269012318470d908978f6
odb: embed base source in the "files" backend
* cb506a8a69c953f7b87bb3ae099e0bed8218d3ab
odb: introduce "files" source
... and probably more related.
Signed-off-by: Christian Hesse <mail@eworm.de>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
These would be largely invalid anyway (save, I suppose, for Linux file
paths that technically can contain new lines).
The actual problem is that these get printed back out into cached -- and
trusted -- cgitrc files, and if the fields have newlines, the git-config
way of less trusted users configuring repos on a shared system can be
abused to inject newlines, which then can be used to smuggle global
options (including filters, which execute code) into the cached cgitrc.
So now, only ever duplicate up to the newline, when dealing with these
inputs.
Reported-by: Adrian Denkiewicz <adrian@doyensec.com>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
There's no reason to pass around function pointers. It was never used
for anything beyond one function.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
lock_slot() opens the lock file with O_RDWR|O_CREAT but without
O_TRUNC. If a previous cgit process was killed between lock_slot() and
unlock_slot() (e.g. by a CGI timeout or OOM), the stale lock file
remains on disk with the old content, and the kernel releases the fcntl
lock.
The next process to claim the same cache slot then opens this stale lock
file, acquires the fcntl lock, writes its key and generated content on
top of the old bytes. If the new response is shorter than what was
previously in the file, trailing bytes from the old response survive
beyond the end of the new content. fstat() in fill_slot() reports the
total file size (including the stale tail), and print_slot() faithfully
sends all of it -- producing a response that is the correct page
followed by a fragment of whatever previously occupied that lock file.
Fix this by truncating the lock file after acquiring the lock and before
writing the new key.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
We don't get any return value from compile_grep_patterns calling
compile_regexp_failed, causing the default die routine to print to
stderr and then for cgit to exit ungracefully.
Instead override the default die routine to show a normal error page.
Perhaps compile_grep_patterns ought to change upstream to return an
error. But this commit here will handle future issues as well, so
perhaps not a bad idea to do anyway.
Link: https://lists.zx2c4.com/pipermail/cgit/2026-March/004982.html
Link: https://lists.zx2c4.com/pipermail/cgit/2026-March/004983.html
Reported-by: Adrian C. <anrxc@sysphere.org>
Reported-by: Aiden Woodruff <aiden@aidenw.net>
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
This exists for other CPU heavy operations like blame, but doesn't for
the follow functionality. Add it for that.
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
In git 2.36, the logic for diff_flush() changed, resulting in our object
getting freed, which led to following renames producing garbage commits
until it eventually crashed.
Fixes: bb02e24 ("git: update to v2.36.0")
Signed-off-by: Jason A. Donenfeld <Jason@zx2c4.com>
|
|
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
... specified in the URL, with a curved arrow ahead of the line number.
Signed-off-by: Loïc <lagiraudiere+cgit@free.fr>
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Signed-off-by: Christian Barcenas <christian@cbarcenas.com>
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Signed-off-by: Christian Barcenas <christian@cbarcenas.com>
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Since the favicon setting defaults to "/favicon.ico", the user can
only unset it with favicon=. However, that would write an empty string
as the config value. Previously, such empty string always satisfied
the if condition. In this case it is better to omit the <link> entirely.
Signed-off-by: Christian Barcenas <christian@cbarcenas.com>
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
This fixes an error which was introduced by
2f50b47c72cbc4270bbd12ae7f520486d5f42736. Git 2.42.0 added a new argument
to config_fn_t, and it was added to gitconfig_config(), but not named.
This causes compile warnings/errors. This commit fixes that by naming the
new parameter, and marking it unused.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.53.0, this requires changes for these
upstream commits:
* bdbebe5714b25dc9d215b48efbb80f410925d7dd
refs: introduce wrapper struct for `each_ref_fn`
* 589127caa73090040200989ff4d24c3d54f473f2
packfile: move list of packs into the packfile store
* 5a5c7359f77ecd1bc4b0e172563161d602f131d3
refs: drop `current_ref_iter` hack
* b6e4cc8c32850315323961659e553d1d14591f7f
tag: support arbitrary repositories in parse_tag()
* 84f0e60b28de69d1ccb7a51b729af6202b6cf4c8
packfile: move packfile store into object source
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.52.0, this requires changes for these
upstream commits:
* 9f6dfe43c8a55b833ae16486bcafe29b543461f9
string-list: align string_list_split() with its _in_place() counterpart
* 78237ea53d6546aeab7adb2c7547a1177311ccde
packfile: split up responsibilities of `reprepare_packed_git()`
* 751808b2a18acba76b824aed4d8b7442bd7f5fca
packfile: refactor `get_packed_git()` to work on packfile store
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.51.2, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.51.1, this requires changes for these
upstream commits:
* e1d062e8ba0b72f49e9ef9713cc7011c330baab8
odb: drop deprecated wrapper functions
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.51.0, this requires changes for these
upstream commits:
* 8f49151763cb81adf4bcec53c1ae67057081b02d
object-store: rename files to "odb.{c,h}"
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.50.1, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.50.0, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.49.0, this requires changes for these
upstream commits:
* 88dd321cfedc6ee190dfafe4670a83ea33cdf4a3
path: drop `git_path()` in favor of `repo_git_path()`
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.48.0, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.48.0, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.47.1, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.47.0, this requires changes for these
upstream commits:
* e8207717f1623325fe1c95338fb03c1104ed5687
refs: add referent to each_ref_fn
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.46.2, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.46.1, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.46.0, this requires changes for these
upstream commits:
* e7da9385708accf518a80a1e17969020fb361048
global: introduce `USE_THE_REPOSITORY_VARIABLE` macro
* 9da95bda74cf10e1475384a71fd20914c3b99784
hash: require hash algorithm in `oidread()` and `oidclr()`
* 30aaff437fddd889ba429b50b96ea4c151c502c5
refs: pass repo when peeling objects
* c8f815c2083c4b340d4148a15d45c55f2fcc7d3f
refs: remove functions without ref store
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
fix error that is given because of macro overlapping cgit_filter member:
../filter.c:388:10: error: no member named '__fprintf_chk' in
'struct cgit_filter'
388 | filter->fprintf(filter, f, prefix);
| ~~~~~~ ^
/usr/include/bits/stdio2.h:92:3: note: expanded from macro 'fprintf'
92 | __fprintf_chk (stream, __USE_FORTIFY_LEVEL - 1, __VA_ARGS__)
| ^
1 error generated.
Signed-off-by: Denis Pronin <dannftk@yandex.ru>
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.45.2, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.45.1, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.45.0, this requires changes for these
upstream commits:
* 9720d23e8caf4adee44b3a32803a9bb0480118bd
date: make DATE_MODE thread-safe
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.44.0, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.43.2, no additional changes required.
(Git v2.43.1 fails to build, thus skipping.)
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.43.0, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.42.1, no additional changes required.
Signed-off-by: Christian Hesse <mail@eworm.de>
|
|
Update to git version v2.42.0, this requires changes for these
upstream commits:
* bc5c5ec0446895f5c4139cd470066beb3c4ac6d5
cache.h: remove this no-longer-used header
* aba070683295a20bdf4f49146384984961c794b2
path: move related function to path
* a4e7e317f8f27f861321e6eb08b9c8c0f3ab570c
config: add ctx arg to config_fn_t
Signed-off-by: Christian Hesse <mail@eworm.de>
|
