summaryrefslogtreecommitdiff
path: root/repository.h
AgeCommit message (Collapse)AuthorFilesLines
2026-08-15repository: move fetch_if_missing into struct repositoryTian Yuchen1-0/+6
The global variable 'fetch_if_missing' controls whether a missing object check should prompt a lazy fetch from a promisor remote. In order to continue the libification effort, move it into 'struct repository' and initialize it to 1 by default to keep the previous behavior. builtin/fetch-pack.c, builtin/fsck.c, and builtin/rev-list.c are entered via commands marked RUN_SETUP in git.c:commands[]. Their 'repo' parameter is only NULL when '-h' is given outside of a repository, in which case either show_usage_if_asked() or parse_options()'s own '-h' handling exits the process before returning. We can therefore drop their UNUSED markers and assign to 'repo' directly. builtin/index-pack.c is entered via RUN_SETUP_GENTLY, so its 'repo' pointer can be NULL any time it is run outside of a repository, not only with '-h'. We keep a NULL check there and fall back to 'the_repository'. builtin/pack-objects.c needs two adjustments to make 'repo' reach every 'fetch_if_missing' call site: 'read_stdin_packs()' now takes a 'struct repository *'; 'option_parse_missing_action()', which is registered as an OPT_CALLBACK, receives a 'repo' through the option's 'value' field now. Additionally, update the partial clone documentation to reflect that this is now a per-repository flag. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Ayush Chandekar <ayu.chandekar@gmail.com> Mentored-by: Olamide Caleb Bello <belkid98@gmail.com> Signed-off-by: Tian Yuchen <cat@malon.dev> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-30Merge branch 'ty/migrate-excludes-file'Junio C Hamano1-1/+1
The 'excludes_file' and various other global configuration variables (including 'editor_program', 'pager_program', 'askpass_program', and 'push_default') have been migrated into the per-repository structure. * ty/migrate-excludes-file: repository: adjust the comment of config_values_private_ environment: move object_creation_mode into repo_config_values environment: move autorebase into repo_config_values environment: move push_default into repo_config_values environment: migrate apply_default_whitespace and apply_default_ignorewhitespace environment: move askpass_program into repo_config_values environment: move pager_program into repo_config_values environment: move editor_program into repo_config_values environment: move excludes_file into repo_config_values repository: introduce repo_config_values_clear()
2026-07-14repository: adjust the comment of config_values_private_Tian Yuchen1-1/+1
The configurations in 'struct config_values_private_' are not all parsed in 'git_default_config()'. For example, 'pager_program' is now parsed in 'pager.c'. Therefore, update the comment. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Ayush Chandekar <ayu.chandekar@gmail.com> Mentored-by: Olamide Caleb Bello <belkid98@gmail.com> Signed-off-by: Tian Yuchen <cat@malon.dev> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-07setup: move prefix into repositoryPatrick Steinhardt1-0/+8
The repository prefix is currently stored in the startup info. This feels somewhat awkward though, as it is inherently a property of a given repository. Move the prefix into the repository accordingly. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-11environment: split up concerns of `is_bare_repository_cfg`Patrick Steinhardt1-0/+7
The `is_bare_repository_cfg` variable tracks two different pieces of information: - It tracks whether the user has invoked git with the "--bare" flag, which makes us treat any discovered Git repository as if it was a bare repository. - Otherwise it tracks whether the discovered `the_repository` is bare. This makes the flag extremely confusing and creates a bit of a challenge when handling multiple repositories in the same process. Split up the concerns of this variable into two pieces: - `startup_info.force_bare_repository` tracks whether the user has passed the "--bare" flag. This is used as a hint to treat newly set up repositories as bare regardless of whether or not they have a worktree. - `struct repository::bare_cfg` tracks whether or not a repository is considered bare. This takes into account both whether the user has passed "--bare" and the discovered state of the repository itself. Whether or not a repository is bare is now resolved when checking the repository's format, and is then later applied to the repository itself via `apply_repository_format()`. This enables a subsequent change where we make `is_bare_repository()` not depend on global state anymore. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-05repository: stop initializing the object database in `repo_set_gitdir()`Patrick Steinhardt1-3/+0
The function `repo_set_gitdir()` obviously sets the Git directory for a given repository. Less obviously though, the function also configures a couple of auxiliary settings. One such thing is that we create the object database in this function. This logic only happens conditionally though, as `set_git_dir()` may be called multiple times during repository setup, and we don't want to create the object database multiple times. This is somewhat tangled and hard to follow. Remove the logic from `repo_set_gitdir()` and instead initialize the object database outside of it. This leads to some duplication right now, but that duplication will be removed in a subsequent step where we will start initializing the object database as part of applying the repo's format. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-19setup: stop using `the_repository` in `upgrade_repository_format()`Patrick Steinhardt1-1/+1
Stop using `the_repository` in `upgrade_repository_format()` and instead accept the repository as a parameter. The injection of `the_repository` is thus bumped one level higher, where callers now pass it in explicitly. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-19setup: stop using `the_repository` in `set_git_work_tree()`Patrick Steinhardt1-0/+1
Stop using `the_repository` in `set_git_work_tree()` and instead accept the repository as a parameter. The injection of `the_repository` is thus bumped one level higher, where callers now pass it in explicitly. Similar as with the preceding commit, we track whether the worktree has been initialized already via a global variable so that we can die in case the repository is re-initialized with a different worktree path. Store this info in the `struct repository` instead so that we correctly handle this per repository. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-05-19setup: stop using `the_repository` in `setup_work_tree()`Patrick Steinhardt1-0/+1
Stop using `the_repository` in `setup_work_tree()` and instead accept the repository as a parameter. The injection of `the_repository` is thus bumped one level higher, where callers now pass it in explicitly. Note that the function tracks two bits of information via global variables. This of course doesn't make much sense anymore now that we can set up worktrees for arbitrary repositories: - We track whether the worktree has already been initialized and, if so, we skip the call to `chdir_notify()` and setenv(3p). It does not make much sense to store this info in the repository, as we _would_ want to update the environment when switching between worktrees back and forth. So instead of storing this info in the repository, we drop this state entirely and live with the fact that we may execute the logic twice. It should ultimately be idempotent though and thus not be much of a problem. - We track whether the worktree configuration is bogus. If so, and if later on some caller tries to setup the worktree, then we'll die instead. This is indeed information that we can move into the repository itself. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-04-10hook: add hook.<event>.enabled switchAdrian Ratiu1-0/+4
Add a hook.<event>.enabled config key that disables all hooks for a given event, when set to false, acting as a high-level switch above the existing per-hook hook.<friendly-name>.enabled. Event-disabled hooks are shown in "git hook list" with an "event-disabled" tab-separated prefix before the name: $ git hook list test-hook event-disabled hook-1 event-disabled hook-2 With --show-scope: $ git hook list --show-scope test-hook local event-disabled hook-1 When a hook is both per-hook disabled and event-disabled, only "event-disabled" is shown: the event-level switch is the more relevant piece of information, and the per-hook "disabled" status will surface once the event is re-enabled. Using an event name as a friendly-name (e.g. hook.<event>.enabled) can cause ambiguity, so a fatal error is issued when using a known event name and a warning is issued for unknown event name, since a collision cannot be detected with certainty for unknown events. Suggested-by: Patrick Steinhardt <ps@pks.im> Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-04-10hook: add per-event jobs configAdrian Ratiu1-0/+3
Add a hook.<event>.jobs count config that allows users to override the global hook.jobs setting for specific hook events. This allows finer-grained control over parallelism on a per-event basis. For example, to run `post-receive` hooks with up to 4 parallel jobs while keeping other events at their global default: [hook] post-receive.jobs = 4 Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-04-10hook: parse the hook.jobs configAdrian Ratiu1-0/+3
The hook.jobs config is a global way to set hook parallelization for all hooks, in the sense that it is not per-event nor per-hook. Finer-grained configs will be added in later commits which can override it, for e.g. via a per-event type job options. Next commits will also add to this item's documentation. Parse hook.jobs config key in hook_config_lookup_all() and store its value in hook_all_config_cb.jobs, then transfer it into r->jobs after the config pass completes. This is mostly plumbing and the cached value is not yet used. Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-12Merge branch 'bc/sha1-256-interop-02'Junio C Hamano1-2/+2
The code to maintain mapping between object names in multiple hash functions is being added, written in Rust. * bc/sha1-256-interop-02: object-file-convert: always make sure object ID algo is valid rust: add a small wrapper around the hashfile code rust: add a new binary object map format rust: add functionality to hash an object rust: add a build.rs script for tests rust: fix linking binaries with cargo hash: expose hash context functions to Rust write-or-die: add an fsync component for the object map csum-file: define hashwrite's count as a uint32_t rust: add additional helpers for ObjectID hash: add a function to look up hash algo structs rust: add a hash algorithm abstraction rust: add a ObjectID struct hash: use uint32_t for object_id algorithm conversion: don't crash when no destination algo repository: require Rust support for interoperability
2026-03-10Merge branch 'ar/config-hooks'Junio C Hamano1-0/+6
Allow hook commands to be defined (possibly centrally) in the configuration files, and run multiple of them for the same hook event. * ar/config-hooks: hook: add -z option to "git hook list" hook: allow out-of-repo 'git hook' invocations hook: allow event = "" to overwrite previous values hook: allow disabling config hooks hook: include hooks from the config hook: add "git hook list" command hook: run a list of hooks to prepare for multihook support hook: add internal state alloc/free callbacks
2026-03-05Merge branch 'ob/core-attributesfile-in-repository'Junio C Hamano1-0/+7
The core.attributesfile is intended to be set per repository, but were kept track of by a single global variable in-core, which has been corrected by moving it to per-repository data structure. * ob/core-attributesfile-in-repository: environment: move "branch.autoSetupMerge" into `struct repo_config_values` environment: stop using core.sparseCheckout globally environment: stop storing `core.attributesFile` globally
2026-03-04Merge branch 'kn/ref-location'Junio C Hamano1-1/+7
Allow the directory in which reference backends store their data to be specified. * kn/ref-location: refs: add GIT_REFERENCE_BACKEND to specify reference backend refs: allow reference location in refstorage config refs: receive and use the reference storage payload refs: move out stub modification to generic layer refs: extract out `refs_create_refdir_stubs()` setup: don't modify repo in `create_reference_database()`
2026-02-25refs: allow reference location in refstorage configKarthik Nayak1-1/+7
The 'extensions.refStorage' config is used to specify the reference backend for a given repository. Both the 'files' and 'reftable' backends utilize the $GIT_DIR as the reference folder by default in `get_main_ref_store()`. Since the reference backends are pluggable, this means that they could work with out-of-tree reference directories too. Extend the 'refStorage' config to also support taking an URI input, where users can specify the reference backend and the location. Add the required changes to obtain and propagate this value to the individual backends. Add the necessary documentation and tests. Traditionally, for linked worktrees, references were stored in the '$GIT_DIR/worktrees/<wt_id>' path. But when using an alternate reference storage path, it doesn't make sense to store the main worktree references in the new path, and the linked worktree references in the $GIT_DIR. So, let's store linked worktree references in '$ALTERNATE_REFERENCE_DIR/worktrees/<wt_id>'. To do this, create the necessary files and folders while also adding stubs in the $GIT_DIR path to ensure that it is still considered a Git directory. Ideally, we would want to pass in a `struct worktree *` to individual backends, instead of passing the `gitdir`. This allows them to handle worktree specific logic. Currently, that is not possible since the worktree code is: - Tied to using the global `the_repository` variable. - Is not setup before the reference database during initialization of the repository. Add a TODO in 'refs.c' to ensure we can eventually make that change. Helped-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Karthik Nayak <karthik.188@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-19hook: include hooks from the configAdrian Ratiu1-0/+6
Teach the hook.[hc] library to parse configs to populate the list of hooks to run for a given event. Multiple commands can be specified for a given hook by providing "hook.<friendly-name>.command = <path-to-hook>" and "hook.<friendly-name>.event = <hook-event>" lines. Hooks will be started in config order of the "hook.<name>.event" lines and will be run sequentially (.jobs == 1) like before. Running the hooks in parallel will be enabled in a future patch. The "traditional" hook from the hookdir is run last, if present. A strmap cache is added to struct repository to avoid re-reading the configs on each rook run. This is useful for hooks like the ref-transaction which gets executed multiple times per process. Examples: $ git config --get-regexp "^hook\." hook.bar.command=~/bar.sh hook.bar.event=pre-commit # Will run ~/bar.sh, then .git/hooks/pre-commit $ git hook run pre-commit Signed-off-by: Emily Shaffer <emilyshaffer@google.com> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-17environment: stop storing `core.attributesFile` globallyOlamide Caleb Bello1-0/+7
The `core.attributeFile` config value is parsed in git_default_core_config(), loaded eagerly and stored in the global variable `git_attributes_file`. Storing this value in a global variable can lead to it being overwritten by another repository when more than one Git repository run in the same Git process. Create a new struct `repo_config_values` to hold this value and other repository dependent values parsed by `git_default_config()`. This will ensure the current behaviour remains the same while also enabling the libification of Git. An accessor function 'repo_config_values()' s created to ensure that we do not access an uninitialized repository, or an instance of a different repository than the current one. Suggested-by: Phillip Wood <phillip.wood123@gmail.com> Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Usman Akinyemi <usmanakinyemi202@gmail.com> Helped-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Olamide Caleb Bello <belkid98@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-07hash: use uint32_t for object_id algorithmbrian m. carlson1-2/+2
We currently use an int for this value, but we'll define this structure from Rust in a future commit and we want to ensure that our data types are exactly identical. To make that possible, use a uint32_t for the hash algorithm. Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-05Merge branch 'ar/submodule-gitdir-tweak'Junio C Hamano1-0/+1
Avoid local submodule repository directory paths overlapping with each other by encoding submodule names before using them as path components. * ar/submodule-gitdir-tweak: submodule: detect conflicts with existing gitdir configs submodule: hash the submodule name for the gitdir path submodule: fix case-folding gitdir filesystem collisions submodule--helper: fix filesystem collisions by encoding gitdir paths builtin/credential-store: move is_rfc3986_unreserved to url.[ch] submodule--helper: add gitdir migration command submodule: allow runtime enabling extensions.submodulePathConfig submodule: introduce extensions.submodulePathConfig builtin/submodule--helper: add gitdir command submodule: always validate gitdirs inside submodule_name_to_gitdir submodule--helper: use submodule_name_to_gitdir in add_submodule
2026-01-12submodule: introduce extensions.submodulePathConfigAdrian Ratiu1-0/+1
The idea of this extension is to abstract away the submodule gitdir path implementation: everyone is expected to use the config and not worry about how the path is computed internally, either in git or other implementations. With this extension enabled, the submodule.<name>.gitdir repo config becomes the single source of truth for all submodule gitdir paths. The submodule.<name>.gitdir config is added automatically for all new submodules when this extension is enabled. Git will throw an error if the extension is enabled and a config is missing, advising users how to migrate. Migration is manual for now. E.g. to add a missing config entry for an existing "foo" module: git config submodule.foo.gitdir .git/modules/foo Suggested-by: Junio C Hamano <gitster@pobox.com> Suggested-by: Phillip Wood <phillip.wood123@gmail.com> Suggested-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Adrian Ratiu <adrian.ratiu@collabora.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-25odb: handle changing a repository's commondirPatrick Steinhardt1-0/+1
The function `repo_set_gitdir()` is called in two situations: - To initialize the repository with its discovered location. As part of this we also set up the new object database. - To update the repository's discovered location in case the process changes its working directory so that we update relative paths. This means we also have to update any relative paths that are potentially used in the object database. In the context of the object database we ideally wouldn't ever have to worry about the second case: if all paths used by our object database sources were absolute, then we wouldn't have to update them. But unfortunately, the paths aren't only used to locate files owned by the given source, but we also use them for reporting purposes. One such example is `repo_get_object_directory()`, where we cannot just change semantics to always return absolute paths, as that is likely to break tooling out there. One solution to this would be to have both a "display path" and an "internal path". This would allow us to use internal paths for all internal matters, but continue to use the potentially-relative display paths so that we don't break compatibility. But converting the codebase to honor this split is quite a messy endeavour, and it wouldn't even help us with the goal to get rid of the need to update the display path on chdir(3p). Another solution would be to rework "setup.c" so that we never have to update paths in the first place. In that case, we'd only initialize the repository once we have figured out final locations for all directories. This would be a significant simplification of that subsystem indeed, but the current logic is so messy that it would take significant investments to get there. Meanwhile though, while object sources may still use relative paths, the best thing we can do is to handle the reparenting of the object source paths in the object database itself. This can be done by registering one callback for each object database so that we get notified whenever the current working directory changes, and we then perform the reparenting ourselves. Ideally, this wouldn't even happen on the object database level, but instead handled by each object database source. But we don't yet have proper pluggable object database sources, so this will need to be handled at a later point in time. The logic itself is rather simple: - We register the callback when creating the object database. - We unregister the callback when releasing it again. - We split up `set_git_dir_1()` so that it becomes possible to skip recreating the object database. This is required because the function is called both when the current working directory changes, but also when we set up the repository. Calling this function without skipping creation of the ODB will result in a bug in case it's already created. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-25odb: move logic to disable ref updates into repoPatrick Steinhardt1-1/+8
Our object database sources have a field `disable_ref_updates`. This field can obviously be set to disable reference updates, but it is somewhat curious that this logic is hosted by the object database. The reason for this is that it was primarily added to keep us from accidentally updating references while an ODB transaction is ongoing. Any objects part of the transaction have not yet been committed to disk, so new references that point to them might get corrupted in case we never end up committing the transaction. As such, whenever we create a new transaction we set up a new temporary ODB source and mark it as disabling reference updates. This has one (and only one?) upside: once we have committed the transaction, the temporary source will be dropped and thus we clean up the disabled reference updates automatically. But other than that, it's somewhat misdesigned: - We can have multiple ODB sources, but only the currently active source inhibits reference updates. - We're mixing concerns of the refdb with the ODB. Arguably, the decision of whether we can update references or not should be handled by the refdb. But that wouldn't be a great fit either, as there can be one refdb per worktree. So we'd again have the same problem that a "global" intent becomes localized to a specific instance. Instead, move the setting into the repository. While at it, convert it into a boolean. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-08-26config: warn on core.commentString=autoPhillip Wood1-0/+3
As support for this setting was deprecated in the last commit print a warning (or die when WITH_BREAKING_CHANGES is enabled) if it is set. Avoid bombarding the user with warnings by only printing it (a) when running commands that call "git commit" and (b) only once per command. Some scaffolding is added to repo_read_config() to allow it to detect deprecated config settings and warn about them. As both "core.commentChar" and "core.commentString" set the comment character we record which one of them is used and tailor the warning message appropriately. Note the odd combination of die_message() followed by die(NULL) is to allow the next commit to insert a call to advise() in the middle. Signed-off-by: Phillip Wood <phillip.wood@dunelm.org.uk> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-15Merge branch 'ps/object-store'Junio C Hamano1-2/+2
Code clean-up around object access API. * ps/object-store: odb: rename `read_object_with_reference()` odb: rename `pretend_object_file()` odb: rename `has_object()` odb: rename `repo_read_object_file()` odb: rename `oid_object_info()` odb: trivial refactorings to get rid of `the_repository` odb: get rid of `the_repository` when handling submodule sources odb: get rid of `the_repository` when handling the primary source odb: get rid of `the_repository` in `for_each()` functions odb: get rid of `the_repository` when handling alternates odb: get rid of `the_repository` in `odb_mkstemp()` odb: get rid of `the_repository` in `assert_oid_type()` odb: get rid of `the_repository` in `find_odb()` odb: introduce parent pointers object-store: rename files to "odb.{c,h}" object-store: rename `object_directory` to `odb_source` object-store: rename `raw_object_store` to `object_database`
2025-07-14Merge branch 'ps/use-reftable-as-default-in-3.0'Junio C Hamano1-0/+6
The reftable ref backend has matured enough; Git 3.0 will make it the default format in a newly created repositories by default. * ps/use-reftable-as-default-in-3.0: setup: use "reftable" format when experimental features are enabled BreakingChanges: announce switch to "reftable" format
2025-07-07repository: move 'repository_format_precious_objects' to repo scopeAyush Chandekar1-0/+1
The 'extensions.preciousObjects' setting when set true, prevents operations that might drop objects from the object storage. This setting is populated in the global variable 'repository_format_precious_objects'. Move this global variable to repo scope by adding it to 'struct repository and also refactor all the occurences accordingly. This change is part of an ongoing effort to eliminate global variables, improve modularity and help libify the codebase. Mentored-by: Christian Couder <christian.couder@gmail.com> Mentored-by: Ghanshyam Thakkar <shyamthakkar001@gmail.com> Signed-off-by: Ayush Chandekar <ayu.chandekar@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-07BreakingChanges: announce switch to "reftable" formatPatrick Steinhardt1-0/+6
The "reftable" format has come a long way and has matured nicely since it has been merged into git via 57db2a094d5 (refs: introduce reftable backend, 2024-02-07). It fixes longstanding issues that cannot be fixed with the "files" format in a backwards-compatible way and performs significantly better in many use cases. Announce that we will switch to the "reftable" format in Git 3.0 for newly created repositories and wire up the change, hidden behind the WITH_BREAKING_CHANGES preprocessor define. This switch is dependent on support in the larger Git ecosystem. Most importantly, libraries like JGit, libgit2 and Gitoxide should support the reftable backend so that we don't break all applications and tools built on top of those libraries. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-07-01object-store: rename `raw_object_store` to `object_database`Patrick Steinhardt1-2/+2
The `raw_object_store` structure is the central entry point for reading and writing objects in a repository. The main purpose of this structure is to manage object directories and provide an interface to access and write objects in those object directories. Right now, many of the functions associated with the raw object store implicitly rely on `the_repository` to get access to its `objects` pointer, which is the `raw_object_store`. As we want to generally get rid of using `the_repository` across our codebase we will have to convert this implicit dependency on this global variable into an explicit parameter. This conversion can be done by simply passing in an explicit pointer to a repository and then using its `->objects` pointer. But there is a second effort underway, which is to make the object subsystem more selfcontained so that we can eventually have pluggable object backends. As such, passing in a repository wouldn't make a ton of sense, and the goal is to convert the object store interfaces such that we always pass in a reference to the `raw_object_store` instead. This will expose the `raw_object_store` type to a lot more callers though, which surfaces that this type is named somewhat awkwardly. The "raw_" prefix makes readers wonder whether there is a non-raw variant of the object store, but there isn't. Furthermore, we nowadays want to name functions in a way that they can be clearly attributed to a specific subsystem, but calling them e.g. `raw_object_store_has_object()` is just too unwieldy, even when dropping the "raw_" prefix. Instead, rename the structure to `object_database`. This term is already used a lot throughout our codebase, and it cannot easily be mistaken for "object directories", either. Furthermore, its acronym ODB is already well-known and works well as part of a function's name, like for example `odb_has_object()`. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-12-02worktree: add `relativeWorktrees` extensionCaleb White1-0/+1
A new extension, `relativeWorktrees`, is added to indicate that at least one worktree in the repository has been linked with relative paths. This ensures older Git versions do not attempt to automatically prune worktrees with relative paths, as they would not not recognize the paths as being valid. Suggested-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Caleb White <cdwhite3@pm.me> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-12repo-settings: split out declarations into a standalone headerPatrick Steinhardt1-50/+1
While we have "repo-settings.c", we do not have a corresponding "repo-settings.h" file. Instead, this functionality is part of the "repository.h" header, making it hard to discover. Split the declarations out of "repository.h" and create a standalone header file with them. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-12environment: make `get_git_work_tree()` accept a repositoryPatrick Steinhardt1-0/+1
The `get_git_work_tree()` function retrieves the path of the work tree of `the_repository`. Make it accept a `struct repository` such that it can work on arbitrary repositories and make it part of the repository subsystem. This reduces our reliance on `the_repository` and clarifies scope. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-12environment: make `get_graft_file()` accept a repositoryPatrick Steinhardt1-0/+1
The `get_graft_file()` function retrieves the path to the graft file of `the_repository`. Make it accept a `struct repository` such that it can work on arbitrary repositories and make it part of the repository subsystem. This reduces our reliance on `the_repository` and clarifies scope. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-12environment: make `get_index_file()` accept a repositoryPatrick Steinhardt1-0/+1
The `get_index_file()` function retrieves the path to the index file of `the_repository`. Make it accept a `struct repository` such that it can work on arbitrary repositories and make it part of the repository subsystem. This reduces our reliance on `the_repository` and clarifies scope. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-12environment: make `get_object_directory()` accept a repositoryPatrick Steinhardt1-0/+1
The `get_object_directory()` function retrieves the path to the object directory for `the_repository`. Make it accept a `struct repository` such that it can work on arbitrary repositories and make it part of the repository subsystem. This reduces our reliance on `the_repository` and clarifies scope. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-12environment: make `get_git_common_dir()` accept a repositoryPatrick Steinhardt1-0/+1
The `get_git_common_dir()` function retrieves the path to the common directory for `the_repository`. Make it accept a `struct repository` such that it can work on arbitrary repositories and make it part of the repository subsystem. This reduces our reliance on `the_repository` and clarifies scope. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-09-12environment: make `get_git_dir()` accept a repositoryPatrick Steinhardt1-0/+2
The `get_git_dir()` function retrieves the path to the Git directory for `the_repository`. Make it accept a `struct repository` such that it can work on arbitrary repositories and make it part of the repository subsystem. This reduces our reliance on `the_repository` and clarifies scope. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-07-08Merge branch 'tb/path-filter-fix'Junio C Hamano1-1/+1
The Bloom filter used for path limited history traversal was broken on systems whose "char" is unsigned; update the implementation and bump the format version to 2. * tb/path-filter-fix: bloom: introduce `deinit_bloom_filters()` commit-graph: reuse existing Bloom filters where possible object.h: fix mis-aligned flag bits table commit-graph: new Bloom filter version that fixes murmur3 commit-graph: unconditionally load Bloom filters bloom: prepare to discard incompatible Bloom filters bloom: annotate filters with hash version repo-settings: introduce commitgraph.changedPathsVersion t4216: test changed path filters with high bit paths t/helper/test-read-graph: implement `bloom-filters` mode bloom.h: make `load_bloom_filter_from_graph()` public t/helper/test-read-graph.c: extract `dump_graph_info()` gitformat-commit-graph: describe version 2 of BDAT commit-graph: ensure Bloom filters are read with consistent settings revision.c: consult Bloom filters for root commits t/t4216-log-bloom.sh: harden `test_bloom_filters_not_used()`
2024-06-25repo-settings: introduce commitgraph.changedPathsVersionTaylor Blau1-1/+1
A subsequent commit will introduce another version of the changed-path filter in the commit graph file. In order to control which version to write (and read), a config variable is needed. Therefore, introduce this config variable. For forwards compatibility, teach Git to not read commit graphs when the config variable is set to an unsupported version. Because we teach Git this, commitgraph.readChangedPaths is now redundant, so deprecate it and define its behavior in terms of the config variable we introduce. This commit does not change the behavior of writing (Git writes changed path filters when explicitly instructed regardless of any config variable), but a subsequent commit will restrict Git such that it will only write when commitgraph.changedPathsVersion is a recognized value. Signed-off-by: Jonathan Tan <jonathantanmy@google.com> Signed-off-by: Junio C Hamano <gitster@pobox.com> Signed-off-by: Taylor Blau <me@ttaylorr.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-14refs: avoid include cycle with "repository.h"Patrick Steinhardt1-1/+6
There is an include cycle between "refs.h" and "repository.h" via "commit.h", "object.h" and "hash.h". This has the effect that several definitions of structs and enums will not be visible once we merge "hash-ll.h" back into "hash.h" in the next commit. The only reason that "repository.h" includes "refs.h" is the definition of `enum ref_storage_format`. Move it into "repository.h" and have "refs.h" include "repository.h" instead to fix the cycle. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-14global: introduce `USE_THE_REPOSITORY_VARIABLE` macroPatrick Steinhardt1-0/+2
Use of the `the_repository` variable is deprecated nowadays, and we slowly but steadily convert the codebase to not use it anymore. Instead, callers should be passing down the repository to work on via parameters. It is hard though to prove that a given code unit does not use this variable anymore. The most trivial case, merely demonstrating that there is no direct use of `the_repository`, is already a bit of a pain during code reviews as the reviewer needs to manually verify claims made by the patch author. The bigger problem though is that we have many interfaces that implicitly rely on `the_repository`. Introduce a new `USE_THE_REPOSITORY_VARIABLE` macro that allows code units to opt into usage of `the_repository`. The intent of this macro is to demonstrate that a certain code unit does not use this variable anymore, and to keep it from new dependencies on it in future changes, be it explicit or implicit For now, the macro only guards `the_repository` itself as well as `the_hash_algo`. There are many more known interfaces where we have an implicit dependency on `the_repository`, but those are not guarded at the current point in time. Over time though, we should start to add guards as required (or even better, just remove them). Define the macro as required in our code units. As expected, most of our code still relies on the global variable. Nearly all of our builtins rely on the variable as there is no way yet to pass `the_repository` to their entry point. For now, declare the macro in "biultin.h" to keep the required changes at least a little bit more contained. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-06-06refs: convert ref storage format to an enumPatrick Steinhardt1-6/+4
The ref storage format is tracked as a simple unsigned integer, which makes it harder than necessary to discover what that integer actually is or where its values are defined. Convert the ref storage format to instead be an enum. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-17refs: retrieve worktree ref stores via associated repositoryPatrick Steinhardt1-0/+6
Similar as with the preceding commit, the worktree ref stores are always looked up via `the_repository`. Also, again, those ref stores are stored in a global map. Refactor the code so that worktrees have a pointer to their repository. Like this, we can move the global map into `struct repository` and stop using `the_repository`. With this change, we can now in theory look up worktree ref stores for repositories other than `the_repository`. In practice, the worktree code will need further changes to look up arbitrary worktrees. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-05-17refs: pass repo when retrieving submodule ref storePatrick Steinhardt1-0/+8
Looking up submodule ref stores has two deficiencies: - The initialized subrepo will be attributed to `the_repository`. - The submodule ref store will be tracked in a global map. This makes it impossible to have submodule ref stores for a repository other than `the_repository`. Modify the function to accept the parent repository as parameter and move the global map into `struct repository`. Like this it becomes possible to look up submodule ref stores for arbitrary repositories. Note that this also adds a new reference to `the_repository` in `resolve_gitlink_ref()`, which is part of the refs interfaces. This will get adjusted in the next patch. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-18repository: drop `initialize_the_repository()`Patrick Steinhardt1-1/+1
Now that we have dropped `the_index`, `initialize_the_repository()` doesn't really do a lot anymore except for setting up the pointer for `the_repository` and then calling `initialize_repository()`. The former can be replaced by statically initializing the pointer though, which basically makes this function moot. Convert callers to instead call `initialize_repository(the_repository)` and drop `initialize_thee_repository()`. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-04-18repository: drop `the_index` variablePatrick Steinhardt1-3/+0
All users of `the_index` have been converted to use either a custom `struct index_state *` or the index provided by `the_repository`. We can thus drop the globally-accessible declaration of this variable. In fact, we can go further than that and drop `the_index` completely now and have it be allocated dynamically in `initialize_repository()` as all the other data structures in it are. This concludes the quest to make Git `the_index` free, which has started with 4aab5b46f4 (Make read-cache.c "the_index" free., 2007-04-01). Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2024-03-28Merge branch 'eb/hash-transition'Junio C Hamano1-0/+4
Work to support a repository that work with both SHA-1 and SHA-256 hash algorithms has started. * eb/hash-transition: (30 commits) t1016-compatObjectFormat: add tests to verify the conversion between objects t1006: test oid compatibility with cat-file t1006: rename sha1 to oid test-lib: compute the compatibility hash so tests may use it builtin/ls-tree: let the oid determine the output algorithm object-file: handle compat objects in check_object_signature tree-walk: init_tree_desc take an oid to get the hash algorithm builtin/cat-file: let the oid determine the output algorithm rev-parse: add an --output-object-format parameter repository: implement extensions.compatObjectFormat object-file: update object_info_extended to reencode objects object-file-convert: convert commits that embed signed tags object-file-convert: convert commit objects when writing object-file-convert: don't leak when converting tag objects object-file-convert: convert tag objects when writing object-file-convert: add a function to convert trees between algorithms object: factor out parse_mode out of fast-import and tree-walk into in object.h cache: add a function to read an OID of a specific algorithm tag: sign both hashes commit: export add_header_signature to support handling signatures on tags ...
2024-02-26Merge branch 'ps/reftable-backend'Junio C Hamano1-2/+3
Integrate the reftable code into the refs framework as a backend. * ps/reftable-backend: refs/reftable: fix leak when copying reflog fails ci: add jobs to test with the reftable backend refs: introduce reftable backend
2024-02-07refs: introduce reftable backendPatrick Steinhardt1-2/+3
Due to scalability issues, Shawn Pearce has originally proposed a new "reftable" format more than six years ago [1]. Initially, this new format was implemented in JGit with promising results. Around two years ago, we have then added the "reftable" library to the Git codebase via a4bbd13be3 (Merge branch 'hn/reftable', 2021-12-15). With this we have landed all the low-level code to read and write reftables. Notably missing though was the integration of this low-level code into the Git code base in the form of a new ref backend that ties all of this together. This gap is now finally closed by introducing a new "reftable" backend into the Git codebase. This new backend promises to bring some notable improvements to Git repositories: - It becomes possible to do truly atomic writes where either all refs are committed to disk or none are. This was not possible with the "files" backend because ref updates were split across multiple loose files. - The disk space required to store many refs is reduced, both compared to loose refs and packed-refs. This is enabled both by the reftable format being a binary format, which is more compact, and by prefix compression. - We can ignore filesystem-specific behaviour as ref names are not encoded via paths anymore. This means there is no need to handle case sensitivity on Windows systems or Unicode precomposition on macOS. - There is no need to rewrite the complete refdb anymore every time a ref is being deleted like it was the case for packed-refs. This means that ref deletions are now constant time instead of scaling linearly with the number of refs. - We can ignore file/directory conflicts so that it becomes possible to store both "refs/heads/foo" and "refs/heads/foo/bar". - Due to this property we can retain reflogs for deleted refs. We have previously been deleting reflogs together with their refs to avoid file/directory conflicts, which is not necessary anymore. - We can properly enumerate all refs. With the "files" backend it is not easily possible to distinguish between refs and non-refs because they may live side by side in the gitdir. Not all of these improvements are realized with the current "reftable" backend implementation. At this point, the new backend is supposed to be a drop-in replacement for the "files" backend that is used by basically all Git repositories nowadays. It strives for 1:1 compatibility, which means that a user can expect the same behaviour regardless of whether they use the "reftable" backend or the "files" backend for most of the part. Most notably, this means we artificially limit the capabilities of the "reftable" backend to match the limits of the "files" backend. It is not possible to create refs that would end up with file/directory conflicts, we do not retain reflogs, we perform stricter-than-necessary checks. This is done intentionally due to two main reasons: - It makes it significantly easier to land the "reftable" backend as tests behave the same. It would be tough to argue for each and every single test that doesn't pass with the "reftable" backend. - It ensures compatibility between repositories that use the "files" backend and repositories that use the "reftable" backend. Like this, hosters can migrate their repositories to use the "reftable" backend without causing issues for clients that use the "files" backend in their clones. It is expected that these artificial limitations may eventually go away in the long term. Performance-wise things very much depend on the actual workload. The following benchmarks compare the "files" and "reftable" backends in the current version: - Creating N refs in separate transactions shows that the "files" backend is ~50% faster. This is not surprising given that creating a ref only requires us to create a single loose ref. The "reftable" backend will also perform auto compaction on updates. In real-world workloads we would likely also want to perform pack loose refs, which would likely change the picture. Benchmark 1: update-ref: create refs sequentially (refformat = files, refcount = 1) Time (mean ± σ): 2.1 ms ± 0.3 ms [User: 0.6 ms, System: 1.7 ms] Range (min … max): 1.8 ms … 4.3 ms 133 runs Benchmark 2: update-ref: create refs sequentially (refformat = reftable, refcount = 1) Time (mean ± σ): 2.7 ms ± 0.1 ms [User: 0.6 ms, System: 2.2 ms] Range (min … max): 2.4 ms … 2.9 ms 132 runs Benchmark 3: update-ref: create refs sequentially (refformat = files, refcount = 1000) Time (mean ± σ): 1.975 s ± 0.006 s [User: 0.437 s, System: 1.535 s] Range (min … max): 1.969 s … 1.980 s 3 runs Benchmark 4: update-ref: create refs sequentially (refformat = reftable, refcount = 1000) Time (mean ± σ): 2.611 s ± 0.013 s [User: 0.782 s, System: 1.825 s] Range (min … max): 2.597 s … 2.622 s 3 runs Benchmark 5: update-ref: create refs sequentially (refformat = files, refcount = 100000) Time (mean ± σ): 198.442 s ± 0.241 s [User: 43.051 s, System: 155.250 s] Range (min … max): 198.189 s … 198.670 s 3 runs Benchmark 6: update-ref: create refs sequentially (refformat = reftable, refcount = 100000) Time (mean ± σ): 294.509 s ± 4.269 s [User: 104.046 s, System: 190.326 s] Range (min … max): 290.223 s … 298.761 s 3 runs - Creating N refs in a single transaction shows that the "files" backend is significantly slower once we start to write many refs. The "reftable" backend only needs to update two files, whereas the "files" backend needs to write one file per ref. Benchmark 1: update-ref: create many refs (refformat = files, refcount = 1) Time (mean ± σ): 1.9 ms ± 0.1 ms [User: 0.4 ms, System: 1.4 ms] Range (min … max): 1.8 ms … 2.6 ms 151 runs Benchmark 2: update-ref: create many refs (refformat = reftable, refcount = 1) Time (mean ± σ): 2.5 ms ± 0.1 ms [User: 0.7 ms, System: 1.7 ms] Range (min … max): 2.4 ms … 3.4 ms 148 runs Benchmark 3: update-ref: create many refs (refformat = files, refcount = 1000) Time (mean ± σ): 152.5 ms ± 5.2 ms [User: 19.1 ms, System: 133.1 ms] Range (min … max): 148.5 ms … 167.8 ms 15 runs Benchmark 4: update-ref: create many refs (refformat = reftable, refcount = 1000) Time (mean ± σ): 58.0 ms ± 2.5 ms [User: 28.4 ms, System: 29.4 ms] Range (min … max): 56.3 ms … 72.9 ms 40 runs Benchmark 5: update-ref: create many refs (refformat = files, refcount = 1000000) Time (mean ± σ): 152.752 s ± 0.710 s [User: 20.315 s, System: 131.310 s] Range (min … max): 152.165 s … 153.542 s 3 runs Benchmark 6: update-ref: create many refs (refformat = reftable, refcount = 1000000) Time (mean ± σ): 51.912 s ± 0.127 s [User: 26.483 s, System: 25.424 s] Range (min … max): 51.769 s … 52.012 s 3 runs - Deleting a ref in a fully-packed repository shows that the "files" backend scales with the number of refs. The "reftable" backend has constant-time deletions. Benchmark 1: update-ref: delete ref (refformat = files, refcount = 1) Time (mean ± σ): 1.7 ms ± 0.1 ms [User: 0.4 ms, System: 1.2 ms] Range (min … max): 1.6 ms … 2.1 ms 316 runs Benchmark 2: update-ref: delete ref (refformat = reftable, refcount = 1) Time (mean ± σ): 1.8 ms ± 0.1 ms [User: 0.4 ms, System: 1.3 ms] Range (min … max): 1.7 ms … 2.1 ms 294 runs Benchmark 3: update-ref: delete ref (refformat = files, refcount = 1000) Time (mean ± σ): 2.0 ms ± 0.1 ms [User: 0.5 ms, System: 1.4 ms] Range (min … max): 1.9 ms … 2.5 ms 287 runs Benchmark 4: update-ref: delete ref (refformat = reftable, refcount = 1000) Time (mean ± σ): 1.9 ms ± 0.1 ms [User: 0.5 ms, System: 1.3 ms] Range (min … max): 1.8 ms … 2.1 ms 217 runs Benchmark 5: update-ref: delete ref (refformat = files, refcount = 1000000) Time (mean ± σ): 229.8 ms ± 7.9 ms [User: 182.6 ms, System: 46.8 ms] Range (min … max): 224.6 ms … 245.2 ms 6 runs Benchmark 6: update-ref: delete ref (refformat = reftable, refcount = 1000000) Time (mean ± σ): 2.0 ms ± 0.0 ms [User: 0.6 ms, System: 1.3 ms] Range (min … max): 2.0 ms … 2.1 ms 3 runs - Listing all refs shows no significant advantage for either of the backends. The "files" backend is a bit faster, but not by a significant margin. When repositories are not packed the "reftable" backend outperforms the "files" backend because the "reftable" backend performs auto-compaction. Benchmark 1: show-ref: print all refs (refformat = files, refcount = 1, packed = true) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 2.0 ms 1729 runs Benchmark 2: show-ref: print all refs (refformat = reftable, refcount = 1, packed = true) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 1.8 ms 1816 runs Benchmark 3: show-ref: print all refs (refformat = files, refcount = 1000, packed = true) Time (mean ± σ): 4.3 ms ± 0.1 ms [User: 0.9 ms, System: 3.3 ms] Range (min … max): 4.1 ms … 4.6 ms 645 runs Benchmark 4: show-ref: print all refs (refformat = reftable, refcount = 1000, packed = true) Time (mean ± σ): 4.5 ms ± 0.2 ms [User: 1.0 ms, System: 3.3 ms] Range (min … max): 4.2 ms … 5.9 ms 643 runs Benchmark 5: show-ref: print all refs (refformat = files, refcount = 1000000, packed = true) Time (mean ± σ): 2.537 s ± 0.034 s [User: 0.488 s, System: 2.048 s] Range (min … max): 2.511 s … 2.627 s 10 runs Benchmark 6: show-ref: print all refs (refformat = reftable, refcount = 1000000, packed = true) Time (mean ± σ): 2.712 s ± 0.017 s [User: 0.653 s, System: 2.059 s] Range (min … max): 2.692 s … 2.752 s 10 runs Benchmark 7: show-ref: print all refs (refformat = files, refcount = 1, packed = false) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 1.9 ms 1834 runs Benchmark 8: show-ref: print all refs (refformat = reftable, refcount = 1, packed = false) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.4 ms … 2.0 ms 1840 runs Benchmark 9: show-ref: print all refs (refformat = files, refcount = 1000, packed = false) Time (mean ± σ): 13.8 ms ± 0.2 ms [User: 2.8 ms, System: 10.8 ms] Range (min … max): 13.3 ms … 14.5 ms 208 runs Benchmark 10: show-ref: print all refs (refformat = reftable, refcount = 1000, packed = false) Time (mean ± σ): 4.5 ms ± 0.2 ms [User: 1.2 ms, System: 3.3 ms] Range (min … max): 4.3 ms … 6.2 ms 624 runs Benchmark 11: show-ref: print all refs (refformat = files, refcount = 1000000, packed = false) Time (mean ± σ): 12.127 s ± 0.129 s [User: 2.675 s, System: 9.451 s] Range (min … max): 11.965 s … 12.370 s 10 runs Benchmark 12: show-ref: print all refs (refformat = reftable, refcount = 1000000, packed = false) Time (mean ± σ): 2.799 s ± 0.022 s [User: 0.735 s, System: 2.063 s] Range (min … max): 2.769 s … 2.836 s 10 runs - Printing a single ref shows no real difference between the "files" and "reftable" backends. Benchmark 1: show-ref: print single ref (refformat = files, refcount = 1) Time (mean ± σ): 1.5 ms ± 0.1 ms [User: 0.4 ms, System: 1.0 ms] Range (min … max): 1.4 ms … 1.8 ms 1779 runs Benchmark 2: show-ref: print single ref (refformat = reftable, refcount = 1) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.4 ms … 2.5 ms 1753 runs Benchmark 3: show-ref: print single ref (refformat = files, refcount = 1000) Time (mean ± σ): 1.5 ms ± 0.1 ms [User: 0.3 ms, System: 1.1 ms] Range (min … max): 1.4 ms … 1.9 ms 1840 runs Benchmark 4: show-ref: print single ref (refformat = reftable, refcount = 1000) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 2.0 ms 1831 runs Benchmark 5: show-ref: print single ref (refformat = files, refcount = 1000000) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 2.1 ms 1848 runs Benchmark 6: show-ref: print single ref (refformat = reftable, refcount = 1000000) Time (mean ± σ): 1.6 ms ± 0.1 ms [User: 0.4 ms, System: 1.1 ms] Range (min … max): 1.5 ms … 2.1 ms 1762 runs So overall, performance depends on the usecases. Except for many sequential writes the "reftable" backend is roughly on par or significantly faster than the "files" backend though. Given that the "files" backend has received 18 years of optimizations by now this can be seen as a win. Furthermore, we can expect that the "reftable" backend will grow faster over time when attention turns more towards optimizations. The complete test suite passes, except for those tests explicitly marked to require the REFFILES prerequisite. Some tests in t0610 are marked as failing because they depend on still-in-flight bug fixes. Tests can be run with the new backend by setting the GIT_TEST_DEFAULT_REF_FORMAT environment variable to "reftable". There is a single known conceptual incompatibility with the dumb HTTP transport. As "info/refs" SHOULD NOT contain the HEAD reference, and because the "HEAD" file is not valid anymore, it is impossible for the remote client to figure out the default branch without changing the protocol. This shortcoming needs to be handled in a subsequent patch series. As the reftable library has already been introduced a while ago, this commit message will not go into the details of how exactly the on-disk format works. Please refer to our preexisting technical documentation at Documentation/technical/reftable for this. [1]: https://public-inbox.org/git/CAJo=hJtyof=HRy=2sLP0ng0uZ4=S-DpZ5dR1aF+VHVETKG20OQ@mail.gmail.com/ Original-idea-by: Shawn Pearce <spearce@spearce.org> Based-on-patch-by: Han-Wen Nienhuys <hanwen@google.com> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>