summaryrefslogtreecommitdiff
path: root/object-file.h
AgeCommit message (Collapse)AuthorFilesLines
2026-08-05odb/streaming: rename `struct odb_read_stream`Patrick Steinhardt1-1/+1
Rename `struct odb_read_stream` to just `struct odb_stream`. This prepares for unification of the two different types of streams, as these provide the same functionality with the preceding refactorings. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-18object-file: move logic to write loose objectsPatrick Steinhardt1-18/+4
The logic to write loose objects is split up across "object-file.c" and "odb/source-loose.c". This split is somewhat weird, but it is the result of two things: - `force_object_loose()` used to reach into internals of how exactly we write objects. - The logic of writing objects is intertwined with potentially starting a transaction. We have refactored `force_object_loose()` over preceding commits to work via generic interfaces now, so this reason doesn't exist anymore. But the second reason still does, as our management of "files" transactions and their ad-hoc creation is still very messy. This area definitely requires further work, and that work is indeed ongoing. That being said, we can already move the writing logic into the "loose" backend rather easily. All we have to do is to expose two functions that relate to the transactions. Expose these two functions and move the writing logic into the "loose" backend accordingly so that it becomes more self-contained. Note that this requires us to drop a reference to `the_repository` in favor of using the source's repository in `start_loose_object_common()`. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-18object-file: move `force_object_loose()`Patrick Steinhardt1-4/+0
In the preceding commits we have refactored `force_object_loose()` to not call internal functions anymore for writing the object. Instead, it now only uses generic functions that are accessible to all callers. Consequently, we can now easily move the function to its only caller, which is git-pack-objects(1). Do so. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-18odb: support setting mtime when writing objectsPatrick Steinhardt1-3/+5
The function `force_object_loose()` is used to loosen packed objects before repacking. It passes the pack's mtime along so that the newly written loose object inherits the same timestamp. This matters for object pruning, which uses the mtime to determine whether an object is old enough to be pruned. In a subsequent commit, `force_object_loose()` will be converted to use the generic `odb_source_write_object()` interface instead of calling `write_loose_object()` directly. But the generic interface doesn't yet support setting a specific mtime, which makes it impossible to implement the logic as of now. Prepare for the change by introducing a new `mtime` parameter to this function that we plumb through the stack. If set, the backends are instructed to set the object's mtime accordingly. If unset, the backends are expected to use the current time instead. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-18odb: compute object hash in `odb_write_object_ext()`Patrick Steinhardt1-4/+0
Same as in a preceding commit, compute the object hash in `odb_write_object_ext()` so that we can unify this logic. Besides unification, this change also allows us to lift the object existence check out of the "loose" backend into the generic layer, which will happen in the next commit. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-18Merge branch 'jt/receive-pack-use-odb-transactions' into HEADJunio C Hamano1-3/+5
'git receive-pack' has been refactored to use ODB transaction interfaces instead of directly managing 'tmp_objdir' for staging incoming objects, bringing it closer to being ODB backend agnostic. * jt/receive-pack-use-odb-transactions: builtin/receive-pack: stage incoming objects via ODB transactions builtin/receive-pack: drop redundant tmpdir env odb/transaction: introduce ODB transaction flags odb/transaction: add transaction env interface odb/transaction: propagate commit errors odb/transaction: propagate begin errors object-file: propagate files transaction errors object-file: drop check for inflight transactions object-file: embed transaction flush logic in commit function object-file: rename files transaction fsync function object-file: rename files transaction prepare function
2026-07-10odb/transaction: introduce ODB transaction flagsJustin Tobler1-1/+3
The temporary directory used by git-receive-pack(1) to write objects is managed slightly differently than how it is done via ODB transactions: - The temporary directory is eagerly created upfront, instead of waiting for the first object write. - The prefix name of the temporary directory is "incoming" instead of "bulk-fsync". In a subsequent commit, git-receive-pack(1) will use ODB transactions instead of `tmp_objdir` directly. To provide a means to configure the same transaction behavior, introduce `enum odb_transaction_flags` and the ODB_TRANSACTION_RECEIVE flag intended as a signal for ODB transactions using the "files" backend to be set up for git-receive-pack(1). Transaction call sites are updated accordingly to provide the required flag parameter. Signed-off-by: Justin Tobler <jltobler@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-10object-file: propagate files transaction errorsJustin Tobler1-1/+2
The "files" transaction backend may encounter errors related to managing the temporary directory used to stage objects, but silently ignores these errors. Instead return errors encountered in the `odb_transaction_files_{prepare,begin,commit}()` interfaces to allow callers to handle them as needed. Signed-off-by: Justin Tobler <jltobler@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-07-10object-file: drop check for inflight transactionsJustin Tobler1-2/+1
ODB transactions are started via `odb_transaction_begin()` and contain validation to avoid starting multiple transactions at the same time. The "files" backend also has the same logic, but is redundant due to the generic layer already handling it. Drop this validation from the "files" backend accordingly. Signed-off-by: Justin Tobler <jltobler@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-16object-file.c: use size_t for header lengthsPhilip Oakley1-3/+3
Continue walking the code path for the >4GB `hash-object --literally` test. The `hash_object_file_literally()` function internally uses both `hash_object_file()` and `write_object_file_prepare()`. Both function signatures use `unsigned long` rather than `size_t` for the mem buffer sizes. Use `size_t` instead, for LLP64 compatibility. While at it, convert those function's object's header buffer length to `size_t` for consistency. The value is already upcast to `uintmax_t` for print format compatibility. Note: The hash-object test still does not pass. A subsequent commit continues to walk the call tree's lower level hash functions to identify further fixes. Signed-off-by: Philip Oakley <philipoakley@iee.email> Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: wire up `write_object_stream()` callbackPatrick Steinhardt1-1/+11
Wire up the `write_object_stream()` callback. Note that we don't move the implementation into "odb/source-loose.c". This is because most of the logic to write loose objects is still contained in "object-file.c", and detangling that requires us to do some refactorings as explained in the preceding commit. So for now, the implementation of writing an object stream is still located in "object-file.c". Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01object-file: refactor writing objects to use loose sourcePatrick Steinhardt1-3/+3
The "object-file" subsystem still hosts the majority of logic used to write loose objects. Eventually, we'll want to move this logic into "odb/source-loose.c", but this isn't yet easily possible because a lot of the writing logic is still being shared with `force_object_loose()`. We will eventually detangle this logic so that we can indeed move all of it into the "loose" source. Meanwhile though, refactor the code so that it operates on a `struct odb_source_loose` directly to already make the dependency explicit. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: wire up `write_object()` callbackPatrick Steinhardt1-6/+8
Move `odb_source_loose_write_object()` from "object-file.c" into "odb/source-loose.c" and wire it up as the `write_object()` callback of the loose source. As in preceding commits, this requires us to expose a couple of generic functions from "object-file.c" as they are used in both subsystems now. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: wire up `freshen_object()` callbackPatrick Steinhardt1-3/+0
Move `odb_source_loose_freshen_object()` from "object-file.c" into "odb/source-loose.c" and wire it up as the `freshen_object()` callback of the loose source. As part of the move, `check_and_freshen_source()` is inlined into the callback function, as it has no other callers anymore. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: drop `odb_source_loose_has_object()`Patrick Steinhardt1-8/+0
The function `odb_source_loose_has_object()` checks whether a specific object exists as a loose object on disk by using lstat(3p). This interface is somewhat redundant, as we typically check for object existence in a generic way via `odb_source_read_object_info()`. In fact, these two calls are redundant in case the latter is called in a specific way: when called without an object info request and without the `OBJECT_INFO_QUICK` flag, then we will end up doing the same call to lstat(3p) in `read_object_info_from_path()`. Drop the function and adapt callers to instead use the generic interface so that its calling conventions align with that of other sources. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: wire up `count_objects()` callbackPatrick Steinhardt1-14/+0
Move `odb_source_loose_count_objects()` and its associated helpers from "object-file.c" into "odb/source-loose.c" and wire it up as the `count_objects()` callback of the loose source. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: wire up `find_abbrev_len()` callbackPatrick Steinhardt1-12/+0
Move `odb_source_loose_find_abbrev_len()` and its associated helpers from "object-file.c" into "odb/source-loose.c" and wire it up as the `find_abbrev_len` callback of the loose source. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: wire up `for_each_object()` callbackPatrick Steinhardt1-19/+13
Move `odb_source_loose_for_each_object()` and its associated helpers from "object-file.c" into "odb/source-loose.c" and wire it up as the `for_each_object()` callback of the loose source. Again, as in the preceding commit, we are forced to expose a couple of functions from "object-file.c" that are now used by both subsystems. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: wire up `read_object_stream()` callbackPatrick Steinhardt1-5/+26
Move `odb_source_loose_read_object_stream()` and its associated helpers from "object-file.c" into "odb/source-loose.c" and wire it up as the `read_object_stream()` callback of the loose source. As part of the move we are also forced to expose a couple of functions from "object-file.h" that parse object headers in a somewhat-generic way, as those functions are now used by both subsystems. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: wire up `read_object_info()` callbackPatrick Steinhardt1-5/+6
Move `odb_source_loose_read_object_info()` from "object-file.c" into "odb/source-loose.c" and wire it up as the `read_object_info()` callback of the loose source. Callers that previously invoked it directly now go through the generic `odb_source_read_object_info()` interface instead. The function `read_object_info_from_path()` cannot be moved along with it because it is still called by `for_each_object_wrapper_cb()`. It is therefore kept in place, but adjusted to take a loose source to clarify that it's always operating on this structure. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: wire up `reprepare()` callbackPatrick Steinhardt1-3/+0
Move `odb_source_loose_reprepare()` from "object-file.c" into "odb/source-loose.c" and wire it up as the `reprepare()` callback of the loose source. While at it, make `odb_source_loose_clear_cache()` static, as it is no longer needed outside of its file. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: start converting to a proper `struct odb_source`Patrick Steinhardt1-2/+0
Start converting `struct odb_source_loose` into a proper pluggable `struct odb_source` by embedding the base struct and assigning it the new `ODB_SOURCE_LOOSE` type. Furthermore, wire up lifecycle management of this source by implementing the `free` callback and taking ownership of the chdir notifications. Note that the loose source is not yet functional as a standalone `struct odb_source`, as it's missing all of the callback implementations. These will be wired up in subsequent commits. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-06-01odb/source-loose: move loose source into "odb/" subsystemPatrick Steinhardt1-20/+1
In subsequent patches we'll be turning `struct odb_source_loose` into a proper `struct odb_source`. As a first step towards this goal, move its struct out of "object-file.c" and into "odb/source-loose.c". This detaches the implementation of the loose object source from the generic object file code, following the same convention already used by the "files" and "in-memory" sources. No functional changes are intended. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-31odb: use enum for `odb_write_object` flagsPatrick Steinhardt1-1/+2
We've got a couple of functions that accept `odb_write_object()` flags, but all of them accept the flags as an `unsigned` integer. In fact, we don't even have an `enum` for the flags field. Introduce this `enum` and adapt functions accordingly according to our coding style. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-20object-name: move logic to compute loose abbreviation lengthPatrick Steinhardt1-0/+12
The function `repo_find_unique_abbrev_r()` takes as input an object ID as well as a minimum object ID length and returns the minimum required prefix to make the object ID unique. The logic that computes the abbreviation length for loose objects is deeply tied to the loose object storage format. As such, it would fail in case a different object storage format was used. Prepare for making this logic generic to the backend by moving the logic into a new `odb_source_loose_find_abbrev_len()` function that is part of "object-file.c". Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-20object-name: move logic to iterate through loose prefixed objectsPatrick Steinhardt1-7/+0
The logic to iterate through loose objects that have a certain prefix is currently hosted in "object-name.c". This logic reaches into specifics of the loose object source, so it breaks once a different backend is used for the object storage. Move the logic to iterate through loose objects with a prefix into "object-file.c". This is done by extending the for-each-object options to support an optional prefix that is then honored by the loose source. Naturally, we'll also have this support in the packfile store. This is done in the next commit. Furthermore, there are no users of the loose cache outside of "object-file.c" anymore. As such, convert `odb_source_loose_cache()` to have file scope. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-20odb: introduce `struct odb_for_each_object_options`Patrick Steinhardt1-1/+1
The `odb_for_each_object()` function only accepts a bitset of flags. In a subsequent commit we'll want to change object iteration to also support iterating over only those objects that have a specific prefix. While we could of course add the prefix to the function signature, or alternatively introduce a new function, both of these options don't really seem to be that sensible. Instead, introduce a new `struct odb_for_each_object_options` that can be passed to a new `odb_for_each_object_ext()` function. Splice through the options structure into the respective object database sources. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-12object-file: generalize counting objectsPatrick Steinhardt1-2/+3
Generalize the function introduced in the preceding commit to not only be able to approximate the number of loose objects, but to also provide an accurate count. The behaviour can be toggled via a new flag. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-12object-file: extract logic to approximate object countPatrick Steinhardt1-0/+13
In "builtin/gc.c" we have some logic that checks whether we need to repack objects. This is done by counting the number of objects that we have and checking whether it exceeds a certain threshold. We don't really need an accurate object count though, which is why we only open a single object directory shard and then extrapolate from there. Extract this logic into a new function that is owned by the loose object database source. This is done to prepare for a subsequent change, where we'll introduce object counting on the object database source level. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-03-05Merge branch 'ps/fsck-stream-from-the-right-object-instance'Junio C Hamano1-1/+3
"fsck" iterates over packfiles and its access to pack data caused the list to be permuted, which caused it to loop forever; the code to access pack data by "fsck" has been updated to avoid this. * ps/fsck-stream-from-the-right-object-instance: pack-check: fix verification of large objects packfile: expose function to read object stream for an offset object-file: adapt `stream_object_signature()` to take a stream t/helper: improve "genrandom" test helper
2026-03-02Merge branch 'ps/object-info-bits-cleanup'Junio C Hamano1-1/+1
A couple of bugs in use of flag bits around odb API has been corrected, and the flag bits reordered. * ps/object-info-bits-cleanup: odb: convert `odb_has_object()` flags into an enum odb: convert object info flags into an enum odb: drop gaps in object info flag values builtin/fsck: fix flags passed to `odb_has_object()` builtin/backfill: fix flags passed to `odb_has_object()`
2026-03-02Merge branch 'ps/odb-for-each-object'Junio C Hamano1-10/+12
Revamp object enumeration API around odb. * ps/odb-for-each-object: odb: drop unused `for_each_{loose,packed}_object()` functions reachable: convert to use `odb_for_each_object()` builtin/pack-objects: use `packfile_store_for_each_object()` odb: introduce mtime fields for object info requests treewide: drop uses of `for_each_{loose,packed}_object()` treewide: enumerate promisor objects via `odb_for_each_object()` builtin/fsck: refactor to use `odb_for_each_object()` odb: introduce `odb_for_each_object()` packfile: introduce function to iterate through objects packfile: extract function to iterate through objects of a store object-file: introduce function to iterate through objects object-file: extract function to read object info from path odb: fix flags parameter to be unsigned odb: rename `FOR_EACH_OBJECT_*` flags
2026-02-23object-file: adapt `stream_object_signature()` to take a streamPatrick Steinhardt1-1/+3
The function `stream_object_signature()` is responsible for verifying whether the given object ID matches the actual hash of the object's contents. In contrast to `check_object_signature()` it does so in a streaming fashion so that we don't have to load the full object into memory. In a subsequent commit we'll want to adapt one of its callsites to pass a preconstructed stream. Prepare for this by accepting a stream as input that the caller needs to assemble. While at it, improve the error reporting in `parse_object_with_flags()` to tell apart the two failure modes. Helped-by: Jeff King <peff@peff.net> Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-12odb: convert object info flags into an enumPatrick Steinhardt1-1/+2
Convert the object info flags into an enum and adapt all functions that receive these flags as parameters to use the enum instead of an integer. This serves two purposes: - The function signatures become more self-documenting, as callers don't have to wonder which flags they expect. - The compiler can warn when a wrong flag type is passed. Note that the second benefit is somewhat limited. For example, when or-ing multiple enum flags together the result will be an integer, and the compiler will not warn about such use cases. But where it does help is when a single flag of the wrong type is passed, as the compiler would generate a warning in that case. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-02odb: prepare `struct odb_transaction` to become genericJustin Tobler1-6/+0
An ODB transaction handles how objects are stored temporarily and eventually committed. Due to object storage being implemented differently for a given ODB source, the ODB transactions must be implemented in a manner specific to the source the objects are being written to. To provide generic transactions, `struct odb_transaction` is updated to store a commit callback that can be configured to support a specific ODB source. For now `struct odb_transaction_files` is the only transaction type and what is always returned when starting a transaction. Signed-off-by: Justin Tobler <jltobler@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-02-02object-file: rename transaction functionsJustin Tobler1-3/+3
In a subsequent commit, ODB transactions are made more generic to facilitate each ODB source providing its own transaction handling. Rename `object_file_transaction_{begin,commit}()` to `odb_transaction_files_{begin,commit}()` to better match the future source specific transaction implementation. Signed-off-by: Justin Tobler <jltobler@gmail.com> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-26odb: drop unused `for_each_{loose,packed}_object()` functionsPatrick Steinhardt1-11/+0
We have converted all callers of `for_each_loose_object()` and `for_each_packed_object()` to use their new replacement functions instead. We can thus remove them now. Do so and inline `packfile_store_for_each_object_internal()` now that it only has a single callsite again. This makes it a bit easier to follow the callback indirection that is happening there. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-26odb: introduce `odb_for_each_object()`Patrick Steinhardt1-3/+4
Introduce a new function `odb_for_each_object()` that knows to iterate through all objects part of a given object database. This function is essentially a simple wrapper around the object database sources. Subsequent commits will adapt callers to use this new function. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-26object-file: introduce function to iterate through objectsPatrick Steinhardt1-0/+11
We have multiple divergent interfaces to iterate through objects of a specific backend: - `for_each_loose_object()` yields all loose objects. - `for_each_packed_object()` (somewhat obviously) yields all packed objects. These functions have different function signatures, which makes it hard to create a common abstraction layer that covers both of these. Introduce a new function `odb_source_loose_for_each_object()` to plug this gap. This function doesn't take any data specific to loose objects, but instead it accepts a `struct object_info` that will be populated the exact same as if `odb_source_loose_read_object()` was called. The benefit of this new interface is that we can continue to pass backend-specific data, as `struct object_info` contains a union for these exact use cases. This will allow us to unify how we iterate through objects across both loose and packed objects in a subsequent commit. The `for_each_loose_object()` function continues to exist for now, but it will be removed at the end of this patch series. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-26odb: fix flags parameter to be unsignedPatrick Steinhardt1-1/+2
The `flags` parameter accepted by various `for_each_object()` functions is a bitfield of multiple flags. Such parameters are typically unsigned in the Git codebase, but we use `enum odb_for_each_object_flags` in some places. Adapt these function signatures to use the correct type. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2026-01-26odb: rename `FOR_EACH_OBJECT_*` flagsPatrick Steinhardt1-1/+1
Rename the `FOR_EACH_OBJECT_*` flags to have an `ODB_` prefix. This prepares us for a new upcoming `odb_for_each_object()` function and ensures that both the function and its flags have the same prefix. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-23streaming: move logic to read loose objects streams into backendPatrick Steinhardt1-37/+5
Move the logic to read loose object streams into the respective subsystem. This allows us to make a couple of function declarations private. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-03object-file: refactor writing objects via a streamPatrick Steinhardt1-10/+4
We have two different ways to write an object into the database: - We either provide the full buffer and write the object all at once. - Or we provide an input stream that has a `read()` function so that we can chunk the object. The latter is especially used for large objects, where it may be too expensive to hold the complete object in memory all at once. While we already have `odb_write_object()` at the ODB-layer, we don't have an equivalent for streaming an object. Introduce a new function `odb_write_object_stream()` to address this gap so that callers don't have to be aware of the inner workings of how to stream an object to disk with a specific object source. Rename `stream_loose_object()` to `odb_source_loose_write_stream()` to clarify its scope. This matches our modern best practices around how to name functions. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-03object-file: rename `write_object_file()`Patrick Steinhardt1-5/+5
Rename `write_object_file()` to `odb_source_loose_write_object()` so that it becomes clear that this is tied to a specific loose object source. This matches our modern naming schema for functions. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-03object-file: refactor freshening of objectsPatrick Steinhardt1-0/+3
When writing an object that already exists in our object database we skip the write and instead only update mtimes of the object, either in its packed or loose object format. This logic is wholly contained in "object-file.c", but that file is really only concerned with loose objects. So it does not really make sense that it also contains the logic to freshen a packed object. Introduce a new `odb_freshen_object()` function that sits on the object database level and two functions `packfile_store_freshen_object()` and `odb_source_loose_freshen_object()`. Like this, the format-specific functions can be part of their respective subsystems, while the backend agnostic function to freshen an object sits at the object database layer. Note that this change also moves the logic that iterates through object sources from the object source layer into the object database layer. This change is intentional: object sources should ideally only have to worry about themselves, and coordination of different sources should be handled on the object database level. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-03object-file: rename `has_loose_object()`Patrick Steinhardt1-8/+8
Rename `has_loose_object()` to `odb_source_loose_has_object()` so that it becomes clear that this is tied to a specific loose object source. This matches our modern naming schema for functions. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-03object-file: read objects via the loose object sourcePatrick Steinhardt1-7/+8
When reading an object via `loose_object_info()` or `map_loose_object()` we hand in the whole repository. We then iterate through each of the object sources to figure out whether that source has the object in question. This logic is reversing responsibility though: a specific backend should only care about one specific source, where the object sources themselves are then managed by the object database. Refactor the code accordingly by passing an object source to both of these functions instead. The different sources are then handled by either `do_oid_object_info_extended()`, which sits on the object database level, and by `open_istream_loose()`. The latter function arguably is still at the wrong level, but this will be cleaned up at a later point in time. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-03object-file: move loose object map into loose sourcePatrick Steinhardt1-0/+3
The loose object map is used to map from the repository's canonical object hash to the compatibility hash. As the name indicates, this map is only used for loose objects, and as such it is tied to a specific loose object source. Same as with preceding commits, move this map into the loose object source accordingly. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-03object-file: hide internals when we need to reprepare loose sourcesPatrick Steinhardt1-3/+3
There are two different situations where we have to clear the cache of loose objects: - When freeing the loose object source itself to avoid memory leaks. - When repreparing the loose object source so that any potentially- stale data is getting evicted from the cache. The former is already handled by `odb_source_loose_free()`. But the latter case is still done manually by in `odb_reprepare()`, so we are leaking internals into that code. Introduce a new `odb_source_loose_reprepare()` function as an equivalent to `packfile_store_prepare()` to hide these implementation details. Furthermore, while at it, rename the function `odb_clear_loose_cache()` to `odb_source_loose_clear()`. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>
2025-11-03object-file: move loose object cache into loose sourcePatrick Steinhardt1-2/+14
Our loose objects use a cache that (optionally) stores all objects for each of the opened sharding directories. This cache is located in the `struct odb_source`, but now that we have `struct odb_source_loose` it makes sense to move it into the latter structure so that all state that relates to loose objects is entirely self-contained. Do so. While at it, rename corresponding functions to have a prefix that relates to `struct odb_source_loose`. Note that despite this prefix, the functions still accept a `struct odb_source` as input. This is done intentionally: once we introduce pluggable object databases, we will continue to accept this struct but then do a cast inside these functions to `struct odb_source_loose`. This design is similar to how we do it for our ref backends. Signed-off-by: Patrick Steinhardt <ps@pks.im> Signed-off-by: Junio C Hamano <gitster@pobox.com>