adsfolio.pdf.catalog

Central local catalog for PDF attachments and observed file locations.

class adsfolio.pdf.catalog.CatalogObservation(attachment_id, location_id, version_id, matched_by)

Bases: object

Result of registering one observed filesystem location.

Parameters:

• attachment_id (int)

• location_id (int)

• version_id (int | None)

• matched_by (str)

attachment_id: int

location_id: int

matched_by: str

version_id: int | None

class adsfolio.pdf.catalog.LegacyMigrationReport(imported)

Bases: object

Summary of one legacy-root migration.

Parameters:

imported (int)

imported: int

class adsfolio.pdf.catalog.PdfCatalog(path=None)

Bases: object

Store PDF attachment identity independently of filesystem paths.

Parameters:

path (Path | None)

ads_bibtex_entries()

Return saved ADS BibTeX as {bibcode: (entry, fingerprint)}.

A None entry means that ADS returned no entry for that Bibcode.

Return type:

dict[str, tuple[str | None, str]]

attach_document_file(document_id, path, *, sha256, size_bytes, mtime_ns, device, inode)

Observe a PDF and link it without replacing another document identity.

Parameters:

• document_id (int)

• path (pathlib.Path)

• sha256 (str)

• size_bytes (int)

• mtime_ns (int)

• device (int)

• inode (int)

Return type:

dict[str, Any]

classmethod canonical_doi(value)

Return a DOI in comparable form: lowercase, without URL or prefix.

Parameters:

value (Any)

Return type:

str

change_token()

Return a value that changes after any write to this catalog.

The first value identifies this open catalog in the process. total_changes counts rows that this connection changed. PRAGMA data_version changes after another connection commits. Equal tokens therefore mean that no committed or pending write occurred between the two calls.

Return type:

tuple[int, int, int]

clear_watch_pending(root, relative_path)

Clear the watch mark of one active location. Change nothing else.

Returns:

True when a marked row was changed.

Return type:

bool

Parameters:

• root (pathlib.Path)

• relative_path (str)

close()

Close the catalog connection.

Return type:

None

content_versions(attachment_id)

Return content versions in observation order.

Parameters:

attachment_id (int)

Return type:

list[dict[str, Any]]

count_ads_documents()

Return the number of cached provider documents.

Return type:

int

count_files(*, root=None, status=None, resolved=None, search=None, search_exact=False, search_wildcards=False, search_fields=(), include_missing=False, document_id=None, attachment_id=None, location_id=None, sha256=None)

Count the locations that query_files returns for the same filters.

The count is done in SQL. Only a search reads metadata JSON. Digest collections are not accepted; call query_files for those.

Parameters:

• root (pathlib.Path | None)

• status (str | None)

• resolved (bool | None)

• search (str | None)

• search_exact (bool)

• search_wildcards (bool)

• search_fields (collections.abc.Collection[str])

• include_missing (bool)

• document_id (int | None)

• attachment_id (int | None)

• location_id (int | None)

• sha256 (str | None)

Return type:

int

delete_locations(root, location_ids)

Delete selected locations and orphaned attachments explicitly.

Parameters:

• root (pathlib.Path)

• location_ids (list[int])

Return type:

int

delete_meta(key)

Delete catalog metadata by key.

Parameters:

key (str)

Return type:

None

discover_root(root, *, recursive=True, cancelled=None)

Discover PDFs and verify known identity candidates by content.

Changed tracked files are hashed before their filesystem facts are updated. A new path is also hashed when its filesystem identity has a known digest. New paths in an existing library are queued for watching; the first import retains the baseline behavior. The digest decides whether the file is that content again (see _observe_file). Unchanged files are never hashed. A file that vanishes before hashing is skipped; any other read error aborts the scan before the catalog changes.

Parameters:

• root (pathlib.Path)

• recursive (bool)

• cancelled (collections.abc.Callable[[], bool] | None)

Return type:

int

document_for_attachment(attachment_id)

Return the canonical document linked to an attachment.

Parameters:

attachment_id (int)

Return type:

dict[str, Any] | None

classmethod document_identity(metadata)

Return the bibcode, canonical DOI, and arXiv ID of document metadata.

Missing identifiers are empty strings. The arXiv ID has no version.

Parameters:

metadata (collections.abc.Mapping[str, Any])

Return type:

tuple[str, str, str]

ensure_root(root)

Register a library root and return its catalog identifier.

Parameters:

root (pathlib.Path)

Return type:

int

find_ads_document_by_bibcode(bibcode)

Return a cached provider document by Bibcode.

Parameters:

bibcode (str)

Return type:

dict[str, Any] | None

find_ads_documents_by_bibcodes(bibcodes)

Return cached provider documents keyed by lowercase Bibcode.

Parameters:

bibcodes (list[str])

Return type:

dict[str, dict[str, Any]]

find_ads_documents_by_doi(doi)

Return cached provider documents matching one DOI.

Parameters:

doi (str)

Return type:

list[dict[str, Any]]

find_ads_documents_by_dois(dois)

Return cached provider documents keyed by normalized DOI.

Parameters:

dois (list[str])

Return type:

dict[str, list[dict[str, Any]]]

get_document(document_id)

Return one canonical document, or None when its ID is unknown.

Parameters:

document_id (int)

Return type:

dict[str, Any] | None

get_file(root, relative_path)

Return one location record and its attachment metadata.

Prefer an exact path match, including a missing location. Otherwise, return a unique active Unicode alias proven by its directory entry.

Parameters:

• root (pathlib.Path)

• relative_path (str)

Return type:

dict[str, Any] | None

get_meta(key)

Return catalog metadata by key.

Parameters:

key (str)

Return type:

str | None

list_documents()

Return canonical documents, including references without PDFs.

Return type:

list[dict[str, Any]]

mark_location_missing(root, relative_path)

Retain one absent file location as history after an external move.

The path must no longer exist. Filesystem access errors stop the update. Return False when no active catalog location owns the path.

Parameters:

• root (pathlib.Path)

• relative_path (str)

Return type:

bool

mark_missing_locations(root, seen_paths, recursive=True)

Mark active root locations absent from a completed scan as missing.

Parameters:

• root (pathlib.Path)

• seen_paths (set[str])

• recursive (bool)

Return type:

int

migrate_legacy_root(root)

Import legacy root state without changing its source files.

Parameters:

root (pathlib.Path)

Return type:

adsfolio.pdf.catalog.LegacyMigrationReport

move_file(root, relative_path, new_relative_path)

Move a location while retaining its attachment and version history.

Parameters:

• root (pathlib.Path)

• relative_path (str)

• new_relative_path (str)

Return type:

bool

observe_file(root, relative_path, *, sha256, size_bytes, mtime_ns, device=None, inode=None, metadata=None, status=None)

Register one filesystem observation atomically.

Parameters:

• root (pathlib.Path)

• relative_path (str)

• sha256 (str | None)

• size_bytes (int)

• mtime_ns (int)

• device (int | None)

• inode (int | None)

• metadata (collections.abc.Mapping[str, Any] | None)

• status (str | None)

Return type:

adsfolio.pdf.catalog.CatalogObservation

preserve_annotation_revision(root, relative_path, expected_sha256, *, apply=True)

Preserve an identified record when only PDF annotations changed.

The previous identified bytes must occur as an exact, cataloged prefix. Both revisions must have the same non-annotation PDF structure. An existing identity proof keeps its verification level. A legacy record without proof does not gain one.

Parameters:

• root (Path) – Library root containing the PDF.

• relative_path (str) – PDF path relative to the root.

• expected_sha256 (str) – Digest of the current PDF contents.

• apply (bool, optional) – Update the catalog when True. Otherwise, only check continuity.

Returns:

The preserved record, or None when continuity cannot be proven.

Return type:

dict or None

query_files(*, root=None, status=None, resolved=None, search=None, search_exact=False, search_wildcards=False, search_fields=(), include_missing=False, document_id=None, attachment_id=None, location_id=None, sha256=None)

Return location records matching local catalog filters.

Parameters:

• include_missing (bool) – Include historical locations whose files are no longer observed. Normal queries return only current locations.

• document_id (int | None) – Restrict locations to one canonical document when supplied.

• attachment_id (int | None) – Restrict results to one attachment or one location identifier.

• location_id (int | None) – Restrict results to one attachment or one location identifier.

• search (str | None) – Keep locations that match the query search. search_fields names the fields that plain terms search. See adsfolio.pdf.search.document_matches for the rules.

• search_exact (bool) – Keep locations that match the query search. search_fields names the fields that plain terms search. See adsfolio.pdf.search.document_matches for the rules.

• search_wildcards (bool) – Keep locations that match the query search. search_fields names the fields that plain terms search. See adsfolio.pdf.search.document_matches for the rules.

• search_fields (collections.abc.Collection[str]) – Keep locations that match the query search. search_fields names the fields that plain terms search. See adsfolio.pdf.search.document_matches for the rules.

• sha256 (str | collections.abc.Collection[str] | None) – One digest, or a collection of digests, of the current content. An empty collection matches nothing. A collection runs one indexed query per distinct digest and merges the rows, so its size is not limited by the number of SQL variables.

• root (pathlib.Path | None)

• status (str | None)

• resolved (bool | None)

Return type:

list[dict[str, Any]]

query_location_ids(*, root=None, search, search_exact=False, search_wildcards=False, search_fields=())

Return current location identifiers that match search in SQL.

It applies the same search rule as query_files.

Parameters:

• root (pathlib.Path | None)

• search (str)

• search_exact (bool)

• search_wildcards (bool)

• search_fields (collections.abc.Collection[str])

Return type:

set[int]

register_verified_replacement(root, relative_path, metadata)

Register verified new contents without losing an absent PDF’s history.

A different name rejoins an old attachment only when exactly one matching attachment has no live locations. Existing editions remain separate. A different paper at the same path receives a new attachment. File bytes are never changed. Unknown filesystem state stops the write.

Parameters:

• root (pathlib.Path)

• relative_path (str)

• metadata (collections.abc.Mapping[str, Any])

Return type:

adsfolio.pdf.catalog.CatalogObservation | None

replace_file_metadata(root, relative_path, metadata)

Replace local metadata for one attachment without changing identity.

Parameters:

• root (pathlib.Path)

• relative_path (str)

• metadata (collections.abc.Mapping[str, Any])

Return type:

bool

replace_file_metadata_if_unchanged(root, relative_path, metadata, expected_record, *, expected_document=<object object>, changed_fields=None, allow_reviewed_stale_recovery=False)

Replace metadata only when the catalog record matches a snapshot.

Parameters:

• root (pathlib.Path) – Library root that contains the location.

• relative_path (str) – Path of the location within the root.

• metadata (collections.abc.Mapping[str, Any]) – Replacement metadata, with the same semantics as replace_file_metadata.

• expected_record (collections.abc.Mapping[str, Any]) – Unmodified record returned by get_file or query_files. Keep its nested metadata separate from the replacement metadata.

• expected_document (collections.abc.Mapping[str, Any] | None) – Unmodified snapshot returned by get_document. Pass None when the record has no linked document. Omit this argument to skip the document check.

• changed_fields (collections.abc.Collection[str] | None) – Fields to merge into an existing document. Omit this argument to infer changes from expected_record and metadata. An empty collection leaves existing document metadata unchanged. The full replacement still updates the attachment and identifies the target document. A new document receives the full replacement metadata.

• allow_reviewed_stale_recovery (bool) – Allow a manually reviewed conflict to replace stale metadata. The proof must match the current PDF hash and identifiers.

Returns:

True if the replacement succeeds. False if the location is missing or a supplied snapshot is incomplete or no longer matches. Get fresh snapshots before retrying after False.

Return type:

bool

Raises:

• RuntimeError – If this connection already has an open transaction.

• TypeError – If changed_fields is a string or contains a non-string value.

• sqlite3.OperationalError – If SQLite cannot acquire the write lock before its timeout.

Notes

The comparison and replacement share one SQLite write transaction. The digest and file facts come from the catalog, not the PDF on disk. The record snapshot does not include linked document metadata. The optional document check covers only the currently linked document. It does not guard a different document selected by replacement metadata. The replacement updates the attachment and may update its document. Other locations can share the attachment or document. Document fields use the existing merge rules for locks and empty values. For non-stale records, the catalog retains the current status when metadata omits status and leaves identifiers and identity proof unchanged. Explicit status and identity changes use the existing replacement rules. The comparison cannot detect changes that restore the snapshot values. Other connections can write again after this call commits.

save_reference_documents(documents, *, source_text=None)

Save one reference batch atomically and update attached metadata.

Parameters:

• documents (list[collections.abc.Mapping[str, Any]]) – Document mappings containing metadata, optional bibcode, and optional existing id. Duplicate citation keys and DOIs fail the whole batch; existing documents are never merged implicitly.

• source_text (str | None) – Original BibTeX to store once for the batch, addressed by its SHA-256 digest in catalog_meta.

Return type:

list[int]

property schema_version: int

Return the catalog schema version.

search_texts(root)

Return the raw text columns that the search filter compares.

Each row holds the location identifier, relative path, stored metadata JSON, and current digest of one current location in root.

Parameters:

root (pathlib.Path)

Return type:

list[tuple[int, str, str, str | None]]

set_ads_document_metrics(bibcode, metrics)

Store citation metrics inside a cached provider document.

Parameters:

• bibcode (str)

• metrics (collections.abc.Mapping[str, int | None])

Return type:

bool

set_document_cite_key(document_id, key)

Save or clear the typed citation key of one document.

Parameters:

• document_id (int) – Catalog document ID.

• key (str) – The key. An empty string removes the typed key.

Raises:

ValueError – If document_id is unknown.

Return type:

None

set_meta(key, value)

Set catalog metadata by key.

Parameters:

• key (str)

• value (str)

Return type:

None

store_ads_bibtex(entries)

Save ADS BibTeX results as {bibcode: (entry, fingerprint)}.

Save a None entry when ADS returned none, so the same metadata is not requested again.

Parameters:

entries (collections.abc.Mapping[str, tuple[str | None, str]])

Return type:

None

update_file_status(root, relative_path, status)

Update the attachment status for one observed location.

Parameters:

• root (pathlib.Path)

• relative_path (str)

• status (str)

Return type:

bool

upsert_ads_document(document, *, source)

Insert or update one cached ADS document.

Parameters:

• document (collections.abc.Mapping[str, Any])

• source (str)

Return type:

None

upsert_ads_documents(documents, *, source)

Insert or update cached provider documents atomically.

Parameters:

• documents (list[collections.abc.Mapping[str, Any]])

• source (str)

Return type:

None

adsfolio.pdf.catalog.file_facts_changed(previous, stat)

Return whether size, modification time, or inode differ from the record.

The device number is not compared. macOS can give a volume a new device number after a restart, while every file on it stays the same. A copy or a replacement file gets a new inode.

Parameters:

• previous (collections.abc.Mapping[str, Any])

• stat (os.stat_result)

Return type:

bool

adsfolio.pdf.catalog.CATALOG_FILENAME = 'pdf-library.sqlite'

str(object=’’) -> str str(bytes_or_buffer[, encoding[, errors]]) -> str

Create a new string object from the given object. If encoding or errors is specified, then the object must expose a data buffer that will be decoded using the given encoding and error handler. Otherwise, returns the result of object.__str__() (if defined) or repr(object). encoding defaults to ‘utf-8’. errors defaults to ‘strict’.