fix(prowlarr): rank releases by author instead of querying for it (#1293) (#1295)

MyAnonamouse is the only indexer Shelfmark treats as enriched, and it
alone was sent {title} {author} while every other indexer got the title
on its own. MAM matches all search terms conjunctively, so whenever the
metadata provider spelled the author differently to the tracker -
Hardcover says Timothy Ferriss, MAM lists Tim Ferriss - the search came
back empty and the UI reported No releases found for this book, with the
release sitting on the tracker the whole time.

The enriched flag is a statement about responses: MAM returns clean
author and bookTitle attributes, which is why it earns format detection
and preferential ordering. Using that same flag to shape the request is
the actual defect, and it is why turning the flag off recovers the
search but takes format detection down with it.

So the query is title-only for every indexer now, and the author orders
the results rather than narrowing them. MAM already hands us its author
field, so agreement is judged on data we hold instead of by an AND we
cannot control. The ranking is three-way on purpose - agrees, no
metadata, disagrees - so an indexer reporting no author does not sort
below one reporting the wrong author.

A wrong verdict costs a release its position, never its visibility: a
transliteration such as Dostoevsky against Dostoyevsky sorts last
instead of vanishing. That is what makes the loose token comparison safe
to ship without a tuning knob.

Falling back to a title-only query on zero results was the alternative.
It only rescues total failure - if two of six editions happen to use the
provider's spelling, the search returns those two, no fallback fires,
and the user quietly gets a truncated list. It also spends a round trip
inside the search deadline and stacks a retry on an indexer that may
still be solving a challenge (#1249).

Manual queries skip author ranking: they are the user's own words and
should not be reordered against the metadata they were typed to
override.

shelfmark/release_sources/prowlarr/source.py

@@ -41,6 +41,8 @@ from shelfmark.release_sources.prowlarr.api import (
 )
 from shelfmark.release_sources.prowlarr.cache import cache_release
 from shelfmark.release_sources.prowlarr.utils import (
+    AUTHOR_UNKNOWN,
+    author_affinity,
     build_source_id,
     coerce_float_like,
     coerce_int_like,
@@ -1008,10 +1010,17 @@ class ProwlarrSource(ReleaseSource):
                 if time.monotonic() > deadline:
                     _raise_timeout_error(f"Prowlarr search timed out after {int(search_budget)}s")
 
-            def search_indexers(
-                query: str, cats: list[int] | None, *, enriched_query: str | None = None
-            ) -> _IndexerSearchOutcome:
-                """Search indexers with given categories via Torznab/Newznab."""
+            def search_indexers(query: str, cats: list[int] | None) -> _IndexerSearchOutcome:
+                """Search indexers with given categories via Torznab/Newznab.
+
+                Every indexer gets the same title-only query. Enriched indexers used
+                to be sent "{title} {author}", but an indexer that ANDs its search
+                terms (MyAnonamouse) returns nothing whenever the metadata provider
+                spells the author differently to the tracker - "Timothy Ferriss" vs
+                "Tim Ferriss" - and the UI reports the book as missing (#1293). The
+                author still decides ordering below, where a spelling difference
+                costs a release its position rather than its existence.
+                """
                 outcome = _IndexerSearchOutcome(results=[])
                 target_indexer_ids = self._get_search_indexer_ids(client, indexer_ids, cats)
                 if not target_indexer_ids:
@@ -1019,16 +1028,11 @@ class ProwlarrSource(ReleaseSource):
 
                 for indexer_id in target_indexer_ids:
                     _check_timeout()
-                    indexer_query = (
-                        enriched_query
-                        if indexer_id in enriched_indexer_ids_set and enriched_query
-                        else query
-                    )
                     outcome.attempted += 1
                     try:
                         raw = client.torznab_search(
                             indexer_id=indexer_id,
-                            query=indexer_query,
+                            query=query,
                             categories=cats,
                             search_type="book",
                         )
@@ -1053,14 +1057,11 @@ class ProwlarrSource(ReleaseSource):
             for idx, variant in enumerate(variants, start=1):
                 _check_timeout()
                 query = variant.title
-                enriched_query = variant.query  # title + author
 
                 if len(variants) > 1:
                     logger.debug("Prowlarr query %s/%s: '%s'", idx, len(variants), query)
 
-                outcome = search_indexers(
-                    query=query, cats=categories, enriched_query=enriched_query
-                )
+                outcome = search_indexers(query=query, cats=categories)
 
                 # Auto-expand: if no results with categories and auto-expand enabled, retry without.
                 # Only when every indexer actually answered: a failed search says nothing about
@@ -1077,9 +1078,7 @@ class ProwlarrSource(ReleaseSource):
                         "Prowlarr: no results for query '%s' with category filter, auto-expanding search",
                         query,
                     )
-                    expanded = search_indexers(
-                        query=query, cats=None, enriched_query=enriched_query
-                    )
+                    expanded = search_indexers(query=query, cats=None)
                     outcome.results = expanded.results
                     outcome.attempted += expanded.attempted
                     outcome.failed += expanded.failed
@@ -1117,6 +1116,10 @@ class ProwlarrSource(ReleaseSource):
 
             results: list[Release] = []
             enriched_source_ids: set[str] = set()
+            affinity_by_source_id: dict[str, int] = {}
+            # A manual query is the user's own words; ranking it against the
+            # metadata author would second-guess what they typed.
+            wanted_author = "" if plan.manual_query else plan.author
 
             for raw_result in all_results:
                 result_with_seed_settings = _apply_indexer_seed_settings(
@@ -1136,13 +1139,20 @@ class ProwlarrSource(ReleaseSource):
                 if idx_id_int is not None and idx_id_int in indexer_priority:
                     release.extra["indexer_priority"] = indexer_priority[idx_id_int]
                 results.append(release)
+                affinity_by_source_id[release.source_id] = author_affinity(
+                    wanted_author, release.extra.get("author")
+                )
 
                 if is_enriched:
                     enriched_source_ids.add(release.source_id)
 
+            # Indexer priority first: it is an explicit user preference. Author
+            # agreement then orders what one indexer returned, so the editions that
+            # match the requested author lead and the rest stay reachable below.
             results.sort(
                 key=lambda r: (
                     _release_indexer_rank(r, indexer_priority),
+                    affinity_by_source_id.get(r.source_id, AUTHOR_UNKNOWN),
                     0 if r.source_id in enriched_source_ids else 1,
                 )
             )

shelfmark/release_sources/prowlarr/utils.py

@@ -14,6 +14,20 @@ if TYPE_CHECKING:
 
 _INTEGER_LIKE_PATTERN = re.compile(r"^[+-]?\d+$")
 _FLOAT_LIKE_PATTERN = re.compile(r"^[+-]?(?:\d+(?:\.\d*)?|\.\d+)$")
+_AUTHOR_TOKEN_PATTERN = re.compile(r"\w+", re.UNICODE)
+_AUTHOR_NOISE_TOKENS = frozenset(
+    {"jr", "sr", "ii", "iii", "iv", "phd", "md", "dr", "mr", "mrs", "ms", "et", "al", "and", "the"}
+)
+
+# Ordering tiers for author agreement between the requested book and what an
+# indexer reported. Lower sorts first.
+AUTHOR_MATCH = 0
+AUTHOR_UNKNOWN = 1
+AUTHOR_MISMATCH = 2
+
+# A mononym ("Homer") can only ever agree on one token; a longer name needs a
+# given name and a surname to agree before it counts as the same person.
+_AUTHOR_TOKENS_REQUIRED = 2
 
 
 def coerce_int_like(value: object) -> int | None:
@@ -32,6 +46,49 @@ def coerce_int_like(value: object) -> int | None:
     return int(normalized)
 
 
+def _author_tokens(value: object) -> list[str]:
+    """Split an author string into comparable lowercase name tokens."""
+    if not isinstance(value, str):
+        return []
+    tokens = [token.lower() for token in _AUTHOR_TOKEN_PATTERN.findall(value)]
+    return [token for token in tokens if token not in _AUTHOR_NOISE_TOKENS]
+
+
+def _author_tokens_compatible(wanted: str, offered: str) -> bool:
+    """Treat an abbreviated given name as the name it abbreviates."""
+    return wanted == offered or wanted.startswith(offered) or offered.startswith(wanted)
+
+
+def author_affinity(wanted: object, offered: object) -> int:
+    """Rank how far an indexer's author field is from the requested author.
+
+    Shelfmark ranks on this rather than filtering on it, so a wrong verdict only
+    costs a release its position in the list, never its visibility. That is what
+    makes the loose token comparison safe: "Tim"/"Timothy" and "T."/"Timothy"
+    agree, while a transliteration ("Dostoevsky"/"Dostoyevsky") is merely sorted
+    last instead of being hidden.
+
+    Three-way on purpose: an indexer that reports no author at all must not sort
+    below one that reports a wrong author, so "no metadata" ranks between
+    agreement and disagreement rather than counting as either.
+    """
+    wanted_tokens = _author_tokens(wanted)
+    offered_tokens = _author_tokens(offered)
+    if not wanted_tokens or not offered_tokens:
+        return AUTHOR_UNKNOWN
+
+    matched = sum(
+        1
+        for wanted_token in wanted_tokens
+        if any(
+            _author_tokens_compatible(wanted_token, offered_token)
+            for offered_token in offered_tokens
+        )
+    )
+    required = min(_AUTHOR_TOKENS_REQUIRED, len(wanted_tokens))
+    return AUTHOR_MATCH if matched >= required else AUTHOR_MISMATCH
+
+
 def build_source_id(result: dict) -> str:
     """Build the Release.source_id for a raw Prowlarr result.
 

tests/prowlarr/test_author_matching.py

@@ -0,0 +1,186 @@
+"""Author agreement ranks Prowlarr results; it never narrows the query (#1293).
+
+MyAnonamouse - the only indexer Shelfmark treats as enriched - used to receive
+"{title} {author}". MAM ANDs its search terms, so any difference between the
+metadata provider's author spelling and the tracker's ("Timothy Ferriss" vs
+"Tim Ferriss") returned nothing at all and the UI reported the book as missing.
+"""
+
+import pytest
+
+from shelfmark.metadata_providers import BookMetadata
+from shelfmark.release_sources.prowlarr.source import ProwlarrSource
+from shelfmark.release_sources.prowlarr.utils import (
+    AUTHOR_MATCH,
+    AUTHOR_MISMATCH,
+    AUTHOR_UNKNOWN,
+    author_affinity,
+)
+
+MAM_INDEXER_ID = 1
+
+
+class TestAuthorAffinity:
+    @pytest.mark.parametrize(
+        ("wanted", "offered"),
+        [
+            ("Timothy Ferriss", "Tim Ferriss"),
+            ("Tim Ferriss", "Timothy Ferriss"),
+            ("T. Ferriss", "Timothy Ferriss"),
+            ("Ursula K. Le Guin", "Ursula Le Guin"),
+            ("Iain M. Banks", "Iain Banks"),
+            ("Frank Herbert", "Frank Herbert, Brian Herbert"),
+            ("Robert Jordan Jr.", "Robert Jordan"),
+            ("homer", "Homer"),
+        ],
+    )
+    def test_same_author_spelled_differently_agrees(self, wanted, offered):
+        assert author_affinity(wanted, offered) == AUTHOR_MATCH
+
+    @pytest.mark.parametrize(
+        ("wanted", "offered"),
+        [
+            ("Timothy Ferriss", "Frank Herbert"),
+            ("Frank Herbert", "Brian Herbert"),
+            ("Homer", "Virgil"),
+        ],
+    )
+    def test_different_author_disagrees(self, wanted, offered):
+        assert author_affinity(wanted, offered) == AUTHOR_MISMATCH
+
+    @pytest.mark.parametrize(
+        ("wanted", "offered"),
+        [
+            ("Timothy Ferriss", None),
+            ("Timothy Ferriss", ""),
+            ("", "Tim Ferriss"),
+            (None, "Tim Ferriss"),
+            ("Timothy Ferriss", {"name": "Tim Ferriss"}),
+        ],
+    )
+    def test_missing_metadata_is_neither_agreement_nor_disagreement(self, wanted, offered):
+        # An indexer that reports no author must not sort below one that reports
+        # the wrong author, so this tier sits between the two.
+        assert author_affinity(wanted, offered) == AUTHOR_UNKNOWN
+        assert AUTHOR_MATCH < AUTHOR_UNKNOWN < AUTHOR_MISMATCH
+
+    def test_a_surname_alone_is_not_enough_for_a_full_name(self):
+        # "Ferriss" appearing under some other given name is a different person.
+        assert author_affinity("Timothy Ferriss", "Bruce Ferriss") == AUTHOR_MISMATCH
+
+
+class _EnrichedIndexerClient:
+    """Stands in for a Prowlarr with MyAnonamouse enabled."""
+
+    def __init__(self, search_results=None):
+        self.queries: list[str] = []
+        self.search_results = search_results or []
+        self.indexer_timeout = 90
+
+    def get_enabled_indexers_detailed(self, *, raise_on_error=False):
+        del raise_on_error
+        return [
+            {
+                "id": MAM_INDEXER_ID,
+                "enable": True,
+                "implementation": "MyAnonamouse",
+                "capabilities": {
+                    "categories": [
+                        {"id": 7000, "subCategories": []},
+                        {"id": 3030, "subCategories": []},
+                    ]
+                },
+            }
+        ]
+
+    def torznab_search(
+        self, *, indexer_id, query, categories=None, search_type="book", limit=100, offset=0
+    ):
+        del indexer_id, categories, search_type, limit, offset
+        self.queries.append(query)
+        return self.search_results
+
+    def get_enriched_indexer_ids(self, restrict_to=None, indexers=None):
+        del restrict_to, indexers
+        return [MAM_INDEXER_ID]
+
+    def get_indexer_seed_settings(self, restrict_to=None):
+        del restrict_to
+        return {}
+
+
+def _mam_result(guid: str, author: str | None) -> dict:
+    return {
+        "guid": guid,
+        "title": "The Tao of Seneca",
+        "author": author,
+        "indexerId": MAM_INDEXER_ID,
+        "indexer": "MyAnonamouse",
+        "protocol": "torrent",
+        "size": 1048576,
+        "seeders": 10,
+        "leechers": 1,
+        "categories": [{"id": 7020}],
+        "infoUrl": f"https://tracker.example/{guid}",
+    }
+
+
+def _search(monkeypatch, client, *, manual_query=None):
+    import shelfmark.release_sources.prowlarr.source as prowlarr_source
+    from shelfmark.core.search_plan import build_release_search_plan
+
+    values = {"PROWLARR_INDEXERS": "", "PROWLARR_AUTO_EXPAND": False}
+    monkeypatch.setattr(
+        prowlarr_source.config, "get", lambda key, default=None: values.get(key, default)
+    )
+
+    source = ProwlarrSource()
+    monkeypatch.setattr(source, "_get_client", lambda: client)
+
+    book = BookMetadata(
+        provider="hardcover",
+        provider_id="123",
+        title="The Tao of Seneca",
+        authors=["Timothy Ferriss"],
+    )
+    plan = build_release_search_plan(book, languages=["en"], manual_query=manual_query)
+    return source.search(book, plan, content_type="ebook")
+
+
+class TestEnrichedIndexerQuery:
+    def test_enriched_indexer_is_queried_without_the_author(self, monkeypatch):
+        client = _EnrichedIndexerClient()
+
+        _search(monkeypatch, client)
+
+        assert client.queries == ["The Tao of Seneca"]
+        assert not any("Ferriss" in query for query in client.queries)
+
+
+class TestAuthorOrdering:
+    def test_matching_author_leads_and_mismatch_stays_visible(self, monkeypatch):
+        client = _EnrichedIndexerClient(
+            search_results=[
+                _mam_result("other-author", "Frank Herbert"),
+                _mam_result("no-author", None),
+                _mam_result("right-author", "Tim Ferriss"),
+            ]
+        )
+
+        results = _search(monkeypatch, client)
+
+        # Ranked, not filtered: the wrong author is last but still reachable.
+        assert [r.extra["author"] for r in results] == ["Tim Ferriss", None, "Frank Herbert"]
+
+    def test_manual_query_is_not_reordered_against_the_metadata_author(self, monkeypatch):
+        client = _EnrichedIndexerClient(
+            search_results=[
+                _mam_result("other-author", "Frank Herbert"),
+                _mam_result("right-author", "Tim Ferriss"),
+            ]
+        )
+
+        results = _search(monkeypatch, client, manual_query="tao seneca")
+
+        assert client.queries == ["tao seneca"]
+        assert [r.extra["author"] for r in results] == ["Frank Herbert", "Tim Ferriss"]