da6cdfbc51
urlparse().path keeps the leading slash in front of a Windows drive, so file:///C:/docs/a.pdf read as \C:\docs\a.pdf and the ingester reported "File does not exist" for every file it discovered. url2pathname is the stdlib conversion that strips it, per platform. Four sites each decided both "is this local" and "what path is this": FSSource._uri_to_path and supports, resolve_adhoc_fetcher, create_document_from_source and check_source_accessible, and convert. is_local_uri and uri_to_path in haiku.rag.uri own those two decisions now, which closes two more cases of the same root cause. A bare C:\docs\a.pdf parses with scheme "c", so add-src raised "No source adapter for URI scheme 'c'" and convert silently treated the path as raw text. And convert and check_source_accessible never percent-decoded at all, so a file named a[b] c.md read as missing on Linux and macOS too. A file URI's host is reattached after conversion rather than passed to url2pathname, which as of 3.14 rejects a non-local authority off Windows. file:////server/share is the empty-authority spelling of a UNC path, its host being the first path segment, so that host is normalised into the authority before conversion. Output is identical on 3.12, 3.13 and 3.14. The ad-hoc FS fetcher roots at the path's own anchor rather than "/", which on Windows is only the current drive. test_uri.py runs on ubuntu, macos and windows across 3.13 and 3.14 without the project installed: --noconftest because the repo conftest imports dependencies that job does not need, and -o addopts= to drop the repository's -n auto. The Windows legs are what cover the drive conversion. Fixes #574.
41 lines
1.7 KiB
Python
41 lines
1.7 KiB
Python
import re
|
|
from pathlib import Path
|
|
from urllib.parse import urlparse
|
|
from urllib.request import url2pathname
|
|
|
|
_WINDOWS_ABSOLUTE_PATH = re.compile(r"^[A-Za-z]:[\\/]")
|
|
|
|
|
|
def is_local_uri(uri: str) -> bool:
|
|
"""True for a ``file://`` URI or a bare filesystem path.
|
|
|
|
``urlparse("C:/docs/a.pdf")`` reports the drive letter as scheme ``c``.
|
|
Require the separator after it so URI-like text such as ``x:content`` is
|
|
not mistaken for a path.
|
|
"""
|
|
scheme = urlparse(uri).scheme
|
|
return scheme in ("", "file") or bool(_WINDOWS_ABSOLUTE_PATH.match(uri))
|
|
|
|
|
|
def uri_to_path(uri: str) -> Path:
|
|
"""Filesystem path for a ``file://`` URI or a bare path.
|
|
|
|
``file://`` URIs percent-encode special characters, and on Windows carry a
|
|
leading slash before the drive (``file:///C:/docs``) that ``Path`` would
|
|
keep. ``url2pathname`` handles both, per platform.
|
|
"""
|
|
parsed = urlparse(uri)
|
|
if parsed.scheme == "file":
|
|
host, path = parsed.netloc, parsed.path
|
|
# file:////server/share spells a UNC path with an empty authority, the
|
|
# host being the first path segment (RFC 8089 appendix E.3.2).
|
|
if not host and path.startswith("//"):
|
|
host, _, path = path[2:].partition("/")
|
|
# url2pathname needs the host urlparse split off to build a UNC path,
|
|
# but Python 3.14 rejects one handed to it on a non-Windows platform.
|
|
# localhost denotes the current machine and is intentionally omitted.
|
|
authority = f"//{host}" if host.lower() not in ("", "localhost") else ""
|
|
return Path(f"{authority}{url2pathname('/' + path.lstrip('/'))}")
|
|
if is_local_uri(uri):
|
|
return Path(uri)
|
|
raise ValueError(f"Not a local URI: {uri}")
|