The index model
An index server earns its keep the day you have a package that must not come from the public internet. This page explains peryx's answer. An index is the list of packages a client installs from (pip and uv call it an index-url); a registry is the same idea under a different name. peryx builds every index from two independent choices, a role (what the index does) and an ecosystem (which packaging format it speaks), and one composition rule. That rule is a security control before it is a convenience.
Two axes: role and ecosystem
Every index peryx serves is a triple: a role, an ecosystem, and a key (its name). The two axes are independent.
- role is what the index does. There are three:
- cached is a read-through cache of one upstream index. "Upstream" means the index peryx fetches from, e.g. pypi.org. On a first request peryx fetches from upstream, stores the result, and serves it; later requests come from local disk. (This was called a "mirror".)
- hosted is an authoritative store you upload to. Nothing upstream; the packages live here because you published them. (This was called a "local" index.)
- virtual is an ordered aggregation of other indexes served under one URL. Its member list is called
layers. (This was called an "overlay".)
- ecosystem is which packaging format the index speaks: pypi and oci today. It fixes the wire protocol (the
PyPI Simple API, the
OCI
/v2/distribution API) and the artifact shapes (wheels and sdists; manifests and blobs). See ecosystems and the standards each one implements.
The axes are orthogonal at creation but coupled at aggregation: a virtual index may only combine members of the same ecosystem. The roles below work the same in every ecosystem; the diagrams use a PyPI example, and each ecosystem's page shows the same shapes in its own protocol.
Prior art
The index servers teams run in production converged on the same role shape, and peryx adopts it:
- Artifactory aggregates local and remote repositories into a virtual repository behind one URL, with a default deployment target for writes and local-before-remote resolution.
- Nexus groups hosted and proxy repositories the same way; the member order decides who wins, and the documentation recommends hosted first.
The shared pattern: a read-through cache primitive, a writable hosted primitive, and an ordered composition served at one URL where your own content wins over upstream. peryx names these cached, hosted, and virtual.
peryx's three roles
- A cached index proxies and caches one upstream, with its own credentials.
- A hosted index stores uploads;
upload_tokengates writes andvolatilegates deletion. - A virtual index serves an ordered list of other indexes under one route. PyPI defaults to first-match per filename and unioned versions; its source policy can instead select hosted candidates at project level or disable cached fallback. Uploads land in the virtual index's designated hosted layer. A layer can be another virtual index.
The default filename-level mode lets you override one broken wheel of an upstream release while its sdist and other
wheels continue to come from the cached layer. PyPI virtual indexes can instead use private-first, where any hosted
candidate for the normalized project hides every cached candidate, or no-fallback, where the virtual index does not
query its immediate cached members. See fallback_mode.
Why shadowing is a security control
The usual way to mix private and public packages is client-side: a private index in --extra-index-url, pypi.org as the
default. pip treats both indexes as equals and installs whichever offers the higher version. Anyone who registers your
internal package's name on pypi.org with version 99.0 now wins the race. This is
dependency confusion, the technique that
compromised PyTorch's nightly channel and, in its original
disclosure, three dozen major companies. Client-side mitigations exist (uv's explicit index pinning, for one) but must
be repeated in every project, for every tool, forever.
A virtual index moves the decision server-side. Give the client one index-url and configure the virtual index with
private-first: a name with hosted candidates then does not expose cached candidates. The rule works with Simple API
clients such as pip and uv because it lives where the indexes meet. Use protected_names as well when an absent or
deleted private project must remain blocked upstream.
The default fallback mode keeps filename-level merging for compatibility and is not a project-level
dependency-confusion boundary. no-fallback is stricter at the configured virtual boundary, but a nested virtual member
uses its own mode, and a client can still bypass this policy by configuring another index or --extra-index-url.
Removal semantics
peryx distinguishes hiding an artifact from destroying it, and keeps both (the PyPI names appear here; each ecosystem maps them to its own protocol):
- Yank (PEP 592) marks a file so resolvers skip it while exact-pin installs still succeed. It is reversible and is the right tool for a bad release that someone may already depend on.
- Delete removes uploaded records outright and is only allowed on
volatilehosted indexes. For a virtual index this un-shadows the upstream file. The content-addressed blob stays, since another index may reference the same digest. - Upstream files cannot be modified on their cached index, so yanking or deleting one through a virtual index
records an override (
yankedorhidden) on the virtual index's hosted layer. The cached index's own route is untouched, the override applies wherever that hosted layer serves, andrestoreclears a hidden marker. This is how a broken upstream release is pulled from your resolvers within seconds, reversibly, without forking the cache.
The default topology
Out of the box peryx runs one trio per ecosystem: a cached index of the public upstream, an empty hosted index, and a
virtual index combining them. For PyPI that is a pypi.org cache behind root/pypi; for OCI, a
Docker Hub cache behind root/oci. Each virtual URL serves the whole public index for its
ecosystem, and the day you need to host a private artifact you add a token to that ecosystem's hosted index; nothing
about the client setup changes.