Struct Gateway 

pub struct Gateway {
    pub bindings: Bindings,
    pool: Arc<AccountPool>,
    client: Client,
    org_pins: Mutex<HashMap<String, OrgPin>>,
    rotation: Mutex<RotationState>,
    rotation_generation: Sender<u64>,
}

The gateway’s route handler: bindings + the account pool + rotation state + the upstream client.

Fields

bindings: Bindings

pool: Arc<AccountPool>

client: Client

org_pins: Mutex<HashMap<String, OrgPin>>

Each org’s walk order, from the config the caller loaded. Read when a session has no binding and selection must run.

rotation: Mutex<RotationState>

Exhaustion marks and cooldowns, shared with the workspace’s rate_limit_event reports. The numbers inside arrive from [gateway] at boot and again on a config reload; the cooldown and streak maps are the running state and are not replaced by a reload that leaves the numbers alone.

rotation_generation: Sender<u64>

Bumped by every cooldown write that can change what Self::select_for answers, so a reader outside this crate can announce the change instead of holding a roster snapshot the clock has moved past.

Implementations

impl Gateway

pub fn new(pool: Arc<AccountPool>, client: Client) -> Self

The caller builds client through ProviderHost, so the forward leg carries the TLS trust and the bounds every other outbound path does: host.streaming_http_client(FORWARD_CONNECT_TIMEOUT, FORWARD_IDLE_TIMEOUT).

pub fn rotation_watch(&self) -> Receiver<u64>

Subscribe to the rotation’s write notifications: every cooldown that starts (or is replaced) bumps the generation, so a reader wakes and re-reads what selection would answer.

pub fn soonest_cooldown_end(&self, now: SystemTime) -> Option<SystemTime>

The soonest live cooldown’s end, or None when nothing is cooling. A reader sleeps until it to announce the thaw, which the gateway cannot publish - the clock drives it, not a write.

fn note_rotation_changed(&self)

pub fn set_rotation_numbers(&self, numbers: RotationNumbers)

The rotation numbers from [gateway]: the 429 streak count and window, and the cooldown when no reset time is known. Called at boot and again by a config reload that moved them.

pub fn set_org_pins(&self, pins: impl IntoIterator<Item = (String, OrgPin)>)

Publish each org’s walk order (primary pin then fallbacks). Called at construction and again by a config reload.

pub fn org_pins(&self) -> Vec<(String, OrgPin)>

The published walk order per org, sorted by org name so a view renders in a stable order (the store underneath is a HashMap).

pub fn select_for( &self, org: &str, model: &str, ) -> Result<AccountKey, SelectFailure>

Which account serves model for org: the org pin lookup, the cooling filter and the declared-model walk, WITHOUT binding. This is the one selection decision - the forward binds what it returns, and a spawn or the launchpad reads it as the answer.

pub fn cooling_account_serving( &self, org: &str, model: &str, ) -> Option<AccountKey>

The account a spawn lands on first once the walk thaws: among the cooling accounts declaring model, the one whose cooldown ends soonest. That is the walk’s answer at the first moment any of them is free, which pin order alone does not give - a later-pinned account can thaw first.

A prediction made while the walk refuses, not a selection: an account still Loading when the cooldown lifts is skipped by the walk, so the read can name one that has not settled. A caller uses it only where select_for fails; the walk’s own answer replaces it as soon as the walk succeeds. Selection never calls it.

fn walk(&self, org: &str, model: &str) -> Result<AccountKey, SelectFailure>

fn select_and_bind( &self, org: &str, project: &str, session: &str, model: &str, ) -> Result<AccountKey, SelectFailure>

Select the account for an unbound session and bind it, so the session keeps it.

fn credential_for(&self, account: &AccountKey) -> Option<(String, String)>

Read the real credential and upstream base out of the bound account’s env. None when the account carries no credential in the variable its provider authenticates with.

fn probe_reset_for(&self, account: &AccountKey) -> Option<Duration>

The reset the account’s own usage probe reports for its exhausted window, when still ahead. Absent for every non-window-billed account.

fn cooldown_for(&self, account: &AccountKey, headers: &HeaderMap) -> Duration

The cooldown for a failing response: the response’s Retry-After, else the account’s own probe reset, else the configured default.

fn select_or_fail( &self, org: &str, project: &str, session: &str, model: &str, ) -> Result<AccountKey, Box<Response<StreamBody>>>

Selection, or the loud response its failure produces.

fn note_response( &self, account: &AccountKey, org: &str, project: &str, session: &str, status: StatusCode, headers: &HeaderMap, )

Apply the rotation triggers the held response proves, then stream it back untouched - the CLI’s retry carries the same session id, so rotating the binding here is what routes the retry to the next account.

fn rotate_off( &self, account: &AccountKey, org: &str, project: &str, session: &str, until: SystemTime, )

Mark account exhausted until until and drop the session’s binding, so the next request for it re-selects with the exhausted account skipped.

pub fn report_rate_limit( &self, org: &str, project: &str, session: &str, reset_at: Option<u64>, )

The workspace’s rate_limit_event report: the CLI told us the bound account’s window is not allowed, and the frame’s reset time is the cooldown. Both the exhausted account and the probe agree on a reset time or its absence; nothing else is read. A session with no binding (gateway not in use) has nothing to rotate.

pub fn is_cooling(&self, account: &AccountKey) -> bool

Whether the rotation is holding account out of selection right now - the cooldown a probe at a window’s cap writes.

A view reads it to state the selection signal without stating the usage: a probe landing at the cap moves this and nothing on the account’s loading row.

pub fn report_probe_limit(&self, account: &AccountKey, reset_at: Option<u64>)

The usage probe’s verdict: a snapshot with any window at the cap and a reset ahead proves exhaustion until that reset. Every session bound to the account rotates.

Trait Implementations

impl RouteHandler for Gateway

fn route<'life0, 'async_trait>( &'life0 self, request: Request<Bytes>, ) -> Pin<Box<dyn Future<Output = Response<StreamBody>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,