Package net.thevpc.naru.ext.models.cache

Class NaruModelCaching

java.lang.Object

net.thevpc.naru.ext.models.cache.NaruModelCaching

public final class NaruModelCaching
extends Object

Runtime entry point for prompt caching: given a request and a provider's declared support, decide what prefix (if any) can be reused, and remember what was actually sent.

Every step is defensive. A missing session extension, an unrecognised mode, a model that changed under us, an expired resource — all of them resolve to "send the whole thing", which is always correct and merely slower. Nothing in here is allowed to fail a model call.

Field Summary

Fields

Modifier and Type	Field	Description
static final long	RESOURCE_SAFETY_MARGIN_MILLIS	Milliseconds before a stateful resource's stated expiry at which we stop trusting it.

Method Summary

Modifier and Type	Method	Description
static NaruCachePlanView	apply(NaruSession session, String providerName, String modelName, NaruCachingMode mode, NaruModelRequest request)	Plan, and remember the chain in the same turn.
static void	commit(NaruSession session, String providerName, String modelName, NaruCachingMode mode, NaruCacheableContext context)	Record the prefix actually sent, so the next turn can detect a hit.
static NaruCachePlanView	plan(NaruSession session, String providerName, String modelName, NaruCachingMode mode, NaruModelRequest request)	Plan a turn.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

RESOURCE_SAFETY_MARGIN_MILLIS

public static final long RESOURCE_SAFETY_MARGIN_MILLIS

Milliseconds before a stateful resource's stated expiry at which we stop trusting it. Expiry is reported by the provider and a request can take seconds; starting a turn on a resource that dies mid-flight wastes the whole turn.

See Also:

• Constant Field Values

Method Details

plan

public static NaruCachePlanView plan(NaruSession session,
 String providerName,
 String modelName,
 NaruCachingMode mode,
 NaruModelRequest request)

Plan a turn.

Parameters:

providerName - provider identity, so one session can talk to several models

modelName - concrete model, so switching models cannot reuse a prefix

mode - provider's declared support; NONE short-circuits

commit

public static void commit(NaruSession session,
 String providerName,
 String modelName,
 NaruCachingMode mode,
 NaruCacheableContext context)

Record the prefix actually sent, so the next turn can detect a hit.

Called after a successful response only. Committing on failure would record a prefix the provider never actually stored, and the next turn would then claim a hit that turns out to cost a full-price miss.

apply

public static NaruCachePlanView apply(NaruSession session,
 String providerName,
 String modelName,
 NaruCachingMode mode,
 NaruModelRequest request)

Plan, and remember the chain in the same turn.

For providers whose cache is purely prefix-based, where the chain is knowable before the call. Providers with a stateful resource commit only after they have the server's confirmation.