Module ops 

LLM advantage and KL ops — forwarding re-exports from rlox-rl-ops.

The implementations live in rlox-rl-ops so that rlox-sandbox can depend on that slim crate without pulling in all of rlox-core. Everything that was previously defined here is still publicly accessible under the same path (rlox_core::llm::ops::*, rlox_core::llm::ops::f32_ops::*, etc.) so existing callers — including rlox-python PyO3 bindings — compile and behave identically without any change.

The only item that remains defined here rather than re-exported is DPOPair, which is a training-plane data container unrelated to advantage estimation.

Modules

f32_ops

f64_ops

Structs

DPOPair

A DPO preference pair holding tokenized prompt, chosen, and rejected sequences.

Functions

compute_batch_group_advantages

Batched GRPO group advantages: process all groups in a single call.

compute_batch_token_kl

Batched token-level KL divergence: process all sequences in a single call.

compute_batch_token_kl_schulman

Batched token-level KL divergence using the Schulman (2020) estimator.

compute_group_advantages

GRPO group advantage: (reward - mean) / std. Returns zeros if std < 1e-8.

compute_token_kl

Token-level KL divergence: sum(exp(log_p) * (log_p - log_q)).

compute_token_kl_schulman

Token-level KL divergence using the Schulman (2020) estimator: sum(exp(log_p - log_q) - (log_p - log_q) - 1).