Active Inference (AIF) frames adaptive behavior as the minimization of expected free energy (EFE), combining epistemic and pragmatic objectives within a single variational principle. We frame AIF as policy optimization and show that, for closed-loop control policies, EFE minimization can be formulated as a convex Markov decision process (MDP). In this formulation, the pragmatic terms are linear in the predictive state marginals and therefore equivalent to reward maximization in a latent MDP, while the epistemic value introduces a nonlinear component that distinguishes EFE minimization from standard reinforcement learning. This perspective further reveals the epistemic drive of active inference as a policy-dependent (performative) reward. We analyze finite-horizon, discounted, and average-reward formulations of EFE and derive a mirror descent (MD) algorithm that locally linearizes the objective around the current state marginals, yielding a policy-dependent reward that is compatible with actor-critic methods and dynamic programming. Finally, we argue that coupling world-model learning with policy optimization gives active inference the structure of performative reinforcement learning, providing a route toward grounding active inference within modern reinforcement learning and optimization theory, including convergence analysis and principled policy improvement guarantees.

We show that physical embodiment in active matter systems enables information processing capabilities that exceed what is possible through explicit flow sensing alone. Using microswimmers as a model system, we demonstrate that the body itself acts as a computational resource, coupling sensory and motor degrees of freedom in ways that simplify the control problem.

We introduce a framework for constrained RL where constraint satisfaction is enforced at every step. A state-action continuation probability models constraint violations as effective horizon reductions. Building on Control as Inference, we develop two off-policy algorithms: max-entropy AS-SAC, which treats violations as absorbing states, and KL-regularized VT-MPO, which halts reward accumulation on violation. VT-MPO achieves competitive performance with 4× fewer environment steps on humanoid locomotion and identifies optimal reward–constraint tradeoff regimes in Safety Gymnasium.

We present ALPARC, an open-source Python toolbox for generating artificial languages with precise control over phonological and acoustic rhythmicity. The toolbox introduces a phonological rhythmicity index to quantify confounding periodicity of phonological features at the transitional-probability rate, and uses a pseudo-random-walk approach to ensure unconfounded stationarity across streams of arbitrary length. ALPARC enables researchers to create controlled stimuli for studying statistical and rhythmic learning, with parametric control over syllable inventories, durations, stress patterns, and transitional probabilities.

Reinforcement Learning (RL) agents can solve diverse tasks but often exhibit unsafe behavior. Constrained Markov Decision Processes (CMDPs) address this by enforcing safety constraints, yet existing methods either sacrifice reward maximization or allow unsafe training. We introduce Constrained Trust Region Policy Optimization (C-TRPO), which reshapes the policy space geometry to ensure trust regions contain only safe policies, guaranteeing constraint satisfaction throughout training. We analyze its theoretical properties and connections to TRPO, Natural Policy Gradient (NPG), and Constrained Policy Optimization (CPO). Experiments show that C-TRPO reduces constraint violations while maintaining competitive returns.

In constrained Markov decision processes, enforcing constraints during training is often thought of as decreasing the final return. Recently, it was shown that constraints can be incorporated directly in the policy geometry, yielding an optimization trajectory close to the central path of a barrier method, which does not compromise final return. Building on this idea, we introduce Central Path Proximal Policy Optimization (C3PO), a simple modification of PPO that produces policy iterates, which stay close to the central path of the constrained optimization problem. Compared to existing on-policy methods, C3PO delivers improved performance with tighter constraint enforcement, suggesting that central path-guided updates offer a promising direction for constrained policy optimization.