KPP Energy Model and Sampling

What does KPP Energy Model and Sampling do?

Responsible for integrating PyTorch models with the Kaiwu sampler. Model parameters, objective functions, and optimizers remain in PyTorch; the Kaiwu solver or existing sampling pipeline is connected via a compatible sampler. Do not automatically convert ordinary machine learning tasks into energy model tasks.

Selecting Components

First, select the structure based on the input, output, and relationships to be modeled, and explain how the chosen component fulfills the task requirements. When the user specifies a structure, follow it; when selecting independently, determine whether the task requires direct relationships among observed variables, hidden factors, hierarchical representations, latent variable reconstruction, context-aware generation, or feature selection, then choose the corresponding component. Model size is not the default priority; first verify if the capability and performance match, then compare training and sampling costs among viable options.

Required capability	Component	Rationale
Explain observed patterns with hidden factors, or learn visible/hidden bipartite structure	RestrictedBoltzmannMachine	Hidden representations contribute to the distribution learning required by the task; binary input alone is insufficient to justify using an RBM
Directly model pairwise interactions of binary nodes, or sample with fixed nodes	BoltzmannMachine	Nodes and their interactions correspond to problem variables; verify with task metrics whether the model is sufficient to capture the required relationships
Learn hierarchical features through multiple RBM layers	UnsupervisedDBN	The task requires representations learned layer by layer, or the user explicitly requests an RBM stack
Encoding, reconstruction, and a sampleable energy-based latent variable prior	QVAE	Both reconstruction and latent variable prior serve the task; a plain autoencoder should not be converted to QVAE
Context-dependent discrete candidate generation with energy guidance	QDiffusion	The proposal provides the required context, and the energy guides candidates based on task-relevant criteria
Select a subset of features from an existing PyTorch model's input	FeatureSelectionWrapper	The task requires input feature selection using a periodically updated binary mask

Binary data, number of labels, or the sequential order of examples alone cannot determine the selection. Validate the selection using task-required outcomes such as association, reconstruction, completion, or prediction; mere successful import or correct output shape does not imply a match. When the task requires higher-order relationships or context, pairwise statistics alone are insufficient to conclude the model is adequate. For different training structures of BM and RBM, see Model Paradigms; only read the parts relevant to the current task.

These are different components of the same KPP package; there is no need to install six separate plugins. Although feature selection internally solves a QUBO, it serves PyTorch model training and falls under this skill; pure facility location, scheduling, Max-Cut, and other tasks that directly find optimal solutions without an associated model use the Kaiwu SDK skill. If a single task simultaneously includes combinatorial optimization and energy training, load both skills and use their respective interpreters.

Minimum Closed Loop for Application Tasks

1. Use representative data to check the task performance of the selected structure, and record the real training and generation scale and time. Use this to plan the full run, and within the total budget, reserve time for model saving, loading in a new process, generation, and report verification.
2. Establish the complete entry point required by the task as early as possible. After each verification, save the best-performing and reusable model, configuration, and report; subsequent experiments use new output directories, and the delivery directory always retains a runnable version.
3. Batch generation accumulates by actual row count: when appending a batch, add that batch's row count, then truncate once the requested quantity is reached. First verify quantity, width, and values with a small request; batch count is not the same as sample count.
4. Separately check direct copy of original records, natural overlap from model sampling, and manual filtering. Common combinations may overlap in independent generation; filtering overlapping records or forcing uniqueness changes the distribution; if used, recheck business metrics such as proportions and associations.
5. Validate according to the publicly stated business requirements of the task. After saving the model, regenerate in a new process using only the delivered model and request input, and report the quality and time for that single run. Deliver upon reaching the current goal; if not met, preserve the current outcome, report honestly, and continue trying within the remaining budget, provided the above checks can be completed.

Check, Auto-Install, Verify

1. Reuse the interpreter specified by the user or available in the project, and check the versions of Python, kaiwu-torch-plugin, kaiwu, and torch. Run python -m pip check once in the same environment, then run scripts/verify_kpp.py in --imports-only mode from the skill directory. If the check results already exist from this run, reuse them and proceed to the task without additional environment scanning or trial general training.
2. Only check the installation plan and install when packages are missing; in case of dependency conflicts, offline packages, Python version issues, or wheel support problems, follow the Installation Guide and select a compatible environment based on actual metadata.
3. Reuse the compatible sampler provided by the task. Only check for appropriate licenses when selecting SDK SA/CIM; do not treat a missing SDK license as a reason that all local samplers cannot train. License initialization and non-interactive limitations are described in the installation guide.
4. Verify the selected model using the actual task program; only run the verification script in default mode when there is no task program and SDK sampler training needs to be checked. KPP_IMPORT_OK only proves import succeeded; KPP_TRAIN_OK only proves the script's small-scale training worked.

Inspection logs, temporary files, and verification copies remain in the task directory. Cloud CIM is used only when the task explicitly requires it and quota authorization is already in place. When calling Kaiwu CIM, task_name must start with kpp-skill to facilitate backend statistics; for example, kpp-skill-rbm-train. If an existing task name does not have this prefix, prepend kpp-skill-.

Training and Generation Contract

Public import paths for BM and RBM:

from kaiwu.torch_plugin import BoltzmannMachine, RestrictedBoltzmannMachine

When using SDK SA, import SimulatedAnnealingOptimizer from kaiwu.classical. BM constructor: BoltzmannMachine(num_nodes, quadratic_coef=None, linear_bias=None, device=None); RBM constructor: RestrictedBoltzmannMachine(num_visible, num_hidden, quadratic_coef=None, linear_bias=None, device=None). Set learning rate, weight decay, etc., via the PyTorch optimizer; do not pass learning_rate, k, batch_size, or other training parameters to the model constructor.

• RBM: get_hidden(batch, bernoulli=True) and sample(sampler) return the joint visible+hidden state, with width num_visible + num_hidden. The positive-phase batch and negative-phase solution pool may have different row counts; average them separately; do not force equal sample sizes. objective(positive, negative) returns a scalar. Directly write positive = model.get_hidden(batch, bernoulli=True); do not unpack the return value or concatenate the visible layer again. get_hidden defaults to bernoulli=False, returning probabilities for the hidden part; this is valid for RBM positive-phase expectations; only use bernoulli=True when delivering binary joint samples. Do not treat probabilistic states as already sampled binary samples. Configure the solution pool capacity according to the actual sampler; for SDK SA, use SimulatedAnnealingOptimizer(size_limit=...); do not invent set_config or add undefined num_solutions parameters to sample.
• SDK Sampling: sampler.solve returns a ±1 two-dimensional array with trailing reference spins. KPP's sample normalizes reference spins, removes the trailing column, and converts to a 0/1 float tensor internally. Do not remove columns or convert the return value of sample again. When the user provides fixed test solutions, solve returns the entire set of solutions unchanged; do not assign solutions to different conditions or duplicate the first row to pad sample count. Conditional sampling must simultaneously verify the fixed columns and the free columns of each converted solution; do not only check the shape.
• Training Loop: each step: optimizer.zero_grad() → positive and negative phases → objective → backward() → step(); objective and backward() must not be placed inside torch.no_grad(). sample() returns a gradient-free sampling tensor; gradients come from the energy recomputation in objective; whether sample() is inside no_grad does not affect training. Keep model, data, and manual tensors on the same device. Check positive and negative phase widths, binary nature of the negative phase, finite loss/gradient, and actual parameter updates.
• BM / DBN: Use condition_sample(sampler, s_visible) for conditional sampling; it always fixes the first s_visible.shape[1] nodes, calling the solver once per conditional row. When known positions differ across rows, rearrange per row: permute linear_bias and the symmetrized quadratic_coef so that known positions come first, build a temporary BoltzmannMachine with the permuted parameters, call condition_sample, then inverse-permute the columns back to the original order. Before using DBN, call create_rbm_layer(input_dim). After training, call mark_as_trained() before using forward/transform.
• QVAE: Can inject encoder/decoder/bm/sampler via constructor, or subclass and implement the four _create_* factory methods. config includes num_latent_units / loss_type / dist_beta / kl_beta / weight_decay. Bernoulli mean correction is optional; when used, call set_dataset_mean and set_train_bias after construction; do not assume that setting the mean will automatically update the bias. The mean can be a scalar or a per-dimension tensor; follow the current interface. Use two optimizers for encoder+decoder and BM; BM update uses bm_loss(q.detach(), weight_decay); see Model Paradigms. Final evaluation retains the input mean correction and output bias from training. q is the encoder logits; (q > 0).float() is a hard decision, not the posterior mean. The difference between native reconstruction and mean is described in the QVAE section of the reference documentation. Random reconstruction or decoding the true posterior mean is acceptable; compute and save metrics from the actual final logits, specifying the method used; do not treat the quality of a small training run as a pass condition for the interface.
• QDiffusion: proposal input [B, L], output [B, L, V] logits. For context completion, use self-attention or similar structures so that missing positions can read from other positions; per-position embedding + MLP can only learn positional preferences, and even with high accuracy, does not satisfy context completion requirements. EnergyModel subclass implements score_conditioned(noisy, candidate, attention_mask) output [B, 1]. Import SequenceTokenSpec from kaiwu.torch_plugin.qdiffusion; it is not exported at the top level. When only "vocabulary size V" is given, keep the output width V; special IDs must fall within this vocabulary and be distinct from content tokens. Do not interpret V as content vocabulary and expand to V+4. If content IDs fill V entirely or the mapping of special IDs is unclear, list the ambiguous mappings in the plan; do not adopt an expanded output width or reduce the user's existing content categories before confirmation. When training the proposal from scratch, first complete independent completion training, then freeze it and train the energy; the native objective internally truncates the proposal gradient, so it cannot be used to pretrain the proposal. Use generator.objective({"targets": batch}) for training and generator.generate(init, max_steps=...) for generation. When freeze_proposal=True, only the energy side is optimized. Full integration conditions are in Model Paradigms.
• Feature Selection (MAIFS): from kaiwu.torch_plugin import FeatureSelectionWrapper, QuadraticLinearSolver. Constructor signature: FeatureSelectionWrapper(model, feature_dim, lambda_reg=0.0, cardinality_k=None, min_selected_features=None, max_selected_features=None, solver="local_search", mask_update_epochs=None, input_feature_axis=-1, solver_kwargs=None). Training uses sel.fit_weights(loader, loss_fn, optimizer, train_epochs): loader yields (x, y), loss_fn returns a scalar, optimizer acts on sel.model.parameters(). The mask is refreshed every mask_update_epochs epochs; if None, it is only refreshed when update_mask(loader, loss_fn, hessian_mode="full"|"diagonal", max_samples=1000) is called manually. Read results via selected_indices(), num_selected(), get_support(). solver="local_search" does not require a license; "sa" and "kaiwu_cim" use the Kaiwu SDK and require a license. The time cost of update_mask is mainly in QUBO solving and count projection, growing quickly with the number of features (locally, 400 features takes about 13–15 seconds, derivative computation takes less than 1 second); hessian_mode and max_samples only affect the derivative part; reserve time based on the actual number of features. QuadraticLinearSolver().solve(Q, b) only converts the QUBO term into an Ising matrix with one auxiliary spin; it does not perform the actual solving.

Delivery and Acceptance

For new tasks, deliver a complete script including data, imports, configuration, training loop, and result output; when modifying existing code, change only the parts required by the task. If only a plan or environment record is required, save the specified document; for multi-round requirements, update the delivery and verify the file has been written. The QVAE/QDiffusion sections in the reference documentation are structural illustrations containing context variables; they cannot be used as directly runnable scripts. When the API is uncertain, locate the current kaiwu.torch_plugin.__file__ and inspect the actual signature and source code. Report the real interpreter, dependency versions, and execution results; passing RBM does not imply passing other models, GPU, or cloud CIM.