Quantum enhanced knowledge distillation

In a context where quantum computing still faces the limitations of Noisy Intermediate-Scale Quantum (NISQ) hardware, the challenge is clear: how can quantum Machine Learning be made truly useful today? The answer comes from a technique already well-known in the classical AI world, but rarely explored at scale in the quantum domain: Knowledge Distillation (KD).

In essence, KD is a process where a powerful and complex model (the “teacher”) guides the training of a simpler model (the “student”), transferring not just labels but structured insights learned during classification. The breakthrough is that this knowledge transfer can now take place from a classical model (like an MLP) to a hybrid quantum-classical model, effectively bridging two previously distant worlds.

The case study focuses on a non-linearly separable multi-class classification problem built on an extended XOR dataset. The teacher model is an MLP with over 1200 parameters, while the student can either be a simplified classical neural network or a hybrid structure with variational quantum circuits. Thanks to KD, the student model learns more efficiently: performance improves significantly, even in complex scenarios, with a substantial reduction in parameter count (from 195 in the classical version to just 74 in some quantum architectures).

Performance metrics like the average F1 score confirm the value of the approach: even the lightest quantum models, typically penalized due to limited parameters and sensitivity to initialization, clearly benefit from distillation. The tested configurations, from universal circuits to setups with selective measurement-based compression, prove that this method works and paves the way for more effective quantum systems, even on constrained hardware.

This form of “cross-domain teaching” is not just a pragmatic solution for the present; it’s a strategic vision for the future. Until quantum computers reach full maturity, classical AI can serve as a guide. And when knowledge transfer is optimised through compatible structures, hybrid models become competitive, sustainable, and far more intelligent.