Physics-Guided Self Supervised Representation Learning for Temporal Hyperspectral Imaging

Recent advances in self-supervised learning have led to the development of powerful foundation models for computer vision and remote sensing. However, relatively little work has explored learning joint spatial, spectral, and temporal representations from hyperspectral Earth observation data. Existing approaches are largely focused on multispectral imagery or single-date hyperspectral acquisitions and are primarily designed to produce encoders for downstream supervised tasks such as classification or segmentation. As a result, the potential of hyperspectral time series for representation learning remains largely unexplored. This work addresses that gap by developing a self-supervised masked autoencoder (MAE) architecture specifically designed for temporal hyperspectral imagery, with the objective of learning physically meaningful latent representations rather than solely optimizing downstream task performance.

Our motivation extends beyond constructing a transferable feature encoder. Instead, we view the learned latent space as a nonlinear transformation of the original hyperspectral observations that can reveal the underlying structure of complex environmental processes. The objective is to compress high-dimensional spectral measurements into a compact manifold while preserving the information required to distinguish different physical and biological phenomena. For example, healthy vegetation, drought stress, nutrient deficiency, and disease are expected to occupy distinct regions of this latent space and follow different trajectories over time as their physiological state evolves. Learning these trajectories directly from large volumes of unlabeled data provides an opportunity to study vegetation dynamics, ecosystem processes, and land surface change in ways that are difficult to achieve using the original hyperspectral feature space alone.

To support this objective, the proposed architecture incorporates physically meaningful metadata directly into the representation learning process. Rather than treating spectral bands as independent channels, the model embeds the physical wavelength associated with each band, enabling it to learn relationships based on spectral proximity and reflectance physics rather than arbitrary band indices. Temporal information, including acquisition timestamps, is similarly embedded to provide context for seasonal variability, vegetation phenology, and long-term environmental change. The framework is also designed to accommodate additional remote sensing metadata, allowing information about the observation itself to influence the learned representations. By integrating these complementary sources of information, the model learns representations that are grounded not only in image appearance but also in the physical processes governing how hyperspectral observations are acquired.

Ultimately, this work aims to shift the role of foundation models in hyperspectral remote sensing from simply producing transferable encoders toward learning interpretable latent spaces that capture the structure and evolution of Earth’s surface. These latent representations serve as nonlinear feature transformations that enable visualization and analysis of high-dimensional hyperspectral temporal data, offering insight into how natural and anthropogenic processes organize and evolve over time. At the same time, they provide a strong foundation for downstream remote sensing applications, bridging the gap between representation learning and scientific understanding of hyperspectral temporal observations.