MDLMPE: Distribution Aware Positional Encoding for Masked Diffusion Language Models

Summary

Masked diffusion language models (MDLMs) enable parallel generation and bidirectional context modeling, but their positional context differs fundamentally from that of autoregressive (AR) models.