MPT (Modified Transformer) uses ALiBi or Rotary embeddings. This patch fixes rotary position cache invalidation and attention mask expansion for variable-length sequences in a custom MPT block.
Recent research has focused on "coil-only" configurations that eliminate the need for heavy permanent magnets, making the patches lighter and more flexible for use on complex surfaces. Software and AI: MPT Model Patching patch mpt