This is a collection of utilities to help adopt language models in Swift apps. It tries to follow the Python transformers API and abstractions whenever possible, but it also aims to provide an idiomatic Swift interface and does not assume prior familiarity with transformers or tokenizers.

Rationale and Overview

Please, check our post.


  • Tokenizers. Utilities to convert text to tokens and back. Follows the abstractions in tokenizers and transformers.js. Usage example:

import Tokenizers

func testTokenizer() async throws {
    let tokenizer = try await AutoTokenizer.from(pretrained: "pcuenq/Llama-2-7b-chat-coreml")
    let inputIds = tokenizer("Today she took a train to the West")
    assert(inputIds == [1, 20628, 1183, 3614, 263, 7945, 304, 278, 3122])

However, you don’t usually need to tokenize the input text yourself – the Generation code will take care of it.

  • Hub. Utilities to download configuration files from the Hub, used to instantiate tokenizers and learn about language model characteristics.

  • Generation. Algorithms for text generation. Currently supported ones are greedy search and top-k sampling.

  • Models. Language model abstraction over a Core ML package.

Supported Models

This package has been tested with autoregressive language models such as:

  • GPT, GPT-Neox, GPT-J.
  • SantaCoder.
  • StarCoder.
  • Falcon.
  • Llama 2.

Encoder-decoder models such as T5 and Flan are currently not supported. They are high up in our priority list.

Other Tools

Roadmap / To Do

  • exporters – Core ML conversion tool.
    • Allow max sequence length to be specified.
    • Allow discrete shapes
    • Return logits from converted Core ML model
    • Use coremltools @ main for latest fixes. In particular, this merged PR makes it easier to use recent versions of transformers.
  • Generation
    • Nucleus sampling (we currently have greedy and top-k sampling)
    • Use new top-k implementation in Accelerate.
    • Support discrete shapes in the underlying Core ML model by selecting the smallest sequence length larger than the input.
  • Optimization: cache past key-values.
  • Encoder-decoder models (T5)
  • Demo app
    • Allow system prompt to be specified.
    • How to define a system prompt template?
    • Test a code model (to stretch system prompt definition)
  • License

    Apache 2.


    View Github