# `Vllm.Tokenizers.TokenizerLike`
[🔗](https://github.com/nshkrdotcom/vllm/blob/v0.3.0/lib/snakebridge_generated/vllm/tokenizers/tokenizer_like.ex#L7)

Wrapper for Python class TokenizerLike.

# `t`

```elixir
@opaque t()
```

# `all_special_ids`

```elixir
@spec all_special_ids(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}
```

# `all_special_tokens`

```elixir
@spec all_special_tokens(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}
```

# `apply_chat_template`

```elixir
@spec apply_chat_template(SnakeBridge.Ref.t(), [term()], [term()], keyword()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}
```

Python method `TokenizerLike.apply_chat_template`.

## Parameters

- `messages` (list(term()))
- `tools` (term() default: None)
- `kwargs` (term())

## Returns

- `term()`

# `bos_token_id`

```elixir
@spec bos_token_id(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}
```

# `convert_ids_to_tokens`

```elixir
@spec convert_ids_to_tokens(SnakeBridge.Ref.t(), [integer()], [term()], keyword()) ::
  {:ok, [String.t()]} | {:error, Snakepit.Error.t()}
```

Python method `TokenizerLike.convert_ids_to_tokens`.

## Parameters

- `ids` (list(integer()))
- `skip_special_tokens` (boolean() default: False)

## Returns

- `list(String.t())`

# `convert_tokens_to_ids`

```elixir
@spec convert_tokens_to_ids(SnakeBridge.Ref.t(), term(), keyword()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}
```

Python method `TokenizerLike.convert_tokens_to_ids`.

## Parameters

- `tokens` (term())

## Returns

- `term()`

# `convert_tokens_to_string`

```elixir
@spec convert_tokens_to_string(SnakeBridge.Ref.t(), [String.t()], keyword()) ::
  {:ok, String.t()} | {:error, Snakepit.Error.t()}
```

Python method `TokenizerLike.convert_tokens_to_string`.

## Parameters

- `tokens` (list(String.t()))

## Returns

- `String.t()`

# `decode`

```elixir
@spec decode(SnakeBridge.Ref.t(), term(), [term()], keyword()) ::
  {:ok, String.t()} | {:error, Snakepit.Error.t()}
```

Python method `TokenizerLike.decode`.

## Parameters

- `ids` (term())
- `skip_special_tokens` (boolean() default: False)

## Returns

- `String.t()`

# `encode`

```elixir
@spec encode(SnakeBridge.Ref.t(), String.t(), [term()], keyword()) ::
  {:ok, [integer()]} | {:error, Snakepit.Error.t()}
```

Python method `TokenizerLike.encode`.

## Parameters

- `text` (String.t())
- `truncation` (term() default: None)
- `max_length` (term() default: None)
- `add_special_tokens` (boolean() default: True)

## Returns

- `list(integer())`

# `eos_token_id`

```elixir
@spec eos_token_id(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}
```

# `from_pretrained`

```elixir
@spec from_pretrained(SnakeBridge.Ref.t(), term(), [term()], keyword()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}
```

Python method `TokenizerLike.from_pretrained`.

## Parameters

- `path_or_repo_id` (term())
- `args` (term())
- `trust_remote_code` (boolean() keyword-only default: False)
- `revision` (term() keyword-only default: None)
- `download_dir` (term() keyword-only default: None)
- `kwargs` (term())

## Returns

- `term()`

# `get_added_vocab`

```elixir
@spec get_added_vocab(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, %{optional(String.t()) => integer()}} | {:error, Snakepit.Error.t()}
```

Python method `TokenizerLike.get_added_vocab`.

## Returns

- `%{optional(String.t()) => integer()}`

# `get_vocab`

```elixir
@spec get_vocab(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, %{optional(String.t()) => integer()}} | {:error, Snakepit.Error.t()}
```

Python method `TokenizerLike.get_vocab`.

## Returns

- `%{optional(String.t()) => integer()}`

# `is_fast`

```elixir
@spec is_fast(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
```

# `max_token_id`

```elixir
@spec max_token_id(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}
```

# `new`

```elixir
@spec new(
  [term()],
  keyword()
) :: {:ok, SnakeBridge.Ref.t()} | {:error, Snakepit.Error.t()}
```

Constructs `TokenizerLike`.

## Parameters

- `args` (term())
- `kwargs` (term())

# `num_special_tokens_to_add`

```elixir
@spec num_special_tokens_to_add(
  SnakeBridge.Ref.t(),
  keyword()
) :: {:ok, integer()} | {:error, Snakepit.Error.t()}
```

Python method `TokenizerLike.num_special_tokens_to_add`.

## Returns

- `integer()`

# `pad_token_id`

```elixir
@spec pad_token_id(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}
```

# `truncation_side`

```elixir
@spec truncation_side(SnakeBridge.Ref.t()) ::
  {:ok, term()} | {:error, Snakepit.Error.t()}
```

# `vocab_size`

```elixir
@spec vocab_size(SnakeBridge.Ref.t()) :: {:ok, term()} | {:error, Snakepit.Error.t()}
```

---

*Consult [api-reference.md](api-reference.md) for complete listing*