Instructions to use KitsuVp/NeoLLM with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use KitsuVp/NeoLLM with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="KitsuVp/NeoLLM", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("KitsuVp/NeoLLM", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use KitsuVp/NeoLLM with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "KitsuVp/NeoLLM"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KitsuVp/NeoLLM",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/KitsuVp/NeoLLM

SGLang

How to use KitsuVp/NeoLLM with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "KitsuVp/NeoLLM" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KitsuVp/NeoLLM",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "KitsuVp/NeoLLM" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "KitsuVp/NeoLLM",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use KitsuVp/NeoLLM with Docker Model Runner:
```
docker model run hf.co/KitsuVp/NeoLLM
```

Add pipeline tag, library name, and links to paper/code/project

by nielsr HF Staff - opened 2 days ago

base: refs/heads/main

←

from: refs/pr/2

Discussion Files changed

-4

nielsr

2 days ago

This PR adds the missing pipeline_tag (text-generation) and library_name (transformers) metadata to the model card, which makes the model discoverable and usable with the Transformers library. It also adds explicit links to the paper (Next-Latent Prediction Transformers Learn Compact World Models), the official GitHub repository, and the project page at the top of the README for easier navigation.

Add pipeline tag, library name, and links to paper/code/project322035ca

KitsuVp

Owner 1 day ago

Thanks for the contribution! I’ll add the pipeline_tag and library_name metadata in the next update.

Regarding the NextLat links, the objective is optional and is currently disabled for this training run. The model card already mentions it under Training objectives and training-time regularizers and includes the paper in the References section alongside the other papers.

When I use NextLat in a future training run, I can include links to its repository and project page in the relevant section, along with its loss and other related training information. I’d prefer not to place those links at the top, since NeoLLM incorporates techniques from many paper

KitsuVp changed pull request status to closed about 23 hours ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment