In this super light Mistral Devstral Provides COLAB-friendly guide tutorials designed for users facing disk space constraints. Running large language models can be a challenge in environments with limited storage and memory, but this tutorial shows how to deploy a powerful Devstral-Small model. Active quantification by using BitsandBytes, Cache Management, and efficient token generation, this tutorial allows you to build a fast, interactive and disk-conscious lightweight assistant. Whether you’re debugging code on the go, writing widgets, or prototyping, this setting ensures your footprint gets the highest performance at the least.
!pip install -q kagglehub mistral-common bitsandbytes transformers --no-cache-dir
!pip install -q accelerate torch --no-cache-dir
import shutil
import os
import gcThe tutorial starts with the required lightweight packages, such as KaggleHub, Mistral-Common, BitsandBytes, and Transformers, to ensure that no cache is stored to minimize disk usage. It also includes acceleration and torch for efficient model loading and reasoning. To further optimize the space, use Python’s Shutil, OS and GC modules to clear any pre-existing caches or temporary directories.
def cleanup_cache():
"""Clean up unnecessary files to save disk space"""
cache_dirs = ['/root/.cache', '/tmp/kagglehub']
for cache_dir in cache_dirs:
if os.path.exists(cache_dir):
shutil.rmtree(cache_dir, ignore_errors=True)
gc.collect()
cleanup_cache()
print("🧹 Disk space optimized!")To maintain a minimum disk footprint throughout the execution, the Clearup_cache() function is defined as deleting redundant cache directories such as /root/.cache and /tmp/kaggleHub. This active cleaning helps free up space before and after key operations. After being called, this function confirms that the disk space has been optimized, thereby enhancing the tutorial’s focus on resource efficiency.
import warnings
warnings.filterwarnings("ignore")
import torch
import kagglehub
from mistral_common.protocol.instruct.messages import UserMessage
from mistral_common.protocol.instruct.request import ChatCompletionRequest
from mistral_common.tokens.tokenizers.mistral import MistralTokenizer
from transformers import AutoModelForCausalLM, BitsAndBytesConfigTo ensure smooth execution without dispersing warning messages, we use Python’s warning module to suppress all runtime warnings. It then imports the required libraries for model interactions, including a torch for tensor computing, a KaggleHub for streaming models, and a transformer for loading quantized LLMs. Specific courses specific to the class such as Usermessage, ChatCompletionRequest, and MistralTokenizer are also packaged to handle tokenized and request formats tailored for Devstral’s architecture.
class LightweightDevstral:
def __init__(self):
print("📦 Downloading model (streaming mode)...")
self.model_path = kagglehub.model_download(
'mistral-ai/devstral-small-2505/Transformers/devstral-small-2505/1',
force_download=False
)
quantization_config = BitsAndBytesConfig(
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_quant_type="nf4",
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_storage=torch.uint8,
load_in_4bit=True
)
print("⚡ Loading ultra-compressed model...")
self.model = AutoModelForCausalLM.from_pretrained(
self.model_path,
torch_dtype=torch.float16,
device_map="auto",
quantization_config=quantization_config,
low_cpu_mem_usage=True,
trust_remote_code=True
)
self.tokenizer = MistralTokenizer.from_file(f'{self.model_path}/tekken.json')
cleanup_cache()
print("✅ Lightweight assistant ready! (~2GB disk usage)")
def generate(self, prompt, max_tokens=400):
"""Memory-efficient generation"""
tokenized = self.tokenizer.encode_chat_completion(
ChatCompletionRequest(messages=[UserMessage(content=prompt)])
)
input_ids = torch.tensor([tokenized.tokens])
if torch.cuda.is_available():
input_ids = input_ids.to(self.model.device)
with torch.inference_mode():
output = self.model.generate(
input_ids=input_ids,
max_new_tokens=max_tokens,
temperature=0.6,
top_p=0.85,
do_sample=True,
pad_token_id=self.tokenizer.eos_token_id,
use_cache=True
)[0]
del input_ids
torch.cuda.empty_cache() if torch.cuda.is_available() else None
return self.tokenizer.decode(output[len(tokenized.tokens):])
print("🚀 Initializing lightweight AI assistant...")
assistant = LightweightDevstral()We define the LightWeaightDevstral class, a core component of the tutorial, which handles model loading and text generation in a resource-efficient way. It first uses KaggleHub to stream the Devstral-Small-2505 model, avoiding redundant downloads. The model then loads positive 4-bit quantization via BitsandBytesConfig, which greatly reduces memory and disk usage while still being inferred. Initialize the custom token from the local JSON file and clear the cache immediately. Generation methods adopt memory security practices such as torch.inference_mode() and empty_cache() to efficiently generate responses, making the assistant even suitable for environments with tight hardware constraints.
def run_demo(title, prompt, emoji="🎯"):
"""Run a single demo with cleanup"""
print(f"n{emoji} {title}")
print("-" * 50)
result = assistant.generate(prompt, max_tokens=350)
print(result)
gc.collect()
if torch.cuda.is_available():
torch.cuda.empty_cache()
run_demo(
"Quick Prime Finder",
"Write a fast prime checker function `is_prime(n)` with explanation and test cases.",
"🔢"
)
run_demo(
"Debug This Code",
"""Fix this buggy function and explain the issues:
```python
def avg_positive(numbers):
total = sum([n for n in numbers if n > 0])
return total / len([n for n in numbers if n > 0])
```""",
"🐛"
)
run_demo(
"Text Tool Creator",
"Create a simple `TextAnalyzer` class with word count, char count, and palindrome check methods.",
"🛠️"
)
Here we use the run_demo() function to demonstrate the coding capabilities of the model through a compact demo suite. Each demo sends a prompt to the Devstral Assistant and prints the generated response, then immediately performs a memory cleanup to prevent stacking in multiple runs. These examples include writing effective Prime checking capabilities, debugging Python snippets with logical flaws, and building mini Textanalyzer classes. These demonstrations highlight the practicality of the model as a lightweight, conscious coding assistant capable of real-time code generation and interpretation.
def quick_coding():
"""Lightweight interactive session"""
print("n🎮 QUICK CODING MODE")
print("=" * 40)
print("Enter short coding prompts (type 'exit' to quit)")
session_count = 0
max_sessions = 5
while session_count We introduced the Quick Coding Mode, a lightweight interactive interface that allows users to submit short encoding prompts directly to Devstral Assistant. Designed to limit memory usage, the session limits interactions to five prompts, each with aggressive memory cleanup to ensure continuous response in low-resource environments. The Assistant responds with concise, truncated code suggestions, making this mode great for rapid prototyping, debugging, or exploring coding concepts without overwhelming the disk or memory capacity of your notebook.
def check_disk_usage():
"""Monitor disk usage"""
import subprocess
try:
result = subprocess.run(['df', '-h', '/'], capture_output=True, text=True)
lines = result.stdout.split('n')
if len(lines) > 1:
usage_line = lines[1].split()
used = usage_line[2]
available = usage_line[3]
print(f"💾 Disk: {used} used, {available} available")
except:
print("💾 Disk usage check unavailable")
print("n🎉 Tutorial Complete!")
cleanup_cache()
check_disk_usage()
print("n💡 Space-Saving Tips:")
print("• Model uses ~2GB vs original ~7GB+")
print("• Automatic cache cleanup after each use")
print("• Limited token generation to save memory")
print("• Use 'del assistant' when done to free ~2GB")
print("• Restart runtime if memory issues persist")Finally, we provide cleaning routines and a useful disk usage monitor. Using Python’s subprocess module uses the DF -H command, which shows the used and available disk space, confirming the lightweight nature of the model. After restarting Cleanup_Cache() to ensure minimal residue, the script ends with a practical set of space-saving tips.
In short, we can now take advantage of the functionality of Mistral’s Devstral model in space-constrained environments like Google Colab without compromising usability or speed. The model is loaded in a highly compressed format, performing efficient text generation and ensuring memory is cleared immediately after use. By including interactive coding modes and a demo suite, users can quickly and seamlessly test their ideas.
Check Code. All credits for this study are to the researchers on the project. Also, please stay tuned for us twitter And don’t forget to join us 100K+ ml reddit And subscribe Our newsletter.
Asif Razzaq is CEO of Marktechpost Media Inc. As a visionary entrepreneur and engineer, ASIF is committed to harnessing the potential of artificial intelligence to achieve social benefits. His recent effort is to launch Marktechpost, an artificial intelligence media platform that has an in-depth coverage of machine learning and deep learning news that can sound both technically, both through technical voices and be understood by a wide audience. The platform has over 2 million views per month, demonstrating its popularity among its audience.
