8 Commits

Include renames

Author	SHA1	Message	Date
cocoon	fde8f60a0d	fix: use torch.topk instead of calling top_k parameter as function in sample_from_logits ... In sample_from_logits(), the branch for sample_logits=False incorrectly called 'top_k' as if it were a function (top_k(probs, k=1, dim=-1)), but 'top_k' is an integer parameter in this scope. This raises: TypeError: 'int' object is not callable Fix: replace with torch.topk(probs, k=1, dim=-1) which is the correct PyTorch API for greedy (argmax) token selection. Also fix a docstring typo in decode_s2(): 'torch.torch.Tensor' -> 'torch.Tensor' Closes #231	2026-04-09 16:03:01 +00:00
Alex Liao	369bc0a70e	Auto-detect device for easier getting started	2025-12-20 11:52:32 -08:00
Maxim	b62f780de2	Refactor auto_regressive_inference to reduce memory allocations and cpu-gpy syncs.	2025-10-29 17:37:05 -03:00
Maxim	64569135b2	Remove unnecessary CUDA cache clearing - no memory leak reproducible now.	2025-10-29 12:04:43 -03:00
Maxim	21eb36afc7	Do not collect tokenizer metrics during inference	2025-10-29 12:04:43 -03:00
Pengxiao Song	e027051b38	fix: add torch.cuda.empty_cache() during autoregressive inference ... Without releasing cached GPU memory, usage will keep growing during autoregressive prediction, leading to significant memory increase or OOM. Calling torch.cuda.empty_cache() prevents this accumulation.	2025-09-02 10:26:27 +08:00
quant	38a643b761	update kronos model code	2025-09-01 21:22:27 +08:00
shiyu-coder	9f946dec6b	initial	2025-07-01 10:57:41 +08:00