Alex Liao
369bc0a70e
Auto-detect device for easier getting started
2025-12-20 11:52:32 -08:00
Maxim
b62f780de2
Refactor auto_regressive_inference to reduce memory allocations and cpu-gpy syncs.
2025-10-29 17:37:05 -03:00
Maxim
64569135b2
Remove unnecessary CUDA cache clearing - no memory leak reproducible now.
2025-10-29 12:04:43 -03:00
Maxim
21eb36afc7
Do not collect tokenizer metrics during inference
2025-10-29 12:04:43 -03:00
Maxim
dccfa764fc
Use pytorch SDPA implementation
2025-10-29 12:04:43 -03:00
YuCry
a7e294cc56
fix: define missing split_token in HierarchicalEmbedding
2025-10-25 23:37:30 +08:00
ShiYu
87157161d4
Bug fix
2025-09-16 10:35:14 +08:00
ShiYu
ac69e16750
Bug fix
2025-09-16 10:32:03 +08:00
Pengxiao Song
e027051b38
fix: add torch.cuda.empty_cache() during autoregressive inference
...
Without releasing cached GPU memory, usage will keep growing during autoregressive prediction, leading to significant memory increase or OOM. Calling torch.cuda.empty_cache() prevents this accumulation.
2025-09-02 10:26:27 +08:00
quant
38a643b761
update kronos model code
2025-09-01 21:22:27 +08:00
shiyu-coder
9f946dec6b
initial
2025-07-01 10:57:41 +08:00
Alex Liao