emese / folyó
hu / en
folyó
folyó
4B · 128k context
Parameters4.0B
Context length131,072 tokens
ArchitectureLLaMA-style decoder-only transformer
Layers32
Attention heads32 (8 KV heads, GQA)
Hidden dim3,072
Vocabulary32,000 (SentencePiece Unigram)
NormalizationRMSNorm (pre-norm)
Positional encodingRoPE
ActivationSwiGLU
Training dataFull Hungarian web corpus (~35B tokens)
LicenseMIT