KV Compaction kabakaba to Attention Matching dzi

\u003ch2\u003eKV ƒe ƒoƒo kabakaba to susu ƒe sɔsɔ dzi\u003c/h2\u003e \u003cp\u003eNyati sia na gɔmesese veviwo kple nyatakaka tso eƒe tanya ŋu, si kpena ɖe sidzedze mama kple gɔmesese ŋu.\u003c/p\u003e \u003ch3\u003eNu vevi siwo wotsɔna yia teƒe bubuwo\u003c/h3\u003e \u003cp\u003eNuxlẽlawo ateŋu akpɔ mɔ be yewoakpɔ viɖe:\u003c/p\u003e \u003cul\u003e \u003cli\u003eNya si ŋu woƒo nu tsoe gɔmesese deto\u003c/li\u003e \u003cli\u003eDɔwɔwɔ ŋutɔŋutɔ kple xexeame ŋutɔŋutɔ ƒe vevienyenye\u003c/li\u003e \u003cli\u003eEŋutinunyalawo ƒe nukpɔsusuwo kple numekuku\u003c/li\u003e \u003cli\u003eNyatakaka yeye siwo ku ɖe ŋgɔyiyi siwo le edzi yim fifia ŋu\u003c/li\u003e \u003c/ul\u003e \u003ch3\u003eAsixɔxɔ ƒe Nyagbɔgblɔ\u003c/h3\u003e \u003cp\u003eNyatakaka nyuiwo abe esia ene kpena ɖe sidzedze tutuɖo ŋu eye wòdoa nyametsotsowɔwɔ si ŋu wonya nu tsoe ɖe ŋgɔ le akpa vovovowo me.\u003c/p\u003e

Nyabiase Siwo Wobiana Enuenu

Nukae nye KV compaction eye nukatae wòle vevie na gbegbɔgblɔ ƒe kpɔɖeŋu gãwo?

KV (key-value) compaction fia alesi woɖea KV cache ƒe lolome dzi kpɔtɔna si gbegbɔgblɔ ƒe kpɔɖeŋu siwo wotu ɖe transformer dzi léna ɖe te le nutsotso me. Esi nya siwo ƒo xlãe ƒe didime le dzidzim ɖe edzi la, KV cache la ɖua ŋkuɖodzinu vevi aɖe, si wɔnɛ be dzidzime dzina ɖe edzi eye wòɖea seɖoƒe na dɔwɔwɔ. Nuƒoƒoƒu nyuie na be kpɔɖeŋuwo te ŋu kpɔa nɔnɔme didiwo gbɔ ŋkuɖodzinu ƒe gazazã si sɔ manɔmee, si naa ŋuɖoɖo ƒe duƒuƒu kple scalability nyona ɖe edzi tẽ na AI-ŋusẽ ƒe dɔwɔwɔwo kple mɔ̃wo.

Aleke susu ƒe sɔsɔ nana be nutete ƒe duƒuƒu nyona ɖe edzi ne wotsɔe sɔ kple mɔnu xoxowo?

KV cache pruning si wozãna tsã la nɔa te ɖe heuristics abe recency alo frequency scores dzi, si ateŋu atsɔ token siwo gakpɔtɔ nye attention-relevant aƒu gbe. Susu ƒe sɔsɔ ɖekae zãa kpɔɖeŋua ŋutɔ ƒe susu ƒe nɔnɔmewo boŋ tsɔ dea dzesi KV ƒe nuŋɔŋlɔ siwo mehiã ŋutɔŋutɔ o. To compaction nyametsotsowo tsɔtsɔ sɔ kple susu ƒe kpekpeme ŋutɔŋutɔwo me la, mɔnua kpɔa cache dzi ɖeɖe kpɔtɔ kabakaba, si sɔ pɛpɛpɛ wu kple nyonyome ƒe gbegblẽ suetɔ kekeake, si wɔnɛ be wòxɔ asi ŋutɔ le latency-sensitive production environments.

Ðe woateŋu azã mɔnu sia ɖe xexeame ŋutɔŋutɔ ƒe AI dɔwɔnuwo kple mɔ̃wo dzia?

Ẽ — KV ƒe nutete kabakaba to susu ƒe tsɔtsɔ sɔ kple wo nɔewo dzi sɔ ŋutɔ na nuwɔwɔ AI ɖoɖowo. Mɔ̃wo abe Mewayz, siwo naa modules siwo wu 207 siwo wotsɔ wɔ ɖekae hena $19/ɣleti ko, ateŋu awɔ asitɔtrɔ mawo ŋudɔ atsɔ awɔ AI dɔwɔwɔ ƒe agba siwo wɔa dɔ nyuie wu le woƒe dɔwɔnuwo katã me. Nutsotso ƒe gazazã dzi ɖeɖe kpɔtɔ fia be ŋuɖoɖo kabakaba, akɔntabubu ƒe gazazãwo dzi ɖeɖe kpɔtɔ, kple ŋutete be woado alɔ zãlawo ƒe kadodo didi, siwo sesẽ wu evɔ womatsɔ dɔwɔwɔ alo kakaɖedzi asa vɔe o.

Ðe mehiã xɔtunu tɔxɛwo be makpɔ viɖe tso KV ƒe nutete ƒe mɔnuwo mea?

Menye kokoko o. Togbɔ be GPU deŋgɔwo naa dɔwɔwɔa kabakaba hã la, susu ƒe tsɔtsɔ sɔ kple wo nɔewo koŋ nye kɔmpiutadziɖoɖowo ƒe nyonyome si ate ŋu ahe viɖewo vɛ le xɔtunu ƒe ɖoɖo vovovowo me. Dɔwɔla siwo tsɔa AI ƒe nɔnɔmewo dea woƒe dɔwɔwɔ me — le kpɔɖeŋu me, wozãa mɔ̃wo abe Mewayz (207 modules, $19/mo) — kpɔa viɖe le mɔ si mele tẽ o nu esime kpɔɖeŋu subɔsubɔ si le ete la va le bɔbɔe wu, si wɔnɛ be AI ƒe ŋutete siwo wɔa dɔ nyuie wu la te ŋu wɔa dɔ nyuie evɔ mehiã be woatsɔ gadede xɔtuɖoɖowo me ɖe adzɔgbe o.

ƒe nyawo