Download

and setup

It was a mad 3-month sprint to release all this as a single person. I will appreciate your understanding if some parts are not yet polished enough, and I will welcome any kind of support from now on.

Here is a github repo. It also contains a binary that should run straight away from MacOS.

github.com/kolinko/effort

huggingface.co/kolinko/mistral-buckets

Also, if you're a researcher, a GPU developer, or you'd like to implement the algorithm in your project (llama.cpp, MLX - looking at you!) - please reach out to kolinko@gmail.com Thanks!

Index

- The Main Page

- The Basics

- Introducing bucketMul

- The GPU implementation

- MoE, quantization and the others.

- Pesky details (or: Help Needed!)

- About the Author(s)

- Download and Run

- Citations, notes and so on