Support for gemma3 from google

[bigtan]

请更新ollama，已支持gemma3

Error: llama runner process has terminated: this model is not supported by your version of Ollama. You may need to upgrade

[puffer-duck]

Need to upgrade to Ollama v0.6, this should add support for gemma3

[bigtan]

@puffer-duck But v0.6 not support intel card accelerate, am i right?

[ForstJean]

"error": {
"message": "llama runner process has terminated: this model is not supported by your version of Ollama. You may need to upgrade"

[tombii]

Yes we need this. Can Intel publish patches to Ollama so that we can compile ourselves? Or setup automatic nightly builds that follow the latest version of Ollama.
Just got my A770 but can still return it, it seems LLM/Ollama is not straight forward with Intel still :(

[digitalextremist]

Note this was brought up here: #12950

That is generally about the version disparity versus just gemma3 ( but now mentioning that model as another reason )

[yizhangliu]

Please！

[cyita]

Hi All,

Gemma3 is now supported in ipex-llm llamacpp! (Ollama support is in progress—we'll provide updates once it's ready.)

Important Notes:

The 27B Gemma3 q4_k_m model requires >16GB VMem.

• In text mode, you can use -c 128 to optimize memory usage.
• In vision mode (llama-gemma3-cli), you may need two Arc GPUs or a GPU with larger memory.

Get Started:

Please follow the following steps to try it out:

1. Download the latest ipex-llm llamacpp portable zip:

https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md

2. Get mmproj.gguf & gemma3 gguf model files

Please download the pre-quantized version from HF: https://huggingface.co/collections/ggml-org/gemma-3-67d126315ac810df1ad9e913

(You must download both the text model and the mmproj file)

Note: Vision capability is available on these model sizes: 4b, 12b and 27b

3. Run gemma3

3.1 Linux

ngl=99
thread=8

3.1.1 Text only

./llama-cli -m $model_path --no-context-shift -n 128 --prompt "It is done, and submitted. You can play 'Survival of the Tastiest' on Android, and on the web. Playing on the web works, but you have to simulate multiple touch for table moving and that can be a bit confusing. There is a lot I'd like to talk about. I will go through every topic, insted of making the typical what went right/wrong list. Concept Working over the theme was probably one of the hardest tasks which I had to face. Originally, I had an idea of what kind of game I wanted to develop, gameplay wise - something with a lot of enemies/actors, simple graphics, maybe set in space, controlled from a top-down view. I was confident that I could fit any theme around it. In the end, the problem with a theme like 'Evolution' in a game is that evolution is unassisted. It happens through several seemingly random mutations over time, with the most apt permutation surviving. This genetic car simulator is, in my opinion, a great example of actual evolution of a species facing a challenge. But is it a game? In a game, you need to control something to reach an objective. That control goes against what evolution is supposed to be like. If you allow the user to pick how to evolve something, it's not evolution anymore - it's the equivalent of intelligent design, the fable invented by creationists to combat the idea of evolution. Being agnostic and a Pastafarian, that's not something that rubbed me the right way. Hence, my biggest dillema when deciding what to create was not with what I wanted to create, but with what I did not. I didn't want to create an 'intelligent design' simulator and wrongly call it evolution. This is a problem, of course, every other contestant also had to face. And judging by the entries submitted, not many managed to work around it. I'd say the only real solution was through the use of artificial selection, somehow. So far, I haven't seen any entry using this at its core gameplay. Alas, this is just a fun competition and after a while I decided not to be as strict with the game idea, and allowed myself to pick whatever I thought would work out. My initial idea was to create something where humanity tried to evolve to a next level, but had some kind of foe trying to stop them from doing so. I kind of had this image of human souls flying in space towards a monolith or a space baby (all based in 2001: A Space Odyssey of course) but I couldn't think of compelling (read: serious) mechanics for that. Borgs were my next inspiration, as their whole hypothesis fit pretty well into the evolution theme. But how to make it work? Are you the borg, or fighting the Borg? The third and final idea came to me through my girlfriend, who somehow gave me the idea of making something about the evolution of Pasta. The more I thought about it the more it sounded like it would work, so I decided to go with it. Conversations with my inspiring co-worker Roushey (who also created the 'Mechanical Underdogs' signature logo for my intros) further matured the concept, as it involved into the idea of having individual pieces of pasta flying around and trying to evolve until they became all-powerful. A secondary idea here was that the game would work to explain how the Flying Spaghetti Monster came to exist - by evolving from a normal dinner table. So the idea evolved more or less into this: you are sitting a table. You have your own plate, with is your 'base'. There are 5 other guests at the table, each with their own plate. Your plate can spawn little pieces of pasta. You do so by 'ordering' them through a menu. Some pastas are better than others; some are faster, some are stronger. They have varying 'costs', which are debited from your credits (you start with a number of credits). Once spawned, your pastas start flying around. Their instinct is to fly to other plates, in order to conquer them (the objective of the game is having your pasta conquer all the plates on the table). But they are really autonomous, so after being spawned, you have no control over your pasta (think DotA or LoL creeps). Your pasta doesn't like other people's pasta, so if they meet, they shoot sauce at each other until one dies. You get credits for other pastas your own pasta kill. Once a pasta is in vicinity of a plate" -t $thread -e -ngl $ngl --color -c 2048 --temp 0

3.1.2 Single turn (Vision)

./llama-gemma3-cli -m $model_path --mmproj $mmproj_path -ngl $ngl -t $thread -p "What is in this image?" --image $image_path

3.2.2 Chat mode (Vision)

./llama-gemma3-cli -m $model_path --mmproj $mmproj_path -ngl $ngl -t $thread

3.2 WIN

3.2.1 Text only

llama-cli.exe -m %MODEL_PATH% -n 128 --prompt "It is done, and submitted. You can play 'Survival of the Tastiest' on Android, and on the web. Playing on the web works, but you have to simulate multiple touch for table moving and that can be a bit confusing. There is a lot I'd like to talk about. I will go through every topic, insted of making the typical what went right/wrong list. Concept Working over the theme was probably one of the hardest tasks which I had to face. Originally, I had an idea of what kind of game I wanted to develop, gameplay wise - something with a lot of enemies/actors, simple graphics, maybe set in space, controlled from a top-down view. I was confident that I could fit any theme around it. In the end, the problem with a theme like 'Evolution' in a game is that evolution is unassisted. It happens through several seemingly random mutations over time, with the most apt permutation surviving. This genetic car simulator is, in my opinion, a great example of actual evolution of a species facing a challenge. But is it a game? In a game, you need to control something to reach an objective. That control goes against what evolution is supposed to be like. If you allow the user to pick how to evolve something, it's not evolution anymore - it's the equivalent of intelligent design, the fable invented by creationists to combat the idea of evolution. Being agnostic and a Pastafarian, that's not something that rubbed me the right way. Hence, my biggest dillema when deciding what to create was not with what I wanted to create, but with what I did not. I didn't want to create an 'intelligent design' simulator and wrongly call it evolution. This is a problem, of course, every other contestant also had to face. And judging by the entries submitted, not many managed to work around it. I'd say the only real solution was through the use of artificial selection, somehow. So far, I haven't seen any entry using this at its core gameplay. Alas, this is just a fun competition and after a while I decided not to be as strict with the game idea, and allowed myself to pick whatever I thought would work out. My initial idea was to create something where humanity tried to evolve to a next level, but had some kind of foe trying to stop them from doing so. I kind of had this image of human souls flying in space towards a monolith or a space baby (all based in 2001: A Space Odyssey of course) but I couldn't think of compelling (read: serious) mechanics for that. Borgs were my next inspiration, as their whole hypothesis fit pretty well into the evolution theme. But how to make it work? Are you the borg, or fighting the Borg? The third and final idea came to me through my girlfriend, who somehow gave me the idea of making something about the evolution of Pasta. The more I thought about it the more it sounded like it would work, so I decided to go with it. Conversations with my inspiring co-worker Roushey (who also created the 'Mechanical Underdogs' signature logo for my intros) further matured the concept, as it involved into the idea of having individual pieces of pasta flying around and trying to evolve until they became all-powerful. A secondary idea here was that the game would work to explain how the Flying Spaghetti Monster came to exist - by evolving from a normal dinner table. So the idea evolved more or less into this: you are sitting a table. You have your own plate, with is your 'base'. There are 5 other guests at the table, each with their own plate. Your plate can spawn little pieces of pasta. You do so by 'ordering' them through a menu. Some pastas are better than others; some are faster, some are stronger. They have varying 'costs', which are debited from your credits (you start with a number of credits). Once spawned, your pastas start flying around. Their instinct is to fly to other plates, in order to conquer them (the objective of the game is having your pasta conquer all the plates on the table). But they are really autonomous, so after being spawned, you have no control over your pasta (think DotA or LoL creeps). Your pasta doesn't like other people's pasta, so if they meet, they shoot sauce at each other until one dies. You get credits for other pastas your own pasta kill. Once a pasta is in vicinity of a plate" -t 8 -e -ngl 99 --color --ctx-size 1200 --no-mmap

3.2.2 Single turn (Vision)

llama-gemma3-cli.exe -m %MODEL_PATH% --mmproj %MMPROJ_PATH% -ngl 99 -t 8 -p "What is in this image?" --image %IMAGE_PATH%

3.2.3 Chat mode (Vision)

llama-gemma3-cli.exe -m %MODEL_PATH% --mmproj %MMPROJ_PATH% -ngl 99 -t 8 

[ForstJean]

Hi All,

Gemma3 is now supported in ipex-llm llamacpp! (Ollama support is in progress—we'll provide updates once it's ready.)

Important Notes:

The 27B Gemma3 q4_k_m model requires >16GB VMem.

• In text mode, you can use -c 128 to optimize memory usage.
• In vision mode (llama-gemma3-cli), you may need two Arc GPUs or a GPU with larger memory.

Get Started:

Please follow the following steps to try it out:

1. Download the latest ipex-llm llamacpp portable zip:

https://github.com/intel/ipex-llm/blob/main/docs/mddocs/Quickstart/llamacpp_portable_zip_gpu_quickstart.md

2. Get mmproj.gguf & gemma3 gguf model files

Please download the pre-quantized version from HF: https://huggingface.co/collections/ggml-org/gemma-3-67d126315ac810df1ad9e913

(You must download both the text model and the mmproj file)

Note: Vision capability is available on these model sizes: 4b, 12b and 27b

3. Run gemma3

3.1 Linux

ngl=99 thread=8

3.1.1 Text only

./llama-cli -m $model_path --no-context-shift -n 128 --prompt "It is done, and submitted. You can play 'Survival of the Tastiest' on Android, and on the web. Playing on the web works, but you have to simulate multiple touch for table moving and that can be a bit confusing. There is a lot I'd like to talk about. I will go through every topic, insted of making the typical what went right/wrong list. Concept Working over the theme was probably one of the hardest tasks which I had to face. Originally, I had an idea of what kind of game I wanted to develop, gameplay wise - something with a lot of enemies/actors, simple graphics, maybe set in space, controlled from a top-down view. I was confident that I could fit any theme around it. In the end, the problem with a theme like 'Evolution' in a game is that evolution is unassisted. It happens through several seemingly random mutations over time, with the most apt permutation surviving. This genetic car simulator is, in my opinion, a great example of actual evolution of a species facing a challenge. But is it a game? In a game, you need to control something to reach an objective. That control goes against what evolution is supposed to be like. If you allow the user to pick how to evolve something, it's not evolution anymore - it's the equivalent of intelligent design, the fable invented by creationists to combat the idea of evolution. Being agnostic and a Pastafarian, that's not something that rubbed me the right way. Hence, my biggest dillema when deciding what to create was not with what I wanted to create, but with what I did not. I didn't want to create an 'intelligent design' simulator and wrongly call it evolution. This is a problem, of course, every other contestant also had to face. And judging by the entries submitted, not many managed to work around it. I'd say the only real solution was through the use of artificial selection, somehow. So far, I haven't seen any entry using this at its core gameplay. Alas, this is just a fun competition and after a while I decided not to be as strict with the game idea, and allowed myself to pick whatever I thought would work out. My initial idea was to create something where humanity tried to evolve to a next level, but had some kind of foe trying to stop them from doing so. I kind of had this image of human souls flying in space towards a monolith or a space baby (all based in 2001: A Space Odyssey of course) but I couldn't think of compelling (read: serious) mechanics for that. Borgs were my next inspiration, as their whole hypothesis fit pretty well into the evolution theme. But how to make it work? Are you the borg, or fighting the Borg? The third and final idea came to me through my girlfriend, who somehow gave me the idea of making something about the evolution of Pasta. The more I thought about it the more it sounded like it would work, so I decided to go with it. Conversations with my inspiring co-worker Roushey (who also created the 'Mechanical Underdogs' signature logo for my intros) further matured the concept, as it involved into the idea of having individual pieces of pasta flying around and trying to evolve until they became all-powerful. A secondary idea here was that the game would work to explain how the Flying Spaghetti Monster came to exist - by evolving from a normal dinner table. So the idea evolved more or less into this: you are sitting a table. You have your own plate, with is your 'base'. There are 5 other guests at the table, each with their own plate. Your plate can spawn little pieces of pasta. You do so by 'ordering' them through a menu. Some pastas are better than others; some are faster, some are stronger. They have varying 'costs', which are debited from your credits (you start with a number of credits). Once spawned, your pastas start flying around. Their instinct is to fly to other plates, in order to conquer them (the objective of the game is having your pasta conquer all the plates on the table). But they are really autonomous, so after being spawned, you have no control over your pasta (think DotA or LoL creeps). Your pasta doesn't like other people's pasta, so if they meet, they shoot sauce at each other until one dies. You get credits for other pastas your own pasta kill. Once a pasta is in vicinity of a plate" -t $thread -e -ngl $ngl --color -c 2048 --temp 0

3.1.2 Single turn (Vision)

./llama-gemma3-cli -m $model_path --mmproj $mmproj_path -ngl $ngl -t $thread -p "What is in this image?" --image $image_path

3.2.2 Chat mode (Vision)

./llama-gemma3-cli -m $model_path --mmproj $mmproj_path -ngl $ngl -t $thread

3.2 WIN

3.2.1 Text only

llama-cli.exe -m %MODEL_PATH% -n 128 --prompt "It is done, and submitted. You can play 'Survival of the Tastiest' on Android, and on the web. Playing on the web works, but you have to simulate multiple touch for table moving and that can be a bit confusing. There is a lot I'd like to talk about. I will go through every topic, insted of making the typical what went right/wrong list. Concept Working over the theme was probably one of the hardest tasks which I had to face. Originally, I had an idea of what kind of game I wanted to develop, gameplay wise - something with a lot of enemies/actors, simple graphics, maybe set in space, controlled from a top-down view. I was confident that I could fit any theme around it. In the end, the problem with a theme like 'Evolution' in a game is that evolution is unassisted. It happens through several seemingly random mutations over time, with the most apt permutation surviving. This genetic car simulator is, in my opinion, a great example of actual evolution of a species facing a challenge. But is it a game? In a game, you need to control something to reach an objective. That control goes against what evolution is supposed to be like. If you allow the user to pick how to evolve something, it's not evolution anymore - it's the equivalent of intelligent design, the fable invented by creationists to combat the idea of evolution. Being agnostic and a Pastafarian, that's not something that rubbed me the right way. Hence, my biggest dillema when deciding what to create was not with what I wanted to create, but with what I did not. I didn't want to create an 'intelligent design' simulator and wrongly call it evolution. This is a problem, of course, every other contestant also had to face. And judging by the entries submitted, not many managed to work around it. I'd say the only real solution was through the use of artificial selection, somehow. So far, I haven't seen any entry using this at its core gameplay. Alas, this is just a fun competition and after a while I decided not to be as strict with the game idea, and allowed myself to pick whatever I thought would work out. My initial idea was to create something where humanity tried to evolve to a next level, but had some kind of foe trying to stop them from doing so. I kind of had this image of human souls flying in space towards a monolith or a space baby (all based in 2001: A Space Odyssey of course) but I couldn't think of compelling (read: serious) mechanics for that. Borgs were my next inspiration, as their whole hypothesis fit pretty well into the evolution theme. But how to make it work? Are you the borg, or fighting the Borg? The third and final idea came to me through my girlfriend, who somehow gave me the idea of making something about the evolution of Pasta. The more I thought about it the more it sounded like it would work, so I decided to go with it. Conversations with my inspiring co-worker Roushey (who also created the 'Mechanical Underdogs' signature logo for my intros) further matured the concept, as it involved into the idea of having individual pieces of pasta flying around and trying to evolve until they became all-powerful. A secondary idea here was that the game would work to explain how the Flying Spaghetti Monster came to exist - by evolving from a normal dinner table. So the idea evolved more or less into this: you are sitting a table. You have your own plate, with is your 'base'. There are 5 other guests at the table, each with their own plate. Your plate can spawn little pieces of pasta. You do so by 'ordering' them through a menu. Some pastas are better than others; some are faster, some are stronger. They have varying 'costs', which are debited from your credits (you start with a number of credits). Once spawned, your pastas start flying around. Their instinct is to fly to other plates, in order to conquer them (the objective of the game is having your pasta conquer all the plates on the table). But they are really autonomous, so after being spawned, you have no control over your pasta (think DotA or LoL creeps). Your pasta doesn't like other people's pasta, so if they meet, they shoot sauce at each other until one dies. You get credits for other pastas your own pasta kill. Once a pasta is in vicinity of a plate" -t 8 -e -ngl 99 --color --ctx-size 1200 --no-mmap

3.2.2 Single turn (Vision)

llama-gemma3-cli.exe -m %MODEL_PATH% --mmproj %MMPROJ_PATH% -ngl 99 -t 8 -p "What is in this image?" --image %IMAGE_PATH%

3.2.3 Chat mode (Vision)

llama-gemma3-cli.exe -m %MODEL_PATH% --mmproj %MMPROJ_PATH% -ngl 99 -t 8 

Thank you for your efforts. I see that the portable zip of ollama in the pre-release has been updated to 20250313, but it seems that the ollama portable zip still cannot run gemma3 properly.

[heylobc]

will it be possible to run on server or python? many thanks

[cyita]

will it be possible to run on server or python? many thanks

We will release the Ollama portable zip with gemma3 support soon.

[sgwhat]

Hi All, you may install our latest version of ipex-llm ollama via pip install --pre --upgrae ipex-llm[cpp] to run gemma3 as below:

1. Run Ollama with GGUF Model on ModelScope

   ollama run modelscope.cn/lmstudio-community/gemma-3-4b-it-GGUF:Q4_K_M
   

2. Run Ollama with GGUF Model on HuggingFace Hub

• You may pull a GGUF model from HuggingFace Hub and then create an Ollama model with Modelfile.

  # In the Modelfile
  FROM /path/to/gemma-3-4b-it-Q4_K_M.gguf
  
  # Create an Ollama model
  ollama create gemma3-gguf -f Modelfile
  

You may see ipex-llm ollama quickstart for more details.

[yizhangliu]

@sgwhat No Ollama Portable Zip ?

[wjr1985]

I've tried the official gemma3 models in 4b and 12b, as well as the q4_K_M versions from ollama, and then also lmstudio-community/gemma-3-4b-it-GGUF:Q4_K_M from huggingface and none of them seem to work after setting up a new conda environment from the quickstart link and running pip install --pre --upgrade ipex-llm[cpp]. I've attached the logs from running ./ollama run https://huggingface.co/lmstudio-community/gemma-3-4b-it-GGUF:Q4_K_M. thank you!

ollama-gemma.txt

[ExplodingDragon]

emm ... Any new progress?