Codestral via Azure ML Studio autocomplete struggling

[TyDunn]

Discussed in #7304

^{Originally posted by MESC2020 August 21, 2025}
Context
I'm using Codestral-2501 through Azure Machine Learning Studio. The provided FIM Endpoints are working as expected.

Issue
However, autocomplete suggestion is very basic.

Example 1
For instance, I'm trying to do the tutorial continue_tutorial.py

#                            Autocomplete provides inline code suggestions as you type.

# 1. Place cursor after `sorting_algorithm:` below and press [Enter]
# 2. Press [Tab] to accept the Autocomplete suggestion

# Basic assertion for sorting_algorithm

but Codestral returns this as suggestion:
assert sorting_algorithm([3, 1,

In Continue Console, everything seems alright.

Example 2
Or if I have this as my code:
def addTwoNumbers(l1, l2)

Codestral's suggestion is: :

Attempt to fix
I saw that there's a json configuration "multilineCompletions": "always" but json is depcrecated and it seems like yaml does not have this as configuration?

Question
So my question now: Continue specifically recommends to use codestral and yet it does not do the intended job.
Any idea why?

Edit:
Maybe you @GHuyHuynh have any idea?

[MESC2020]

@TyDunn May I add the additional finding that same issue appears through Mistral's direct API (so not just Azure related). In my test, continue sends two requests to Mistral, one /chat and one /fim where later one is (sometimes) cancelled and autocomplete suggest is very limited / not multiline. You can find more details in issue #7178

[TyDunn]

@MESC2020 Appreciate the additional info. @jpoly1219 is going to look into this

[jpoly1219]

@TyDunn @MESC2020 Thank you for your patience! This should be fixed in the recent changes I merged into main. I fixed the issue of Completion request being sent alongside FIM, and also confirmed that the expected completion is not cut out in the middle given enough timeout.

Fig 1: Assertion completes as expected.

In general, the cutting off people are seeing is because the autocomplete timeout is set to too low of a value. If you give a large amount in the Continue settings, like 2000ms for example, you will see that multiline completions work as intended. This timeout is purely setting the upper bound, so setting this to a high value won't make your autocomplete take 2 seconds for every request.

Fig 2: Autocomplete cancels with 150ms. You can see that we only render the first line of the completion.

Fig 3: Autocomplete finishes fully with 2000ms.

The way multiline autocomplete works is that we let the model try its best to output all it can in a given timeout period, and if it fails to finish the generation, we only return the first line, which is almost guaranteed to be complete.

Multiline completions just take longer, so setting it to always will often time out if not given enough leeway.

Whether this is good DX is something we can discuss with the community, but AFAIK this was a conscious decision made by the team after testing autocomplete with all sorts of configuration.

Fig 4: Set this to a higher value. Maybe not 2000, but 500 should suffice.

[MESC2020]

@jpoly1219 Thank you for fixing the double request bug.
Your explanation regarding how autocomplete works with timeouts makes sense and I'm surprised I didn't think about that sooner.

As you said, the Multiline Autocompletions = always-config gives the sense of enforcement i.e. no timeout.
But this might be purely subjective.

[github-actions]

This issue hasn't been updated in 90 days and will be closed after an additional 10 days without activity. If it's still important, please leave a comment and share any new information that would help us address the issue.