Setting default response headers in the next.config are not overwritten by response headers set in a middleware. Based on the docs and previous deployments in a standalone builds I'd expect this to work.
I think the offending lines in OpenNext are here:
  
  
    
        
           | 
           headers = {  | 
        
        
           | 
             ...middlewareEventOrResult.responseHeaders,  | 
        
        
           | 
             ...headers,  | 
        
        
           | 
           };  | 
        
    
   
 
I could send a PR, however I think this is a breaking change for existing users, so might not be ideal to land in v3?