Use environment variables AND config

It appears that when I use my own config file, environment variables set in the docker-compose.yaml are not respected. I'd like to keep most of my config in a file for version control, but I'd also like to have sensitive things like RPCUSER and RPCPASSWORD as environment variables, out of plain-text version control. Is there a way to achieve this?