Ansible Vault allows the vault password file to be set with an environment variable. git-diff-ansible-vault should do the same.
http://docs.ansible.com/ansible/playbooks_vault.html#running-a-playbook-with-vault
You can also set ANSIBLE_VAULT_PASSWORD_FILE environment variable, e.g. ANSIBLE_VAULT_PASSWORD_FILE=~/.vault_pass.txt and Ansible will automatically search for the password in that file.