Skip to content

Building vim from source with Anaconda

Jump to bottom Edit New page
Louis Maddox edited this page Jul 8, 2019 · 28 revisions

Background

I successfully installed Black, the Python code autoformatter, and found that a requirement for it to work inside vim (which is a nice feature of vim-go) is that your installation of vim must be compiled with Python 3.6 or later.

  • The error message received was a failure to build an f-string (which was of course introduced in 3.6): 
    Error detected while processing /home/louis/.vim/vimplug/black/plugin/black.vim: 
line  135:
  File "<string>", line 24
    return venv_path / 'lib' / f'python{pyver[0]}.{pyver[1]}' / 'site-packages'
                                                            ^
SyntaxError: invalid syntax

Attempted to install Python 3-built vim via APT package repository

I was nervous about messing up my vim installation, so first of all I backed up my ~/.vim/ directory to my dotfiles repo in case an installation somehow managed to wipe them (for the record it didn't, but better to be safe than sorry).

My first approach was to upgrade the standard vim, but at the time of writing this did not include Python 3 support.

sudo apt-get install --only-upgrade vim
vim --version

Next, I read that there was a version called vim-nox, which is described in the version info as "Huge version without GUI.", i.e. it's built with most of the possible features, most importantly of which are:

+python/dyn
+python3/dyn

i.e. it's built with both Python 2 and Python 3 supported (but apparently you can't run both in the same vim session, if you for some reason wanted to).

Importantly however (at the time of writing) this package does not seem to have been built with a version of Python later than 3.5, so it can't support Black. Oddly, the build info given doesn't give any idea of what version of Python it was built with:

Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H     -Wall -Wextra -g -O2 -fPIE -Wformat -Werror=format-security -fstack-protector-all -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1       
Linking: gcc   -L. -Wl,-Bsymbolic-functions -Wl,-z,relro -L/build/buildd/ruby1.9.1-1.9.3.484/debian/lib -rdynamic -Wl,-export-dynamic -Wl,-E  -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now -Wl,-z,nodlopen -Wl,-z,nodump -Wl,-z,noexecstack -Wl,-z,noexecheap -Wl,--as-needed -o vim        -lm -ltinfo -lnsl  -lselinux   -lacl -lattr -lgpm -ldl   -Wl,-E  -fstack-protector -L/usr/local/lib  -L/usr/lib/perl/5.18/CORE -lperl -ldl -lm -lpthread -lcrypt

Unlike the info given in this issue on the Black bug tracker, which [on the last line] shows it was built from /usr/lib/python3.5/:

Compilation: gcc -c -I. -Iproto -DHAVE_CONFIG_H -Wdate-time -g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=1
Linking: gcc -Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now -Wl,--as-needed -o vim -lm -ltinfo -lnsl -lselinux -lacl -lattr -lgpm -ldl -L/usr/lib/python3.5/config-3.5m-x86_64-linux-gnu -lpython3.5m -lpthread -ldl -lutil -lm

Attempted to upgrade system Python to build vim via the Deadsnakes PPA

Since it didn't seem possible to do this 'the easy way' (i.e. the safest way) through the standard APT package listings, I tried to get versions of Python from the Deadsnakes PPA (as recommended on various Q&A pages, e.g. here):

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt-get update
sudo apt-get install python3.6
sudo apt-get install python3.7

However there was no config directory underneath /usr/lib/python3.6 / /usr/lib/python3.7.

For comparison, my system Python 2.7 installation has a config directory at /usr/lib/python2.7/config-x86_64-linux-gnu/

I was following the guide here on building vim from source, and it does mention to "pay attention here check directory correct" when passing a value to the configure script with the --with-python-config-dir flag, so I looked at all files in the /usr/lib directory but neither the 3.6/3.7 versions had a config directory as far as I could tell.

I'm not going to reinstall it to check*, but now I think about it, there may have been python3.6-config and python3.7-config programs installed into /usr/lib which I could have used to provide the location of the config directories, but I didn't see them (though having said that, I didn't attempt to install them from the Deadsnakes PPA either).

  • * simply because apt-get purge didn't remove these /usr/lib/python3.* directories when it 'uninstalled' the packages, so I was uncomfortable about reinstalling something that didn't uninstall cleanly.

Building vim with Anaconda: rationale

Lastly, I am now trying to build from Anaconda, which is the source of python on my system PATH.

See:

  • Vim installation and Anaconda 
    ./configure --with-features=huge --enable-pythoninterp=dynamic --enable-python3interp=dynamic
  • ...and compare to: Building Vim from source (the guide I mentioned in the previous section), which notably also specifies this flag: 
    --with-python3-config-dir=/usr/lib/python3.5/config

The first guide also mentions being able to use these values on the config flags:

--with-python-config-dir=$(shell python-config --configdir)
--with-python3-config-dir=$(shell python3-config --configdir)

(ignore the part about 'Debian-based systems', I think they just mean 'Debian-like' Linux installations [like Ubuntu-based distros], as compared to Windows)

Building vim with Anaconda: guide

Firstly, (going off the YCM guidance), install all the prerequisite libraries:

sudo apt install libncurses5-dev libgnome2-dev libgnomeui-dev \
libgtk2.0-dev libatk1.0-dev libbonoboui2-dev \
libcairo2-dev libx11-dev libxpm-dev libxt-dev python-dev \
python3-dev ruby-dev lua5.1 liblua5.1-dev libperl-dev git
  • N.B. "On Ubuntu 16.04, liblua5.1-dev is the lua dev package name not lua5.1-dev."

Get all the information about vim so you know where to put it when you reinstall:

  • which vim --> /usr/bin/vim
  • ls -l $(which vim) --> /usr/bin/vim -> /etc/alternatives/vim
  • ls -l /etc/alternatives/vim --> /etc/alternatives/vim -> /usr/bin/vim.nox

Now remove vim - I think it's better to use purge than remove:

sudo apt purge vim vim-runtime gvim vim-tiny vim-common vim-gui-common vim-nox

but remember that you can add --dry-run at the end of the line to preview what happens.

Also note that in order to prevent apt update from overwriting the compiled version, you have to 'put a hold' on vim (as described here):

sudo apt-mark hold vim

Then follow the official instructions to install vim from its GitHub repo, for my needs this amounts to:

cd ~/opt/
git clone https://github.com/vim/vim.git
cd vim
git pull
cd src
make distclean
make
sudo make install
  • The distclean line is annotated on the official vim instructions as "if you build Vim before", which I assume means that multiple installations will leave old versions of the files lying around (which could lead to the installer picking up and using some of the old config files rather than generating new ones?)

    if make built it, then clean should delete it. if configure built it, then distclean should delete it.

Main page

Add a custom sidebar
Clone this wiki locally