AttributeError: 'TFBertEmbeddings' object has no attribute 'word_embeddings'

[git-lu]

I'm having this issue running the quick-start for classification and regression python -m lit_nlp.examples.glue_demo --port=5432 --quickstart

I've tried downgrading pytorch to version 0.4.1, but then I get another error, saying BoolTensors are not defined, so i'm guessing it must be used with pytorch > 1.x.
These are my installed packages in the environment:
lit-nlp.txt

Any help will be very much appreciated!

[jameswex]

I believe the issue is your transformers version. We pin to 4.1.1 for our demos. See https://github.com/PAIR-code/lit/blob/main/environment.yml for the conda environment and package versions we run these demos with.

[git-lu]

Yes, I was hoping it wasn't that. So now my issue is different.
I'm trying to load this model, from this library and I'm getting

which is an issue that it's solved upgrading transformers>=4.6.1

I'm pretty new with this stuff, so I don't know if this can be solved...

This is my model spec:

from lit_nlp import dev_server
from lit_nlp.examples.models.glue_models import GlueModel
from lit_nlp import server_flags
from absl import app
from tass_data import TASSSentimentData, TASSSentLabels

class BetoSentiment(GlueModel):
  """Beto Emotions Classification model on TASS."""

  def __init__(self, *args, **kw):
    super().__init__(
        *args,
        text_a_name="tweet",
        text_b_name=None,
        labels=TASSSentLabels,
        **kw)

def main(_):
    datasets = {
        'tass_sentiment_2020_train': TASSSentimentData('data/sent/train/'),

    }
    models = {
        'beto_sent': BetoSentiment('finiteautomata/beto-sentiment-analysis')
    }
    lit_demo = dev_server.Server(
        models, datasets, **server_flags.get_flags())
    dev_server.Server(models, datasets, )
    return lit_demo.serve()


if __name__ == "__main__":
    app.run(main)

And this is my dataset specs,

import os
from pathlib import Path

import pandas as pd
from lit_nlp.api import dataset as lit_dataset
from lit_nlp.api import types as lit_types


TASSSentLabels = ['POS', 'NEG', 'NEU']

def parse_sent_label(label):
    parsing = {
        'P':'POS',
        'N':'NEG',
        'NEU':'NEU'
    }
    return parsing[label]
    

class TASSSentimentData(lit_dataset.Dataset):
    """Loader for TASS text data."""

    def __init__(self, folder_path):
        os.chdir(folder_path)
        self._examples = []
        self._countries = []
        for file in os.listdir():
            df = pd.read_csv(file, sep='\t', names=['id', 'tweet', 'label'])
            country = Path(file).name
            self._countries.append(country)
            for _, row in df.iterrows():
                self._examples.append(
                    {
                        'tweet': row['tweet'],
                        'label': parse_sent_label(row['label']),
                    }
                )

    def spec(self):
        return {
            'tweet': lit_types.TextSegment(),
            'label': lit_types.CategoryLabel(vocab=TASSSentLabels)
        }

And the data can be downloaded from here http://tass.sepln.org/2020/?page_id=74 , it's the first task :)

Thank you for the quick answer!

[jameswex]

Thanks for all the info. So seems like you'll need to use a newer transformers version and update our glue_models.py code to work with that updated version. Per AndreasMadsen/python-textualheatmap#5, you could try replacing ".word_embeddings" with ".weights" on line 339 in glue_models.py and see if that helps (though there may be other incompatibilities once you get beyond that)

[krlng]

Tried that advice, but ran into some unaligned vectors error afterwards:

Shapes of all inputs must match: values[0].shape = [30522,128] != values[1].shape = [2,128] [Op:Pack] name: packed

Are plan to update the dependencies?

[jameswex]

Not that we pin to transformers 4.1.1 just for our demos, due to us having that same version internally as well. If you want to use LIT with your own model code as opposed to the code in our examples/ dir then you can depend on a more recent version of transformers without issue.