Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Import archive error 'withheld_in_countries' #63

Open
pauloxnet opened this issue Jan 1, 2022 · 0 comments
Open

Import archive error 'withheld_in_countries' #63

pauloxnet opened this issue Jan 1, 2022 · 0 comments

Comments

@pauloxnet
Copy link

Importing the twitter archive I received this error:

$ twitter-to-sqlite import archive.db twitter-2021-12-31-<hash>.zip 
birdwatch-note-rating: not yet implemented
birdwatch-note: not yet implemented
branch-links: not yet implemented
community-tweet: not yet implemented
contact: not yet implemented
device-token: not yet implemented
direct-message-mute: not yet implemented
mute: not yet implemented
periscope-account-information: not yet implemented
periscope-ban-information: not yet implemented
periscope-broadcast-metadata: not yet implemented
periscope-comments-made-by-user: not yet implemented
periscope-expired-broadcasts: not yet implemented
periscope-followers: not yet implemented
periscope-profile-description: not yet implemented
professional-data: not yet implemented
protected-history: not yet implemented
reply-prompt: not yet implemented
screen-name-change: not yet implemented
smartblock: not yet implemented
spaces-metadata: not yet implemented
sso: not yet implemented
Traceback (most recent call last):
  File "/home/paulox/.virtualenvs/dogsheep/bin/twitter-to-sqlite", line 8, in <module>
    sys.exit(cli())
  File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1128, in __call__
    return self.main(*args, **kwargs)
  File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1053, in main
    rv = self.invoke(ctx)
  File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1659, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 1395, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/click/core.py", line 754, in invoke
    return __callback(*args, **kwargs)
  File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/twitter_to_sqlite/cli.py", line 759, in import_
    archive.import_from_file(db, filename, content)
  File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/twitter_to_sqlite/archive.py", line 246, in import_from_file
    db[table_name].insert_all(rows, pk=pk, replace=True)
  File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 2625, in insert_all
    self.insert_chunk(
  File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 2406, in insert_chunk
    result = self.db.execute(query, params)
  File "/home/paulox/.virtualenvs/dogsheep/lib/python3.9/site-packages/sqlite_utils/db.py", line 422, in execute
    return self.conn.execute(sql, parameters)
sqlite3.OperationalError: table archive_tweet has no column named withheld_in_countries

I found only a single tweet with the key withheld_in_countries in tweet.js that seems the problems:

[
{
    "tweet" : {
      "retweeted" : false,
      "source" : "<a href=\"http://twitter.com/download/android\" rel=\"nofollow\">Twitter for Android</a>",
      "entities" : {
        "hashtags" : [
          {
            "text" : "NowOnAndroid",
            "indices" : [
              "64",
              "77"
            ]
          }
        ],
        "symbols" : [ ],
        "user_mentions" : [
          {
            "name" : "Periscope",
            "screen_name" : "PeriscopeCo",
            "indices" : [
              "3",
              "15"
            ],
            "id_str" : "1111111111",
            "id" : "222222222"
          }
        ],
        "urls" : [
          {
            "url" : "https://t.co/xxxxxxxxx",
            "expanded_url" : "https://vine.co/v/xxxxxxxxx",
            "display_url" : "vine.co/v/xxxxxxxxxx",
            "indices" : [
              "78",
              "101"
            ]
          }
        ]
      },
      "display_text_range" : [
        "0",
        "101"
      ],
      "favorite_count" : "0",
      "id_str" : "1111111111111111111111",
      "truncated" : false,
      "retweet_count" : "0",
      "withheld_in_countries" : [
        "TR"
      ],
      "id" : "000000000000000000",
      "possibly_sensitive" : false,
      "created_at" : "Fri Aug 14 06:04:03 +0000 2015",
      "favorited" : false,
      "full_text" : "RT @periscopeco: Travel the world. LIVE. The Global Map is here #NowOnAndroid https://t.co/NZXdsPWROk",
      "lang" : "en"
    }
  }
  ]

I solved the error removing the key from the tweet.js but I'm reporting this error to improve the project.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant