Data

Tools

Indexes

Applications

Experiments

Open Source Science

getoldtweets3

A Python 3 library and a corresponding command line utility for accessing old tweets

https://github.com/mottl/getoldtweets3

Science Score: 10.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ○
    codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ○
    Academic publication links
  • ✓
    Committers with academic emails
    1 of 17 committers (5.9%) from academic institutions
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (11.5%) to scientific vocabulary

Keywords

python3 twitter twitter-api
Last synced: 6 months ago · JSON representation

Repository

A Python 3 library and a corresponding command line utility for accessing old tweets

Basic Info
  • Host: GitHub
  • Owner: Mottl
  • License: mit
  • Language: Python
  • Default Branch: master
  • Homepage:
  • Size: 121 KB
Statistics
  • Stars: 365
  • Watchers: 23
  • Forks: 125
  • Open Issues: 46
  • Releases: 0
Archived Fork of Jefferson-Henrique/GetOldTweets-python
Topics
python3 twitter twitter-api
Created over 7 years ago · Last pushed over 2 years ago
Metadata Files
Readme License

README.md

GetOldTweets3

A Python 3 library and a corresponding command line utility for accessing old tweets.

Python 3x Build Status pypi Downloads

GetOldTweets3 is an improvement fork of the original Jefferson Henrique's GetOldTweets-python. It fixes known issues and adds features such as counting retweets, searching over multiple users accounts, etc. GetOldTweets3 supports only Python 3.

Details

Twitter Official API has the bother limitation of time constraints, you can't get older tweets than a week. Some tools provide access to older tweets but in the most of them you have to spend some money before. I was searching other tools to do this job but I didn't found it, so after analyze how Twitter Search through browser works I understand its flow. Basically when you enter on Twitter page a scroll loader starts, if you scroll down you start to get more and more tweets, all through calls to a JSON provider. After mimic we get the best advantage of Twitter Search on browsers, it can search the deepest oldest tweets.

Installation

Use pip install GetOldTweets3
or pip install -e git+https://github.com/Mottl/GetOldTweets3#egg=GetOldTweets3

Command line utility

GetOldTweets3: exports tweets to a specified csv file ("output_got.csv" by default).

Examples

Get help: bash GetOldTweets3 -h

Example 1 - Get tweets by query search: bash GetOldTweets3 --querysearch "europe refugees" --maxtweets 10

Example 1 - Get the last 10 top tweets by username: bash GetOldTweets3 --username "barackobama" --toptweets --maxtweets 10

Example 3 - Get tweets by the username and bound dates (until date is not included) and preserve emojis as unicode: bash GetOldTweets3 --username "barackobama" --since 2015-09-10 --until 2015-09-12 --maxtweets 10 --emoji unicode

Example 4 - Get tweets by several usernames: bash GetOldTweets3 --username "BarackObama,AngelaMerkeICDU" --usernames-from-file userlist.txt --maxtweets 10 (check https://github.com/Mottl/influencers for some prepared lists of usernames)

Example 5 - Get tweets by language: bash GetOldTweets3 --querysearch "bitcoin" --lang cn --maxtweets 10

Example 6 - Get tweets by place: bash GetOldTweets3 --querysearch "bitcoin" --near "Berlin, Germany" --within 25km --maxtweets 10

Example 7 - Get tweets by geo coordinates: bash GetOldTweets3 --querysearch "museum" --near "55.75, 37.61" --within 40km --maxtweets 10

Example 8 - Get tweets by minimum number of replies: bash GetOldTweets3 --querysearch "bitcoin" --minreplies 10 --maxtweets 10

Example 9 - Get tweets by minimum number of favorites: bash GetOldTweets3 --querysearch "bitcoin" --minfaves 10 --maxtweets 10

Example 10 - Get tweets by minimum number of retweets: bash GetOldTweets3 --querysearch "bitcoin" --minretweets 10 --maxtweets 10

Example 11 - Get tweets by excluding tweets with any word of a list: bash GetOldTweets3 --querysearch "bitcoin" --exclude-words-from-file excludewords.txt --maxtweets 10

where words to exclude are separated by a whitespace character in the excludewords.txt file.

Python classes

  • Tweet: Model class that describes a specific tweet.

    • id (str)
    • permalink (str)
    • username (str)
    • to (str)
    • text (str)
    • date (datetime) in UTC
    • retweets (int)
    • favorites (int)
    • mentions (str)
    • hashtags (str)
    • geo (str)

  • TweetManager: A manager class to help getting tweets in Tweet's model.

    • getTweets (TwitterCriteria): Return the list of tweets retrieved by using an instance of TwitterCriteria.

  • TwitterCriteria: A collection of search parameters to be used together with TweetManager.

    • setUsername (str or iterable): An optional specific username(s) from a twitter account (with or without "@").
    • setSince (str. "yyyy-mm-dd"): A lower bound date (UTC) to restrict search.
    • setUntil (str. "yyyy-mm-dd"): An upper bound date (not included) to restrict search.
    • setQuerySearch (str): A query text to be matched.
    • setTopTweets (bool): If True only the Top Tweets will be retrieved.
    • setNear(str): A reference location area from where tweets were generated.
    • setWithin (str): A distance radius from "near" location (e.g. 15mi).
    • setMaxTweets (int): The maximum number of tweets to be retrieved. If this number is unsetted or lower than 1 all possible tweets will be retrieved.

Examples

Get tweets by username(s): ``` python import GetOldTweets3 as got

tweetCriteria = got.manager.TweetCriteria().setUsername("barackobama whitehouse")\ .setMaxTweets(2) tweet = got.manager.TweetManager.getTweets(tweetCriteria)[0] print(tweet.text) ```

Get tweets by query search: python tweetCriteria = got.manager.TweetCriteria().setQuerySearch('europe refugees')\ .setSince("2015-05-01")\ .setUntil("2015-09-30")\ .setMaxTweets(1) tweet = got.manager.TweetManager.getTweets(tweetCriteria)[0] print(tweet.text)

Get tweets by username and bound dates and preserve emojis as unicode: python tweetCriteria = got.manager.TweetCriteria().setUsername("barackobama")\ .setSince("2015-09-10")\ .setUntil("2016-01-01")\ .setMaxTweets(1)\ .setEmoji("unicode") tweet = got.manager.TweetManager.getTweets(tweetCriteria)[0] print(tweet.text)

Get the last 10 top tweets by username: python tweetCriteria = got.manager.TweetCriteria().setUsername("barackobama")\ .setTopTweets(True)\ .setMaxTweets(10) tweet = got.manager.TweetManager.getTweets(tweetCriteria)[0] print(tweet.text)

Owner

  • Name: Dmitry Mottl
  • Login: Mottl
  • Kind: user
  • Location: #[global_allocator]

GitHub Events

Total
  • Watch event: 2
Last Year
  • Watch event: 2

Committers

Last synced: over 2 years ago

All Time
  • Total Commits: 114
  • Total Committers: 17
  • Avg Commits per committer: 6.706
  • Development Distribution Score (DDS): 0.377
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Dmitry Mottl d****l@g****m 71
Jefferson Henrique j****i@h****m 21
Ian i****n@p****m 3
Daniel Dandurand d****7@g****m 3
Norman Walsh n****w@n****m 2
Brad Rowe b****e@l****m 2
Jonathan Abdo j****o@g****m 2
Mattias Östmar m****r@g****m 1
Fernando Martinelli Ramacciotti f****i@g****m 1
Alessandro Ogier a****r@g****m 1
Jordan Taylor j****r@g****u 1
RolandColored k****g@g****m 1
BubaVV m****m@g****m 1
Riaan F Venter m****g@r****o 1
mawic m****r@g****e 1
Henry Dorfman h****n@g****m 1
MichaelKarpe 2****e 1
Committer Domains (Top 20 + Academic)
gmx.de: 1 rfv.io: 1 gatech.edu: 1 nwalsh.com: 1 phaerus.com: 1

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 87
  • Total pull requests: 15
  • Average time to close issues: 23 days
  • Average time to close pull requests: about 1 month
  • Total issue authors: 74
  • Total pull request authors: 14
  • Average comments per issue: 4.93
  • Average comments per pull request: 1.53
  • Merged pull requests: 5
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
  • irfanprabaswara (3)
  • afzal646 (3)
  • ekalhor (3)
  • humasak (2)
  • santoshbs (2)
  • JaimeBadiola (2)
  • PaulNoo (2)
  • mesapedrazas (2)
  • heeringa0 (2)
  • bhavyadeep111 (2)
  • Geor23es (1)
  • hiperjp (1)
  • Nitigya-Handa (1)
  • greencoder88 (1)
  • shahmohamadi (1)
Pull Request Authors
  • giulionf (2)
  • Mottl (1)
  • marmistrz (1)
  • mawic (1)
  • TheRealKeyboardWarrior (1)
  • ndw (1)
  • amauricio (1)
  • aogier (1)
  • jtaylor351 (1)
  • MichaelKarpe (1)
  • hans-ekbrand (1)
  • JannikSeuss (1)
  • hiperjp (1)
  • avivfaraj (1)
Top Labels
Issue Labels
question (3) bug (2) enhancement (2)
Pull Request Labels

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 368 last-month
  • Total dependent packages: 0
  • Total dependent repositories: 8
  • Total versions: 10
  • Total maintainers: 1
pypi.org: getoldtweets3

Get old tweets from Twitter

  • Versions: 10
  • Dependent Packages: 0
  • Dependent Repositories: 8
  • Downloads: 368 Last month
Rankings
Stargazers count: 3.4%
Forks count: 4.2%
Dependent repos count: 5.2%
Average: 6.5%
Downloads: 9.9%
Dependent packages count: 10.0%
Maintainers (1)
Mottl
Last synced: 6 months ago