Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/atharvapathak/twitter_sentiment_analysis_project

Twitter sentiment analysis is the process of analyzing tweets posted on the Twitter platform to determine the overall sentiment expressed within them. It involves using natural language processing (NLP) and machine learning techniques to classify tweets.

https://github.com/atharvapathak/twitter_sentiment_analysis_project

Science Score: 23.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ○
    .zenodo.json file
  • ○
    DOI references
  • ✓
    Academic publication links
    Links to: arxiv.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (11.1%) to scientific vocabulary

Keywords

api bag-of-words bert cnn data gbm nltk rnn spacy twitter
Last synced: 5 months ago · JSON representation

Repository

Twitter sentiment analysis is the process of analyzing tweets posted on the Twitter platform to determine the overall sentiment expressed within them. It involves using natural language processing (NLP) and machine learning techniques to classify tweets.

Basic Info
  • Host: GitHub
  • Owner: atharvapathak
  • License: mit
  • Language: Python
  • Default Branch: main
  • Homepage:
  • Size: 20.8 MB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Topics
api bag-of-words bert cnn data gbm nltk rnn spacy twitter
Created almost 2 years ago · Last pushed almost 2 years ago
Metadata Files
Readme License

README.md


1. Technologies Used

  1. Tweepy API
  2. NLTK
  3. BERT Model
  4. Tensorflow
  5. Seaborn
  6. Streamlit

2. Project Description

2.1 Data Extraction and Preprocessing

We scraped data for each illness using the Tweepy API, based on keywords and phrases for each category. Additionally, we scraped tweets that didn't contain these keywords. This data acted as the ‘neutral’ data. The data was cleaned using libraries like regex, NLTK. Links, emojis, emoticons, and symbols were removed.

2.2 DL Model

We explored Transformer models and found that BERT(Bidirectional Encoder Representations from Transformers) was better-suited for sentiment analysis. We used a pretrained BERT model and fine-tuned it on our training data. We trained a model for each class.
The output given by the final layer was not fed to any activation function; it was instead given as input to a custom function to normalize and standardize the data. The function is given below:


2.3 Visualisation and Deployment

We used Seaborn to display the caculated level of Loneliness, Stress, and Anxiety for each user across time, thus enabling us to see how the user's mental state varied over time. Moreover, we estimate the weighted average for each category, over previous tweets [0:LOW,1:HIGH]. Additonally, you can also view each specific tweet and its scores. Deployment was done using Streamlit.

3. Files

  • Cleaning Tweets.py - Script to clean scraped tweets
  • Extracting Targeted Tweets.py - Script to scrape a user's Twitter information
  • Streamlit Deployment.py - Script to deploy the project
  • Streamlit Deployment.ipynb - Jupyter Notebook to deploy the project
  • Extracted Tweets - Training Data
  • Training Models:
    • Anxiety Model.py
    • Lonely Model.py
    • Stress Model.py

4. References

5. License

MIT

Owner

  • Login: atharvapathak
  • Kind: user

GitHub Events

Total
Last Year

Committers

Last synced: 11 months ago

All Time
  • Total Commits: 6
  • Total Committers: 1
  • Avg Commits per committer: 6.0
  • Development Distribution Score (DDS): 0.0
Past Year
  • Commits: 0
  • Committers: 0
  • Avg Commits per committer: 0.0
  • Development Distribution Score (DDS): 0.0
Top Committers
Name Email Commits
Atharva Pathak a****w@g****m 6

Issues and Pull Requests

Last synced: 9 months ago

All Time
  • Total issues: 0
  • Total pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Total issue authors: 0
  • Total pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
Past Year
  • Issues: 0
  • Pull requests: 0
  • Average time to close issues: N/A
  • Average time to close pull requests: N/A
  • Issue authors: 0
  • Pull request authors: 0
  • Average comments per issue: 0
  • Average comments per pull request: 0
  • Merged pull requests: 0
  • Bot issues: 0
  • Bot pull requests: 0
View more stats
Top Authors
Issue Authors
Pull Request Authors
Top Labels
Issue Labels
Pull Request Labels