Science Score: 44.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • CITATION.cff file
    Found CITATION.cff file
  • codemeta.json file
    Found codemeta.json file
  • .zenodo.json file
    Found .zenodo.json file
  • DOI references
  • Academic publication links
  • Academic email domains
  • Institutional organization owner
  • JOSS paper metadata
  • Scientific vocabulary similarity
    Low similarity (13.7%) to scientific vocabulary
Last synced: 11 months ago · JSON representation ·

Repository

Basic Info
  • Host: GitHub
  • Owner: fahaddeshmukh
  • License: mit
  • Language: Jupyter Notebook
  • Default Branch: master
  • Size: 28.4 MB
Statistics
  • Stars: 0
  • Watchers: 1
  • Forks: 0
  • Open Issues: 0
  • Releases: 0
Created almost 3 years ago · Last pushed almost 3 years ago
Metadata Files
Readme License Citation

Readme.md

Semantic Duplicate Detection: Quora Question Pairs Analysis using Machine Learning

This repository contains files for a machine learning (ML)-based solution to the Quora Question Pairs problem, hosted on Kaggle. This challenge falls within the scope of natural language processing (NLP) and involves the task of identifying whether pairs of questions from the Quora platform are duplicates.

Directories

The project is structured as follows:

  • bin: This directory contains the source code files required to run the analysis.

    • quora.ipynb: This Jupyter Notebook file provides an alternative interface for running the analysis interactively. It contains the same code as src.py but provides easy accessablity along with a narrative.
  • data: This directory contains the dataset used for the analysis.

    • train.csv: The CSV file that contains the original dataset, that contains questions. This file is required for running the analysis. Please ensure that it is located in this directory.
  • docs: This directory contains any documentation related to the project.

  • plots: This directory contains the EDA plots used for the analysis.

    Installation

To use this script, you need to have Python installed on your system (3.8 or higher). You also need to install the required dependencies by running the following command:

pip install -r requirements.txt

Requirements

The script requires the following Python packages:

  • pandas
  • numpy
  • matplotlib
  • seaborn
  • scikit-learn
  • lightgbm
  • spacy
  • FuzzyWuzzy

You can install these packages by running the command mentioned in the "Installation" section.

License

This project is licensed under the MIT License. See the LICENSE file for details.

Citation Information

Citation information for this project can be found in the Citation files in the directory. See the CITATION file for details.

Contact Information

For any questions, suggestions, or issues, please feel free to contact:

  • Name: Fahad Deshmukh
  • Email: deshmukh@uni-potsdam.de

Owner

  • Name: Fahad Deshmukh
  • Login: fahaddeshmukh
  • Kind: user
  • Location: Dortmund, Germany.

I'm a Data Science student at Uni Potsdam and I'm passionate about ML and DL techniques. This Github showcases some of my academic and personal projects.

Citation (CITATION.cff)

# This CITATION.cff file was generated with cffinit.
# Visit https://bit.ly/cffinit to generate yours today!

cff-version: 1.2.0
title: >-
  Semantic Duplicate Detection: Quora Question Pairs
  Analysis using Machine Learning
message: >-
  If you use this software, please cite it using the
  metadata from this file.
type: software
authors:
  - given-names: Fahad
    family-names: Deshmukh
    email: deshmukh@uni-potsdam.de
    affiliation: University of Potsdam
identifiers:
  - type: url
    value: 'https://github.com/fahaddeshmukh/Question-pair-nlp'
    description: github repo
abstract: >-

  This project presents a machine learning (ML)-based
  solution for the Quora Question Pairs challenge hosted on
  Kaggle. With a focus on natural language processing (NLP),
  the project addresses the task of identifying whether
  pairs of questions from the Quora platform are duplicates.
  The primary objective is to construct binary
  classification models capable of determining whether two
  provided questions are semantically similar or represent
  duplicates.


  The project involves a dataset comprising over 400,000
  observations across six columns: 'ID', 'QID1', 'QID2',
  'Question1', 'Question2', and 'Is_duplicate'. Through
  systematic processing, exploratory data analysis, and
  feature engineering, the project seeks to derive insights
  for model development. The 'Is_duplicate' column,
  containing boolean values indicating question duplicity,
  serves as the dependent variable.


  The project encompasses loading and preprocessing the
  dataset, conducting exploratory data analysis, and
  performing feature engineering using NLP techniques. The
  engineered features are then subjected to additional
  exploratory analysis. Predictive modeling forms the
  culminating phase of the project, where various binary
  classification models are trained and evaluated to
  determine the most effective solution.


  By addressing the Quora Question Pairs challenge, this
  project contributes to the field of NLP and
  classification, demonstrating the application of machine
  learning techniques in detecting semantic similarity
  within textual data. The outcomes shed light on the
  efficacy of different models and feature engineering
  strategies in tackling real-world NLP tasks.
keywords:
  - NLP
  - Machine Learning
  - Python
  - jupyter notebbok
  - Semantic duplicate
license: MIT
date-released: '2023-08-10'

GitHub Events

Total
Last Year

Dependencies

requirements.txt pypi
  • fuzzywuzzy ==0.18.0
  • matplotlib ==3.4.3
  • nltk ==3.6.3
  • numpy ==1.21.2
  • pandas ==1.3.3
  • scikit-learn ==0.24.2
  • spacy ==3.1.3