Data

Tools

Indexes

Applications

Experiments

Open Source Science

https://github.com/ydataai/ydata-synthetic

Synthetic data generators for tabular and time-series data

https://github.com/ydataai/ydata-synthetic

Science Score: 49.0%

This score indicates how likely this project is to be science-related based on various indicators:

  • ○
    CITATION.cff file
  • ✓
    codemeta.json file
    Found codemeta.json file
  • ✓
    .zenodo.json file
    Found .zenodo.json file
  • ✓
    DOI references
    Found 1 DOI reference(s) in README
  • ✓
    Academic publication links
    Links to: arxiv.org, acm.org
  • ○
    Committers with academic emails
  • ○
    Institutional organization owner
  • ○
    JOSS paper metadata
  • ○
    Scientific vocabulary similarity
    Low similarity (15.8%) to scientific vocabulary

Keywords

datageneration datagenerator deep-learning gan gan-architectures gans generative-adversarial-network machine-learning python3 pytorch synthetic-data tensorflow2 time-series timeseries training-data

Keywords from Contributors

data-profilers data-quality eda exploratory-data-analysis pandas-profiling pandas-dataframe html-report exploration big-data-analytics data-exploration
Last synced: 5 months ago · JSON representation

Repository

Synthetic data generators for tabular and time-series data

Basic Info
  • Host: GitHub
  • Owner: ydataai
  • License: mit
  • Language: Jupyter Notebook
  • Default Branch: dev
  • Homepage: https://docs.sdk.ydata.ai
  • Size: 16.3 MB
Statistics
  • Stars: 1,570
  • Watchers: 32
  • Forks: 252
  • Open Issues: 59
  • Releases: 31
Topics
datageneration datagenerator deep-learning gan gan-architectures gans generative-adversarial-network machine-learning python3 pytorch synthetic-data tensorflow2 time-series timeseries training-data
Created almost 6 years ago · Last pushed 6 months ago
Metadata Files
Readme License Support

README.md

YData Synthetic Logo

Join us on Discord

YData Synthetic

A package to generate synthetic tabular and time-series data leveraging the state of the art generative models.

🎊 The exciting features:

These are must try features when it comes to synthetic data generation: - A new streamlit app that delivers the synthetic data generation experience with a UI interface. A low code experience for the quick generation of synthetic data - A new fast synthetic data generation model based on Gaussian Mixture. So you can quickstart in the world of synthetic data generation without the need for a GPU. - A conditional architecture for tabular data: CTGAN, which will make the process of synthetic data generation easier and with higher quality!

Synthetic data

What is synthetic data?

Synthetic data is artificially generated data that is not collected from real world events. It replicates the statistical components of real data without containing any identifiable information, ensuring individuals' privacy.

Why Synthetic Data?

Synthetic data can be used for many applications: - Privacy compliance for data-sharing and Machine Learning development - Remove bias - Balance datasets - Augment datasets

Looking for an end-to-end solution to Synthetic Data Generation?
YData Fabric enables the generation of high-quality datasets within a full UI experience, from data preparation to synthetic data generation and evaluation.
Check out the Community Version.

ydata-synthetic

This repository contains material related with architectures and models for synthetic data, from Generative Adversarial Networks (GANs) to Gaussian Mixtures. The repo includes a full ecosystem for synthetic data generation, that includes different models for the generation of synthetic structure data and time-series. All the Deep Learning models are implemented leveraging Tensorflow 2.0. Several example Jupyter Notebooks and Python scripts are included, to show how to use the different architectures.

Are you ready to learn more about synthetic data and the bext-practices for synthetic data generation?

Quickstart

The source code is currently hosted on GitHub at: https://github.com/ydataai/ydata-synthetic

Binary installers for the latest released version are available at the Python Package Index (PyPI). commandline pip install ydata-synthetic

The UI guide for synthetic data generation

YData synthetic has now a UI interface to guide you through the steps and inputs to generate structure tabular data. The streamlit app is available form v1.0.0 onwards, and supports the following flows: - Train a synthesizer model - Generate & profile synthetic data samples

Installation

commandline pip install ydata-synthetic[streamlit]

Quickstart

Use the code snippet below in a python file (Jupyter Notebooks are not supported): ```python from ydatasynthetic import streamlitapp

streamlit_app.run() ```

Or use the file streamlit_app.py that can be found in the examples folder.

commandline python -m streamlit_app

The below models are supported: - CGAN - WGAN - WGANGP - DRAGAN - CRAMER - CTGAN

Watch the video

Examples

Here you can find usage examples of the package and models to synthesize tabular data. - Fast tabular data synthesis on adult census income dataset Open in Colab - Tabular synthetic data generation with CTGAN on adult census income dataset Open in Colab - Time Series synthetic data generation with TimeGAN on stock dataset Open in Colab - Time Series synthetic data generation with DoppelGANger on FCC MBA dataset Open in Colab - More examples are continuously added and can be found in /examples directory.

Datasets for you to experiment

Here are some example datasets for you to try with the synthesizers:

Tabular datasets

Sequential datasets

Project Resources

In this repository you can find the several GAN architectures that are used to create synthesizers:

Tabular data

Sequential data

Contributing

We are open to collaboration! If you want to start contributing you only need to: 1. Search for an issue in which you would like to work. Issues for newcomers are labeled with good first issue. 2. Create a PR solving the issue. 3. We would review every PRs and either accept or ask for revisions.

Support

For support in using this library, please join our Discord server. Our Discord community is very friendly and great about quickly answering questions about the use and development of the library. Click here to join our Discord community!

FAQs

Have a question? Check out the Frequently Asked Questions about ydata-synthetic. If you feel something is missing, feel free to book a beary informal chat with us.

License

MIT License

Owner

  • Name: YData
  • Login: ydataai
  • Kind: organization
  • Email: hello@ydata.ai

Accelerating AI with improved data

GitHub Events

Total
  • Create event: 19
  • Commit comment event: 1
  • Issues event: 4
  • Watch event: 152
  • Delete event: 11
  • Member event: 1
  • Issue comment event: 5
  • Push event: 18
  • Pull request review event: 1
  • Pull request event: 7
  • Fork event: 19
Last Year
  • Create event: 19
  • Commit comment event: 1
  • Issues event: 4
  • Watch event: 152
  • Delete event: 11
  • Member event: 1
  • Issue comment event: 5
  • Push event: 18
  • Pull request review event: 1
  • Pull request event: 7
  • Fork event: 19

Committers

Last synced: 9 months ago

All Time
  • Total Commits: 235
  • Total Committers: 23
  • Avg Commits per committer: 10.217
  • Development Distribution Score (DDS): 0.6
Past Year
  • Commits: 14
  • Committers: 3
  • Avg Commits per committer: 4.667
  • Development Distribution Score (DDS): 0.357
Top Committers
Name Email Commits
Fabiana f****e@y****i 94
renovate[bot] 2****] 50
Francisco Santos 3****s 17
dependabot[bot] 4****] 12
Gonçalo Martins Ribeiro g****o@y****i 12
Alexandre Quemy a****y@y****i 9
Miriam Seoane Santos 6****s 9
Luís Portela Afonso l****a@y****i 6
Vasco Ramos v****s@y****i 5
ricardodcpereira r****a@y****i 4
ubabe53 5****3 3
Fabes201 3****1 2
Arunn Thevapalan a****k@g****m 2
Fabiana Clemente f****e@F****l 1
Alex Strick van Linschoten s****l 1
Archit Yadav a****7@g****m 1
Carlos Gavidia-Calderon c****c@g****m 1
Ceshine Lee s****k@g****m 1
Claudio f****c@g****m 1
Guan Wang c****u@g****m 1
Uppu Rajesh Kumar 7****i 1
ljmatkins l****s@g****m 1
mglcampos m****s 1
Committer Domains (Top 20 + Academic)
ydata.ai: 6

Issues and Pull Requests

Last synced: 6 months ago

All Time
  • Total issues: 54
  • Total pull requests: 142
  • Average time to close issues: 3 months
  • Average time to close pull requests: 2 months
  • Total issue authors: 52
  • Total pull request authors: 20
  • Average comments per issue: 1.46
  • Average comments per pull request: 0.41
  • Merged pull requests: 78
  • Bot issues: 1
  • Bot pull requests: 72
Past Year
  • Issues: 4
  • Pull requests: 8
  • Average time to close issues: N/A
  • Average time to close pull requests: 39 minutes
  • Issue authors: 4
  • Pull request authors: 3
  • Average comments per issue: 0.0
  • Average comments per pull request: 0.0
  • Merged pull requests: 2
  • Bot issues: 0
  • Bot pull requests: 4
View more stats
Top Authors
Issue Authors
  • pbezz1 (2)
  • datatalking (2)
  • range-yugen (1)
  • AnthonyFang623 (1)
  • wjjessen (1)
  • tsiakmaki (1)
  • ednaruiz (1)
  • DavidJardim (1)
  • akashshah59 (1)
  • alabarga (1)
  • majidliaquat (1)
  • renovate[bot] (1)
  • rroosshhaann (1)
  • lmqhello (1)
  • Ahmadshahzad2 (1)
Pull Request Authors
  • renovate[bot] (58)
  • fabclmnt (21)
  • dependabot[bot] (19)
  • aquemy (14)
  • miriamspsantos (11)
  • vascoalramos (6)
  • ricardodcpereira (4)
  • jfsantos-ds (3)
  • datatalking (2)
  • jaiwiwjwjwisn (2)
  • gupta-vivek (2)
  • T0217 (2)
  • gmartinsribeiro (2)
  • cptanalatriste (1)
  • alexbarros (1)
Top Labels
Issue Labels
bug (23) question (9) enhancement (9) help wanted (2) information requested (2) documentation (1)
Pull Request Labels
dependencies (19) github_actions (10) python (8) documentation (1) enhancement (1) Regular (1)

Packages

  • Total packages: 1
  • Total downloads:
    • pypi 4,234 last-month
  • Total dependent packages: 2
  • Total dependent repositories: 1
  • Total versions: 32
  • Total maintainers: 1
pypi.org: ydata-synthetic

Synthetic data generation methods with different synthetization methods.

  • Versions: 32
  • Dependent Packages: 2
  • Dependent Repositories: 1
  • Downloads: 4,234 Last month
Rankings
Stargazers count: 1.9%
Forks count: 3.6%
Downloads: 4.3%
Dependent packages count: 7.3%
Average: 7.8%
Dependent repos count: 22.1%
Maintainers (1)
ydata
Last synced: 6 months ago

Dependencies

requirements.txt pypi
  • easydict ==1.9
  • matplotlib ==3.3.2
  • numpy ==1.19.
  • pandas ==1.2.
  • pmlb ==1.0.
  • pytest ==6.2.
  • scikit-learn ==1.0.
  • seaborn ==0.11.
  • tensorflow ==2.4.
  • tqdm <5.0
  • typeguard ==2.13.
.github/workflows/master.yml actions
  • actions/checkout v3 composite
  • actions/create-release v1 composite
  • cycjimmy/semantic-release-action v3 composite
.github/workflows/prerelease.yml actions
  • AButler/upload-release-assets v2.0 composite
  • actions/checkout v3 composite
  • actions/download-artifact v3 composite
  • actions/setup-python v4 composite
  • actions/upload-artifact v3 composite
  • pypa/gh-action-pypi-publish v1.6.4 composite
.github/workflows/pull_request.yml actions
  • actions/cache v3 composite
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
.github/workflows/docs.yaml actions
  • actions/cache v3 composite
  • actions/checkout v3 composite
  • actions/setup-python v4 composite
  • oprypin/find-latest-tag v1.1.1 composite
requirements-docs.txt pypi
  • mike ==1.1.2
  • mkdocs ==1.4.2
  • mkdocs-material ==9.0.9
  • mkdocs-material-extensions ==1.1.1
  • mkdocstrings ==0.20.0
setup.py pypi