You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Join us on [](https://tiny.ydata.ai/dcai-ydata-synthetic)
11
12
12
-
# ydata-synthetic is now ydata-sdk
13
-
**Better, faster, easier**
14
-
[YData SDK](https://docs.sdk.ydata.ai/latest/) is the leading Python package for data professional that provides connectors, metadata management, data quality profiling and synthetic data generation.
15
-
16
-
17
-
## from ydata-synthetic to ydata-sdk
18
-
With the update of `ydata-synthetic`to `ydata-sdk`, users will now have access to a single API that automatically selects and optimizes
19
-
the best generative model for their data. This streamlined approach eliminates the need to choose between
20
-
various models manually, as the API intelligently identifies the optimal model based on the specific dataset and use case.
21
-
22
-
Instead of having to manually select from models such as:
-[TimeGAN](https://papers.nips.cc/paper/2019/file/c9efe5f26cd17ba6216bbe2a7d26d490-Paper.pdf) (specifically for *time-series* data)
33
-
-[DoppelGANger](https://dl.acm.org/doi/pdf/10.1145/3419394.3423643) (specifically for *time-series* data)
34
-
35
-
The new API handles model selection automatically, optimizing for the best performance in fidelity, utility, and privacy.
36
-
This significantly simplifies the synthetic data generation process, ensuring that users get the highest quality output without
37
-
the need for manual intervention and tiring hyperparameter tuning.
38
-
39
-
Are you ready to learn more about synthetic data and the best-practices for synthetic data generation?
40
-
For more materials on [synthetic data generation with Python see the documentation](https://docs.fabric.ydata.ai/latest/sdk/).
41
-
42
-
## Quickstart
43
-
Binary installers for the latest released version are available at the [Python Package Index (PyPI).](https://pypi.org/project/ydata-sdk/)
44
-
```commandline
45
-
pip install ydata-sdk
46
-
```
47
-
48
-
## The good old YData Synthetic
49
-
`YData-Synthetic` was an pioneering open-source package developed in 2020 with the primary goal of educating users about generative models for synthetic data generation.
50
-
Even though the journey was fun, and we have learned a lot from the community it is now time to upgrade `ydata-synthetic`.
51
-
Heading towards the future of synthetic data generation we recommend users to transition to `ydata-sdk`, which provides a superior experience with enhanced performance, precision, and ease of use, making it the preferred tool for synthetic data generation and a perfect introduction to Generative AI.
13
+
# YData Synthetic
14
+
A package to generate synthetic tabular and time-series data leveraging the state of the art generative models.
52
15
16
+
## 🎊 The exciting features:
17
+
> These are must try features when it comes to synthetic data generation:
18
+
> - A new streamlit app that delivers the synthetic data generation experience with a UI interface. A low code experience for the quick generation of synthetic data
19
+
> - A new fast synthetic data generation model based on Gaussian Mixture. So you can quickstart in the world of synthetic data generation without the need for a GPU.
20
+
> - A conditional architecture for tabular data: CTGAN, which will make the process of synthetic data generation easier and with higher quality!
21
+
53
22
## Synthetic data
54
23
### What is synthetic data?
55
24
Synthetic data is artificially generated data that is not collected from real world events. It replicates the statistical components of real data without containing any identifiable information, ensuring individuals' privacy.
@@ -63,18 +32,68 @@ Synthetic data can be used for many applications:
63
32
64
33
> **Looking for an end-to-end solution to Synthetic Data Generation?**<br>
65
34
> [YData Fabric](https://ydata.ai/products/synthetic_data) enables the generation of high-quality datasets within a full UI experience, from data preparation to synthetic data generation and evaluation.<br>
35
+
> Check out the [Community Version](https://ydata.ai/ydata-fabric-free-trial).
36
+
66
37
38
+
# ydata-synthetic
39
+
This repository contains material related with architectures and models for synthetic data, from Generative Adversarial Networks (GANs) to Gaussian Mixtures.
40
+
The repo includes a full ecosystem for synthetic data generation, that includes different models for the generation of synthetic structure data and time-series.
41
+
All the Deep Learning models are implemented leveraging Tensorflow 2.0.
42
+
Several example Jupyter Notebooks and Python scripts are included, to show how to use the different architectures.
43
+
44
+
Are you ready to learn more about synthetic data and the bext-practices for synthetic data generation?
45
+
46
+
## Quickstart
47
+
The source code is currently hosted on GitHub at: https://github.com/ydataai/ydata-synthetic
48
+
49
+
Binary installers for the latest released version are available at the [Python Package Index (PyPI).](https://pypi.org/project/ydata-synthetic/)
50
+
```commandline
51
+
pip install ydata-synthetic
52
+
```
67
53
68
54
### The UI guide for synthetic data generation
69
55
70
-
YData Fabric offers an UI interface to guide you through the steps and inputs to generate structure data.
71
-
Read more about [YData Fabric](https://ydata.ai/products/fabric).
56
+
YData synthetic has now a UI interface to guide you through the steps and inputs to generate structure tabular data.
57
+
The streamlit app is available form *v1.0.0* onwards, and supports the following flows:
58
+
- Train a synthesizer model
59
+
- Generate & profile synthetic data samples
60
+
61
+
#### Installation
62
+
63
+
```commandline
64
+
pip install ydata-synthetic[streamlit]
65
+
```
66
+
#### Quickstart
67
+
Use the code snippet below in a python file (Jupyter Notebooks are not supported):
68
+
```python
69
+
from ydata_synthetic import streamlit_app
70
+
71
+
streamlit_app.run()
72
+
```
73
+
74
+
Or use the file streamlit_app.py that can be found in the [examples folder](https://github.com/ydataai/ydata-synthetic/tree/master/examples/streamlit_app.py).
75
+
76
+
```commandline
77
+
python -m streamlit_app
78
+
```
79
+
80
+
The below models are supported:
81
+
- CGAN
82
+
- WGAN
83
+
- WGANGP
84
+
- DRAGAN
85
+
- CRAMER
86
+
- CTGAN
87
+
88
+
[](https://youtu.be/ep0PhwsFx0A)
72
89
73
90
### Examples
74
91
Here you can find usage examples of the package and models to synthesize tabular data.
75
-
- Tabular [synthetic data generation on Titanic Kaggle dataset](https://github.com/ydataai/ydata-sdk/blob/main/examples/synthesizers/regular_quickstart.py)
76
-
- Time Series [synthetic data generation]('https://github.com/ydataai/ydata-sdk/blob/main/examples/synthesizers/time_series_quickstart.py')
77
-
- More examples are continuously added and can be found in [examples directory](https://github.com/ydataai/ydata-sdk/tree/main/examples).
92
+
- Fast tabular data synthesis on adult census income dataset [](https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/regular/models/Fast_Adult_Census_Income_Data.ipynb)
93
+
- Tabular synthetic data generation with CTGAN on adult census income dataset [](https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/regular/models/CTGAN_Adult_Census_Income_Data.ipynb)
94
+
- Time Series synthetic data generation with TimeGAN on stock dataset [](https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/timeseries/TimeGAN_Synthetic_stock_data.ipynb)
95
+
- Time Series synthetic data generation with DoppelGANger on FCC MBA dataset [](https://colab.research.google.com/github/ydataai/ydata-synthetic/blob/master/examples/timeseries/DoppelGANger_FCC_MBA_Dataset.ipynb)
96
+
- More examples are continuously added and can be found in `/examples` directory.
78
97
79
98
### Datasets for you to experiment
80
99
Here are some example datasets for you to try with the synthesizers:
@@ -89,7 +108,7 @@ Here are some example datasets for you to try with the synthesizers:
89
108
90
109
## Project Resources
91
110
92
-
Find below useful literature of how to generate synthetic data and available generative models:
111
+
In this repository you can find the several GAN architectures that are used to create synthesizers:
93
112
94
113
### Tabular data
95
114
-[GAN](https://arxiv.org/abs/1406.2661)
@@ -106,9 +125,17 @@ Find below useful literature of how to generate synthetic data and available gen
We are open to collaboration! If you want to start contributing you only need to:
130
+
1. Search for an issue in which you would like to work. Issues for newcomers are labeled with good first issue.
131
+
2. Create a PR solving the issue.
132
+
3. We would review every PRs and either accept or ask for revisions.
109
133
110
134
## Support
111
135
For support in using this library, please join our Discord server. Our Discord community is very friendly and great about quickly answering questions about the use and development of the library. [Click here to join our Discord community!](https://tiny.ydata.ai/dcai-ydata-synthetic)
112
136
113
137
## FAQs
114
-
Have a question? Check out the [Frequently Asked Questions](https://ydata.ai/resources/10-most-asked-questions-on-ydata-synthetic) about Synthetic Data. If you feel something is missing, feel free to [reach out on the Discord community](https://tiny.ydata.ai/dcai-ydata-synthetic).
138
+
Have a question? Check out the [Frequently Asked Questions](https://ydata.ai/resources/10-most-asked-questions-on-ydata-synthetic) about `ydata-synthetic`. If you feel something is missing, feel free to [book a beary informal chat with us](https://meetings.hubspot.com/fabiana-clemente).
![renovate[bot]](https://avatars.githubusercontent.com/in/2740?v=4&size=40){kind=avatar}
0 commit comments