readme.md

Awesome Tensorlayer - A curated list of dedicated resources

You have just found TensorLayer! High performance DL and RL library for industry and academic.

Contribute

Contributions welcome! Read the contribution guidelines first.

Contents

• Tips and Tricks
• 1. Basics Examples
• 2. Computer Vision
• 3. Natural Language Processing
• 4. Reinforcement Learning
• 5. Adversarial Learning
• 6. Pretrained Models
• 7. Auto Encoders
• 8. Data and Model Managment Tools

Tips and Tricks

• Tricks to use TensorLayer is a third party repository to collect tricks to use TensorLayer better.

1. Basics Examples

Get start with TensorLayer.

1.1 MNIST - Hello World

Training MNIST with Dropout is the Hello World in deep learning.

• Using Dropout in Tensorlayer - Method 1 using DropoutLayer and network.all_drop to switch training and testing.

• Using Dropout in Tensorlayer - Method 2 using DropoutLayer and is_train to switch training and testing.

1.2 CIFAR10 - Data Augmentation

In deep learning, data augmentation is a key fator to improve the performance. While, a complex data augmentation method and large dataset will slow down the training, therefore, TensorFlow provides TFRecord and DatasetAPI for fast data processing, see TensorFlow-Importing Data for more information.

• Placeholder - Slow.

• TFRecord - Fast.

• TF DatasetAPI - Fast.

1.3 Distributed Training

Our distributed training is powered by Uber Horovod, to the best of our knowledge, it is the best solution for TensorFlow.

• Our small examples here can help you to understand and test our API easily. Note that, due to the small data size, using more GPUs could not have performance gain, these examples just show you how to use the API.

1.4 More DatasetAPI and TFRecord Examples

• Downloading and Preprocessing PASCAL VOC with TensorLayer VOC data loader. 知乎文章

• Read and Save data in TFRecord Format.

• Read and Save time-series data in TFRecord Format.

• Convert CIFAR10 in TFRecord Format for performance optimization.

• More dataset loader can be found in tl.files.load_xxx

1.5 Connect with other TF wrappers

• Connect with Keras.

• Connect with TF-Slim.

• Connect with TF-Slim Networks an example with the CNN InceptionV3 by [C. Szegedy et al, 2015].

2. Computer Vision

2.1 Computer Vision Applications

• Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization

• OpenPose: Real-time multi-person keypoint detection

• InsignFace - Additive Angular Margin Loss for Deep Face Recognition

• Spatial-Transformer-Nets (STN) trained on MNIST dataset based on the paper by [M. Jaderberg et al, 2015].

• Generative Adversarial Text to Image Synthesis on bird and flower dataset.

• SRGAN - A Super Resolution GAN based on the paper by [C. Ledig et al, 2016].

• Improved CycleGAN using Resize-Convolution.

• U-Net Brain Tumor Segmentation trained on BRATS 2017 dataset based on the paper by [M. Jaderberg et al, 2015] with some modifications.

• Image2Text: im2txt based on the paper by [O. Vinyals et al, 2016].

• DAGAN: Fast Compressed Sensing MRI Reconstruction based on the paper by [G. Yang et al, 2017].

• GAN-CLS for Text to Image Synthesis based on the paper by [S. Reed et al, 2016]

• Unsupervised Image-to-Image Translation with Generative Adversarial Networks, code

• More Computer Vision Application can be found in Adversarial Learning Section

2.2 Pretrained Image Classification Models

• VGG16, VGG19, MobileNet, SqueezeNet, Inception and etc can be found in tensorlayer/pretrained-models and examples/pretrained_cnn

2.3 CNN and Computational Speed or Memory Footprint Bandwitdh Optimization

FP8 (float8) and FP16 (float16)

• Convolutional Network using FP16 (float16) on the MNIST dataset.

Quantization Networks

See examples/quantized_net.

• Binary Networks works on mnist and cifar10.

• Ternary Network works on mnist and cifar10.

• DoReFa-Net works on mnist and cifar10.

• Quantization For Efficient Integer-Arithmetic-Only Inference works on mnist and cifar10.

Our TensorRT Toolbox for production is coming soon

3. Natural Language Processing

3.1 ChatBot

• Seq2Seq Chatbot in 200 lines of code for Seq2Seq.

3.2 Text Generation

• Text Generation with LSTMs - Generating Trump Speech.

• Modelling PennTreebank code1 and code2, see blog post.

3.3 Text Classification

• FastText Classifier running on the IMDB dataset based on the paper by [A. Joulin et al, 2016].

3.4 Word Embedding

• Minimalistic Implementation of Word2Vec based on the paper by [T. Mikolov et al, 2013].

3.5 Spam Detection

• Chinese Spam Detector.

4. Reinforcement Learning

4.1 Actor Critic

• Asynchronous Advantage Actor Critic (A3C) with Continuous Action Space based on this blog post.

• Actor-Critic using TD-error as the Advantage, Reinforcement Learning based on this blog post.

4.2 Policy Network

• Deep Policy Network - Code working with Pong Game on ATARI - Related blog post from Andrej Karpathy.

4.3 Q-Learning

• Deep Q Network with Tables and Neural Networks on the FrozenLake OpenAI Gym - Related blog post.

4.4 Imitation Learning

• DAGGER for (Gym Torcs) by zsdonghao.

4.5 RL Toolbox

• RL Toolbox is a reinfore learning tool box, contains TRPO, A3C for ontinous action space by jjkke88.

5. Adversarial Learning

• SRGAN - A Super Resolution GAN based on the paper by [C. Ledig et al, 2016].

• DCGAN trained on the CelebA dataset based on the paper by [A. Radford et al, 2015].

• CycleGAN improved with resize-convolution based on the paper by [J. Zhu et al, 2017].

• DAGAN: Fast Compressed Sensing MRI Reconstruction based on the paper by [G. Yang et al, 2017].

• GAN-CLS for Text to Image Synthesis based on the paper by [S. Reed et al, 2016]

• Unsupervised Image-to-Image Translation with Generative Adversarial Networks, code

• BEGAN: Boundary Equilibrium Generative Adversarial Networks based on the paper by [D. Berthelot et al, 2017].

6. Pretrained Models

• All official pretrained models can be found here.
• Tricks to use TensorLayer provides useful examples to use tl.models.

7. Auto Encoders

Variational Autoencoder (VAE)

• Variational Autoencoder trained on the CelebA dataset.

• Variational Autoencoder trained on the MNIST dataset.

8. Data and Model Managment Tools

• Why Database?.

• Put Tasks into Database and Execute on Other Agents, see code.

• TensorDB applied on Pong Game on OpenAI Gym: Trainer File and Generator File based on the following blog post.

• TensorDB applied to classification task on MNIST dataset: Master File and Worker File.

How to cite TL in Research Papers ?

If you find this project useful, we would be grateful if you cite the TensorLayer paper：

@article{tensorlayer2017,
    author  = {Dong, Hao and Supratak, Akara and Mai, Luo and Liu, Fangde and Oehmichen, Axel and Yu, Simiao and Guo, Yike},
    journal = {ACM Multimedia},
    title   = {{TensorLayer: A Versatile Library for Efficient Deep Learning Development}},
    url     = {http://tensorlayer.org},
    year    = {2017}
}

ENJOY