readme.md

Awesome Tensorlayer - A curated list of dedicated resources

You have just found TensorLayer! High performance DL and RL library for industry and academic.

Contribute

Contributions welcome! Read the contribution guidelines first.

Contents

• Tutorials - Tips and Tricks
• Basics
• Computer Vision
• Natural Language Processing
• Reinforcement Learning
• Auto Encoders
• Adversarial Learning
• Pretrained Models
• Miscellaneous
• Research Papers using TensorLayer

Tutorials - Tips and Tricks

• Tricks to use TensorLayer is a third party repository to collect tricks to use TensorLayer better.

Basics Examples

Get start with TensorLayer.

MNIST - Hello World

Training MNIST with Dropout is the Hello World in deep learning.

• Using Dropout in Tensorlayer - Method 1 using DropoutLayer and network.all_drop to switch training and testing.

• Using Dropout in Tensorlayer - Method 2 using DropoutLayer and is_train to switch training and testing.

CIFAR10 - Data Augmentation

In deep learning, data augmentation is a key fator

Data Manipulation

Image Preprocessing

• Image Augmentation using Python randomly augment images with flipped or cropped images.

• Downloading and Preprocessing PASCAL VOC using TensorFlow Dataset API with TensorLayer VOC data loader.

TF Records

• Read and Save data in TFRecord Format.

• Read and Save time-series data in TFRecord Format.

• Convert CIFAR10 in TFRecord Format for performance optimization.

Loading Data

• Convolutional Network working on the dataset CIFAR10 using TensorLayer CIFAR10 data loader.

• Convolutional Network working on the dataset CIFAR10 using TFRecords.

Multi Layer Perceptron (MLP)

• Simple MLP Network trained on MNIST dataset.

Keras

• Using Keras Layers with Tensorlayer.

TF-Slim

• Using TF-Slim Layers with Tensorlayer.

• Using TF-Slim Networks with Tensorlayer an example with the CNN InceptionV3 by [C. Szegedy et al, 2015].

Datasets

Process PASCAL VOC

• Downloading and Preprocessing PASCAL VOC using TensorFlow Dataset API with TensorLayer VOC data loader. 知乎文章

More here

Computer Vision

Computer Vision Applications

Style Transfer

• Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization, see here

Pose Estimation

• OpenPose: Real-time multi-person keypoint detection library, see here

Face Recognition

• InsignFace - Additive Angular Margin Loss for Deep Face Recognition

Spatial Transformer Networks

• Spatial-Transformer-Nets (STN) trained on MNIST dataset based on the paper by [M. Jaderberg et al, 2015].

Text-to-Image Synthesis

• Generative Adversarial Text to Image Synthesis on bird and flower dataset.

Improved CycleGAN

• Improved CycleGAN using Resize-Convolution.

Medical Applications

• U-Net Brain Tumor Segmentation trained on BRATS 2017 dataset based on the paper by [M. Jaderberg et al, 2015] with some modifications.

Image Captioning

• Image2Text: im2txt based on the paper by [O. Vinyals et al, 2016].

More Computer Vision Application can be found in Adversarial Learning

Pretrained models for ImageNet Classification such as VGG16, VGG19, MobileNet, SqueezeNet, Inception can be found in tensorlayer/pretrained-models and examples/pretrained_cnn

CNN and Computational Speed or Memory Footprint Bandwitdh Optimization

FP8 (float8) and FP16 (float16)

• Convolutional Network using FP16 (float16) on the MNIST dataset.

Quantizat Networks

See examples/quantized_net.

• Binary Networks works on mnist and cifar10.

• Ternary Network works on mnist and cifar10.

• DoReFa-Net works on mnist and cifar10.

• Quantization For Efficient Integer-Arithmetic-Only Inference works on mnist and cifar10.

Natural Language Processing

• Text Generation with LSTMs - Generating Trump Speech.

• Modelling PennTreebank code1 and code2, see blog post.

Embedding Networks

FastText

• FastText Classifier running on the IMDB dataset based on the paper by [A. Joulin et al, 2016].

Word2Vec

• Minimalistic Implementation of Word2Vec based on the paper by [T. Mikolov et al, 2013].

NLP Applications

Spam Detection

• Chinese Spam Detector.

ChatBot

• Seq2Seq Chatbot in 200 lines of code for Seq2Seq.

Reinforcement Learning

Actor Critic

• Asynchronous Advantage Actor Critic (A3C) with Continuous Action Space based on this blog post.

• Actor-Critic using TD-error as the Advantage, Reinforcement Learning based on this blog post.

Policy Network

• Deep Policy Network - Code working with Pong Game on ATARI - Related blog post from Andrej Karpathy.

Q-Learning

• Deep Q Network with Tables and Neural Networks on the FrozenLake OpenAI Gym - Related blog post.

Imitation Learning

• DAGGER for (Gym Torcs) by zsdonghao.

Toolbox

• RL Toolbox is a reinfore learning tool box, contains trpo, a3c algorithm for continous action space by jjkke88.

Auto Encoders

Variational Autoencoder (VAE)

• Variational Autoencoder trained on the CelebA dataset.

• Variational Autoencoder trained on the MNIST dataset.

Adversarial Learning

State of the art

• DCGAN trained on the CelebA dataset based on the paper by [A. Radford et al, 2015].

• SRGAN - A Super Resolution GAN based on the paper by [C. Ledig et al, 2016].

• CycleGAN improved with resize-convolution based on the paper by [J. Zhu et al, 2017].

• BEGAN: Boundary Equilibrium Generative Adversarial Networks based on the paper by [D. Berthelot et al, 2017].

Applications

Image Reconstruction

• DAGAN: Fast Compressed Sensing MRI Reconstruction based on the paper by [G. Yang et al, 2017].

Text to Image

• GAN-CLS for Text to Image Synthesis based on the paper by [S. Reed et al, 2016]

Image to Image

• Im2Im Translation based on the paper by [H. Dong et al, 2017]

Pretrained Models

• All models implementations available using TF-Slim can be connected to TensorLayer via SlimNetsLayer.

• All pretrained models in here.

Miscellaneous

TensorLayer DB: TensorDB

• What is TensorDB.

• TensorDB applied on Pong Game on OpenAI Gym: Trainer File and Generator File based on the following blog post.

• TensorDB applied to classification task on MNIST dataset: Master File and Worker File.

TensorFlask

• TensorFlask - a simple webservice API to process HTTP POST requests using Flask and TensorFlow/Layer.

How to cite TL in Research Papers ?

If you find this project useful, we would be grateful if you cite the TensorLayer paper：

@article{tensorlayer2017,
    author  = {Dong, Hao and Supratak, Akara and Mai, Luo and Liu, Fangde and Oehmichen, Axel and Yu, Simiao and Guo, Yike},
    journal = {ACM Multimedia},
    title   = {{TensorLayer: A Versatile Library for Efficient Deep Learning Development}},
    url     = {http://tensorlayer.org},
    year    = {2017}
}

Research papers using TensorLayer

• An example research paper by [A. Author et al, 2018]