基于Theano的可扩展深度学习框架:deepy

jopen 9年前

基于Theano的可扩展深度学习框架deepy。

Dependencies

  • Python 2.7 (Better on Linux)
  • numpy
  • theano
  • scipy for L-BFGS and CG optimization

Clean interface

# MNIST Multi-layer model with dropout.  from deepy.dataset import MnistDataset, MiniBatches  from deepy.networks import NeuralClassifier  from deepy.layers import Dense, Softmax, Dropout  from deepy.trainers import MomentumTrainer, LearningRateAnnealer    model = NeuralClassifier(input_dim=28*28)  model.stack(Dense(256, 'relu'),              Dropout(0.2),              Dense(256, 'relu'),              Dropout(0.2),              Dense(10, 'linear'),              Softmax())    trainer = MomentumTrainer(model)    annealer = LearningRateAnnealer(trainer)    mnist = MiniBatches(MnistDataset(), batch_size=20)    trainer.run(mnist, controllers=[annealer])

Examples

Enviroment setting

  • CPU
source bin/cpu_env.sh
  • GPU
source bin/gpu_env.sh

MNIST Handwriting task

  • Simple MLP
python experiments/mnist/mlp.py
  • MLP with dropout
python experiments/mnist/mlp_dropout.py
  • MLP with PReLU and dropout
python experiments/mnist/mlp_prelu_dropout.py
  • Deep convolution
python experiments/mnist/deep_convolution.py
  • Elastic distortion
python experiments/mnist/mlp_elastic_distortion.py
  • Recurrent visual attention model </li> </ul>
    python experiments/attention_models/baseline.py

    Language model

    Penn Treebank benchmark

    • Baseline RNNLM (Full-output layer)
    python experiments/lm/baseline_rnnlm.py
    • Class-based RNNLM
    python experiments/lm/class_based_rnnlm.py
    • LSTM based LM (Full-output layer)
    python experiments/lm/lstm_rnnlm.py

    Char-based language models

    • Char-based LM with LSTM
    python experiments/lm/char_lstm.py
    • Char-based LM with Deep RNN
    python experiments/lm/char_rnn.py

    Deep Q learning

    • Start server
    pip install Flask-SocketIO  python experiments/deep_qlearning/server.py
    • Open this address in browser
    http://localhost:5003

    Auto encoders

    • Recurrent NN based auto-encoder
    python experiments/auto_encoders/rnn_auto_encoder.py
    • Recursive auto-encoder
    python experiments/auto_encoders/recursive_auto_encoder.py

    Train with CG and L-BFGS

    • CG
    python experiments/scipy_training/mnist_cg.py
    • L-BFGS
    python experiments/scipy_training/mnist_lbfgs.py

    Other experiments

    DRAW

    See https://github.com/uaca/deepy-draw

    # Train the model  python mnist_training.py  # Create animation  python animation.py experiments/draw/mnist1.gz

    Highway networks

    python experiments/highway_networks/mnist_baseline.py  python experiments/highway_networks/mnist_highway.py

    Effect of different initialization schemes

    python experiments/initialization_schemes/gaussian.py  python experiments/initialization_schemes/uniform.py  python experiments/initialization_schemes/xavier_glorot.py  python experiments/initialization_schemes/kaiming_he.py

    Other features

    • Auto gradient correction

    Sorry for that deepy is not well documented currently, but the framework is designed in the spirit of simplicity and readability. This will be improved if someone requires.

    项目主页:http://www.open-open.com/lib/view/home/1434971044591