簡介
最近各大廠商有關(guān)深度學習神經(jīng)網(wǎng)絡(luò)的快速模型構(gòu)建的組件包，層出不窮，Uber也不甘其后，推出了基于TensorFlow的工具箱: Ludwig
該工具集已經(jīng)發(fā)布到了GitHub
2019-02-18 15_12_04-uber_ludwig_ Ludwig is a toolbox built on top of TensorFlow that allows to train.png
特點
這個工具箱甚至允許不寫任何代碼就能夠訓練神經(jīng)網(wǎng)絡(luò)模型，示例網(wǎng)頁：Examples
2019-02-18 15_13_15-Examples - Ludwig.png
支持的應用包括：
- Text Classification
- Named Entity Recognition Tagging
- Natural Language Understanding
- Machine Translation （支持Attention機制）
- Chit-Chat Dialogue Modeling through Sequence2Sequence（支持Attention機制）
- Sentiment Analysis
- Image Classification
- Image Captioning
- One-shot Learning with Siamese Networks
- Visual Question Answering
- Kaggle's Titanic: Predicting survivors
- Time series forecasting
- Movie rating prediction
- Multi-label classification
-
Multi-Task Learning
看到支持的類型，我的心情是這樣的：
duang.jpg
安裝
ludwig依賴的組件包：
Cython>=0.25
h5py>=2.6
matplotlib>=3.0
numpy>=1.15,<1.16
pandas>=0.19
scipy>=0.18
scikit-image
scikit-learn
seaborn>=0.7
spacy>=2.0
https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz
tqdm
tabulate>=0.7
tensorflow>=1.12.0
PyYAML>=3.12
pytest
安裝的方式如下，說明ludwig是依賴Spacy的
pip install ludwig
python -m spacy download en
初試牛刀
實現(xiàn)模型的訓練，最簡單的方式，準備訓練集，如情感分析，數(shù)據(jù)集是如下的csv:
2019-02-18 15_23_34-movie_reviews.csv - Excel.png
假如路徑為：./model/movie_reviews.csv
然后準備模型的描述YAML文件，文件路徑為：./model/movie_sentiment_model_definition.yaml
內(nèi)容如下：
需要特別說明的是：
情感分析的例子，按照官方說明的默認配置：
input_features:
-
name: review
type: text
encoder: parallel_cnn
level: word
output_features:
-
name: sentiment
type: category
效果是非常差的，訓練3次之后，準確率在50%之后，就沒有提升了，這與拋硬幣也沒什么區(qū)別啊。。。
2019-02-18 17_02_20-Photos.png
經(jīng)過實踐證明，type設(shè)置為sequence，encoder設(shè)置為rnn是相對靠譜的，訓練5次之后，準確率如下：
2019-02-18 17_02_50-Photos.png
訓練10次之后，準確率如下：
2019-02-18 17_14_22-Photos.png
所以對于文本方面的處理，按照如下設(shè)置，訓練才有一定意義。
input_features:
-
name: review
type: sequence
encoder: rnn
cell_type: lstm
bidirectional: true
num_layers: 2
reduce_output: None
output_features:
-
name: sentiment
type: category
training:
epochs: 100
dropout: True
dropout_rate: 0.1
early_stop: 100
可以發(fā)現(xiàn)，這個配置是RNN文本分類模型相關(guān)，并且設(shè)置為BI-LSTM機制
安裝ludwig完畢之后，訓練這個模型，只需要在控制臺錄入：
ludwig experiment --data_csv ./model/movie_reviews.csv --model_definition_file ./model/movie_sentiment_model_definition.yaml
然后就搞定模型的訓練了。
訓練過程中，對于訓練集中的文本，會生成與訓練集同名的文本結(jié)構(gòu)描述文件，如：movie_reviews.json（因為內(nèi)容太大，所以截圖結(jié)構(gòu)折疊后的樣子）：
2019-02-18 15_47_17-D__reviews.json - Notepad+.png
會自動在項目路徑生成如下目錄結(jié)構(gòu)：results_run_0
此目錄下會有一個模型描述json文件：results_run_0\description.json
我們可以在這里找到我們所熟悉的模型相關(guān)的超參數(shù)：
{
"command": "ludwig experiment ",
"dataset_type": "generic",
"input_data": "./model/movie_reviews.csv",
"ludwig_version": "0.1.0",
"model_definition": {
"combiner": {
"type": "concat"
},
"input_features": [
{
"bidirectional": true,
"cell_type": "lstm",
"encoder": "rnn",
"name": "review",
"num_layers": 2,
"reduce_output": "None",
"tied_weights": null,
"type": "sequence"
}
],
"output_features": [
{
"dependencies": [],
"loss": {
"class_distance_temperature": 0,
"class_weights": 1,
"confidence_penalty": 0,
"distortion": 1,
"labels_smoothing": 0,
"negative_samples": 0,
"robust_lambda": 0,
"sampler": null,
"type": "softmax_cross_entropy",
"unique": false,
"weight": 1
},
"name": "sentiment",
"reduce_dependencies": "sum",
"reduce_input": "sum",
"top_k": 3,
"type": "category"
}
],
"preprocessing": {
"bag": {
"fill_value": "",
"format": "space",
"lowercase": 10000,
"missing_value_strategy": "fill_with_const",
"most_common": false
},
"binary": {
"fill_value": 0,
"missing_value_strategy": "fill_with_const"
},
"category": {
"fill_value": "<UNK>",
"lowercase": false,
"missing_value_strategy": "fill_with_const",
"most_common": 10000
},
"force_split": false,
"image": {
"missing_value_strategy": "backfill"
},
"numerical": {
"fill_value": 0,
"missing_value_strategy": "fill_with_const"
},
"sequence": {
"fill_value": "",
"format": "space",
"lowercase": false,
"missing_value_strategy": "fill_with_const",
"most_common": 20000,
"padding": "right",
"padding_symbol": "<PAD>",
"sequence_length_limit": 256,
"unknown_symbol": "<UNK>"
},
"set": {
"fill_value": "",
"format": "space",
"lowercase": false,
"missing_value_strategy": "fill_with_const",
"most_common": 10000
},
"split_probabilities": [
0.7,
0.1,
0.2
],
"stratify": null,
"text": {
"char_format": "characters",
"char_most_common": 70,
"char_sequence_length_limit": 1024,
"fill_value": "",
"lowercase": true,
"missing_value_strategy": "fill_with_const",
"padding": "right",
"padding_symbol": "<PAD>",
"unknown_symbol": "<UNK>",
"word_format": "space_punct",
"word_most_common": 20000,
"word_sequence_length_limit": 256
},
"timeseries": {
"fill_value": "",
"format": "space",
"missing_value_strategy": "fill_with_const",
"padding": "right",
"padding_value": 0,
"timeseries_length_limit": 256
}
},
"training": {
"batch_size": 64,
"bucketing_field": null,
"decay": false,
"decay_rate": 0.96,
"decay_steps": 10000,
"dropout": true,
"dropout_rate": 0.1,
"early_stop": 100,
"epochs": 100,
"gradient_clipping": null,
"increase_batch_size_on_plateau": 0,
"increase_batch_size_on_plateau_max": 512,
"increase_batch_size_on_plateau_patience": 5,
"increase_batch_size_on_plateau_rate": 2,
"learning_rate": 0.001,
"learning_rate_warmup_epochs": 5,
"optimizer": {
"beta1": 0.9,
"beta2": 0.999,
"epsilon": 1e-08,
"type": "adam"
},
"reduce_learning_rate_on_plateau": 0,
"reduce_learning_rate_on_plateau_patience": 5,
"reduce_learning_rate_on_plateau_rate": 0.5,
"regularization_lambda": 0,
"regularizer": "l2",
"staircase": false,
"validation_field": "combined",
"validation_measure": "loss"
}
},
"random_seed": 42
}
訓練所得的模型結(jié)構(gòu)如下：
2019-02-18 15_38_01-Photos.png
對于模型的預測，訓練完畢之后，假定有如下數(shù)據(jù)需要預測：
./data/test_data.csv
2019-02-18 15_44_37-test_data.csv - Excel.png
因為已經(jīng)獲得訓練之后模型的路徑了，只需要在控制臺錄入：
ludwig predict --data_csv ./data/test_data.csv --model_path ./results/_run_0/model
之后，Ludwig又會給我們生成一個results_0目錄，其中就是預測的結(jié)果，這個預測結(jié)果是根據(jù)之前那個不合理的配置生成的模型，得到的，因此只有預覽的意義：
2019-02-18 15_49_58-Q-Dir 6.17.png
預測的類別名稱：
2019-02-18 15_50_33-sentiment_predictions.csv - Excel.png
預測的概率：
2019-02-18 15_50_55-sentiment_probability.csv - Excel.png
之所以看起來預測結(jié)果這么不靠譜，是因為只訓練了兩次，真正業(yè)務訓練，對于這種大小的訓練集，至少也應該訓練100次以上了。
通過python調(diào)用Ludwig API
這個就很簡單了，按照如下代碼寫即可，加入logging_level=logging.DEBUG這一句的目的是能夠正常輸出訓練模型的日志。
"""
@version: 0.1
@author: Blade He
@site:
@software: PyCharm
@file: main.py
@time: 2019/02/18
"""
from ludwig import LudwigModel
import yaml
import logging
def startjob(csv_file_path = r'./model/movie_reviews.csv',
model_file = r'./model/movie_sentiment_model_definition.yaml',
test_file = r'./data/test_data.csv'):
with open(model_file, encoding='utf-8', mode='r') as file:
model_definition = yaml.load(file.read())
print(model_definition)
ludwig_model = LudwigModel(model_definition)
train_stats = ludwig_model.train(data_csv=csv_file_path,
logging_level=logging.DEBUG)
print(train_stats)
predictions = ludwig_model.predict(data_csv=test_file,
logging_level=logging.DEBUG)
print(predictions)
ludwig_model.close()
if __name__ == '__main__':
startjob()
小結(jié)
Uber這次真的是把神經(jīng)網(wǎng)絡(luò)的應用，做成工具包了，是真正意義上的，無需寫代碼，通過命令行+模型配置文件的形式，就可以訓練與使用神經(jīng)網(wǎng)絡(luò)模型。
這應該就是趨勢吧
