17 minutes read (About 2590 words)

Model sava and load in tensorflow

설정
필요한 라이브러리를 설치하고 텐서플로를 임포트(import)합니다.
1
pip install -q pyyaml h5py  # HDF5 포맷으로 모델을 저장하기 위해서 필요합니다
Note: you may need to restart the kernel to use updated packages.
1
2
3
4
5
6
import os
import tensorflow as tf
from tensorflow import keras
from tensorflow.keras import layers

print(tf.__version__)
2.4.0-dev20200724


예제 데이터셋 받기
1
2
3
4
5
6
7
(train_images, train_labels), (test_images, test_labels) = tf.keras.datasets.mnist.load_data()

train_labels = train_labels[:1000]
test_labels = test_labels[:1000]

train_images = train_images[:1000].reshape(-1, 28 * 28) / 255.0
test_images = test_images[:1000].reshape(-1, 28 * 28) / 255.0
모델링 작업
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Sequential 모델 정의
def create_model():
    model = tf.keras.models.Sequential([
        keras.layers.Dense(512, activation='relu', input_shape=(784,)),
        keras.layers.Dropout(0.2),
        keras.layers.Dense(10)
    ])

    model.compile(optimizer='adam',
                  loss=tf.losses.SparseCategoricalCrossentropy(from_logits=True),
                  metrics=['accuracy'])

    return model


# 모델 객체 생성
model = create_model()

# 출력
model.summary()
Model: "sequential_8"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_16 (Dense)             (None, 512)               401920    
_________________________________________________________________
dropout_8 (Dropout)          (None, 512)               0         
_________________________________________________________________
dense_17 (Dense)             (None, 10)                5130      
=================================================================
Total params: 407,050
Trainable params: 407,050
Non-trainable params: 0
_________________________________________________________________


훈련하는 동안 체크포인트 저장하기
훈련 중간과 훈련 마지막에 체크포인트(checkpoint)를 자동으로 저장하도록 하는 것이 많이 사용하는 방법입니다. 다시 훈련하지 않고 모델을 재사용하거나 훈련 과정이 중지된 경우 이어서 훈련을 진행할 수 있습니다. tf.keras.callbacks.ModelCheckpoint은 이런 작업을 수행하는 콜백(callback)입니다. 이 콜백은 체크포인트 작업을 조정할 수 있도록 여러가지 매개변수를 제공합니다.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
checkpoint_path = "training_1/cp.ckpt"
checkpoint_dir = os.path.dirname(checkpoint_path)

# 모델의 가중치를 저장하는 콜백 만들기
cp_callback = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_path,
                                                 save_weights_only=True,
                                                 verbose=1)

# 새로운 콜백으로 모델 훈련하기
model.fit(train_images, 
          train_labels,  
          epochs=10,
          validation_data=(test_images,test_labels),
          callbacks=[cp_callback])  # 콜백을 훈련에 전달합니다

# 옵티마이저의 상태를 저장하는 것과 관련되어 경고가 발생할 수 있습니다.
# 이 경고는 (그리고 이 노트북의 다른 비슷한 경고는) 이전 사용 방식을 권장하지 않기 위함이며 무시해도 좋습니다.
WARNING:tensorflow:Automatic model reloading for interrupted job was removed from the `ModelCheckpoint` callback in multi-worker mode, please use the `keras.callbacks.experimental.BackupAndRestore` callback instead. See this tutorial for details: https://www.tensorflow.org/tutorials/distribute/multi_worker_with_keras#backupandrestore_callback.
Epoch 1/10
16/32 [==============>...............] - ETA: 0s - loss: 1.8756 - accuracy: 0.3736 
Epoch 00001: saving model to training_1/cp.ckpt
32/32 [==============================] - 1s 30ms/step - loss: 1.5677 - accuracy: 0.5056 - val_loss: 0.6899 - val_accuracy: 0.7870
Epoch 2/10
31/32 [============================>.] - ETA: 0s - loss: 0.4283 - accuracy: 0.8845
Epoch 00002: saving model to training_1/cp.ckpt
32/32 [==============================] - 0s 8ms/step - loss: 0.4276 - accuracy: 0.8844 - val_loss: 0.5193 - val_accuracy: 0.8380
Epoch 3/10
20/32 [=================>............] - ETA: 0s - loss: 0.2892 - accuracy: 0.9208
Epoch 00003: saving model to training_1/cp.ckpt
32/32 [==============================] - 0s 6ms/step - loss: 0.2828 - accuracy: 0.9232 - val_loss: 0.4733 - val_accuracy: 0.8510
Epoch 4/10
19/32 [================>.............] - ETA: 0s - loss: 0.1721 - accuracy: 0.9687
Epoch 00004: saving model to training_1/cp.ckpt
32/32 [==============================] - 0s 6ms/step - loss: 0.1836 - accuracy: 0.9622 - val_loss: 0.4489 - val_accuracy: 0.8490
Epoch 5/10
17/32 [==============>...............] - ETA: 0s - loss: 0.1666 - accuracy: 0.9582
Epoch 00005: saving model to training_1/cp.ckpt
32/32 [==============================] - 0s 6ms/step - loss: 0.1629 - accuracy: 0.9605 - val_loss: 0.4112 - val_accuracy: 0.8580
Epoch 6/10
30/32 [===========================>..] - ETA: 0s - loss: 0.1015 - accuracy: 0.9851
Epoch 00006: saving model to training_1/cp.ckpt
32/32 [==============================] - 0s 7ms/step - loss: 0.1023 - accuracy: 0.9846 - val_loss: 0.4088 - val_accuracy: 0.8650
Epoch 7/10
17/32 [==============>...............] - ETA: 0s - loss: 0.0798 - accuracy: 0.9883
Epoch 00007: saving model to training_1/cp.ckpt
32/32 [==============================] - 0s 6ms/step - loss: 0.0828 - accuracy: 0.9870 - val_loss: 0.4074 - val_accuracy: 0.8680
Epoch 8/10
32/32 [==============================] - ETA: 0s - loss: 0.0718 - accuracy: 0.9899
Epoch 00008: saving model to training_1/cp.ckpt
32/32 [==============================] - 0s 7ms/step - loss: 0.0715 - accuracy: 0.9900 - val_loss: 0.4204 - val_accuracy: 0.8590
Epoch 9/10
28/32 [=========================>....] - ETA: 0s - loss: 0.0588 - accuracy: 0.9915
Epoch 00009: saving model to training_1/cp.ckpt
32/32 [==============================] - 0s 7ms/step - loss: 0.0574 - accuracy: 0.9920 - val_loss: 0.4110 - val_accuracy: 0.8640
Epoch 10/10
31/32 [============================>.] - ETA: 0s - loss: 0.0325 - accuracy: 0.9978
Epoch 00010: saving model to training_1/cp.ckpt
32/32 [==============================] - 0s 6ms/step - loss: 0.0328 - accuracy: 0.9978 - val_loss: 0.3962 - val_accuracy: 0.8660





<tensorflow.python.keras.callbacks.History at 0x7fd0f59b3f10>



이 코드는 tensorflow 체크포인트 파일을 만들고 에포크가 종료될 때마다 업데이트합니다:
1
ls {checkpoint_dir}
checkpoint                   cp.ckpt.index
cp.ckpt.data-00000-of-00001
1
2
3
4
5
# 새로운 모델 생성
model = create_model()

loss, acc = model.evaluate(test_images, test_labels, verbose=2)
print("훈련되지 않은 모델의 정확도: {:5.2f}%".format(100*acc))
32/32 - 0s - loss: 2.3409 - accuracy: 0.1250
훈련되지 않은 모델의 정확도: 12.50%


저장했던 모델을 로드하고 다시 평가해보도록 하겠습니다.
1
2
3
4
5
6
# 가중치 로드
model.load_weights(checkpoint_path)

# 모델 재평가
loss, acc = model.evaluate(test_images, test_labels, verbose=2)
print("복원된 모델의 정확도: {:5.2f}%".format(100*acc))
32/32 - 0s - loss: 0.3962 - accuracy: 0.8660
복원된 모델의 정확도: 86.60%


체크포인트 콜백 매개변수
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# 파일 이름에 에포크 번호를 포함시킵니다(`str.format` 포맷)
checkpoint_path = "training_2/cp-{epoch:04d}.ckpt"
checkpoint_dir = os.path.dirname(checkpoint_path)

# 다섯 번째 에포크마다 가중치를 저장하기 위한 콜백을 만듭니다
cp_callback = tf.keras.callbacks.ModelCheckpoint(
    filepath=checkpoint_path, 
    verbose=1, 
    save_weights_only=True,
    period=5)

# 새로운 모델 객체를 만듭니다
model = create_model()

# `checkpoint_path` 포맷을 사용하는 가중치를 저장합니다
model.save_weights(checkpoint_path.format(epoch=0))

# 새로운 콜백을 사용하여 모델을 훈련합니다
model.fit(train_images, 
          train_labels,
          epochs=50, 
          callbacks=[cp_callback],
          validation_data=(test_images,test_labels),
          verbose=0)
WARNING:tensorflow:`period` argument is deprecated. Please use `save_freq` to specify the frequency in number of batches seen.
WARNING:tensorflow:Automatic model reloading for interrupted job was removed from the `ModelCheckpoint` callback in multi-worker mode, please use the `keras.callbacks.experimental.BackupAndRestore` callback instead. See this tutorial for details: https://www.tensorflow.org/tutorials/distribute/multi_worker_with_keras#backupandrestore_callback.

Epoch 00005: saving model to training_2/cp-0005.ckpt

Epoch 00010: saving model to training_2/cp-0010.ckpt

Epoch 00015: saving model to training_2/cp-0015.ckpt

Epoch 00020: saving model to training_2/cp-0020.ckpt

Epoch 00025: saving model to training_2/cp-0025.ckpt

Epoch 00030: saving model to training_2/cp-0030.ckpt

Epoch 00035: saving model to training_2/cp-0035.ckpt

Epoch 00040: saving model to training_2/cp-0040.ckpt

Epoch 00045: saving model to training_2/cp-0045.ckpt

Epoch 00050: saving model to training_2/cp-0050.ckpt





<tensorflow.python.keras.callbacks.History at 0x7fd0f1d481d0>
1
ls {checkpoint_dir}
checkpoint                        cp-0025.ckpt.index
cp-0000.ckpt.data-00000-of-00001  cp-0030.ckpt.data-00000-of-00001
cp-0000.ckpt.index                cp-0030.ckpt.index
cp-0005.ckpt.data-00000-of-00001  cp-0035.ckpt.data-00000-of-00001
cp-0005.ckpt.index                cp-0035.ckpt.index
cp-0010.ckpt.data-00000-of-00001  cp-0040.ckpt.data-00000-of-00001
cp-0010.ckpt.index                cp-0040.ckpt.index
cp-0015.ckpt.data-00000-of-00001  cp-0045.ckpt.data-00000-of-00001
cp-0015.ckpt.index                cp-0045.ckpt.index
cp-0020.ckpt.data-00000-of-00001  cp-0050.ckpt.data-00000-of-00001
cp-0020.ckpt.index                cp-0050.ckpt.index
cp-0025.ckpt.data-00000-of-00001
1
2
latest = tf.train.latest_checkpoint(checkpoint_dir)
latest
'training_2/cp-0050.ckpt'
1
2
3
4
5
6
7
8
9
# 모델 초기화 및 생성
model = create_model()

# 모델 로드
model.load_weights(latest)

# 모델 복원, 평가
loss, acc = model.evaluate(test_images, test_labels, verbose=2)
print("복원된 모델의 정확도: {:5.2f}%".format(100*acc))
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
32/32 - 0s - loss: 0.4795 - accuracy: 0.8720
복원된 모델의 정확도: 87.20%


수동으로 가중치 저장하기
1
2
3
4
5
6
7
8
9
10
11
12
# 가중치를 저장합니다
model.save_weights('./checkpoints/my_checkpoint')

# 새로운 모델 객체를 만듭니다
model = create_model()

# 가중치를 복원합니다
model.load_weights('./checkpoints/my_checkpoint')

# 모델을 평가합니다
loss,acc = model.evaluate(test_images,  test_labels, verbose=2)
print("복원된 모델의 정확도: {:5.2f}%".format(100*acc))
32/32 - 0s - loss: 0.4795 - accuracy: 0.8720
복원된 모델의 정확도: 87.20%
전체 모델 저장하기
1
2
3
4
5
6
7
# 새로운 모델 객체를 만들고 훈련합니다
model = create_model()
model.fit(train_images, train_labels, epochs=10)

# SavedModel로 전체 모델을 저장합니다
!mkdir -p saved_model
model.save('saved_model/my_model')
Epoch 1/10
32/32 [==============================] - 0s 15ms/step - loss: 1.6664 - accuracy: 0.4644
Epoch 2/10
32/32 [==============================] - 0s 3ms/step - loss: 0.4997 - accuracy: 0.8490
Epoch 3/10
32/32 [==============================] - 0s 3ms/step - loss: 0.2933 - accuracy: 0.9225
Epoch 4/10
32/32 [==============================] - 0s 3ms/step - loss: 0.1953 - accuracy: 0.9644
Epoch 5/10
32/32 [==============================] - 0s 4ms/step - loss: 0.1473 - accuracy: 0.9746
Epoch 6/10
32/32 [==============================] - 0s 4ms/step - loss: 0.1240 - accuracy: 0.9736
Epoch 7/10
32/32 [==============================] - 0s 4ms/step - loss: 0.0863 - accuracy: 0.9785
Epoch 8/10
32/32 [==============================] - 0s 3ms/step - loss: 0.0603 - accuracy: 0.9967
Epoch 9/10
32/32 [==============================] - 0s 3ms/step - loss: 0.0554 - accuracy: 0.9974
Epoch 10/10
32/32 [==============================] - 0s 3ms/step - loss: 0.0374 - accuracy: 0.9988
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.iter
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_1
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.beta_2
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.decay
WARNING:tensorflow:Unresolved object in checkpoint: (root).optimizer.learning_rate
WARNING:tensorflow:A checkpoint was restored (e.g. tf.train.Checkpoint.restore or tf.keras.Model.load_weights) but not all checkpointed values were used. See above for specific issues. Use expect_partial() on the load status object, e.g. tf.train.Checkpoint.restore(...).expect_partial(), to silence these warnings, or use assert_consumed() to make the check explicit. See https://www.tensorflow.org/guide/checkpoint#loading_mechanics for details.
INFO:tensorflow:Assets written to: saved_model/my_model/assets
1
2
3
4
5
# my_model 디렉토리
!ls saved_model

# assests 폴더, saved_model.pb, variables 폴더
!ls saved_model/my_model
my_model
assets         saved_model.pb variables
1
2
3
4
new_model = tf.keras.models.load_model('saved_model/my_model')

# 모델 구조를 확인합니다
new_model.summary()
Model: "sequential_23"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_46 (Dense)             (None, 512)               401920    
_________________________________________________________________
dropout_23 (Dropout)         (None, 512)               0         
_________________________________________________________________
dense_47 (Dense)             (None, 10)                5130      
=================================================================
Total params: 407,050
Trainable params: 407,050
Non-trainable params: 0
_________________________________________________________________
1
2
3
4
5
# 복원된 모델을 평가합니다
loss, acc = new_model.evaluate(test_images,  test_labels, verbose=2)
print('복원된 모델의 정확도: {:5.2f}%'.format(100*acc))

print(new_model.predict(test_images).shape)
32/32 - 0s - loss: 0.4205 - accuracy: 0.0880
복원된 모델의 정확도:  8.80%
(1000, 10)
HDF5 파일로 저장하기
1
2
3
4
5
6
7
# 새로운 모델 객체를 만들고 훈련합니다
model = create_model()
model.fit(train_images, train_labels, epochs=10)

# 전체 모델을 HDF5 파일로 저장합니다
# '.h5' 확장자는 이 모델이 HDF5로 저장되었다는 것을 나타냅니다
model.save('my_model.h5')
Epoch 1/10
32/32 [==============================] - 0s 14ms/step - loss: 1.6326 - accuracy: 0.5135
Epoch 2/10
32/32 [==============================] - 0s 3ms/step - loss: 0.4184 - accuracy: 0.8959
Epoch 3/10
32/32 [==============================] - 0s 3ms/step - loss: 0.3308 - accuracy: 0.9177
Epoch 4/10
32/32 [==============================] - 0s 3ms/step - loss: 0.2427 - accuracy: 0.9320
Epoch 5/10
32/32 [==============================] - 0s 3ms/step - loss: 0.1401 - accuracy: 0.9757
Epoch 6/10
32/32 [==============================] - 0s 3ms/step - loss: 0.1046 - accuracy: 0.9879
Epoch 7/10
32/32 [==============================] - 0s 3ms/step - loss: 0.0840 - accuracy: 0.9864
Epoch 8/10
32/32 [==============================] - 0s 3ms/step - loss: 0.0713 - accuracy: 0.9946
Epoch 9/10
32/32 [==============================] - 0s 3ms/step - loss: 0.0562 - accuracy: 0.9925
Epoch 10/10
32/32 [==============================] - 0s 3ms/step - loss: 0.0405 - accuracy: 0.9994
1
2
3
4
5
# 가중치와 옵티마이저를 포함하여 정확히 동일한 모델을 다시 생성합니다
new_model = tf.keras.models.load_model('my_model.h5')

# 모델 구조를 출력합니다
new_model.summary()
Model: "sequential_25"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
dense_50 (Dense)             (None, 512)               401920    
_________________________________________________________________
dropout_25 (Dropout)         (None, 512)               0         
_________________________________________________________________
dense_51 (Dense)             (None, 10)                5130      
=================================================================
Total params: 407,050
Trainable params: 407,050
Non-trainable params: 0
_________________________________________________________________
1
2
loss, acc = new_model.evaluate(test_images,  test_labels, verbose=2)
print('복원된 모델의 정확도: {:5.2f}%'.format(100*acc))
32/32 - 0s - loss: 0.4255 - accuracy: 0.0890
복원된 모델의 정확도:  8.90%
1
2


You need to set install_url to use ShareThis. Please set it in _config.yml.

Like this article? Support the author with

You forgot to set the business or currency_code for Paypal. Please set it in _config.yml.

Comments

You forgot to set the shortname for Disqus. Please set it in _config.yml.
Follow

Links

Recent

Archives

Tags

fastcampus1
git1
python1

Subscribe to Updates

You need to set client_id and slot_id to show this AD unit. Please set it in _config.yml.
×