Saving and Loading Incremental Models

Usage Guide

To use the incremental model saving and loading function in the Rec SDK TensorFlow training framework, set the following parameters of the init API during framework initialization:

• is_incremental_checkpoint=True: enables the incremental model saving and loading function.
• save_checkpoint_due_time=xx: saves the full model every xx seconds.
• save_delta_checkpoints_secs=xx: saves the incremental model every xx seconds.
• restore_model_version=xx: loads the model whose step is xx. If this parameter is not specified, the latest model is loaded. If no model needs to be loaded, you do not need to set this parameter.

Sample

Sample:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

from mx_rec.util.initialize import init
# set init
init(train_steps=args.train_steps,
     eval_steps=args.eval_steps,
      save_steps=args.save_checkpoints_steps,
      max_steps=args.max_steps,
      use_dynamic=use_dynamic,
      use_dynamic_expansion=use_dynamic_expansion,
      save_checkpoint_due_time=4,
      save_delta_checkpoints_secs=2,
      is_incremental_checkpoint=True,
      restore_model_version=3)