Documentation

Setting the Mixed Precision Mode

This section uses setting precision_mode_v2 to mixed_float16 as an example to describe how to set the mixed precision mode.

In Estimator Mode

  • Automated porting
    1. Check whether init_resource exists in the ported script.
      • If it exists, refer to the following example to pass the session_config configuration to the init_resource function and add the precision_mode_v2 parameter to session_config. 
         1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
        		 
        if __name__ == '__main__':

  session_config = tf.ConfigProto(allow_soft_placement=True)
  custom_op = session_config.graph_options.rewrite_options.custom_optimizers.add()
  custom_op.name = "NpuOptimizer"
  custom_op.parameter_map["precision_mode_v2"].s = tf.compat.as_bytes("mixed_float16")

  (npu_sess, npu_shutdown) = init_resource(config=session_config)
  tf.app.run()
  shutdown_resource(npu_sess, npu_shutdown)
  close_session(npu_sess)

        Note that only the parameters supported in initialize_system can be configured in config of the init_resource function. For other functions, configure them in run_config of the npu_run_config_init function.

      • If it does not exist, go to the next step.
    2. Search for npu_run_config_init in the ported script and locate the run configuration function, such as run_config in the example.
      If the session_config parameter does not exist in the run configuration function, add the parameter according to the following example. If the session_config parameter exists, go to the next step. 
      1
2
3
4
5
6
7
8
9
      		 
      session_config = tf.ConfigProto(allow_soft_placement=True)

run_config = tf.estimator.RunConfig(
  train_distribute=distribution_strategy,
  session_config=session_config,
  save_checkpoints_secs=60*60*24)

classifier = tf.estimator.Estimator(
  model_fn=model_function, model_dir=flags_obj.model_dir, config=npu_run_config_init(run_config=run_config))
    3. Modify the session_config configuration and add precision_mode_v2. 
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
      		 
      session_config = tf.ConfigProto(allow_soft_placement=True)
custom_op = session_config.graph_options.rewrite_options.custom_optimizers.add()
custom_op.name = 'NpuOptimizer'
custom_op.parameter_map["precision_mode_v2"].s = tf.compat.as_bytes("mixed_float16")

run_config = tf.estimator.RunConfig(
  train_distribute=distribution_strategy,
  session_config=session_config,
  save_checkpoints_secs=60*60*24)

classifier = tf.estimator.Estimator(
  model_fn=model_function, model_dir=flags_obj.model_dir, config=npu_run_config_init(run_config=run_config))
  • Manual porting
    In Estimator mode, set the precision mode by using the precision_mode_v2 parameter in NPURunConfig. 
    1
2
3
4
5
6
7
8
    		 
    from npu_bridge.npu_init import *

npu_config=NPURunConfig(
  model_dir=FLAGS.model_dir,
  save_checkpoints_steps=FLAGS.save_checkpoints_steps,
  session_config=tf.ConfigProto(allow_soft_placement=True,log_device_placement=False),
  precision_mode_v2="mixed_float16"
  )

In sess.run Mode

  • Automated porting
    1. Check whether init_resource exists in the ported script.
      • If it exists, refer to the following example to pass the session_config configuration to the init_resource function and add the precision_mode_v2 parameter to session_config. 
         1
 2
 3
 4
 5
 6
 7
 8
 9
10
        		 
        if __name__ == '__main__':
  session_config = tf.ConfigProto(allow_soft_placement=True)
  custom_op = session_config.graph_options.rewrite_options.custom_optimizers.add()
  custom_op.name = 'NpuOptimizer'
  custom_op.parameter_map["precision_mode_v2"].s = tf.compat.as_bytes("mixed_float16")

  (npu_sess, npu_shutdown) = init_resource(config=session_config)
  tf.app.run()
  shutdown_resource(npu_sess, npu_shutdown)
  close_session(npu_sess)

        Note that only the parameters supported in initialize_system can be configured in session_config of the init_resource function. For other functions, configure them in config_proto of the npu_config_proto function.

      • If it does not exist, go to the next step.
    2. Search for npu_config_proto in the ported script, find the run configuration parameter (such as session_config in the following example), and add precision_mode_v2 to the run configuration parameter, as shown in the following: 
      1
2
3
4
5
6
7
8
      		 
      session_config = tf.ConfigProto(allow_soft_placement=True)
custom_op = session_config.graph_options.rewrite_options.custom_optimizers.add()
custom_op.name = 'NpuOptimizer'
custom_op.parameter_map["precision_mode_v2"].s = tf.compat.as_bytes("mixed_float16")
config = npu_config_proto(config_proto=session_config)
with tf.Session(config=config) as sess:
  sess.run(tf.global_variables_initializer())
  interaction_table.init.run()
  • Manual porting
    In sess.run() mode, set the precision mode by using the session configuration option precision_mode_v2. 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
    		 
    import tensorflow as tf
from npu_bridge.npu_init import *

config = tf.ConfigProto(allow_soft_placement=True)

custom_op =  config.graph_options.rewrite_options.custom_optimizers.add()
custom_op.name =  "NpuOptimizer" 
custom_op.parameter_map["use_off_line"].b = True
custom_op.parameter_map["precision_mode_v2"].s = tf.compat.as_bytes("mixed_float16")
config.graph_options.rewrite_options.remapping = RewriterConfig.OFF
config.graph_options.rewrite_options.memory_optimization = RewriterConfig.OFF

with tf.Session(config=config) as sess:
  print(sess.run(cost))

In Keras Mode

  • Automated porting
    1. Check whether init_resource exists in the ported script.
      • If it exists, refer to the following example to pass the session_config configuration to the init_resource function and add the precision_mode_v2 parameter to session_config. 
         1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
        		 
        if __name__ == '__main__':

  session_config = tf.ConfigProto(allow_soft_placement=True )
  custom_op = session_config.graph_options.rewrite_options.custom_optimizers.add()
  custom_op.name = "NpuOptimizer" 
  custom_op.parameter_map["precision_mode_v2"].s = tf.compat.as_bytes("mixed_float16")
  ... ...

  (npu_sess, npu_shutdown) = init_resource(config=session_config)
  tf.app.run()
  shutdown_resource(npu_sess, npu_shutdown)
  close_session(npu_sess)

        Note that only the parameters supported in initialize_system can be configured in config of the init_resource function. For other functions, configure them in config of the set_keras_session_npu_config function.

      • If it does not exist, go to the next step.
    2. Search for the set_keras_session_npu_config function in the script, find the run configuration, for example, config_proto, and add precision_mode_v2 to the run configuration, as shown in the following: 
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
      		 
      import tensorflow as tf
import tensorflow.python.keras as keras
from tensorflow.python.keras import backend as K
from npu_bridge.npu_init import *

config_proto = tf.ConfigProto(allow_soft_placement=True)
custom_op = config_proto.graph_options.rewrite_options.custom_optimizers.add()
custom_op.name = 'NpuOptimizer'
custom_op.parameter_map["precision_mode_v2"].s = tf.compat.as_bytes("mixed_float16")
npu_keras_sess = set_keras_session_npu_config(config=config_proto)

# Preprocess data...
# Construct a model...
# Build the model...
# Train the model...
  • Manual porting

    The configuration method is similar to that of manual porting in sess.run mode. For details, see In sess.run Mode.

This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details