Porting with sess.run

About sess.run

As a low-level API of TensorFlow, sess.run appears more flexible than Estimator. On the flip side, using it for model implementation could be complex.

Develop your training script with the sess.run API as follows:

1. Preprocess data.
2. Construct a model, calculate the loss, and update the gradient.
3. Create a session and initialize resources.
4. Start training.

The following describes how to port the sess.run training script for training on the Ascend AI Processor.

Header File Inclusion

To import NPU-related libraries, add this header file reference in related Python files as follows:

1

from npu_bridge.npu_init import *

After the preceding header file is imported, the training script is executed on the Ascend AI Processor by default.

Data Preprocessing

The code snippet is ready to use in normal cases. Manual tweaking is required only in the following scenario:

  dataset = dataset.batch(batch_size, drop_remainder=True)

 assert num_written_lines == num_actual_predict_examples

Model Construction, Loss Calculation, and Gradient Update

The code snippet is ready-to-use in normal cases. Manual tweaking is required only in the following scenario:

• Replace dropout in the original network with the corresponding CANN API for better performance. You must also pay attention to the impact on the accuracy.
  • If tf.nn.dropout exists, modify it as follows:

    layers = npu_ops.dropout()

  • If tf.layers.dropout, tf.layers.Dropout, tf.keras.layers.Dropout, tf.keras.layers.SpatialDropout1D, tf.keras.layers.SpatialDropout2D, or tf.keras.layers.SpatialDropout3D exists, add the following header file reference:

    from npu_bridge.estimator.npu import npu_convert_dropout

• Replace gelu in the original network with the corresponding CANN API:
  Original TensorFlow code:

  def gelu(x): 
    cdf = 0.5 * (1.0 + tf.tanh(
       (np.sqrt(2 / np.pi) * (x + 0.044715 * tf.pow(x, 3))))) 
    return x*cdf
  layers = gelu()

  Code after porting:

  from npu_bridge.estimator.npu_unary_ops import npu_unary_ops
  layers = npu_unary_ops.gelu(x)

Session Creation and Resource Initialization

When running your training script on the Ascend AI Processor by using sess.run, note the following configurations:

• The following configuration option is deactivated by default and should remain deactivated:

  rewrite_options.disable_model_pruning

• The following configuration options are activated by default and should remain activated:
  • rewrite_options.function_optimization
  • rewrite_options.constant_folding
  • rewrite_options.shape_optimization
  • rewrite_options.arithmetic_optimization
  • rewrite_options.loop_optimization
  • rewrite_options.dependency_optimization
  • rewrite_options.layout_optimizer
• The following configuration option is enabled by default and should be disabled explicitly:
  • rewrite_options.remapping
  • rewrite_options.memory_optimization
• If tf.device code is used in the original network, add the session configuration allow_soft_placement=True to allow TensorFlow to automatically allocate devices.

Original TensorFlow code:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16

# Construct an iterator.
iterator=Iterator.from_structure(train_dataset.output_types, train_dataset.output_shapes)

# Obtain the batch data.
next_batch=iterator.get_next()

# Initialize the iterator.
training_init_op=iterator.make_initializer(train_dataset)
 
# Initialize the variables.
init=tf.global_variables_initializer()
sess=tf.Session()
sess.run(init)
 
#Get the number of training/validation steps per epoch
train_batches_per_epoch=int(np.floor(train_size/batch_size))

Code after porting:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

# Construct an iterator.
iterator=Iterator.from_structure(train_dataset.output_types, train_dataset.output_shapes)

# Obtain the batch data.
next_batch=iterator.get_next()

# Initialize the iterator.
training_init_op=iterator.make_initializer(train_dataset)
 
# Initialize the variables.
init=tf.global_variables_initializer()

# Create a session. If tf.device code is used on the original network, add the session configuration allow_soft_placement=True to allow TensorFlow to automatically allocate devices.
config = tf.ConfigProto(allow_soft_placement=True)
custom_op = config.graph_options.rewrite_options.custom_optimizers.add()
custom_op.name = "NpuOptimizer"
# The remapping and memory_optimization functions of TensorFlow must be explicitly disabled to avoid conflicts with the functions of the NPU.
config.graph_options.rewrite_options.remapping = RewriterConfig.OFF  # Explicitly disable the function.
config.graph_options.rewrite_options.memory_optimization = RewriterConfig.OFF  # Explicitly disable the function.
sess = tf.Session(config=config)
sess.run(init)
 
#Get the number of training/validation steps per epoch
train_batches_per_epoch=int(np.floor(train_size/batch_size))

Training

The code snippet is ready-to-use. See the following example.

1
2
3
4
5
6
7
8
9

# Start epochs.
for epoch in range(num_epochs):
  ##Initialize iterator with the training dataset
  sess.run(training_init_op)
  for step in range(train_batches_per_epoch):  
    #get next batch of data
    img_batch,label_batch=sess.run(next_batch)
    #run the training op
    _,train_loss = sess.run([train_op, loss],feed_dict={x:img_batch, y_:label_batch, is_training:True})

After using the session object created by tf.Session, you need to explicitly call session.close() or use with to create a session (close() is automatically called through the context). For details, see the following examples:

Example 1: Explicit call of sess.close()

1
2
3

sess = tf.Session(config=config)
sess.run(...)
sess.close()

Example 2: Session creation using with

1
2

with tf.Session(config=config) as sess:
    sess.run(...)

If an error is reported during the porting and training, rectify the fault by referring to FAQs or contact technical support.