Process-level Online Recovery

This section describes how to construct a UCE fault of on-chip memory in the training code to adapt to the process-level online recovery verification.

• The modifications in this section are used only to guide you to verify the process-level online recovery function in the test environment. Do not use the instrumented version in the production environment.
• Before performing the operations in this section, ensure that the training can be started properly and process-level online recovery has been configured.
• To ensure the normal use of the process-level online recovery function, ensure that the clocks of the master and worker nodes in the Kubernetes cluster are the same.
• The code in the following sections may be different from the actual code. Use the actual code.

• MindCluster Adaptation
  
• Script Adaptation