Enabling Large Batch Size Training for DNN Models Beyond the Memory Limit While Maintaining Performance

Computer Science

• Deep Learning Model 100%
• Deep Neural Network 100%
• Data Item 66%
• Memory Capacity 66%
• Multiple Device 33%
• Processing Method 33%
• Training Model 33%
• Target Machine 33%
• Graphics Processing Unit 33%

Keyphrases

• Batch Size 100%
• DNN Model 100%
• Deep Learning Model 42%
• Memory Capacity 28%
• Batch Processing 28%
• Memory Size 14%
• Multiple Devices 14%
• Processing Methods 14%
• Micro-batch 14%
• Small Batch 14%
• Training Model 14%
• Hyperparameters 14%
• Normalization Algorithm 14%
• Loss Normalization 14%
• Gradient Accumulation 14%