- When the slope grows exponentially instead of delaying this problem is known as 'Exploding Gradient' - While training RNN your slope becomes either too small or big and makes it difficult to train the model. The slope too small is known as 'Vanishing Gradient'