Good news! On the current trajectory, I am very hopeful nonlinear RNNs could make a comeback. Which would incidentally help to ease the memory pressure for inference tasks.